Responsible AI by Design

Module 7 – Legal & Regulatory Landscape

Frank Rudzicz

2025-09-22

Lecture 7

  • This lecture = external law only: binding obligations, timelines, regulators
  • We skip internal policies, RACI, boards (that was Lecture 5)
  • Format: What applies to me? When? What to file? What to publish? What to prove?

From Governance to Regulation

%%{init: {"theme": "base", "themeVariables": { "fontSize": "32px", "fontFamily": "Inter, Arial, sans-serif" }}}%%
flowchart TD
    A[Governance<br> Internal] --> C[Trust and<br> Risk<br> Management]
    B[Regulation<br> External] --> C
%% Subtle palette (optional; safe in v11)
  classDef left fill:#eaf3ff,stroke:#7aa8e6,stroke-width:3px,color:#0f172a;
  classDef right fill:#ffecef,stroke:#e28190,stroke-width:3px,color:#0f172a;
  classDef hub fill:#ecfdf5,stroke:#10b981,stroke-width:3px,color:#064e3b;
  class A left; class B right; class C hub;

🇨🇦 Canada in 2025:
Where things stand

  • PIPEDA: still the federal private‑sector baseline (until new law passes)
  • Bill C‑27 (CPPA + AIDA): did not pass; ended on the Order Paper when Parliament was prorogued 6 Jan 2025.
    • Expect re‑introduction in modified form!
  • Action: build portable privacy & AI controls you can reuse under any successor bill.

👀 LegisInfo on C-27🔗

👀 Timeline of events🔗

🇨🇦 C‑27 Status → Your Plan

  • Now (binding):
    • Follow PIPEDA🔗 for private‑sector personal info
    • Honour provincial health info laws (PHIA🔗 in NS)
  • Prepare (portable controls):
    • Risk assessments for “high‑impact” AI use cases
    • Data & model lineage records
    • Deployment logs + human appeal process for consequential decisions
    • Vendor attestations about training data sources & evaluation

⚓️ Nova Scotia & Sector Statutes (binding now)

  • FOIPOP (NS)🔗 – access + privacy duties for public bodies
  • PIIDPA (NS)🔗 – data residency by default: public bodies & municipalities (and their service providers) must ensure personal information remains in Canada and is accessed/disclosed only in Canada, except statutory exceptions; applies to contracts from Dec 15, 2006 (municipalities +1 year)
  • PHIA (NS)🔗 – governs custodians of personal health information (consent, safeguards, retention, access)
  • Private sector in NS: generally under PIPEDA (federal)

Implication: If you are a public body or vendor to one → data residency & access controls are mandatory; ensure contract clauses reflect PIIDPA.

Canada: key instruments timeline

HealthCanada ML‑enabled
Medical Devices (5 Feb 2025)

  • Scope: Class II–IV MLMD submissions (new & amendments)
  • Expectations (examples):
    • Data management: representativeness; shift; labeling; privacy
    • Training/validation: performance claims; generalizability; bias analysis
    • Change management: algorithm change protocol (ACP) for future updates
    • Human factors: clinical workflow; user instructions; transparency
  • Evidence: testing reports, post‑market monitoring plans, cybersecurity posture

👀Health Canada Pre-market guidance for machine learning-enabled medical devices🔗

Required Artifacts to Keep
(start now)

  • Processing inventory (systems, purposes, data elements, lawful bases)
  • Cross‑border flow records (public bodies: PIIDPA exceptions invoked, vendor locations)
  • PIAs/DPIAs with AI screening; bias/fairness test plans & results
  • Human‑in‑the‑loop & appeal procedures (logs of decisions & reversals)
  • Vendor contracts: data residency, sub‑processor approval, audit rights, AEDT/NIST clauses where relevant
  • EU deals: technical file for high‑risk; GPAI disclosures (training data categories, evals)

🇪🇺 EU AI Act:
What non‑EU firms must watch

  • Entry into force: 1 Aug 2024; phased application through 2026/27
  • Key dates (current):
    • 2 Feb 2025: bans on unacceptable‑risk + AI literacy duties
    • 2 Aug 2025: governance rules & GPAI obligations start
    • 2 Aug 2026: most high‑risk obligations apply (some embedded products to 2027)
  • If you sell/provide into the EU: keep technical documentation, risk management file; plan conformity assessment (high‑risk); for GPAI: transparency, documentation, copyright & safety disclosures

🇪🇺 EU AI Act risk tiers

EU AI Act is risk-based: obligations (and fines) rise from Minimal → Limited → High-risk, with Unacceptable uses prohibited; key application dates: Feb 2025 (bans), Aug 2025 (GPAI), Aug 2026 (high-risk).

Note

EU AI Act — Risk tiers → duties

  • Unacceptable: banned uses (e.g., social scoring, real-time biometric ID).
  • High-risk: allowed with strict controls—risk mgmt, data governance, human oversight, logging, CE-mark/QMS, post-market monitoring.
  • Limited: transparency only (disclose AI interactions; label deepfakes).
  • Minimal: no mandatory duties (voluntary codes).

Why it matters now

  • In force: 1 Aug 2024; prohibitions Feb 2025; GPAI rules Aug 2025; most high-risk duties Aug 2026.
  • Penalties up to €35m/7% (prohibitions) and €15m/3% (other breaches).
  • Public sector/public services using high-risk AI: Fundamental Rights Impact Assessment (FRIA) required.

🏈 US comparator
(sectoral rules you might face)

  • NYC Local Law 144 (AEDT)🔗 (automated employment decsions) – requires:
    • Annual bias audit by an independent auditor
    • Public posting of audit summary + AEDT distribution date
    • Notice to candidates/employees at least 10 business days before use; provide info on qualifications assessed and an alternative process
    • Use of historical data preferred; test or pooled data allowed with conditions; no inferred demographics
  • NIST AI RMF 1.0🔗 – voluntary; four functions: Govern, Map, Measure, Manage; use as a control catalog for audits and vendor due diligence

Map → Measure → Manage
(NIST AI RMF → evidence)

  • Map: context, intended use, affected populations → risk statement, harm hypotheses
  • Measure: metrics/tests for robustness, bias, privacy → test protocols & datasets, confidence intervals
  • Manage: treatment plans, controls, monitoring, incident playbooks → risk register, KPIs, alerts
  • Govern: policies, roles, culture → charters, accountability matrices, audit trails

🏥 Sector: Healthcare (Canada/NS)

  • PHIA (NS) governs health info custodians
  • Health Canada: 2025 ML‑enabled medical devices pre‑market guidance → documentation expectations for AI/ML SaMD
  • Practical: treat clinical‑decision AI as regulated medical device pathway; align with ISO 13485, IEC 62304 where applicable

💸 Sector: Finance (Canada)

  • OSFI Guideline E‑23 (final Sept 11, 2025)🔗 – comprehensive Model Risk Management for all FRFIs; explicitly includes AI/ML models
  • Effective date: 1 May 2027 (transition runway to build/upgrade MRM)
  • Core expectations:
    • Model inventory (all models, incl. AI), materiality tiers
    • Independent validation & ongoing monitoring
    • Change mgmt for data/model/code; challenger models where appropriate
    • Data quality controls; lineage & documentation
    • Board/management oversight; roles & accountability

👷‍♀️ Sector: Employment/HR

  • Canadian HR & human rights law → non‑discrimination duties
  • If operating in NYC or hiring there: AEDT bias audit + notice (LL 144)
  • Practical: maintain selection‑rate monitoring & publish AEDT audit where required

Contractual Protections You Need

  • Data residency clauses (esp. NS public sector)
  • AI use restrictions (training on client data; derived models)
  • Conformance & audit: right to examine training data provenance, evaluation
  • Indemnities for IP, privacy breaches, bias claims
  • Incident notification SLAs (hours, content, regulatory contact)

Evidence You’ll Be Asked For

  • Decision logs & appeal outcomes for consequential uses
  • PIA/DPIA reports; risk assessments for high‑impact use cases
  • Bias testing protocols & results (method, datasets, thresholds)
  • Data lineage & cross‑border access records
  • Vendor GPAI disclosures (training data categories, evals) for EU deals

Wrap-Up

  • Canada now: PIPEDA, NS PIIDPA/PHIA/FOIPOP where applicable
  • Track successor to C‑27 (CPPA/AIDA); build portable controls now
  • EU AI Act timelines matter if you sell into the EU
  • US: AEDT (NYC) and NIST AI RMF show up via Request For Proposals (RFPs) and audits
  • Compliance = evidence + contracts + operational controls

Activity 7.1

EU AI Act Risk Mapping & Mini Compliance Plan

Instructions: Write concise answers in the template below. Target effort: ~20 minutes.

Scenario (fictional, EU context)
A municipal authority plans to deploy “AidAssist”, an AI system that:

  • Screens housing-benefit applications for eligibility and potential fraud.
  • Produces a risk score and a recommended decision for a caseworker (who can override).
  • Uses a general-purpose model fine-tuned on 5 years of local data (applications + outcomes).
  • Sends applicants a chatbot message explaining missing documents and next steps.
  • The vendor builds the model and supplies the software; the city hosts and operates it.

Activity 7.1 (continued)

Your tasks (checklist):

Activity 7.1 (continued)

Deliverables (submit one page)

  • Risk tier + one-sentence justification.
  • Role(s) (provider/deployer) + 1–2 lines explaining why.
  • Top 6 obligations (bulleted, one line each).
  • FRIA snapshot: 3 rights risks → mitigations (one line each).
  • Milestones split by Aug 2025 vs Aug 2026 (bulleted).
    (Aim for ~250–350 words total.)

Tip

  • Public-benefit eligibility systems typically fall under High-risk.
  • Public sector using high-risk AI → plan a FRIA before first use.
  • Avoid Unacceptable features (e.g., social scoring-like logic, manipulative design).
  • Don’t forget transparency for the chatbot; keep logs/traceability for audits.