Paradoxes of Responsible AI

Frank Rudzicz

What is “Responsible AI”?

There is no single canonical definition, but there is a strong family resemblance across standards, policy, and scholarship:

Responsible AI is the design, development, deployment, and governance of AI systems in ways that are lawful, ethical, and technically robust, so that they support human rights and societal values rather than undermining them.

(adapted from Dignum (2019), High-Level Expert Group on Artificial Intelligence (2019), U.S. Department of Commerce (2023))

What is “Responsible AI”?

Across frameworks, technically, ‘Responsible AI’ often includes:

  • Technical robustness & safety
    Systems should be valid, reliable, safe, secure, and resilient in their intended context of use.
  • Fairness & non-discrimination
    Harmful bias should be identified and managed; outcomes should not systematically disadvantage already marginalised groups.
  • Transparency & explainability
    People should have meaningful insight into how systems work and how decisions are made.
  • Privacy & data governance
    Data collection and use should respect privacy, data protection, and appropriate limits on re-use.
  • Accountability & human oversight
    There should be clear lines of responsibility, routes for appeal, and the ability for humans to intervene or override.
  • Societal & environmental well-being
    AI should be aligned with broader public interests, fundamental rights, and sustainability goals.

Today

  1. 👷 Paradox 1 – Outsourced Infrastructure
    • AI is critical public infrastructure
    • ❌ But we have vendor lock-in and dependence on external platforms
  2. 🕵️‍♀️ Paradox 2 – Fairness vs. Privacy️‍
    • We want systems that are both fair and private
    • ❌ Privacy protections can worsen bias (and vice versa)
  3. 🤪Paradox 3 – Explanations vs. Human Cognition
    • Explanations are supposed to increase trust and help workflows
    • ❌ The problem is between the keyboard and the chair

Note

I’ll be using the term ‘paradox’ somewhat lightly.

Paradox 1: Infrastructure

👷 Depending on things we don’t control

AI is increasingly treated as infrastructure.

  • Large foundation models as a shared layer across sectors
  • Cloud-based APIs for vision, language, and recommendation
  • Integration into critical systems:
    • health records, case management, security, education

But most of this infrastructure is:

  • Built and maintained by a very small set of US companies
  • Hosted on their proprietary clouds
  • Governed by terms of service rather than public law
  • Frequently updated or deprecated outside our control (or knowledge!)

👷 AI as Infrastructure: Some signals

Paradox

Public institutions (universities, hospitals, governments) now rely on infrastructure they don’t own, can’t fully inspect, and struggle to govern.

  • This raises questions about resilience, sustainability, and democratic control (e.g., Whittaker (2021), West, Whittaker, and Crawford (2019)).
  • Discussions in industry and standards bodies emphasize AI as a foundational layer that organizations rely on, similar to electricity or networking.

These debates point to concrete risks:

  • Vendor lock-in: models, data pipelines, and workflows that are expensive to move elsewhere (Robbins and Wynsberghe 2022).
  • Opaque updates: model changes that alter behaviour without notice.
  • Asymmetric information: providers know far more about system performance and failure modes than regulators or users.

👷 Infrastructure Dependence

You might recognize these patterns:

  • Universities and schools
    • Rely on proprietary proctoring, plagiarism detection, or tutoring systems
    • Limited influence on training data, error analysis, or appeals processes
  • Health and social services
    • Third-party risk scores and triage systems integrated into workflows
    • Contracts or NDAs limiting transparency and external auditing
  • Cities and public agencies
    • Predictive analytics and “smart city” platforms operated by vendors
    • Difficulty in exiting contracts without major disruption

These are not just cases of “bad procurement”: they illustrate a structural tension between public accountability and privately controlled AI infrastructure.

👷 Governing Outsourced AI Infrastructure

Questions for Responsible AI as infrastructure:

  • What minimum conditions should be required for externally provided AI?
    • Access to documentation, evaluation reports, and change logs
    • Independent impact assessments and audits
    • Clear exit strategies and data portability
  • How can institutions pool capacity?
    • E.g., shared evaluation labs or public-interest testing centres?
  • What forms of public or civic AI infrastructure might complement or counterbalance commercial offerings?
    • Think 🚊 trains. In 🇨🇦 Canada, we run public trains on private tracks; in 🇪🇺 Europe, they run private trains on public tracks.

Paradox 2: Privacy vs Bias

🕵️‍♀️ Fairness needs data, Privacy limits data

Responsible AI conversations often emphasize both:

  • ⬆️ Fairness / non-discrimination
  • ⬆️ Privacy / data protection

At first glance, these values align. But in practice, they can be in tension:

  • To detect discrimination, we often need sensitive attributes:
  • Yet many legal frameworks discourage or prohibit collecting exactly those attributes.
  • Organisations may avoid collecting the data they would need to prove or improve fairness.

Paradox 2.1

The more we protect privacy by not collecting sensitive data, the harder it becomes to see and correct bias. But the more data we collect for fairness audits, the more we risk privacy harms.

🕵️‍♀️ How Privacy can reduce Fairness

Recent empirical work highlights complex interactions among privacy, accuracy, and fairness in machine learning:

  • Studies on differential privacy (DP) show that injecting noise to protect individuals can unevenly affect groups, sometimes amplifying error rates for already disadvantaged communities (Dadsetan et al. 2024).
  • Work in federated learning finds that privacy-preserving mechanisms and fairness-aware training can interfere with each other, creating unexpected trade-offs between privacy, fairness, and performance. (Wasif et al. 2025)
  • Other research shows that using DP to protect sensitive attributes may limit the ability of fairness-aware optimizers to correct disparities, unless carefully tuned to specific contexts (Gu et al. 2022; Wasif et al. 2025)

Paradox 2.2

Taken together, this suggests there is no free lunch: privacy mechanisms can protect individuals but may also mask or worsen structural inequities if deployed without fairness analysis.

Paradox 3: Explaining in vain

🤪 Explainable AI (XAI)

XAI is often presented as synonymous with trust.

  • Provide global explanations about what features matter overall.
  • Provide local explanations for individual decisions.
  • Improve trust and calibrate reliance so humans can work with AI.

But human cognition is messy:

  • We rely on heuristics and shortcuts.
  • We are susceptible to automation bias (over-trusting AI) and algorithm aversion (rejecting it outright).
  • Explanations can shape perception, not just inform it.

Paradox 3

Explanations designed to make AI more transparent can exacerbate overconfidence, confusion, or bias, especially when they are incomplete, misleading, or poorly aligned with human decision-making.

🤪 Explainability Pitfalls

Empirical studies have uncovered several patterns:

  • Information overload and simulation difficulty
    • Work on interpretable models shows that longer or more detailed explanations can actually make it harder for people to accurately “simulate” model behaviour or spot errors, even when models are transparent (Poursabzi-Sangdeh et al. 2021).
  • Explainability pitfalls
    • Recent HCI research introduces explainability pitfalls: unintended negative downstream effects of adding explanations, where users act against their own interests or align too closely with third-party goals because explanations subtly exploit cognitive shortcuts (Ehsan and Riedl 2021).

Tl;dr: explanations are interventions in human cognition, not neutral windows onto the model.

🤪 Explanations with Humans in Mind

Towards more responsible use of XAI:

  1. Start from the task and user, not the model
  • What decisions are being made?
  • What are the consequences of errors?
  • What does the human need to know to challenge or override the system?
  1. Support contestation, not just curiosity
  • Explanations should help people say no: to question, appeal, or escalate.
  • Provide actionable information (e.g., factors that can be challenged).
  1. Beware of plausible but unfaithful explanations
  • Where possible, empirically check that explanations track true model behaviour.
  • Be explicit about uncertainty and limitations.
  1. Test explanations like we test interfaces
  • User studies with the actual populations affected, not just lab surrogates
  • Measure not only “trust” or satisfaction, but error rates, calibration, and distribution of harms
  1. Combine explanations with governance
  • Explanations are not a substitute for:
    • Impact assessments
    • Independent audits
    • Clear lines of accountability and redress

Responsibility

Cross-cutting Themes

Across these paradoxes, a pattern emerges:

  • The hardest problems are institutional, not purely technical:
    • Dependence on infrastructure we don’t control
    • Trade-offs between privacy and fairness that require value judgments
    • Explanations that interact with human cognition and organisational incentives
  • Technical tools (privacy mechanisms, fairness metrics, XAI methods) are necessary but insufficient.

Cross-cutting Themes

Some cross-cutting questions to carry into your own work:

  • Where are you implicitly treating AI as infrastructure?
  • What assumptions are you making about who controls data and models?
  • How do you handle trade-offs – who gets to decide what counts as “fair enough” or “private enough”?
  • How are explanations (or their absence) shaping power, liability, and trust in your setting?

Final thoughts

🤖 Note

  • Responsible AI is about who controls infrastructure, how we manage trade-offs, and how humans actually think and decide with AI.
  • Addressing these issues will require cross-disciplinary work.




Thank you.

References

Dadsetan, Ali, Dorsa Soleymani, Xijie Zeng, and Frank Rudzicz. 2024. “Can Large Language Models Be Privacy Preserving and Fair Medical Coders?” In ML4H. http://arxiv.org/abs/2412.05533.
Dignum, Virginia. 2019. Responsible Artificial Intelligence: How to Develop and Use AI in a Responsible Way. Cham: Springer. https://doi.org/10.1007/978-3-030-30371-6.
Ehsan, Upol, and Mark O. Riedl. 2021. “Explainability Pitfalls: Beyond Dark Patterns in Explainable AI.” arXiv Preprint arXiv:2109.12480. https://doi.org/10.48550/arXiv.2109.12480.
Gu, Tianhao, Cynthia Resnick, Yizhen Wang, et al. 2022. “Privacy, Accuracy, and Model Fairness Trade-Offs in Machine Learning.” In 2022 IEEE Symposium on Security and Privacy (SP), 1132–49. IEEE. https://doi.org/10.1109/SP46214.2022.9833577.
High-Level Expert Group on Artificial Intelligence. 2019. “Ethics Guidelines for Trustworthy AI.” Brussels: European Commission. https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai.
Poursabzi-Sangdeh, Forough, Daniel G. Goldstein, Jake M. Hofman, Jennifer Wortman Vaughan, and Hanna Wallach. 2021. “Manipulating and Measuring Model Interpretability.” In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1–52. ACM. https://doi.org/10.1145/3411764.3445315.
Robbins, Scott, and Aimee van Wynsberghe. 2022. “Our New Artificial Intelligence Infrastructure: Becoming Locked into an Unsustainable Future.” Sustainability 14 (8): 4829. https://doi.org/10.3390/su14084829.
U.S. Department of Commerce, National Institute of Standards and Technology. 2023. “AI Risk Management Framework (AI RMF 1.0).” NIST AI 100-1. Gaithersburg, MD: National Institute of Standards; Technology. https://doi.org/10.6028/NIST.AI.100-1.
Wasif, Dawood, Dian Chen, Sindhuja Madabushi, Nithin Alluru, Terrence J. Moore, and Jin-Hee Cho. 2025. “Empirical Analysis of Privacy-Fairness-Accuracy Trade-Offs in Federated Learning: A Step Towards Responsible AI.” In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AIES). https://doi.org/10.48550/arXiv.2503.16233.
West, Sarah Myers, Meredith Whittaker, and Kate Crawford. 2019. “Discriminating Systems: Gender, Race, and Power in AI.” New York: AI Now Institute. https://ainowinstitute.org/publication/discriminating-systems-gender-race-and-power-in-ai-2.
Whittaker, Meredith. 2021. “The Steep Cost of Capture.” Interactions 28 (6): 50–55. https://doi.org/10.1145/3488666.