Operationalizing machine learning in healthcare requires more than training an accurate model.
This course examines the chain from clinical purpose and care processes to data, study design, modelling, evaluation, workflow, deployment, monitoring, and accountable use.
Course information
Term
Fall 2026
Instructor
Frank Rudzicz
Class meeting
Thursdays, 17h30-20h30
Course communication
MS Teams for registered students
This public page is the course map and quick-start guide. Current announcements, topic registration, presenter assignments, submissions, and private class work remain in MS Teams or the private studio repository.
AIRO-Sim Project
The organizing project for DGIN 5401
Project in one sentence
The class will build and study a shared, transparent cardiac-surgery risk system using fully synthetic AIRO-Sim data. Each student will lead a distinct research question, steward a shared artefact, and submit an individual research paper; the class will also produce a concise shared system dossier and an integrated demonstration.
Start here
Course Project Handbook
Project purpose, question allocation, checkpoints, assessment, individual and shared responsibilities, repository practice, and final deliverables.
AIRO-Sim Technical Brief
Release-specific tables, joins, prediction snapshots, outcome construction, partitions, text use, system interfaces, monitoring, and final checks.
AIRO-Sim v1.0.0
The frozen public student release, including source files, metadata, documentation, checksums, and starter scripts.
Private studio repository
Class code, versioned contracts, shared artefacts, issues, reviews, models, case-viewer work, and integration. Access details will be posted in MS Teams.
Registered students · link in Teams
AIRO-Sim: obtain, verify, and begin
The course dataset
Release at a glance
8,000 synthetic operations
7,500 synthetic patients
20,000 synthetic notes
2018–2025 synthetic calendar years
The release provides compressed CSV tables, a SQLite database, longitudinal events, naturalistic synthetic notes, reference partitions, sample cases, metadata, documentation, and starter validation/loading scripts. It was built most directly around postoperative delirium and also contains superficial and deep sternal-infection outcomes that may support a class-selected alternative task.
Release documentation
Obtain AIRO-Sim
Use the frozen tag airo-sim-v1.0.0 for course work and record that version in every derived artefact.
- Open the AIRO-Sim v1.0.0 release.
- Under Assets, download Source code (zip) or Source code (tar.gz).
- Extract the archive.
- Open a terminal in the extracted directory containing
PACKAGE_SHA256SUMS.txt,data/,docs/,metadata/, andstarter/.
git clone \
--branch airo-sim-v1.0.0 \
--depth 1 \
https://github.com/Dalhousie-A2I2/AIRO-Sim.git
cd AIRO-SimCloning the tag normally leaves the checkout in a detached-HEAD state. That is appropriate for a frozen data release. Keep project code and derived data in the private studio repository rather than editing the release in place.
Verify the package
Run the checksum command from the release root.
shasum -a 256 -c PACKAGE_SHA256SUMS.txtsha256sum -c PACKAGE_SHA256SUMS.txt$failed = $false
Get-Content PACKAGE_SHA256SUMS.txt | ForEach-Object {
if ($_ -match '^([0-9a-fA-F]{64})\s+\*?(.+)$') {
$expected = $matches[1].ToLower()
$path = $matches[2]
$actual = (Get-FileHash -Algorithm SHA256 -LiteralPath $path).Hash.ToLower()
if ($actual -eq $expected) {
Write-Host "$path`: OK"
} else {
Write-Error "$path`: FAILED"
$failed = $true
}
}
}
if ($failed) { exit 1 }Every listed file should report OK. A failed checksum means that the release is incomplete, altered, or was extracted incorrectly; stop before analysis and obtain a clean copy.
Create a Python environment
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r starter/requirements.txtpy -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -r starter\requirements.txtDo not commit .venv/ to a repository. Record the environment used for final analyses through a dependency file or container.
First successful use
python starter/validate_release.py
python starter/load_data.pyThe validator should report:
Core release checks passed.
Controlled audit anomalies are expected; see the dataset card.Before we fit a model, we should understand the units, keys, partitions, timings, and fields that may confound our model.
Load the data
pandas reads the supplied .csv.gz files directly; they do not need to be decompressed first.
from pathlib import Path
import pandas as pd
ROOT = Path(".")
core = pd.read_csv(ROOT / "data" / "mhc_synthetic_core.csv.gz")
notes = pd.read_csv(ROOT / "data" / "notes.csv.gz")
splits = pd.read_csv(ROOT / "data" / "splits_reference.csv")
dictionary = pd.read_csv(
ROOT / "metadata" / "synthetic_data_dictionary.csv"
)
primary_t0_features = dictionary.loc[
(dictionary["included_in_core"] == 1)
& (dictionary["allowed_primary_T0"] == 1)
& (dictionary["recommended_role"] == "feature"),
"variable_name",
].tolist()
print(core.shape)
print(core["patient_id"].nunique())
print(len(primary_t0_features))import sqlite3
import pandas as pd
with sqlite3.connect("data/airo_sim.sqlite") as connection:
tables = pd.read_sql_query(
"""
SELECT name
FROM sqlite_master
WHERE type = 'table'
ORDER BY name
""",
connection,
)
core = pd.read_sql_query(
"SELECT * FROM mhc_synthetic_core",
connection,
)
print(tables)
print(core.shape)Aspects to understand before modelling
Unit and keys
One row in the core table is one operation. operation_id is the unique operation key; patient_id links repeated operations.
Patient grouping
No patient may appear in both development and held-out partitions. Split and resample at the unit that can leak.
Prediction snapshots
T0, T1, T2, outcome-only, and research-only fields have different permissions. Pay attention to plausible data availability.
Outcomes
The outcome must define positive, negative, missing, and exclusion rules.
Notes and events
Filter note type, note time, event time, and available_snapshot before constructing features. Discharge or post-outcome text cannot enter T0 or T1 prediction.
Held-out evidence
The supplied test and shadow labels are visible, so they are not a blind external test. Prespecify their use and do not tune repeatedly against them.
A first audit
assert len(core) == 8_000
assert core["operation_id"].is_unique
assert core["patient_id"].nunique() == 7_500
assert core.groupby("patient_id")["split_reference"].nunique().max() == 1
print(
core.groupby("split_reference")["comp_neuro_delirium"]
.agg(nonmissing="count", prevalence="mean")
)Then investigate, document, and decide rather than merely asserting:
- row counts, duplicate identifiers, and join multiplicity;
- repeated patients and partition overlap;
- missing outcomes and predictors by partition and relevant subgroup;
- impossible or extreme numeric values;
- parent–child consistency among complication fields;
- planned versus performed procedure differences;
- timestamp and prediction-snapshot consistency;
- structured–note contradictions, omissions, and provenance;
- note-template or near-duplicate leakage; and
- changes in missingness, procedure mix, note style, and predictions in the shadow period.
- Editing release files in place instead of creating versioned derived artefacts.
- Treating
99in the raw table as a clinical category without consulting the dictionary. - Treating missing outcomes as negative outcomes.
- Using identifiers, split labels, synthetic year, discharge information, or outcome-only fields as ordinary predictors.
- Allowing one patient, note template, or near-duplicate note to cross a boundary that is supposed to be held out.
- Importing T1, T2, discharge, or post-outcome information into a T0 study.
- Repeatedly tuning decisions on the test or shadow partition while still calling it held out.
- Describing synthetic event rates, note language, associations, subgroup differences, or model performance as estimates of a real institution.
Lectures and course materials
The lecture decks are linked from their beginning. Use the deck table of contents or overview mode to move to a section.
Lecture 1 From a clinical need to a monitored, accountable system
Course purpose, the shared research studio, seminar practice, intended use, evidence, workflow, and recurring category mistakes.
Lecture 2 Clinical data science foundations: from care to evidence
How care becomes data, how data become a study, how analysis is computed, and how results become defensible evidence.
Lecture 3 Honest models and meaningful evaluation
Study cards, baselines, information budgets, model restraint, calibration, thresholds, uncertainty, transport, action, and audit.
Later lectures ### Materials will be added as the course develops
Future links will be added here.
Forthcoming
Seminar papers
| Date | Theme | Paper | Why we are reading it |
|---|---|---|---|
| 24 September 2026 | Evaluation | Author et al. Paper title | Calibration, validation, and claim boundaries |
How seminars work
A seminar should help a mixed clinical–technical audience faithfully reconstruct and understand the paper’s contributions but also critically test it.
Additional resources
These links are starting points rather than substitutes for critical appraisal or the original scientific record.
Reporting, bias, and lifecycle guidance
- TRIPOD+AI — reporting studies that develop or evaluate clinical prediction models using regression or machine-learning methods.
- PROBAST+AI — assessing quality, risk of bias, and applicability in prediction-model studies.
- DECIDE-AI — reporting early-stage live clinical evaluation of AI decision-support systems.
- FUTURE-AI — lifecycle guidance for trustworthy and deployable AI in healthcare.
Finding and following evidence
- PubMed and Cochrane Library for biomedical literature, reviews, and clinical terminology.
- OpenAlex and Semantic Scholar for citation trails and related work.
- PMLR and OpenReview for machine-learning methods, reviews, and discussion records.
- ClinicalTrials.gov for registrations, protocols, and prospective study records.
- Zotero for a reproducible shared reading and citation workflow.