Teaching / CSCI 4146 & CSCI 6409
Summer 2026 · Dalhousie UniversityProcess of Data Science
A model is only one step in a decision system. This project-driven course follows the full lifecycle: frame the problem, choose metrics, build and audit models, diagnose AI failures, estimate causal effects, and monitor what happens after deployment.
What you will learn
Translate a broad goal into north-star, input, and guardrail metrics using a structured data-science workflow.
Use SHAP, LIME, counterfactual reasoning, slicing, and subgroup analysis to find how and where models fail.
Analyze prompts, logs, embeddings, clusters, and LLM-as-a-judge pipelines to map failure patterns.
Apply causal inference, uplift modeling, and monitoring to support decisions and measure downstream impact.
The full lifecycle
Weekly syllabus
CRISP-DM, problem framing, business impact, and the Python/Jupyter environment
North-star and guardrail metrics; confusion matrices as business tools; dashboard workshop
Statistical significance, power, actionability, regression, classification, and validation pipelines
Calibration, overfitting, transparency, XGBoost/LightGBM, and preparation for model auditing
LIME, SHAP, counterfactuals, feature importance, partial dependence, and interaction effects
Slicing, error attribution, bias discovery, group fairness, proxy variables, and Fairlearn
Failure taxonomies for ML and LLM pipelines; logs, prompts, labels, and conversation analysis
Embeddings, clustering, semantic patterns, LLM judges, root-cause dashboards
Counterfactuals, propensity scores, matching, common support, and synthetic controls
Uplift modeling, heterogeneous treatment effects, Qini curves, ROI, and persuadables
Drift, feedback loops, silent failure, fairness decay, governance, Evidently, and Streamlit
The complete lifecycle, project showcase, industry deployment patterns, and final clinic
Five projects, one connected portfolio
Projects are completed in teams of one to three. Each produces a technical submission and a ten-minute “impression talk” modeled after a concise stakeholder update. Assessment emphasizes technical execution, actionable insight, debugging, communication, and the ability to defend decisions.
Assessment at a glance
This course is offered as CSCI 4146 and CSCI 6409. It assumes prior machine-learning foundations and focuses on the broader system in which models operate.