Data & target
Define examples, labels and sampling so the training problem represents the intended deployment task.
Side 237Side Studies / Research
Subject
Machine Learning
Purpose
Predictive and generative models studied through data, objectives, representations, optimization, generalization and deployment under distribution shift.
Structure
05 movesSystem mapV0
Components → constraints → flows → control → failure
Learn patterns without confusing fit with knowledge.
Machine learning is a pipeline from data-generating process to objective to trained model to deployment, and every transition can introduce failure even when benchmark accuracy is high.
Representation & model class
Choose features or learned architectures that encode useful inductive biases without silently excluding relevant variation.
Objective & optimization
Train by minimizing a surrogate loss while recognizing that optimization success does not guarantee the right behavioral objective.
Generalization
Evaluate on unseen data and diagnose overfitting, leakage, calibration and subgroup performance.
Deployment & shift
Monitor changing inputs, feedback loops, robustness and human use after the model leaves the benchmark environment.
Keep the boundaries visible.
training loss ≠ real-world utility
prediction ≠ causal explanation
benchmark improvement ≠ general capability
Questions that organize the Side.
What distribution is the model actually expected to generalize to?
Which errors are hidden by aggregate performance metrics?
How should objectives change when model outputs alter the future data it receives?
What should carry weight here?
Use held-out and external evaluation, ablations, calibration and stress tests; claims about mechanisms or causality require evidence beyond predictive performance.