Rules and representations.
Knowledge is encoded in symbols, logic, search spaces, constraints and explicit procedures.
Side 13
Artificial
Intelligence
A study of machines that perform tasks requiring representation, search, learning, prediction, generation, planning or control. The Side treats AI as a family of computational approaches rather than one technology or one historical moment.
represent→learn / search→predict / plan→act→evaluate
06paradigms
05learning problems
06evaluation lenses
13Side
AI is not one method.
Different paradigms solve different computational problems. Some encode knowledge explicitly; others learn patterns from examples; many modern systems combine multiple approaches.
Reason under uncertainty.
Models represent uncertain states, observations and dependencies using probability.
Learn from data.
Parameters or decision rules are inferred from examples rather than specified entirely by hand.
Learn layered representations.
Multi-layer neural networks transform data through learned intermediate features.
Learn from consequences.
An agent selects actions, receives feedback and learns policies that improve expected cumulative reward.
Combine strengths.
Search, rules, learned models, tools, memory and external verification can be composed into larger systems.
Intelligence is a behavioral description, not a single internal mechanism.
Two systems can solve the same task through very different representations, training procedures and computational pathways.
What signal teaches the system?
Learning problems differ primarily in the information available during training and the objective being optimized.
| Learning setup | Training signal | Typical objective | Example |
|---|---|---|---|
| Supervised | Inputs paired with target labels or values. | Predict the target on unseen examples. | Classifying images or estimating prices. |
| Unsupervised | Unlabeled observations. | Discover structure, density or compact representations. | Clustering or dimensionality reduction. |
| Self-supervised | Targets generated from the data itself. | Learn representations by predicting hidden or future parts. | Next-token or masked-token prediction. |
| Reinforcement | Rewards generated through interaction. | Maximize expected cumulative reward. | Game play, control and sequential decision problems. |
| Preference learning | Comparisons, rankings or evaluative feedback. | Align behavior with a learned preference signal. | Ranking responses or actions. |
Generalization problemtraining performance ≠ performance on the distribution that matters
data+objective+model class+optimization→learned behavior
Representation through layers.
Neural networks transform inputs through parameterized functions whose weights are adjusted to reduce an objective function.
Encode the observation.
What numerical representation enters?
Tokens, pixels, sensor readings or structured features must first become model-readable values.
Transform through layers.
What activations result?
Each layer combines learned weights with prior activations to produce a new representation.
Measure disagreement with the objective.
How wrong was the output?
The loss function converts model behavior into an optimization signal.
Estimate how parameters affect loss.
Which direction reduces error?
Backpropagation efficiently propagates derivative information through the computation graph.
Change the weights.
How large a step?
An optimizer adjusts parameters, then the process repeats across many examples.
Local spatial structure
Convolutional networks exploit locality and shared filters, especially in image-like data.
Sequential state
Recurrent networks carry information through a sequence using evolving hidden state.
Attention across context
Attention mechanisms allow elements in a sequence to condition on other elements with learned relevance weights.
One training process, many downstream tasks.
Foundation models are trained broadly enough that the same underlying model can support many tasks through prompting, fine-tuning, retrieval, tools or other adaptation.
Pretraining
Large-scale training learns statistical structure from broad data using a generic predictive or generative objective.
Representation
The model develops internal features that can support multiple tasks without being manually specified one by one.
Instruction tuning
Additional training can make behavior more responsive to task descriptions and preferred response formats.
Retrieval
External information can be supplied at inference time so responses need not rely solely on parameters learned during training.
Tool use
A model can call calculators, code interpreters, databases, browsers or other systems when the task benefits from external capability.
Multimodality
Models can be trained across text, images, audio, video or other signals, allowing information to move between modalities.
Fluent generation is not the same as grounded knowledge.
A model can produce well-formed output even when its internal prediction is unsupported. Verification and source grounding remain separate system functions.
From response generation to action loops.
An agentic system repeatedly observes state, chooses an action, changes the environment and evaluates what happened.
goalobserveplan / chooseactinspect / repeat
Decompose before acting.
Large goals can be converted into smaller steps whose dependencies are easier to manage.
Carry useful state forward.
Working, episodic or external memory can preserve information that would otherwise disappear between steps.
Extend beyond the model.
External actions can retrieve facts, perform exact calculations or modify real systems.
Inspect intermediate work.
A system can test outputs against constraints or evidence before proceeding.
Authority is part of architecture.
Read, write and irreversible actions require different levels of control and verification.
Reality pushes back.
Agent performance depends on external state, latency, interfaces, failures and the observability of consequences.
What does “good” mean?
AI evaluation is multidimensional. A system can be strong on average while failing catastrophically on the exact cases that matter.
Capability
Can it do the task?
Measure success on representative tasks, not only polished examples.
Generalization
Does performance survive new cases?
Test outside memorized or highly similar training examples.
Calibration
Does confidence track correctness?
Useful systems should distinguish what they know reliably from what remains uncertain.
Robustness
What happens under perturbation?
Small changes in wording, data quality or environment should not cause uncontrolled failure.
Alignment
Is behavior consistent with intended constraints?
The objective used in training or prompting can diverge from what operators actually want.
System impact
What happens after deployment?
Human adaptation, incentives, misuse, distribution shift and workflow changes belong to evaluation too.
Evaluation habitbenchmark score → representative task → failure distribution → real workflow → monitored consequences
Artificial Intelligence: A Modern ApproachRussell & Norvig · broad field map
Deep LearningGoodfellow, Bengio & Courville · neural foundations
Reinforcement LearningSutton & Barto · sequential learning
Understanding Deep LearningSimon J. D. Prince · modern neural systems