Foundations
What the model actually is, drawn rather than described. Tokens, vectors, attention, and the arithmetic that turns text into a probability.
2 ARTICLES IN THIS TRACK
MODULE 01 · LEARNING IN PUBLIC
The model is
the easy part.
The harness
is the product.
A visual lab about what is actually happening inside a language model, what has to be built around it before it becomes software, and how to read each new wave of AI tooling without buying the pitch.
TOKENS → VECTORS → ATTENTION CONTEXT · TOOLS · LOOPS EVALS OVER VIBES
WHAT SHIPSMODEL + HARNESS
NEXT TOKEN MODEL a function
01 CONTEXT
02 TOOLS
03 MEMORY
04 GUARDRAILS
05 EVALS
THE WORD "HARNESS"
Everything that decides what the model sees, what it is allowed to do, and what happens when it is wrong. The model answers. The harness is the software.
01 / WHAT THIS COVERS
Three tracks,
one argument.
You cannot design a good harness without a mental model of the thing you are wrapping, and you cannot judge a trend without both. So the module runs in three lanes and they keep referring to each other.
The harness
Everything around the model that makes it a product: context assembly, tools, memory, loops, permissions, evals, and cost.
4 ARTICLES IN THIS TRACK
Signal vs. hype
Each new wave — agents, MCP, context engineering — read against what it replaces and what it actually costs to run.
2 ARTICLES IN THIS TRACK
02 / THE WRITING
Start with the machine,
then the machinery.
Article 01 is the foundation the rest of the module leans on. Everything after it assumes you can picture a token becoming a vector and a vector becoming a probability.
ARTICLE 01 · FOUNDATIONS PUBLISHED
Inside a transformer
Text goes in, a probability distribution comes out. A visual walk from a token to the next token, with GPT-3’s 175 billion parameters accounted for one matrix at a time.
- Why the whole thing is one function, run in a loop
- Tokens, embeddings, and why direction carries meaning
- Attention and the MLP, and what each one is for
- Softmax, temperature, and where the 175 billion parameters live
Module index 4 PUBLISHED · 4 IN THE QUEUE
- 01 Inside a transformer ~18 min
- 02 Attention, worked out slowly Foundations
- 03 Harness engineering: making coding agents reliable ~14 min
- 04 Context engineering ate prompt engineering Harness
- 05 Evals are the only feedback you get Harness
- 06 Agents: what survives the demo Signal
- 07 What a production RAG system actually needs ~16 min
- 08 The agent that escaped the exam ~15 min
The queue is a direction, not a schedule. Articles ship when the diagrams hold up.
03 / HOW THIS IS WRITTEN
Four rules
for the whole module.
- 01
Draw it before believing it.
Most AI explanations are analogies stacked on analogies. If a mechanism cannot be drawn — the actual matrices, the actual loop, the actual numbers — it is usually not understood yet, only repeated.
- 02
The model is a component, not the product.
Swapping the model is the cheapest change you can make. What decides whether the thing works is the harness around it: what enters the context, which tools exist, what happens on failure, and who is allowed to approve what.
- 03
Every trend gets read against its cost.
Agents, MCP, long context, context engineering. Each one buys something real and charges for it in latency, tokens, or operability. The interesting question is never whether it is new — it is what it replaces.
- 04
Notes in public, mistakes included.
This module is written while learning and while shipping AI features at work. Where something is uncertain it says so, and where a number comes from a source, the source is named.
WHERE TO BEGIN
One token in.
One token out.
Everything a language model does is that step, repeated. Article 01 takes it apart completely — with diagrams instead of equations, and every number attributed.
Inside a transformer →