A language model like ChatGPT or Claude is not magic. It is one tiny calculation, the one on the right, repeated billions of times and tuned on a huge pile of text.
This site goes from plotting a few dots all the way up to how a chatbot writes a reply. Every lesson is a small page where you poke at the idea and see the code that runs it.
Start at lesson 1One neuron: should I bring an umbrella?
Each input is multiplied by a weight, added up with a bias, then squashed to a number between 0 and 1. Move the weights and watch the decision change.
Stacked the way a neural network is drawn: foundations at the bottom, the full language model at the top. Start at lesson 1 at the bottom and work up. Tick a lesson when it clicks for you.
What it means for a computer to "learn" at all, before any neural networks.
How does a photo, a word or a house become numbers a computer can learn from?
Turn a small table of real things into points on a plot.
Can you predict something just by looking at what it's close to?
Classify a clicked point by letting its neighbours vote.
How do you say, with one number, how wrong a model is?
Drag a line through data and watch the loss change.
How does a model fix itself without anyone telling it the answer?
Roll a ball downhill on the loss surface, one step at a time.
The building block behind every modern AI, and how millions of them learn together.
What is a single artificial neuron actually computing?
Weights, a bias and an activation, with sliders for each.
Why can stacking simple neurons draw curved boundaries that one neuron can't?
Solve the XOR dataset from knn.php with a tiny network.
When the output is wrong, how much is each weight to blame?
Follow the error backwards through a network, number by number.
Why do models memorise instead of learn, and how do we stop it?
Watch overfitting happen, then fix it with more data and a validation set.
How text gets into a network, and the one idea that made modern language models work.
Why do language models read "unbelievable" as un + believ + able?
A byte-pair tokenizer that learns its own word pieces.
How can "king − man + woman" land near "queen"?
Place words in space so similar meanings end up close together.
embeddings.php
What is the one task every LLM is trained on?
A word-counting model that writes sentences, badly but honestly.
nextword.php
In "the cat sat because it was tired", how does the model know "it" means the cat?
Queries, keys and values, shown as a grid of who looks at whom.
attention.php
What happens inside one layer of a model like GPT or Claude?
Put attention, a small network and residual connections together.
transformer.php
Can you build a GPT from scratch, train it, and chat with it?
A real transformer in PHP that learns names, poetry or a tiny chat, then writes back.
Scaling everything up, and turning a text predictor into an assistant.
What changes when the data is most of the internet and the weights number in billions?
Estimate the compute, data and cost of training a real model.
pretraining.php
Why does the same question give different answers each time?
Turn the temperature dial on a probability list and see what gets picked.
sampling.php
How does a model that finishes text learn to follow instructions and be helpful?
Walk through instruction tuning and learning from human feedback.
finetuning.php
Why do models hallucinate, and what are people doing about it?
Test the ideas from every earlier lesson against real failure cases.
limits.php
You start here, at the bottom, with dots on a plot.