Capstone

Welcome to the Capstone, the final stop in the course. Every module built one part of the transformer and proved it correct in isolation. This project puts them all in one place and runs them end to end: you’ll build the complete mini-GPT, train it until it learns, and let it write — a full language model, from the first matrix multiply to the last generated character, entirely by your own hand in NumPy.

You’ll assemble the architecture from the pieces you already have — token and position embeddings, stacked causal transformer blocks of multi-head attention and feed-forward layers, a final layer norm, and the language-modeling head — then run the training loop with Adam, watch the loss fall from the random baseline of about 3.18 toward fluency, and generate a passage with temperature and top-k sampling. A closing report traces each part of the finished model back to the module that taught it.

Start the lesson to build, train, and sample from your own transformer, top to bottom.