https://datatweets.com/courses/transformers-from-scratch/capstone/Recent content in Capstone on DATATWEETSHugoenCopyright (c) 2026 DatatweetsFri, 10 Jul 2026 09:00:00 +0200https://datatweets.com/courses/transformers-from-scratch/capstone/lesson-1-guided-project-build-and-train-your-own-mini-gpt/Fri, 10 Jul 2026 09:00:00 +0200https://datatweets.com/courses/transformers-from-scratch/capstone/lesson-1-guided-project-build-and-train-your-own-mini-gpt/The course capstone. You assemble the full mini-GPT from the pieces built across eight modules — token and position embeddings, causal multi-head attention, stacked transformer blocks, and the language-modeling head — into one 154,456-parameter model. You confirm the forward pass gives (32, 32, 24) logits and an initial loss of 3.20 (right at log 24), then train it with Adam for 1200 steps in about 24 seconds, watching train and validation loss fall from 3.20 to about 0.13. Finally you generate text with greedy and temperature-plus-top-k decoding — coherent Lantern Bay English — and trace the finished model back through every module by name. Everything runs for real in pure NumPy and reproduces byte-for-byte from a fixed seed.