https://datatweets.com/courses/transformers-from-scratch/generation-and-sampling/Recent content in Generation & Sampling on DATATWEETSHugoenCopyright (c) 2026 DatatweetsFri, 10 Jul 2026 09:00:00 +0200https://datatweets.com/courses/transformers-from-scratch/generation-and-sampling/lesson-1-greedy-decoding-and-its-limits/Fri, 10 Jul 2026 09:00:00 +0200https://datatweets.com/courses/transformers-from-scratch/generation-and-sampling/lesson-1-greedy-decoding-and-its-limits/Greedy decoding is the simplest way to sample from a language model: at each step take the single most likely next token, append it, and feed it back. In this lesson you train the char-level MiniGPT on the Lantern Bay corpus, implement generate_greedy in pure NumPy, and watch it produce coherent but strictly deterministic text. You will see the same prompt yield byte-identical output twice, and watch greedy parrot the training passage and loop through it, motivating the temperature sampling you build next.https://datatweets.com/courses/transformers-from-scratch/generation-and-sampling/lesson-2-temperature-sampling/Fri, 10 Jul 2026 09:00:00 +0200https://datatweets.com/courses/transformers-from-scratch/generation-and-sampling/lesson-2-temperature-sampling/Greedy decoding was deterministic and repetitive because it always took the argmax. In this lesson you sample the next token from the model’s softmax distribution instead, then add a temperature T that rescales the logits before softmax to trade coherence against variety. You will watch the same logits sharpen at T=0.5 and flatten at T=2.0, measure the change in entropy, and generate real Lantern Bay text at T=0.5, 1.0, and 1.5 from your own MiniGPT, seeing two draws at T=1.0 differ while low T stays coherent and high T slides into misspellings.https://datatweets.com/courses/transformers-from-scratch/generation-and-sampling/lesson-3-top-k-sampling/Fri, 10 Jul 2026 09:00:00 +0200https://datatweets.com/courses/transformers-from-scratch/generation-and-sampling/lesson-3-top-k-sampling/Temperature sampling from Lesson 2 flattens the next-token distribution, but even after tempering the long tail of unlikely characters keeps a small chance of being drawn, and at higher temperatures those bad tokens occasionally slip through and wreck the text. Top-k sampling fixes this directly: keep only the k highest-probability tokens, set the rest to negative infinity, renormalize, and sample from what remains. In this lesson you implement top_k_filter and a generate function in pure NumPy on your trained MiniGPT, watch which tokens survive top-k for a real distribution, see k=1 reproduce greedy exactly, and compare k=3 against k=10 to feel the safety-versus-diversity trade-off.https://datatweets.com/courses/transformers-from-scratch/generation-and-sampling/lesson-4-top-p-nucleus-sampling/Fri, 10 Jul 2026 09:00:00 +0200https://datatweets.com/courses/transformers-from-scratch/generation-and-sampling/lesson-4-top-p-nucleus-sampling/Top-k sampling uses a fixed cutoff, which is too loose when the model is confident and too tight when it is genuinely uncertain. Top-p (nucleus) sampling fixes this by keeping the smallest set of top tokens whose cumulative probability reaches p, then renormalizing and sampling. In this lesson you train the char-level MiniGPT on the Lantern Bay corpus, implement top_p_filter and a nucleus generate in NumPy, and measure on two real next-token distributions how the nucleus keeps just 1 token when the model is peaked and 4 when it is uncertain, all at the same p = 0.9.https://datatweets.com/courses/transformers-from-scratch/generation-and-sampling/lesson-5-guided-project-a-text-generator/Fri, 10 Jul 2026 09:00:00 +0200https://datatweets.com/courses/transformers-from-scratch/generation-and-sampling/lesson-5-guided-project-a-text-generator/In this guided project you fold every decoding strategy from the module into a single configurable generate() function. You train the char-level MiniGPT on the Lantern Bay corpus to a loss near 0.13, then build one generator that forwards the last block_size tokens, applies temperature, an optional top-k filter, an optional top-p (nucleus) filter, renormalizes, and samples, or takes the argmax when greedy is set. You run the same prompt under greedy, temperature, top-k, and nucleus settings, read the real outputs side by side, and finish by generating a short passage from your own model.