How to Scale Dot-Product Attention by sqrt(d_k) in NumPy

A focused NumPy experiment that measures how query-key width changes dot-product scores, attention concentration, and entropy, then verifies why square-root scaling removes the width effect under standard assumptions.

July 22, 2026 in Machine Learning, Python by Mehdi Lotfinejad10 minutes