LLMs, the whole thing · Part 8
Causal attention and the two masks: one hides the future, one cuts links on purpose
Section 3.5 of Raschka's book on seven real GPT-2 tokens: why masking after softmax breaks the row sums, why -inf on the scores fixes it in one pass, and how dropout is a second, random mask that doubles its survivors.