Broadcasting: how a [4,256] table adds to a whole batch
The positional-embedding sum quietly relies on broadcasting. Here is the whole rule — align from the right, stretch the 1s — and why PyTorch never copies a byte.
Tag
#maths-for-llms
- #agents
- #aggregation
- #ai
- #ai-agents
- #ai-tooling
- #architecture
- #artifacts
- #attention
- #automation
- #banking
- #batch
- #bean-validation
- #bert
- #binding
- #blackboard
- #branching
- #broadcasting
- #cgroups
- #ci-cd
- #classpath
- #claude
- #claude-code
- #cli
- #configuration
- #cosine-similarity
- #csp
- #data-modeling
- #dataviz
- #ddd
- #deep-learning
- #deepseek
- #design-system
- #developer-experience
- #developer-tooling
- #developer-tools
- #devex
- #devops
- #dns
- #docker
- #embabel
- #embeddings
- #events
- #fairness
- #fastmail
- #fine-tuning
- #flyway
- #from-java-to-python
- #frontend
- #fundamentals
- #gdpr
- #git
- #github-actions
- #goap
- #gpt
- #grafana
- #guardrails
- #hibernate-validator
- #hooks
- #http
- #human-in-the-loop
- #ideas
- #java
- #java-to-python
- #jinja2
- #junit
- #jvm
- #kafka
- #keycloak
- #knowledge-base
- #learning
- #linux
- #llm
- #llms
- #llms-txt
- #loki
- #loops
- #lora
- #machine-learning
- #maths-for-llms
- #mcp
- #mockito
- #mongodb
- #multi-agent
- #networking
- #nlp
- #oauth
- #oauth2
- #observability
- #okf
- #opentelemetry
- #otel
- #outbox
- #parking
- #patterns
- #positional-encoding
- #postgres
- #print-css
- #privacy
- #prompt-injection
- #python
- #pytorch
- #react
- #refactoring
- #rocm
- #search-console
- #secrets
- #security
- #self-attention
- #seo
- #shipping
- #skills
- #spring
- #spring-ai
- #spring-batch
- #spring-boot
- #spring-data
- #spring-data-jdbc
- #sse
- #ssh
- #state
- #systemd
- #templating
- #tempo
- #tensors
- #testing
- #tokenization
- #tools
- #traceql
- #tracing
- #traefik
- #training-data
- #transactions
- #transformers
- #tts
- #typescript
- #unsloth
- #uv
- #vectors
- #vite
- #waitfor
- #web-speech-api
- #workflow
- #worktree
Positional embeddings and vector addition
The maths under positional embeddings, from zero: a token becomes what it is plus where it sits, and that 'plus' is a literal element-wise sum of two vectors. Why the differently-sized tables never actually clash, worked by hand, with slides.
Cosine similarity and the dot product
A from-zero tour of the maths under large language models: how words become coordinates, why similarity is an angle, and the single multiply-and-add a computer uses to compare two meanings.