RoPE
The usual way a model is told where each token sits in the sequence, by rotating its representation by an angle proportional to its position.
what it is made of · the vocabulary
rotary embeddingsposition encoding
It matters commercially because it can be stretched after training: most of the models that grew from 8k to 128k context did so by scaling RoPE rather than by retraining, which is also why quality often sags in the last quarter of a very long window.