Transformer
The architecture nearly every modern model uses: layers that let each token look at every other token and decide which ones matter.
what it is made of · the vocabulary
attention is all you need
Introduced in 2017. Its two useful properties are that it trains in parallel, which made scale affordable, and that its cost grows with the square of the sequence length, which is why long context is expensive and why most of the last five years of engineering has gone into working around that one fact.