Structured extraction models that run on 96 GB

19 models with published weights that fit in 96 GB — a MacBook Pro with 96. Counted at four-bit quantisation, weights only, leaving the machine about a third of its memory and a gigabyte for context: room for roughly 102 billion parameters. A mixture of experts is counted in full, because it is held in full even though only a few experts compute.

An Apple configuration for people who intend to run models rather than occasionally try one. Comfortably holds the seventy-billion class with a very long context, or a mixture-of-experts model whose active parameters are few but whose weights are all resident.

Tools that pull structure out of unstructured input — fields from an invoice, a schema from a page, a table from a report. Some are models, some are services with a model inside. Judge them on what they do when the input is malformed, because that is the whole job; anything can parse a clean file.

Wider

Structured extraction models Models that run on 96 GB