Conversation models that run on 32 GB

515 models with published weights that fit in 32 GB — a well-specified laptop. Counted at four-bit quantisation, weights only, leaving the machine about a third of its memory and a gigabyte for context: room for roughly 33 billion parameters. A mixture of experts is counted in full, because it is held in full even though only a few experts compute. 515 in all, a hundred to a page; this is page 6 of 6.

Enough for a thirty-billion-parameter model at four-bit with a long context, or a smaller one at higher precision if quality matters more than size. A practical ceiling for a laptop that also has to be a laptop.

Models that answer in prose across whatever subject you put to them. This is the largest category and the least differentiated: most of them are competent at most things, so the choice usually comes down to price, context length and how the maker behaves about availability. Look at what a model costs per million tokens in and out — the output rate is often four or five times the input rate, and it is the one that decides your bill.

• bert-base-chineseBERT community0.1B≈0.1 GB at 4-bit
• BEN2Prama LLC0.1B≈0.1 GB at 4-bit
• distilgpt2DistilBERT community0.1B≈0.1 GB at 4-bit
• vit-base-patch16-224Google0.1B≈0.1 GB at 4-bit
• nsfw_image_detectionFalcons.ai0.1B≈0.1 GB at 4-bit
• distilbert-base-uncasedDistilBERT community0.1B≈0.0 GB at 4-bit
• t5-smallT5 community0.1B≈0.0 GB at 4-bit
• RMBG-1.4BRIA AI0.0B≈0.0 GB at 4-bit
• detr-resnet-50AI at Meta0.0B≈0.0 GB at 4-bit
• table-transformer-structure-recognitionMicrosoft0.0B≈0.0 GB at 4-bit
• table-transformer-detectionMicrosoft0.0B≈0.0 GB at 4-bit
• segformer_b2_clothesmattmdjaga0.0B≈0.0 GB at 4-bit
• segformer-b0-finetuned-ade-512-512NVIDIA0.0B≈0.0 GB at 4-bit
• Ornith-1.0-9BOrnith0.0B≈0.0 GB at 4-bit
• AutoGLM-Phone-9BZ.ai0.0B≈0.0 GB at 4-bit

Wider