Reading documents models that run on 64 GB

24 models with published weights that fit in 64 GB — a workstation. Counted at four-bit quantisation, weights only, leaving the machine about a third of its memory and a gigabyte for context: room for roughly 67 billion parameters. A mixture of experts is counted in full, because it is held in full even though only a few experts compute.

A workstation, or a well-specified Mac. Seventy-billion-parameter models fit at four-bit, which is the class where local output stops being obviously worse than the hosted models people pay for. Loading takes real time and the machine will be warm.

Models that read documents — scans, photographs of pages, forms, tables — and return text or structure. The interesting difference between them is not whether they can read a clean page, which they all can, but what they do with a table, a stamp, a handwritten margin or a column that breaks across pages. Several are priced per page rather than per token, which makes them easy to budget and hard to compare with the rest.

• chandraDatalab8.8B≈5.7 GB at 4-bit
• NuMarkdown-8B-ThinkingNuMind8.3B≈5.4 GB at 4-bit
• RolmOCRReducto8.3B≈5.4 GB at 4-bit1 also selling it hosted
• olmOCR-7B-0225-previewAi28.3B≈5.4 GB at 4-bit
• llava-v1.6-mistral-7b-hfLlava Hugging Face7.0B≈4.5 GB at 4-bit
• olmOCR-2-7B-1025Allen Institute for AI (Ai2)7.0B≈4.5 GB at 4-bit1 also selling it hosted
• olmOCR-7B-0725-FP8Allen Institute for AI (Ai2)7.0B≈4.5 GB at 4-bit1 also selling it hosted
• olmOCR-7B-0825Allen Institute for AI (Ai2)7.0B≈4.5 GB at 4-bit1 also selling it hosted
• chandra-ocr-2Datalab5.3B≈3.4 GB at 4-bit
• NuExtract3NuMind4.5B≈3.0 GB at 4-bit
• Unlimited-OCRBaidu3.3B≈2.2 GB at 4-bit
• dots.ocrdots studio3.0B≈2.0 GB at 4-bit
• Isaac 0.2 2B PreviewPerceptron2.0B≈1.3 GB at 4-bit1 also selling it hosted
• LightOnOCR-1B-1025LightOn AI1.2B≈0.8 GB at 4-bit
• HunyuanOCRTencent Hunyuan1.1B≈0.7 GB at 4-bit
• LightOnOCR-2-1BLightOn AI1.0B≈0.7 GB at 4-bit
• Isaac 0.2 1BPerceptron1.0B≈0.7 GB at 4-bit1 also selling it hosted
• PaddleOCR-VL-1.5paddlepaddle1.0B≈0.6 GB at 4-bit
• PaddleOCR-VL-1.6paddlepaddle1.0B≈0.6 GB at 4-bit
• PaddleOCR-VL-0.9Bpaddlepaddle0.9B≈0.6 GB at 4-bit1 also selling it hosted
• OvisOCR2ATH-MaaS0.9B≈0.6 GB at 4-bit
• GOT-OCR2_0StepFun0.7B≈0.5 GB at 4-bit
• GOT-OCR-2.0-hfstepfun-ai0.6B≈0.4 GB at 4-bit
• trocr-base-handwrittenMicrosoft0.3B≈0.2 GB at 4-bit

Wider