Reranking search results models that run on 64 GB
10 models with published weights that fit in 64 GB — a workstation. Counted at four-bit quantisation, weights only, leaving the machine about a third of its memory and a gigabyte for context: room for roughly 67 billion parameters. A mixture of experts is counted in full, because it is held in full even though only a few experts compute.
A workstation, or a well-specified Mac. Seventy-billion-parameter models fit at four-bit, which is the class where local output stops being obviously worse than the hosted models people pay for. Loading takes real time and the machine will be warm.
Models that take a handful of results a search already found and put them in the right order. They are the cheap second stage of retrieval: an embedding search casts a wide net fast, a reranker reads the candidates properly. Because they only ever see a few documents, they cost little per query, and they usually improve a weak search more than a better embedding model would.
- Qwen3-VL-Embedding-8BAlibaba8.1B≈5.3 GB at 4-bit
- Qwen3 8B (embeddings)Alibaba8.0B≈5.2 GB at 4-bit6 also selling it hosted
- Qwen3-Reranker-8BAlibaba8.0B≈5.2 GB at 4-bit2 also selling it hosted
- Qwen3-Embedding-4BAlibaba4.0B≈2.6 GB at 4-bit4 also selling it hosted
- Qwen3-Reranker-4BAlibaba4.0B≈2.6 GB at 4-bit1 also selling it hosted
- Qwen3-VL-Embedding-2BAlibaba2.1B≈1.4 GB at 4-bit
- llama-nemotron-rerank-vl-1b-v2NVIDIA1.0B≈0.7 GB at 4-bit1 also selling it hosted
- Qwen3-Embedding-0.6BAlibaba0.6B≈0.4 GB at 4-bit4 also selling it hosted
- Qwen3-Reranker-0.6BAlibaba0.6B≈0.4 GB at 4-bit1 also selling it hosted
- bge-reranker-largeBAAI0.6B≈0.4 GB at 4-bit