Reranking search results models that run on 16 GB
10 models with published weights that fit in 16 GB — the common laptop. Counted at four-bit quantisation, weights only, leaving the machine about a third of its memory and a gigabyte for context: room for roughly 16 billion parameters. A mixture of experts is counted in full, because it is held in full even though only a few experts compute.
The commonest laptop configuration, and the point where a genuinely useful model runs locally: fourteen billion parameters at four-bit fits with room to spare for the context. Expect a capable general assistant that will not match the frontier, and remember that anything else the machine is doing competes for the same memory.
Models that take a handful of results a search already found and put them in the right order. They are the cheap second stage of retrieval: an embedding search casts a wide net fast, a reranker reads the candidates properly. Because they only ever see a few documents, they cost little per query, and they usually improve a weak search more than a better embedding model would.
- Qwen3-VL-Embedding-8BAlibaba8.1B≈5.3 GB at 4-bit
- Qwen3 8B (embeddings)Alibaba8.0B≈5.2 GB at 4-bit6 also selling it hosted
- Qwen3-Reranker-8BAlibaba8.0B≈5.2 GB at 4-bit2 also selling it hosted
- Qwen3-Embedding-4BAlibaba4.0B≈2.6 GB at 4-bit4 also selling it hosted
- Qwen3-Reranker-4BAlibaba4.0B≈2.6 GB at 4-bit1 also selling it hosted
- Qwen3-VL-Embedding-2BAlibaba2.1B≈1.4 GB at 4-bit
- llama-nemotron-rerank-vl-1b-v2NVIDIA1.0B≈0.7 GB at 4-bit1 also selling it hosted
- Qwen3-Embedding-0.6BAlibaba0.6B≈0.4 GB at 4-bit4 also selling it hosted
- Qwen3-Reranker-0.6BAlibaba0.6B≈0.4 GB at 4-bit1 also selling it hosted
- bge-reranker-largeBAAI0.6B≈0.4 GB at 4-bit