Reranking search results models that run on 128 GB
10 models with published weights that fit in 128 GB — a large Mac or a server card. Counted at four-bit quantisation, weights only, leaving the machine about a third of its memory and a gigabyte for context: room for roughly 136 billion parameters. A mixture of experts is counted in full, because it is held in full even though only a few experts compute.
A large Mac or a server card. This is where the biggest openly published models come within reach — and where the distinction between total and active parameters starts to decide everything, because a mixture of experts computes with a fraction of itself and must still be held whole.
Models that take a handful of results a search already found and put them in the right order. They are the cheap second stage of retrieval: an embedding search casts a wide net fast, a reranker reads the candidates properly. Because they only ever see a few documents, they cost little per query, and they usually improve a weak search more than a better embedding model would.
- Qwen3-VL-Embedding-8BAlibaba8.1B≈5.3 GB at 4-bit
- Qwen3 8B (embeddings)Alibaba8.0B≈5.2 GB at 4-bit6 also selling it hosted
- Qwen3-Reranker-8BAlibaba8.0B≈5.2 GB at 4-bit2 also selling it hosted
- Qwen3-Embedding-4BAlibaba4.0B≈2.6 GB at 4-bit4 also selling it hosted
- Qwen3-Reranker-4BAlibaba4.0B≈2.6 GB at 4-bit1 also selling it hosted
- Qwen3-VL-Embedding-2BAlibaba2.1B≈1.4 GB at 4-bit
- llama-nemotron-rerank-vl-1b-v2NVIDIA1.0B≈0.7 GB at 4-bit1 also selling it hosted
- Qwen3-Embedding-0.6BAlibaba0.6B≈0.4 GB at 4-bit4 also selling it hosted
- Qwen3-Reranker-0.6BAlibaba0.6B≈0.4 GB at 4-bit1 also selling it hosted
- bge-reranker-largeBAAI0.6B≈0.4 GB at 4-bit