Pass IndexThe State of AISign in

OTIS Mock AIME 2024-2025

OTIS Mock AIME 2024-2025 is run by Epoch AI (AI Benchmarking Hub). It has ranked 306 models, of which the catalogue holds 174, scoring from 0 to 100 on Accuracy.

measured by Epoch AI (AI Benchmarking Hub) · the board itself

PlaceModelMetricScore
1stof 306GPT-6.1 Terramean_score (max)100
2ndof 306Gemini 4 Argonmean_score (high)100
3rdof 306GPT 6.1 Solmean_score (max)100
4thof 306Claude Sonnet 5.5mean_score (max)100
5thof 306GPT 6 Solmean_score (max)100
6thof 306Claude Opus 5.5mean_score (max)100
8thof 306Claude Fable 5.1mean_score (max)100
9thof 306GPT 6 Astramean_score (max)100
10thof 306Claude Fable 5mean_score (high)100
11thof 306GPT-5.6 Solmean_score (max)100
12thof 306GPT-5.5 Promean_score (xhigh)100
13thof 306GPT-5.5mean_score (xhigh)100
14thof 306GPT-5.6 Terramean_score (max)99.722
16thof 306DeepSeek V4.1 Flash 1017mean_score (max)99.444
18thof 306Qwen 3.8 Maxmean_score (xhigh)99.444
19thof 306Muse Spark 1.3mean_score (xhigh)99.167
20thof 306Grok 4.6mean_score (xhigh)99.167
21stof 306Claude Haiku 5.5mean_score (xhigh)98.889
22ndof 306GPT 6 Lunamean_score (max)98.889
23rdof 306Gemini 3.8 Flashmean_score (high)98.889
24thof 306Claude Opus 5mean_score (max)98.889
25thof 306DeepSeek V4 Pro 0813mean_score (max)98.611
27thof 306DeepSeek V4.1 Flashmean_score (max)98.333
28thof 306GPT-5.6 Lunamean_score (max)98.333
29thof 306Claude Opus 4.8mean_score (max)98.333
30thof 306Muse-Spark-Open-816B-A42Bmean_score (xhigh)98.056
31stof 306Grok 4.7mean_score (xhigh)98.056
32ndof 306Claude Opus 4.7mean_score (xhigh)97.8
37thof 306GPT-5.4mean_score (high)97.778
38thof 306Grok 4.5mean_score (high)97.778
40thof 306Gemini 3.7 Flashmean_score (high)97.222
41stof 306Kimi K3mean_score (max)97.222
42ndof 306DeepSeek V4 Promean_score (max)96.667
43rdof 306Kimi K2.6mean_score96.111
44thof 306GPT-5.2mean_score (high)96.111
46thof 306Gemini 3.1 Pro Previewmean_score95.6
48thof 306Qwen3.7 Maxmean_score (max)95.556
49thof 306Kimi K2.7 Codemean_score95.556
52ndof 306Gemini 3 Flashmean_score (high)95.556
54thof 306Gemini 3.5 Flashmean_score (high)95.556
56thof 306Claude Sonnet 5mean_score (xhigh)94.722
57thof 306DeepSeek V4 Flash (0731)mean_score (max)94.444
58thof 306Claude Opus 4.6mean_score94.444
59thof 306Gemini 3.6 Flashmean_score (high)94.167
60thof 306GLM 5.3 Flashmean_score (max)93.889
62ndof 306GLM 5.1mean_score93.333
64thof 306Qwen3.6 Plusmean_score93.333
65thof 306Qwen 3.7 Plusmean_score93.333
67thof 306Grok 4.3mean_score (high)93.333
69thof 306Gemini 3 Flash Previewmean_score92.778
70thof 306Grok 4.20mean_score92.222
71stof 306Kimi K2.5mean_score92.2
72ndof 306gemini-3-pro-previewmean_score91.389
73rdof 306GPT-5mean_score (high)91.389
74thof 306GLM-5.3mean_score (max)91.111
75thof 306Qwen3.6 Max Previewmean_score (max)91.111
76thof 306Qwen3.6 27Bmean_score91.111
78thof 306Inkling Smallmean_score (xhigh)90
79thof 306Muse Spark 1.2mean_score88.9
80thof 306GPT-5.4 minimean_score (xhigh)88.889
82ndof 306Qwen3.5 397B A17Bmean_score88.889
84thof 306Inklingmean_score (xhigh)88.889
85thof 306GPT OSS 120Bmean_score (high)88.889
86thof 306GPT-5.1mean_score (high)88.611
87thof 306deepseek-reasonermean_score87.817
88thof 306GPT-5.4 nanomean_score (high)87.78
91stof 306Nemotron 3 Ultramean_score86.667
92ndof 306Qwen 3.5 Plusmean_score86.667
93rdof 306Qwen3.6 35B A3Bmean_score86.667
94thof 306Qwen3.7 Flashmean_score86.667
96thof 306Qwen3 235B A22B Thinking 2507mean_score86.667
97thof 306GPT-5 minimean_score (high)86.667
98thof 306GLM 5.2mean_score (max)86.389
99thof 306Claude Opus 4.5mean_score86.111
100thof 306Claude Sonnet 4.6mean_score85.794
103rdof 306o3mean_score (medium)84.444
104thof 306Qwen3.6-Flashmean_score84.444
105thof 306Qwen3.5-Flashmean_score84.444
108thof 306Gemini 2.5 Promean_score84.167
111thof 306GLM-4.7mean_score83.333
118thof 306o4-minimean_score (high)81.667
119thof 306GPT-5 nanomean_score (high)81.111
122ndof 306Gemini 3.1 Flash-Litemean_score (high)80
125thof 306GLM-5mean_score80
129thof 306Claude Sonnet 4.5mean_score77.778
131stof 306Grok 3 Minimean_score (high)77.778
132ndof 306o3-minimean_score (high)76.944
137thof 306Gemma 4 31Bmean_score (minimal)73.333
138thof 306o1mean_score (high)73.333
141stof 306Gemini 2.5 Flashmean_score73.056
142ndof 306MiniMax M3mean_score71.111
143rdof 306Gemini 3.5 Flash-Litemean_score (high)71.111
146thof 306Claude Sonnet 4mean_score71.111
148thof 306Qwen3 30B A3B Thinking 2507mean_score70.278
149thof 306Qwen3.5-35B-A3Bmean_score (none)70
154thof 306Claude Opus 4.1mean_score68.889
156thof 306GPT-5.5 Instantmean_score68.056
157thof 306Seed-OSS-36B-Instructmean_score67.5
158thof 306Qwen3 32Bmean_score66.944
161stof 306Claude Haiku 4.5mean_score66.667
162ndof 306Qwen3 14Bmean_score66.389
163rdof 306DeepSeek-R1 (0528)mean_score66.389
164thof 306GPT OSS 20Bmean_score (medium)65.278
166thof 306Claude Opus 4mean_score64.444
169thof 306Qwen3 30B A3Bmean_score62.778
170thof 306Qwen3 30B A3B Instruct 2507mean_score62.222
173rdof 306Qwen3.5 9Bmean_score (none)61.667
177thof 306QwQ-32Bmean_score59.167
178thof 306GLM-4.7 Flashmean_score58.333
182ndof 306Claude 3.7 Sonnetmean_score57.778
184thof 306Qwen3 8Bmean_score56.111
185thof 306Qwen3.5-4Bmean_score (none)55.833
186thof 306DeepSeek R1 Distill QWEN 32Bmean_score55.556
188thof 306Grok 3mean_score55.556
194thof 306DeepSeek-R1mean_score53.333
195thof 306Qwen3-4B-Instruct-2507mean_score52.222
196thof 306R1 Distill Llama 70Bmean_score51.389
199thof 306DeepSeek R1 Distill QWEN 14Bmean_score50.556
200thof 306deepseek-chatmean_score48.889
202ndof 306o1-minimean_score (high)46.944
208thof 306GPT-4.1 minimean_score44.722
212thof 306DeepSeek R1 0528 Qwen3 8Bmean_score43.889
218thof 306GPT-4.1 fine-tunedmean_score38.333
221stof 306DeepSeek-V3 0324mean_score37.778
226thof 306Mistral Medium 2505mean_score32.222
228thof 306gemini-2.0-flash-001mean_score31.111
230thof 306Magistral Small 2506mean_score30
233rdof 306GPT-4.1 nanomean_score28.889
242ndof 306Gemma 3 27Bmean_score22.5
245thof 306DeepSeek-R1-Distill-Qwen-1.5Bmean_score21.389
246thof 306Llama 4 Maverick 17Bmean_score20.556
247thof 306Qwen-Plusmean_score17.778
248thof 306Gemma 3 12Bmean_score16.667
251stof 306DeepSeek-V3mean_score15.833
252ndof 306Phi-4mean_score13.75
254thof 306Meta-Llama-3.1-405B-Instructmean_score9.722
255thof 306Claude 3.5 Sonnetmean_score8.472
256thof 306Mistral Large 2407mean_score8.472
257thof 306Qwen3-1.7Bmean_score (none)8.056
258thof 306Qwen2.5 72B Instructmean_score8.056
261stof 306Gemma 3 4Bmean_score7.5
262ndof 306AgentRLmean_score7.361
263rdof 306GPT-4o-mini (2024-07-18)mean_score6.944
266thof 306GPT-4 Turbomean_score6.667
268thof 306GPT-4o (2024-08-06)mean_score6.389
269thof 306GPT-4o (2024-05-13)mean_score6.25
270thof 306GPT-4o (2024-11-20)mean_score6.25
271stof 306qwen-turbomean_score6.111
272ndof 306mistral-small-2503mean_score5.833
274thof 306Llama 3.3 70B Instructmean_score5.139
275thof 306Claude 3 Opusmean_score4.722
278thof 306Meta-Llama-3-70B-Instructmean_score4.306
279thof 306Claude Haiku 3.5mean_score4.306
280thof 306Mistral Small 3.1 (25.03)mean_score3.889
282ndof 306Meta-Llama-3.1-70B-Instructmean_score3.611
283rdof 306Granite 4.0 Micromean_score2.778
284thof 306Llama-3.2-90B-Vision-Instructmean_score2.639
285thof 306Qwen2.5 7B Instructmean_score2.5
287thof 306Claude 2.0mean_score2.5
288thof 306Claude 3 Sonnetmean_score2.5
289thof 306GPT-3.5 Turbomean_score2.222
290thof 306Llama 3 8B Instructmean_score1.944
291stof 306Claude 2.1mean_score1.944
292ndof 306Mistral Large (24.02)mean_score1.944
293rdof 306Claude 3 Haikumean_score1.806
294thof 306Llama 3.1 8B Instructmean_score1.667
295thof 306Gemma 2 27Bmean_score1.389
296thof 306gemma-3-1b-itmean_score1.111
297thof 306GPT-4 (0613)mean_score1.111
300thof 306deepseek-llm-67b-chatmean_score83.3
301stof 306Llama 3.2 1B Instructmean_score55.6
303rdof 306gemma-2-9b-itmean_score55.6
304thof 306Mistral-7B-Instruct-v0.3mean_score27.8
306thof 306Llama-2-70b-chat-hfmean_score0

Read from the board on 2026-11-01