Will OpenAI's next-gen math-focused model score at least 95% on the MATH benchmark?
27
Ṁ1802resolved Sep 16
Resolved
NO1D
1W
1M
ALL
Resolve to YES if OpenAI's next generation math-focused model achieves a score of 95% or higher on the MATH benchmark.
If the next generation of general models (e.g. GPT-4), code models (e.g. Codex), or any other models specialized for reasoning are released earlier than the math models and score 95% or higher, it will resolve this question to YES.
Benchmarking on a subset of MATH is acceptable.
Using tools(e.g. calculator) & code is allowed.
Get Ṁ1,000 play money
🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ119 | |
| 2 | Ṁ75 | |
| 3 | Ṁ51 | |
| 4 | Ṁ37 | |
| 5 | Ṁ14 |
Sort by:
Why is this resolving yes? I would have thought no? https://github.com/openai/simple-evals?tab=readme-ov-file#benchmark-results
Related questions
Related questions
By the end of Q1 2025 will an open source model beat OpenAI’s o1 model?
46% chance
Will OpenAI o1 (or any direct iteration) get gold on any International Math Olympiad by the end of 2025?
48% chance
Will OpenAI Release a Model Capable of Reliably performing Gradeschool Math from Reasoning by Jan 1, 2025?
77% chance
By the end of Q2 2025 will an open source model beat OpenAI’s o1 model?
72% chance
Will OpenAI's next major LLM (after GPT-4) surpass 70% accuracy on the GPQA benchmark?
65% chance
Will openAI have the most accurate LLM across most benchmarks by EOY 2024?
39% chance
Will OpenAI's next major LLM (after GPT-4) surpass 74% accuracy on the GPQA benchmark?
81% chance
Will any model get above human level (92%) on the Simple Bench benchmark before September 1st, 2025.
60% chance
Will a single model achieve superhuman performance on all OpenAI gym environments by 2025?
25% chance
Which MATH-AI 23 works will have >50 Google Scholar citations by end of 2026?