
Benchmark Gap #4: Once a single AI model solves >= 95% of miniF2F, MATH, and MMLU STEM, how many months will it be before an AI is listed as a (co) first author on a published math paper?
Mini
9
Ṁ5992050
37
expected
1D
1W
1M
ALL
This question is meant to measure the gap between solving the main math-based benchmarks at the time of market creation, and contributing to real world mathematics.
The co first author requirement is loose: I will also accept an AI being credited with significant contributions to both deciding what to prove and the actual proof (merely contributing to the proof is not enough - I am trying to get at "the AI does the work of a mathematician" not "the AI does the work of a proof assistant"). I would also accept, for instance, the human author of the paper expressing that they would have named the AI as a coauthor if it was human, or saying that the result could not have been obtained without the assistance of the AI.
Get Ṁ1,000 play money
Related questions
Related questions
will a paper released in 2025 by a frontier AI lab have one of their AIs as a co-author?
15% chance
Does AI Pareto-dominate technical but non-mathematician humans at math?
21% chance
Will an AI achieve >85% performance on the FrontierMath benchmark before 2028?
54% chance
Will any AI model score >80% on Epoch's Frontier Math Benchmark in 2025?
3% chance
Will AI contribute as much as a co-author would today to a real research mathematics paper before Jan 1 2026?
25% chance
Benchmark Gap #5: Once a single AI model solves >= 95% of miniF2F, MATH, and MMLU STEM, will it be less than two years before AI models are used as entry-level data science / data analysis / statistics workers?
67% chance
Will an AI score over 80% on FrontierMath Benchmark in 2025
2% chance
Benchmark Gap #8: Once a single AI gets >= 80% on FrontierMath Tier 4, how long until an AI publishes a math paper?
-
Will an AI model write the proof to the Riemann Hypothesis by the end of 2025?
1% chance
Will an AI co-author a mathematics research paper published in a reputable journal before the end of 2026?
26% chance