Will a text model achieve 100% performance on the MMLU in five years?
Mini
3
Ṁ402028
28%
chance
1D
1W
1M
ALL
This video predicts that a model will achieve 100% performance on the Massive Multitask Language Understanding Benchmark.
This will resolve YES if a model is listed as having achieved 100 percent performance on the MMLU on papers with code, in a technical document, or published article by 3:00 PM EST June 28, 2028. This will resolve NO if a model is not listed as ahving achieved 100 percent performance on the MMLU.
Get Ṁ1,000 play money
Related questions
Related questions
What organization will have the top language model on LMSys overall leaderboards December 1st 2024?
Will LLMs' loss function achieve the level of entropy of human text by the end of 2030?
61% chance
Will any open-source model achieve GPT-4 level performance on MMLU through 2024?
83% chance
Will a open source pure Mamaba LLM surpass 82 MMLU on MMLU (5-shot) before end of year 2024?
25% chance
Will an AI model outperform 95% of Manifold users on accuracy before 2026?
54% chance
Will language models be able to solve simple graphical mazes by the end of 2025?
67% chance
By 2027, will it be generally agreed upon that LLM produced text > human text for training LLMs?
64% chance
Will any model get above human level (92%) on the Simple Bench benchmark before September 1st, 2025.
60% chance
Will an open-source text-to-music LLM model be able to create an entire album as convincing as a human by end of 2025?
30% chance
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance