When will SOTA on MMLU STEM be >= 90%?
Mini
7
Ṁ3742032
1D
1W
1M
ALL
4%
Before 2023
8%
2023-2024
36%
2024-2025
24%
2025-2026
14%
2026-2027
4%
2027-2032
8%
2032-never
Get Ṁ1,000 play money
Sort by:
@NcyRocks Sorry: python style ranges, so X - Y means after the start of X and before the start of Y. If I can I'll edit the options to make that clearer
Related questions
Related questions
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance
MMLU 99% #3: Will SOTA for MMLU (average) pass 99% by the start of 2026?
16% chance
MMLU 99% #4: Will SOTA for MMLU (average) pass 99% by the start of 2027?
12% chance
MMLU 99% #2: Will SOTA for MMLU (average) pass 99% by the start of 2025?
12% chance
Will SOTA on MATH in Sep 2024 utilize a hard-coded search/amplification procedure?
42% chance
By 2026, will it be standard practice to sandbox SOTA LLMs?
29% chance
When will no-calculator SOTA on the MATH dataset pass 90%?
Will there be a period of 12 contiguous months during which no new compute-SOTA LM is released, by Jan 1, 2033?
69% chance
BIG-bench accuracy 75% #2: Will SOTA for a single model on BIG-bench pass 75% by the start of 2025?
65% chance
Will the transformer architecture be replaced in SOTA LLMs by 2028?
61% chance