Will any foundation model score 75 or higher on The Foundation Model Transparency Index by the end of 2024? (LLama2: 54)
13
αΉ466Dec 31
45%
chance
1D
1W
1M
ALL
The top-scoring model scores only 54 out of 100. No major foundation model developer is close to providing adequate transparency, revealing a fundamental lack of transparency in the AI industry.
There are 100 indicators that determine this score:
Read more: https://crfm.stanford.edu/fmti/
Get αΉ1,000 play money
Related questions
Related questions
Will the mean score for The Foundation Model Transparency Index be 50% or more by the end of 2025? (current: 37%)
52% chance
Will any foundation models/LLMs be able to reliably come up with novel unparalleled misalignments before EOY 2024?
45% chance
Will any model get above human level (92%) on the Simple Bench benchmark before September 1st, 2025.
60% chance
Will the market of foundation models be more fragmented by EOY 2026?
53% chance
Will we see improvements in the TruthfulQA LLM benchmark in 2024?
74% chance
Will Meta release a Llama 3 405B multi-modal open source before the end of 2024?
46% chance
π OpenLLMs: Will Any Open Source LLM on the HuggingFace OpenLLM Leaderboard Significantly Gain in Avg Score by YE 2024?
49% chance
Will an opensource LLM on huggingface beat an average human at the most common LLM benchmarks by July 1, 2024?
74% chance
will top LLM models use formal methods by EOY 2024?
24% chance
Will an LLM report >50% score on ARC in 2025?
81% chance