Search
News Chat
Login
Search
Top
For You
Business
Crypto
Culture
Environment
Politics
Science
Sports
Tech
Video Games
World
MMLU News
Prediction markets for MMLU
Prediction markets for MMLU
Will the next LLM released by OpenAI be worse than GPT-4 at MMLU?
May 12, 12:41 PM
Jan 1, 6:29 AM
19.11%
chance
22
913
Option
Votes
YES
NO
1274
897
Will any open-source model achieve GPT-4 level performance on MMLU through 2024?
Sep 10, 8:26 PM
Jan 1, 4:59 AM
83.13%
chance
21
1740
Option
Votes
NO
YES
1483
767
Gpt-5 >=91% on MMLU?
Apr 11, 6:23 AM
Dec 31, 6:29 PM
87.59%
chance
41
2566
Option
Votes
NO
YES
1936
700
MMLU 99% #4: Will SOTA for MMLU (average) pass 99% by the start of 2027?
Feb 8, 10:23 PM
Jan 1, 8:00 AM
12.11%
chance
12
1166
Option
Votes
YES
NO
1260
925
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
Feb 8, 10:23 PM
Jan 1, 8:00 AM
44.39%
chance
5
129
Option
Votes
YES
NO
173
125
MMLU 99% #3: Will SOTA for MMLU (average) pass 99% by the start of 2026?
Feb 8, 10:23 PM
Jan 1, 8:00 AM
15.74%
chance
7
362
Option
Votes
YES
NO
247
113
Gpt-5 >=95% on MMLU?
Feb 13, 5:20 PM
Dec 31, 10:59 PM
39.29%
chance
4
43
Option
Votes
YES
NO
134
89
Benchmark Gap #4: Once a single AI model solves >= 95% of miniF2F, MATH, and MMLU STEM, how many months will it be before an AI is listed as a (co) first author on a published math paper?
Dec 20, 8:23 AM
Jan 1, 8:00 AM
15.31%
chance
9
599
Option
Votes
YES
NO
873
225
Will a open source pure Mamaba LLM surpass 82 MMLU on MMLU (5-shot) before end of year 2024?
May 8, 7:44 AM
Jan 1, 7:59 AM
25.01%
chance
3
75
Option
Votes
YES
NO
173
58
When will SOTA on MMLU STEM be >= 90%?
Jul 28, 11:06 PM
Jan 1, 8:00 AM
7
374
Will Mistral release model weights that come close to GPT4 in 2024? (MMLU > 80)
Jan 5, 7:19 PM
Jan 1, 4:59 AM
8
280
MMLU 99% #2: Will SOTA for MMLU (average) pass 99% by the start of 2025?
Feb 8, 10:22 PM
Jan 1, 8:00 AM
15.15%
chance
8
393
Option
Votes
YES
NO
313
141
Articles
Latest stories
Hugging Face Releases FineWeb (15T) and FineWeb-Edu (1.3T, 5.4T) Datasets
Authors
13
1 month
AI
Tech
Allen AI Enhances OLMo LLMs for Improved Reasoning Tasks MMLU and TruthfulQA with Transparency
Authors
5
4 months
AI
Tech
Previous
Next
