FrontierMath Tier 4 was built to defeat machines for years. It lasted fourteen months.
Epoch AI, the research group that built it, announced on Sept. 10, 2026 that AI had solved every problem in the tier1. OpenAI's GPT-6 Astra solved the last one, a problem written by the mathematician Jay Pantone. The tier had launched just 14 months earlier, in July 2025. Back then, the field's best models solved only three of its 50 problems between them2.
That pattern keeps repeating. Ai Lately tracked five hard AI benchmarks from their first scores to the point their builders called them full. Four of the five reached that point within about a year of launch.
One test, built the same way, still has room to climb. The gap between the four and the one is the real story here.
Five Tests, One Pattern
Start with the oldest. MMLU asked AI models to answer college-level quiz questions across 57 subjects, a test of broad knowledge rather than deep reasoning. Dan Hendrycks and his co-authors released it in September 202010.
GPT-3, then the best model around, barely beat guessing at random. Within a few years, that gap had nearly closed.
By May 2024, frontier models had closed most of the MMLU gap, and Wenhu Chen built a tougher successor. "The current models already hit the ceiling? It's time to upgrade MMLU!" he wrote on X11.
His new test, MMLU-Pro, showed the gap clearly. GPT-4o beat GPT-4-turbo by nine points on the harder test, but by only two on the original MMLU11.
The Newest AI Tests Cut Their Lifespan From 4 Years to 1
Launch and ceiling dates for five AI benchmarks, 2020 to 2026
Each pin marks one benchmark's launch or the date its builders called it full. The gap between the two pins shrinks from years, for the oldest tests, to about a year for the two newest.
Sources: Epoch AI on X [1]; Epoch AI on X [3]; ARC Prize on X [8]; ARC Prize [9]; arXiv [10]; Wenhu Chen on X [11]; Epoch AI on X [13]; arXiv [14]; Vals AI [16]. Chart by Kelly Dennis.
The numbers behind this chart
| Date | Event |
|---|---|
| Sept. 2020 | MMLU launches |
| March 2021 | MATH launches |
| Nov. 2023 | GPQA Diamond launches |
| May 2024 | MMLU-Pro replaces it. Built because MMLU had hit its ceiling |
| March 2025 | ARC-AGI-2 launches |
| April 2025 | MATH Level 5 saturates |
| July 2025 | FrontierMath Tier 4. Launches with a single-digit top score |
| April 22, 2026 | ARC-AGI-2 clears 85% |
| Sept. 2026 | GPQA Diamond retires |
| Sept. 10, 2026 | FrontierMath Tier 4 full |
The MATH benchmark's hardest tier followed the same arc. Hendrycks and his colleagues published the 12,500-problem set in March 202112. By April 2025, Epoch AI posted that Level 5, the hardest slice, had become "quite saturated, with both o3 and o4-mini getting 98% accuracy"13.
GPQA Diamond moved faster still. Its questions came from working scientists rather than puzzle writers, and few outsiders could answer them. Researchers led by David Rein published it in November 2023, when GPT-4 answered 39 percent of its graduate-level science questions correctly14.
OpenAI's o1 beat human PhDs on the same test by September 2024, scoring 77.3 percent15. By September 2026, Vals AI's leaderboard put one model at 95.45 percent and declared the test saturated, retiring it from new write-ups16.

FrontierMath Tier 4 compressed the timeline again. Epoch AI built it with research mathematicians to push past the limits of the first three tiers3. At launch, the field's best systems solved three of the fifty problems, about 6 percent2.
After a correction that June, the top score on the tier reached 76 percent, and Epoch called it "approaching saturation"45. Ten weeks later, every one of its problems had fallen1.
Epoch AI@EpochAIResearchEvery FrontierMath Tier 4 problem has now been solved by AI, with GPT-6 Astra solving the last problem standing. Mathematicians often commented that AI found unintended shortcuts when solving their Tier 4 problems. Not so for this last one, which was created by Jay Pantone. pic.twitter.com/ieEZaomhaE
ARC-AGI-2 ran on the same clock. The ARC Prize Foundation announced the test on X in March 2025 as "an unsaturated frontier AGI benchmark that challenges AI reasoning systems." Reasoning models then scored under 4 percent8.
OpenAI's GPT-5.5 crossed the benchmark's 85 percent grand-prize bar 13 months later, on April 22, 20269. By Nov. 17, 2026, OpenAI's GPT-6.2 Astra sat well clear of that bar on the public leaderboard.
Three Hard AI Tests Climb From Single Digits to 97 Percent
Best public score by date for ARC-AGI-2, FrontierMath Tier 4 and GPQA Diamond, 2023 to 2026
Each line follows one test's best known score over time, higher is better. All three climb toward the top right as newer models post stronger results, and two of the three cross 95 percent within about a year and a half of launch.
Each test launched on its own date, so the three lines start at different points on the calendar. The climb itself is the comparison, apart from the calendar gap.
Sources: Import AI [2]; Epoch AI on X [5]; OpenAI [6]; ARC Prize on X [8]; ARC Prize [9]; arXiv [14]; OpenAI [15]; Vals AI [16]. Chart by Kelly Dennis.
The numbers behind this chart
| Period | GPQA Diamond | ARC-AGI-2 | FrontierMath Tier 4 |
|---|---|---|---|
| Nov. 2023 | 39% | ||
| Sept. 2024 | 77.3% | ||
| Mar. 2025 | 4% | ||
| Jul. 2025 | 6% | ||
| Jun. 2026 | 76% | ||
| Apr. 2026 | 85% | ||
| Sept. 2026 | 95.45% | ||
| Sept. 2026 | 97.6% | ||
| Nov. 2026 | 95.8% |

Why the Climb Compressed
Credit the labs' own math homework. OpenAI said GPT-6 Astra's internal score on FrontierMath Tier 4 started at 17 percent and ended at 97.6 percent before shipping6. That climb happened behind closed doors, far from any leaderboard. Labs now treat reasoning and math as a named training goal, and they test against the hardest public benchmarks while they do it.
Mathematicians feel the pull too. Terence Tao, the UCLA mathematician, met OpenAI's James Donovan and Mark Chen at a math-and-AI workshop on March 4, 202617. That kind of meeting barely existed three years earlier. Research mathematicians now help write the very tests meant to outlast their students' models.

The Case for Climbing Higher
Epoch AI, the researchers behind FrontierMath, see this as proof the work matters. It is also a reason to build the next test. The group said so the day it called Tier 4 nearly solved. "We believe the future of math benchmarking lies in open problems drawn from real research, like those we've collected in FrontierMath: Open Problems"5.
The group makes the same case more broadly. A test that stays easy stops teaching anyone anything. "Most AI benchmarks share a common flaw: they saturate too quickly to study long-run trends," Epoch AI wrote in December 202518. The group was pitching a project that stitches many tests into one long scale.
That logic shaped ARC-AGI-2 from day one. The ARC Prize Foundation priced its grand prize to a bar that only genuine reasoning could clear78.
Skeptics Push Back
Some researchers reject the whole chase. Angela Aristidou is a professor at University College London and a Stanford fellow who studies how companies actually use AI. She argues the scores measure the wrong thing.
Aristidou studied radiology tools with strong benchmark scores that slowed doctors down once hospitals installed them. Her argument turns on where a model works, apart from how hard its test reads. She laid out the finding in MIT Technology Review in March 202619. "What appeared as a productivity-enhancing AI tool when tested in a vacuum introduced delays in practice."
Her verdict is blunt. "AI is almost never used in the way it is benchmarked," she wrote in the same piece.
Humanity's Last Exam backs a softer version of her point. Dan Hendrycks, whose Center for AI Safety built the test with Scale AI, announced it with a warning. "State-of-the-art AIs get <10% accuracy and are highly overconfident," he wrote20. Almost two years later, the best public score is still 38.3 percent21.
A 2026 study of 60 widely cited benchmarks, led by Mubashara Akhtar, found that nearly half had already saturated. Expert-built tests resisted the fast climb longer than crowdsourced ones22.
The speed-up is real. One carefully built test still resists it.
What to Do
Before trusting a vendor's benchmark claim, ask two questions this piece's own tracking answers. First, how old is the test? A score on a benchmark past its second birthday may already sit near a ceiling every rival has also reached.
That tells a buyer little about who actually leads. Age alone can hide the real gap between rivals.
Second, who built it, and how? Expert-curated tests such as Humanity's Last Exam held their difficulty far longer than quizzes built by crowdsourcing questions online22. Check an independent tracker such as Epoch AI or Vals AI before repeating a lab's own number.
By the Numbers
- GPQA Diamond opened at 39 percent in 2023 and crossed 95 percent by 20261416.
- FrontierMath Tier 4 went from launch to every problem solved in 14 months, July 2025 to Sept. 10, 202613.
- ARC-AGI-2's 85 percent grand-prize bar fell to GPT-5.5 in 13 months, by April 22, 202689.
- GPT-3 scored 43.9 percent on MMLU in 2020, the test that needed a harder successor within four years10.
- Humanity's Last Exam's best public score sits at 38.3 percent, the one test here still far from its ceiling21.
- Nearly half of 60 widely used benchmarks had saturated, a 2026 study found, with the share rising as a test ages22.
- OpenAI's GPT-6.2 Astra topped ARC-AGI-2's public leaderboard at 95.8 percent as of Nov. 17, 2026, well clear of the grand-prize bar9.
What to Watch
Watch Dec. 4, 2026, when the ARC Prize Foundation posts its 2026 results. That update will confirm whether a model cleared the 85 percent bar on ARC-AGI-2's private evaluation set, beyond its public leaderboard9.
FrontierMath: Open Problems, the harder successor Epoch AI is already building, will show whether its difficulty holds against the same labs5. Humanity's Last Exam has held near the 38 percent mark for most of 2026. A climb past it would be the clearest sign yet that expert curation buys a test more time2122.
Sources
- Epoch AI, "Every FrontierMath Tier 4 problem has now been solved by AI," X, Sept. 10, 2026, https://x.com/EpochAIResearch/status/2098103831502708864
- Jack Clark, "Import AI 420: Prisoner's Dilemma AI; FrontierMath Tier 4; and how to regulate AI companies," Import AI, July 14, 2025, https://jack-clark.net/2025/07/14/import-ai-420-prisoner-dilemma-ai-frontiermath-tier-4-and-how-to-regulate-ai-companies/
- Epoch AI, "Introducing FrontierMath Tier 4," X, July 11, 2025, https://x.com/EpochAIResearch/status/1943744462972215305
- Epoch AI, "FrontierMath: Tiers 1-4," Epoch AI, June 12, 2026, https://epoch.ai/frontiermath/tiers-1-4
- Epoch AI, "FrontierMath: Tiers 1-4 is now approaching saturation," X, June 12, 2026, https://x.com/EpochAIResearch/status/2065488238543036528
- OpenAI, "GPT-6 Astra: A new generation of intelligence," OpenAI, Sept. 8, 2026, https://openai.com/index/gpt-6-astra/
- ARC Prize Foundation, "Announcing ARC-AGI-2 and ARC Prize 2025," ARC Prize, March 24, 2025, https://arcprize.org/blog/announcing-arc-agi-2-and-arc-prize-2025
- ARC Prize, "Today we are announcing ARC-AGI-2, an unsaturated frontier AGI benchmark," X, March 24, 2025, https://x.com/arcprize/status/1904269307284230593
- ARC Prize Foundation, "ARC Prize Leaderboard," ARC Prize, Nov. 17, 2026, https://arcprize.org/leaderboard
- Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, Jacob Steinhardt, "Measuring Massive Multitask Language Understanding," arXiv, Sept. 7, 2020, https://arxiv.org/abs/2009.03300
- Wenhu Chen, "Tired of MMLU? The current models already hit the ceiling?," X, May 15, 2024, https://x.com/WenhuChen/status/1790597967319007564
- Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, Jacob Steinhardt, "Measuring Mathematical Problem Solving With the MATH Dataset," arXiv, March 5, 2021, https://arxiv.org/abs/2103.03874
- Epoch AI, "MATH Level 5 ... now appears to be quite saturated," X, April 18, 2025, https://x.com/EpochAIResearch/status/1913379488286601533
- David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, Samuel R. Bowman, "GPQA: A Graduate-Level Google-Proof Q&A Benchmark," arXiv, Nov. 20, 2023, https://arxiv.org/abs/2311.12022
- OpenAI, "Learning to reason with LLMs," OpenAI, Sept. 12, 2024, https://openai.com/index/learning-to-reason-with-llms/
- Vals AI, "GPQA Diamond Leaderboard and Methodology," Vals AI, Sept. 1, 2026, https://www.vals.ai/benchmarks/gpqa
- Institute for Pure & Applied Mathematics, "Terence Tao at IPAM's Fireside Chat 2026," Wikimedia Commons, March 4, 2026, https://commons.wikimedia.org/wiki/File:Terence_Tao_at_IPAM%27s_Fireside_Chat_2026_(50m16s).jpg
- Epoch AI, "Most AI benchmarks share a common flaw: they saturate too quickly to study long-run trends," X, Dec. 3, 2025, https://x.com/EpochAIResearch/status/1996248575400132794
- Angela Aristidou, "AI benchmarks are broken. Here's what we need instead.," MIT Technology Review, March 31, 2026, https://www.technologyreview.com/2026/03/31/1134833/ai-benchmarks-are-broken-heres-what-we-need-instead/
- Dan Hendrycks, "We're releasing Humanity's Last Exam," X, Jan. 23, 2025, https://x.com/hendrycks/status/1882433928407241155
- Center for AI Safety, Scale AI, "Humanity's Last Exam Leaderboard," Center for AI Safety, Jan. 28, 2026, https://agi.safe.ai/
- Mubashara Akhtar, Anka Reuel, Prajna Soni, et al., "When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation," ICML / arXiv, Aug. 6, 2026, https://arxiv.org/abs/2602.16763




