The plateau doesn't have to be perfectly flat, but it's not a straight line upward anymore either (kind of like our work on transistors, where we've kind of hit the bounds of speed in clock cycles but are improving on miniaturization and power efficiency)
replyIt took 6 years to solve ARC-AGI 1, 1 year to solve ARC-AGI 2 and 6 months to solve ARC-AGI 3.
replyThose version numbers don't necessarily correspond to equal increases in "difficulty", though.
replyCorrect, the benchmark became exponentially more difficult as it progressed from pattern matching puzzles to games.
replyVery true, the sharp increase in difficulty (as measured by human passrate plummeting from 1->2 and again from 2->3) gives an even more stark view of AI capabilities over time.
reply