upvote

points

by twtw996 hours ago |
comments
upvoteby bicx5 hours ago|
next
[-]
That last benchmark seemed like an impressive leg up against Opus until I saw the sneaky footnote that it was actually a Sonnet result. Why even include it then, other than hoping people don't notice?
reply
upvoteby osti5 hours ago|
parent|
next
[-]
It's only that one number that is for sonnet.
reply
upvoteby 0123456789ABCDE4 hours ago|
parent|
[-]
except for the webarena-verified
reply
upvoteby conradkay5 hours ago|
parent|
prev|
[-]
Sonnet was pretty close to (or better than) Opus in a lot of benchmarks, I don't think it's a big deal
reply
upvoteby jitl5 hours ago|
parent|
[-]
wat
reply
upvoteby 0123456789ABCDE4 hours ago|
parent|
[-]
maybe gp's use of the word "lots" is unwarranted

https://artificialanalysis.ai indicates that sonnect 4.6 beats opus 4.6 on GDPval-AA, Terminal-Bench Hard, AA Long context Reasoning, IFBench.

see: https://artificialanalysis.ai/?models=claude-sonnet-4-6%2Ccl...

reply
upvoteby chabes6 hours ago|
prev|
next
[-]
Definitely don’t want to click in at x either.
reply
upvoteby Sabinus3 hours ago|
parent|
prev|
next
[-]
Get a redirect plugin and set it up to send you to xcancel instead of Twitter. I've done it, and it's very convenient.
reply
upvoteby anonym00se16 hours ago|
parent|
prev|
next
[-]
Ditto, but I did anyways and enjoyed that OpenAI doesn't include the dogwater that is Grok on their scorecard.
reply
upvoteby Aboutplants6 hours ago|
prev|
next
[-]
It seems that all frontier models are basically roughly even at this point. One may be slightly better for certain things but in general I think we are approaching a real level playing field field in terms of ability.
reply
upvoteby observationist6 hours ago|
parent|
next
[-]
Benchmarks don't capture a lot - relative response times, vibes, what unmeasured capabilities are jagged and which are smooth, etc. I find there's a lot of difference between models - there are things which Grok is better than ChatGPT for that the benchmarks get inverted, and vice versa. There's also the UI and tools at hand - ChatGPT image gen is just straight up better, but Grok Imagine does better videos, and is faster.

Gemini and Claude also have their strengths, apparently Claude handles real world software better, but with the extended context and improvements to Codex, ChatGPT might end up taking the lead there as well.

I don't think the linear scoring on some of the things being measured is quite applicable in the ways that they're being used, either - a 1% increase for a given benchmark could mean a 50% capabilities jump relative to a human skill level. If this rate of progress is steady, though, this year is gonna be crazy.

reply
upvoteby baq6 hours ago|
parent|
next
[-]
Gemini 3.1 slaps all other models at subtle concurrency bugs, sql and js security hardening when reviewing. (Obviously haven’t tested gpt 5.4 yet.)

It’s a required step for me at this point to run any and all backend changes through Gemini 3.1 pro.

reply
upvoteby observationist5 hours ago|
parent|
next
[-]
I have a few standard problems I throw at AI to see if they can solve them cleanly, like visualizing a neural network, then sorting each neuron in each layer by synaptic weights, largest to smallest, correctly reordering any previous and subsequent connected neurons such that the network function remains exactly the same. You should end up with the last layer ordered largest to smallest, and prior layers shuffled accordingly, and I still haven't had a model one-shot it. I spent an hour poking and prodding codex a few weeks back and got it done, but it conceptually seems like it should be a one-shot problem.
reply
upvoteby thejohnconway1 hours ago|
parent|
[-]
Lol, I’ve had cutting edge models suggest I make an inflexible hole bigger by putting shim in it, and argue their case stubbornly. I don’t know what you’re using to suggest they are anywhere near solving your problem there!
reply
upvoteby adonese5 hours ago|
parent|
prev|
[-]
Which subscription do you have to use it? Via Google ai pro and gemini cli i always get timeouts due to model being under heavy usage. The chat interface is there and I do have 3.1 pro as well, but wondering if the chat is the only way of accessing it.
reply
upvoteby baq5 hours ago|
parent|
[-]
Cursor sub from $DAYJOB.
reply
upvoteby basch4 hours ago|
parent|
prev|
next
[-]
>ChatGPT image gen is just straight up better

Yet so much slower than Gemini / Nano Banana to make it almost unusable for anything iterative.

reply
upvoteby bigyabai6 hours ago|
parent|
prev|
[-]
> If this rate of progress is steady, though, this year is gonna be crazy.

Do you want to make any concrete predictions of what we'll see at this pace? It feels like we're reaching the end of the S-curve, at least to me.

reply
upvoteby observationist6 hours ago|
parent|
[-]
If you look at the difference in quality between gpt-2 and 3, it feels like a big step, but the difference between 5.2 and 5.4 is more massive, it's just that they're both similarly capable and competent. I don't think it's an S curve; we're not plateauing. Million token context windows and cached prompts are a huge space for hacking on model behaviors and customization, without finetuning. Research is proceeding at light speed, and we might see the first continual/online learning models in the near future. That could definitively push models past the point of human level generality, but at the very least will help us discover what the next missing piece is for AGI.
reply
upvoteby ryandrake5 hours ago|
parent|
[-]
For 2026, I am really interested in seeing whether local models can remain where they are: ~1 year behind the state of the art, to the point where a reasonably quantized November 2026 local model running on a consumer GPU actually performs like Opus 4.5.

I am betting that the days of these AI companies losing money on inference are numbered, and we're going to be much more dependent on local capabilities sooner rather than later. I predict that the equivalent of Claude Max 20x will cost $2000/mo in March of 2027.

reply
upvoteby mootothemax4 hours ago|
parent|
[-]
Huh, that’s interesting, I’ve been having very similar thoughts lately about what the near-ish term of this tech looks like.

My biggest worry is that the private jet class of people end up with absurdly powerful AI at their fingertips, while the rest of us are left with our BigMac McAIs.

reply
upvoteby thewebguyd6 hours ago|
parent|
prev|
next
[-]
Kind of reinforces that a model is not a moat. Products, not models, are what's going to determine who gets to stay in business or not.
reply
upvoteby gregpred6 hours ago|
parent|
next
[-]
Memory (model usage over time) is the moat.
reply
upvoteby energy1236 hours ago|
parent|
prev|
[-]
Narrative violation: revenue run rates are increasing exponentially with about 50% gross margins.
reply
upvoteby kseniamorph5 hours ago|
parent|
prev|
next
[-]
makes sense, but i'd separate two things: models converging in ability vs hitting a fundamental ceiling. what we're probably seeing is the current training recipe plateauing — bigger model, more tokens, same optimizer. that would explain the convergence. but that's not necessarily the architecture being maxed out. would be interesting to see what happens when genuinely new approaches get to frontier scale.
reply
upvoteby druskacik6 hours ago|
parent|
prev|
[-]
That has been true for some time now, definitely since Claude 3 release two years ago.
reply
upvoteby swingboy6 hours ago|
prev|
next
[-]
Why do so many people in the comments want 4o so bad?
reply
upvoteby cheema335 hours ago|
parent|
next
[-]
> Why do so many people in the comments want 4o so bad?

You can ask 4o to tell you "I love you" and it will comply. Some people really really want/need that. Later models don't go along with those requests and ask you to focus on human connections.

reply
upvoteby astrange6 hours ago|
parent|
prev|
next
[-]
They have AI psychosis and think it's their boyfriend.

The 5.x series have terrible writing styles, which is one way to cut down on sycophancy.

reply
upvoteby baq6 hours ago|
parent|
[-]
Somebody on Twitter used Claude code to connect… toys… as mcps to Claude chat.

We’ve seen nothing yet.

reply
upvoteby mikkupikku6 hours ago|
parent|
next
[-]
My computer ethics teacher was obsessed with 'teledildonics' 30 years ago. There's nothing new under the sun.
reply
upvoteby Sharlin4 hours ago|
parent|
next
[-]
There are many games these days that support controllable sex toys. There's an interface for that, of course: https://github.com/buttplugio/buttplug. Written in Rust, of course.
reply
upvoteby the_af4 hours ago|
parent|
[-]
> Written in Rust, of course.

Safety is important.

reply
upvoteby vntok5 hours ago|
parent|
prev|
[-]
Was your teacher Ted Nelson?
reply
upvoteby mikkupikku4 hours ago|
parent|
[-]
I wish, dude is a legend.
reply
upvoteby manmal5 hours ago|
parent|
prev|
next
[-]
ding-dong-cli is needed
reply
upvoteby Herring5 hours ago|
parent|
prev|
[-]
what.. :o
reply
upvoteby embedding-shape6 hours ago|
parent|
prev|
next
[-]
Someone correct me if I'm wrong, but seemingly a lot of the people who found a "love interest" in LLMs seems to have preferred 4o for some reason. There was a lot of loud voices about that in the subreddit r/MyBoyfriendIsAI when it initially went away.
reply
upvoteby drittich5 hours ago|
parent|
[-]
I think it's time for an https://hotornot.com for AI models.
reply
upvoteby vntok5 hours ago|
parent|
[-]
botornot?
reply
upvoteby MattGaiser6 hours ago|
parent|
prev|
[-]
The writing with the 5 models feels a lot less human. It is a vibe, but a common one.
reply
upvoteby karmasimida6 hours ago|
prev|
next
[-]
It is a bigger model, confirmed
reply
upvoteby MarcFrame5 hours ago|
prev|
next
[-]
how does 5.4-thinking have a lower FrontierMath score than 5.4-pro?
reply
upvoteby nico12075 hours ago|
parent|
next
[-]
Well 5.4-pro is the more expensive and more advanced version of 5.4-thinking so why wouldn't it?
reply
upvoteby dom965 hours ago|
prev|
[-]
Why do none of the benchmarks test for hallucinations?
reply
upvoteby tedsanders4 hours ago|
parent|
next
[-]
In the text, we did share one hallucination benchmark: Claim-level errors fell by 33% and responses with an error fell by 18%, on a set of error-prone ChatGPT prompts we collected (though of course the rate will vary a lot across different types of prompts).

Hallucinations are the #1 problem with language models and we are working hard to keep bringing the rate down.

(I work at OpenAI.)

reply
upvoteby netule5 hours ago|
parent|
prev|
[-]
[flagged]
reply