upvote
I don't think that's the case, because with instructions per second, there is a clear relationship. Whereas here on one axis they plot GHz and on the other they plot matches of StarCraft won.

There is certainly some influence of the Ghz of your processor on how many matches of StarCraft one can win, but.. well. Bogus science.

reply
My point was that it is a core but not necessarily correlated metric and it sounds like you agree with that.
reply
Okay so, the graph is "informative" because it plots the perplexity value of the various OpenAI models over time?

That's it? That's all this submission has going for it?

Okay.

reply
That was all I'd apprehended of the post when I replied earlier.

The post argues that NTP perplexity is a good first approximation to model capability, and could therefore form the basis for an industry-wide commitment to pause model development at a given perplexity level until there's a consensus that AI alignment and safety has caught up enough that it's safe to continue developing even more capable models.

It's definitely not simply an ad.

reply