upvote
> Didn’t Sol-6.1 top Astra?

On specific coding skewed benchmarks yes.

But Astra is in a different league when it comes to doing research and investigative tasks that require exploration.

For my usecases, Astra is pareto frontier - it dominates Fable 5.1 on cost. Fable 5.1 is a phenomenal model but Astra is cheaper and uses way less tokens and steps to produce similar output.

reply
deleted
reply
> Didn’t Sol-6.1 top Astra

No, Astra is still smarter (https://artificialanalysis.ai/models/?models=gpt-6-astra%2Cg...).

reply
I thought the same.

My assumption is Astra is the biggest model in terms of param size and has the most intellectual horsepower.

But Sol-6.1 is more instruction tuned to perform better on the average use cases for most users, so it benchmarks better and feels better to use.

reply
It's widely suspected that sol 6.1 is a slightly detuned version of astra 6
reply
> It's widely suspected that sol 6.1 is a slightly detuned version of astra 6

My feel is it's the opposite of a "detune".

Sol 6.1 is likely a distillation that's highly instruction tuned and designed to perform better on benchmarks.

reply
Sol 6.1 is incredibly incredibly slow, 20t/s slow (vs the ~70-80 of Terra 6.1 and Luna 6). It wouldn't be so much slower if it was just a distillation, surely? There's something else going on there.
reply
There was some (admittedly flimsy) evidence from a microsoft model description card that it's doing multiple passes through its own weights before token generation to improve output; it might be an architecture change that's making it slow, moreso than raw size
reply
Is it using comparable tokens per task?
reply
> Sol 6.1 is incredibly incredibly slow, 20t/s slow

How are you getting these numbers?

reply
I have a small token/s extension for Pi. It's regularly around ~15-20t/s for Sol 6.1.
reply
But is it slower than Astra?
reply
deleted
reply