I would describe it as "fidelity" and "verbosity" (or just amount of token generation to complete the task, sometimes that works out to scratch space, or literally how large the "solution" is).

If you have something that needs to be done right, might be a bit complicated, up the model size.

You can see this in the pelicans. Big model pelicans are pretty accurate by default. Up the reasoning and only more so, but with more detail. For Astra, it is 105 lines for low, 250 lines for max reasoning.

Small model pelicans will lack the fidelity of a large model. Bits will be out of place etc. For luna, it's 90 lines for low, 150 lines for xhigh.

Additionally the amount of time taken is increased for the larger models. Luna takes 11 seconds on low, and 1:33 for xhigh. Astra is 33 seconds on low, 4 minutes on max.

And naturally, there is the cost. There's some overlap in functionality between luna xhigh and Astra low in the sense that luna really can do quite a suitable job for some tasks. But there are just some tasks that just don't make sense for Luna, even at high reasoning.

The other thing to remember is that sometimes high fidelity isn't ideal. It can lead to overdesigning. My recommendation is to commit early, commit often, and review everything you do, which we've all been doing since before LLMs right?

I mostly just use frontier models as well. Except for one case: when I let the cache expire (I think 5+ mins of inactivity) I'll switch to one of the cheaper models to summarize and write a handoff note, then pick that up with the better model. Picking up a session whose cache expired with something like 200k tokens with the frontier model reflects really poorly on your usage.

This is explained a bit in the API docs but you also have to adjust it based on your own tasks.

https://developers.openai.com/api/docs/guides/reasoning?api-...

For easy problems, just use Luna on max level. It has so much token mileage you can go forever.

Ya it’s annoying to have to manage this.

But effort is basically how much extra internal scratchpad to use and how much extra questions to ask and answer before producing a result, Exploring more hypotheses, validating consistencies, calling more tools.

If you’re happy with your token spend on Astra then keep doing what you’re doing. but if you feel the need to conserve tokens, then you can do that by switching to smaller models like Luna when the task is straight forward.

there is no correct answer for that. One is the difference in size/params. The other is the amount of "rounds" of reasoning generating and reviewing what is generated before the model decides it is good.