upvote
Been using GLM 5.3 Flash via Ollama Cloud basic subscription:

if this is the only model I had to use for the rest of my life, I would be cool with that. It's good at a wide variety of tasks including reverse engineering, design, working with ESP 32 etc.

I think Muse Spark 1.3 is better but GLM 5.3 Flash is an excellent daily driver.

reply
I've been using GLM 5.3 Flash as the planner and DS 4.1 as the task worker over the last month and the results have been very good, predictable, ridiculously cheap.
reply
What reasoning level do you use for each?
reply
I'm the weirdo who is clinging onto GLM 5.2. 5.3 seems to have picked up the bad writing from Opus 5.
reply
GLM 5.2 was the first open model that I was happy to use as my main. It's a fantastic model. 5.3 cost a lot more per task, was slower and didn't get that much better results for me, so I tried 5.3 Flash for a bit, it was ok, but didn't feel as magical as 5.2 and I have now landed on DeepSeek 4.1 Flash as my main.
reply
I don't think that's likely, given the close release dates of both? I wonder if 5.3 just choked on too much RL slop, independently of Opus 5.
reply
Genuinely curious, why not use GPT Sol 6.1 high which has a better cost per task and intelligence ($0.32, 50) or Mimo 2.6 pro? ($0.13, 46) over GLM Max's $2.01, 45?
reply
Not OP, but I've been trying out Sol 6.1 medium with hermes agent, and I have not been able to match usage costs to what I've been getting with glm 5.3 flash high.

Really don't know why, since I've offloaded thinking to the LLMs in this area of my life. Though even at medium, 6.1-Sol is noticeably better at most tasks I throw at it.

reply
I'm OP and I have the Ollama basic $20/mo plan.

Been very happy with it although I will say Muse Spark 1.3 being free on OpenCode is my main model these days. If I run out of free credits on OC (which almost never happens) then I drop into GLM Flash 5.3.

reply
deleted
reply
If you liked GLM 5.3, you would appreciate Deepseek V4 Flash a lot. In my personal opinion and use-cases, it outperforms GLM 5.3.
reply
V4 isn't the newest DeepSeek, and GLM 5.3 non-Flash isn't the same ballpark

- DeepSeek V4.1 Flash has marginally better coding, and smaller experts (8B-In/16B-Out) with more compressed attention and KV cache, so it's faster but will have less expertise per task. It's a 552B model, so while it theoretically may contain more trained data it also requires more RAM (the experts are smaller to compensate for speed loss) making it harder to serve without the distributed KV cache DeepSeek hosts.

- GLM 5.3 Flash has marginally better agentic work, and has video input (DS doesn't), and its experts are 18B so there's more intelligence per task. It's a 320B model so it requires less RAM which will make it cheaper to serve. Considering it's a much smaller model, the higher intelligence indicates their architecture/training is better.

- GLM 5.3 (non-Flash) is far more intelligent than both; it's a 744B model with 40B experts, but it's text-only.

reply
Model sizes aren't directly comparable, and they aren't even that relevant. I care about result quality, cost, and speed, not size. I switched to deepseek because it was giving me better coding results than GLM, and for far less cost. GLM is still better at planning.
reply
Stupid question probably because I’ve gotten out of sync with the new releases but is Flash better or worse than vanilla 5.3? I’ve been using 5.3 extensively (and maybe ironically have used flash only for viewing images which I have as a skill in Pi because 5.3 vanilla seemingly doesn’t support vision). I find it generally competent but still very far behind opus 5.5. I’ve been trying to avoid Anthropic but the lift of opus vs GLM is enough that I end up coming back to it. I feel like currently there’s no comparison.

anyway, tldr I would not be happy if stuck with vanilla 5.3 so is flash better and should I move to that?

reply
GLM 5.3 Flash is better if you need native multi-modal capabilities, higher speed, and lower costs, but worse than vanilla 5.3 in raw reasoning and for complex programming.
reply
Have you compared to deepseek v4.1 flash? It’s so good, at a ridiculous price
reply
> As an aside, it's really unfortunate that artificial analysis only provides comparison at the highest reasoning level

They do other reasoning levels for some models.

Here is Sonnet 5.5 across the full range of efforts from low to max: https://artificialanalysis.ai/?models=claude-sonnet-5-5-low%...

Opus 5.5 low to max: https://artificialanalysis.ai/?models=claude-opus-5-5%2Cclau...

Here is Muse Spark 1.3 Max and Xhigh: https://artificialanalysis.ai/?models=muse-spark-1-3%2Cmuse-...

I'm guessing it depends how much the vendor wants to contribute. Running these benchmarks isn't cheap!

reply
I agree, however It's too easy to game arbitrary limits like "medium". There is no olympics for "just rolled out of bed and slightly hungover"
reply
We don't even need a standard for reasoning levels.

Just accept whatever the provider names as "medium" and compare the accuracy between models vs the time it takes to complete a task.

reply
You don't need to directly compare the same reasoning levels. Today you can compare Sol on Low vs Haiku on Max, for example, but you can't look at GLM 5.3 High, only Max.
reply
For some tasks I genuinely feel Opus 5.5 does better on low than medium or high. The trouble, as always, is figuring out which.

Edit: I realize after commenting that you're not suggesting medium is useless, but just arbitrary and differs between models. Personally I don't want to directly compare mediums between models, I just want to be able to compare across effort levels.

reply
i run opus on medium, sonet on high, and haiku on high for implementation based how complicated is for planing and specing and review opus is on high usually, except if if the task is contained.

did not try to run it on low, will have to try it.

reply
Opus 5.5 on low is good if you are relatively sure a given task is straightforward and time spent gathering background information is purely wasted or possibly even detrimental. Opus 5.5 low seems great at doing the most obvious thing based on the prompt without thinking much about it.
reply
I agree - Opus 5.5 on low is a really great model if it has some direction and the tasks aren't complex.

If the tasks are complex cranking Opus up to high or xhigh and getting it to delegate to Opus low subagents is pretty effective and much faster than running everything at high.

reply
A graph of time (or cost) per task against success rate does it. Just include all the different levels.
reply
This exactly. The three interesting parameters are total time to answer, total cost, and output quality (for any given purpose).
reply
[flagged]
reply
That's the beauty of the line charts; the labels are arbitrary points on the line. You can easily compare the series visually.
reply
I feel like reasoning is a useless metric, and we should instead focus on results, and time to achieved said results. That is a more useful metric overall.
reply
Reasoning level influences both of those things dramatically. So we need to have it available as a parameter for comparisons.
reply
not a metric, it's a parameter used at inference time
reply
Yes, but what metric defines what constitutes a given reasoning level? That's what's arbitrary here, not the label (which is also arbitrary).
reply
this is properly bad as you can run glm on 6000pro, at (in my eyes) pretty good quant, and a q2 on a 5090. im suprised at how good 5.3 flash is at some parts. i still think ds4.1 flash is better but it is more demanding. i hope meta pushes something in a range of 300-400b with a 15-20b active parameters. i know its a crowded space. but it would be fast and probably mostly good enough.
reply
Have you tried two 6000 Pro Blackwells with a model spread between both ?
reply
The most important graph for me on Artificial Analysis is this: https://artificialanalysis.ai/?agentic-speed=intelligence-vs...

It shows the time to complete a task versus intelligence index, which is much more important for my use-case as I do quick back and forths usually. Someone on here suggested it to me a while ago. It also captures speed and token efficiency nicely, since both of these decrease time per task.

reply
> As an aside, it's really unfortunate that artificial analysis only provides comparison at the highest reasoning level, which is mostly unusable for high-volume production use cases. I would really like them to start providing comparisons at low and medium reasoning levels across these models because those tend to be the ones that actually get used in production.

The frustrating thing is they started doing this for the big labs—you can see their benchmarks for all reasoning levels of Sol 6.1 or Haiku 5.5. They just don't seem to do it for any of the open weight models.

reply
Yep, just benchmarked it against GLM5.3 Flash on my own local bench and it's slower in TTFT, tok/s and concurrency. It also scored lower.

Tried different reasoning levels and didn't really see any difference but GLM 5.3 flash only really works high, max or none so I couldn't really compare the lower efforts which appear to work on Spark.

reply
There’s a reason they put the “US” in “the most capable US open weight model”.
reply
My immediate reaction is that it’s weaker on coding than other open-weight models and that coding is a primary use case for running locally (not the only one, but surely a major one). That said, it takes a lot to get full Kimi and GLM running locally.
reply
Artificial analysis puts it next to Mistral's new model.
reply
Ahat's the cost of a b300 versus two mac minis?
reply
How much longer does it take to ask this and wait for response compared to checking yourself?
reply
Which is crazy bc they promised to open source muse 1.2 which is good. And they give us this bait and switch
reply
yeah for botsbench.com side where we do cybersecurity investigation evals of models + harnesses, we've been having to play with:

1) time budget model gets (5min / task) -- this became a real problem with opus/astra/fable at high+

2) tuning which effort level we report on (when not all)

reply
Why didnt thry include OpenAI or Anthropic models? I get they were comparing open weights.
reply
very true. when configuring custom agents for enterprise, we almost always default to none or low reasoning knobs. The added latency from medium or higher reasoning levels compared to the improvement in response quality is a diminished return.
reply
Um a 512GB m5 max Mac Studio is what 25k? A B300 GPU and the computer to use it is what? Double? Triple that? Apples v oranges my friend. Apples v oranges.
reply
it’s quite suspect to claim that a model is ‘underperforming’ so resoundingly, especially after so little time since release.

there’s this objectivist approach of reducing llm quality to a single dimension, furthermore one with measurability bias.

my thesis is that the chinese models beat the american models on benchmarks, on quantitative measures, but probably do worse in subjective measures. to put an example, compare a chinese phone with an iphone, the specs are great, but the OS? the materials? the privacy posture?

so will chinese llms have great specs and benchmarks, but will miss the mark when it comes to the actual quality in practice when your input stops looking like the benchmarks it was trained on.

America number one, it’s the chinese that play catch up with clones, not the other way around

reply
deleted
reply
[flagged]
reply
[dead]
reply
[dead]
reply
AA is a joke.
reply