upvote

points

by vunderba6 hours ago |
comments
upvoteby vunderba1 hours ago|
next
[-]
Well, the results are in, at least for text-to-image (the editing bench will come later).

Qwen-Image 2.1 is definitely a pretty big leap over the last open-weight version, Qwen-Image 1.0, released back in August of last year and managed to score 7 out of 15 as opposed to its predecessor which scored 4 out of 15.

Even though it's significantly smaller, 7b vs 20b, it's multimodal (so you don't need a separate image-to-image model like you did with Qwen-Edit), more coherent, and significantly faster even when outputting at higher 2K resolutions. However, in my testing, I found that I had to play with dialing up the CFG depending on the complexity of the prompt.

I've also added a progress dropdown under Model Performance so you can see how cloud vs. local models have been trending since 2024. Spoiler: June of this year released some of the biggest bangers (Krea 2, Ideogram 4, and the kind of slept-on Boogu-Image 0.1).

Downsides:

- It was clearly trained on at least some level of synthetic training data, and it shows in some of the subpar outputs in terms of fidelity. Some of this you might be able to iron out with a refiner model downstream or a custom LoRA but time will tell.

- They've moved away from the permissive Apache license. Commercial usage is only allowed by request.

Comparisons:

https://genai-showdown.specr.net

If you just want to compare local models only:

http://genai-showdown.specr.net/?models=local

reply
upvoteby Epitaque4 hours ago|
prev|
next
[-]
That benchmark might have some issues. You prompted the models to generate an image of striking a ring against a crucible. Then you (presumably, manually?) scored the images that depicted an anvil higher than the ones striking something resembling a crucible.
reply
upvoteby vunderba4 hours ago|
parent|
[-]
That’s a good catch. Yes, all scoring is done through manual review since relying on a VL model for these kinds of meta-metrics is a sort of loose equivalent of gödel's second incompleteness theorem.

I’ll have to think about this one. When I crafted the prompt, I wasn’t really thinking about the differences between a crucible and an anvil. It was more the visual of an archangel smelting halos for newly arrived heavenly beings.

reply
upvoteby speerer3 hours ago|
parent|
[-]
I'm not sure why one would even strike metal against a crucible! It's a container for liquid metal. One of the outputs shows it being smashed by the manoeuvre, which is probably the most realistic outcome of all of them.

Sorry, I'm not trying to nitpick. I'm just joining in because I'm interested in how the models dealt with the request.

reply
upvoteby vunderba3 hours ago|
parent|
[-]
Well this is HN - original home of the "ummm actually..." - so I appreciate when people pick all the nits. :)

Even though I prompted for a crucible in the prompt, I think the fact that the prompt also contained terms like “blacksmith” and “hammer,” caused it to lean towards anvils over crucibles in some of the pictures (which as you brought up makes more sense anyway).

reply
upvoteby zargon4 hours ago|
prev|
next
[-]
It would make more sense to compare with Qwen Image 2, since that was the last open weights Qwen model.

Edit: This is wrong.

reply
upvoteby vunderba3 hours ago|
parent|
[-]
Wait... is that true? I don't think the original Queen Image 2.0 was ever released beyond an API. At least, I don't remember a public weights release.
reply
upvoteby zargon3 hours ago|
parent|
[-]
I promise I searched on my phone before posting that, but it looks like I misinterpreted an hf search result that was actually Qwen-Image-2512 but the url was truncated to Qwen-Image-2 due to my screen width (ugh). For some reason I was thinking Qwen Image 2.1 was the one that had been available for a while but not released, but you're right that was 2.0.
reply
upvoteby Chance-Device2 hours ago|
prev|
next
[-]
Native transparency isn’t so hard to do by the way, I made an image AE (I don’t say VAE deliberately as none of these are VAEs, I don’t know why they keep being called that since the variational part is completely absent) that supported this about two years ago as a hobby project. I haven’t really been following the space recently, I’m surprised it’s taken so long for this to come out if it’s a first.
reply
upvoteby alightsoul6 hours ago|
prev|
next
[-]
They're trying to cash in but this is just sad
reply
upvoteby thenthenthen6 hours ago|
parent|
[-]
Within a few days this seems a total pivot from Xiaomi’s op RL dashboard and the praise of Chinese open model? What is the sentiment now?
reply
upvoteby Foobar85686 hours ago|
parent|
next
[-]
Qwen can't train anymore with openai reasoning tokens? I kid I kid.
reply
upvoteby cmrdporcupine6 hours ago|
parent|
prev|
[-]
Everything is combined and uneven, including the opinions of hackernews commenters?

There is no single opinion, and clearly no single Chinese approach.

Also Chinese labs are in particular very careful about anything which be used to create pornographic content, which is highly illegal in the PRC.

reply
upvoteby ranger_danger5 hours ago|
parent|
[-]
> Also Chinese labs are in particular very careful about anything which be used to create pornographic content, which is highly illegal in the PRC.

MiniMax H3 is dominating AI porn right now, and they are Chinese.

reply
upvoteby alightsoul5 hours ago|
parent|
[-]
That's because the users don't care about the licence. They did try to take down a porn Lora repo on hugging face. The trick is, don't say you're using minimax, and you can make porn with it.
reply
upvoteby tempesttempo864 hours ago|
prev|
[-]
Could you add new OAI 2.5 image models?
reply
upvoteby vunderba4 hours ago|
parent|
[-]
Can do! GPT-Image-2 already scored unsurprisingly very high: 12 out of 15 on text-to-image, and 10 out of 12 on image-to-image.

The three benchmarks it failed on (D20, Flat Earth, and Banded Snake) are pretty difficult, so I'd be surprised if 2.5 manages to pass them, but I’ll add it for completeness’ sake later this week.

reply