undefined

points

[-]

These models are, in a sense, distillations of language models. They wouldn't be as good without a pre-trained language model, so it's unfair to compare them.

Image models are essentially a mapping of text embedding space -> image. So the clearer and cleaner the text embedding space is the better model you get as you need less weights trying to interpret a text prompt, recent image models even use multimodal versions of LLMs (Flux uses Qwen3 VL) so all the visual separation was done beforehand making the model even more efficient.

edit: Maybe a better analogy is that the image models are only half the model in a encoder-decoder sense, where the LLM is a pretrained frozen encoder and the image model is the decoder.

by roenxi1 hours ago|

prev|

[-]

To make a rather speculative argument - since an LLM should be able to describer an image losslessly, images are a subset of what an LLM needs to understand. It is possible that the statistical model being learned for images is just simpler.

The get a proper LLM they have to model all of human knowledge regarding patterns, physics, biology, social interaction, skills, history, etc, etc. For images they have to recall what a dog looks like in a still photo. The image models aren't even necessarily good at a wide range of novel situations; usually they get used for stock scenes.

Natural evolution has also had a much easier time coming up with vision models (everything with eyeballs) than complex language models (as far as we know only a smallish family of monkeys).

by swyx25 minutes ago|

parent|

[-]

this is the right answer. ultimately text and general llm usage incl coding is far more internally complex than just memorizing the looks and composition of concepts placed in an image. dont forget that most of our reasoning benchmarks basically require the llm to have a search engine and calculator in its own weights.

by gavmor1 hours ago|

prev|

[-]

At first, I considered that diffusion models struggle with reasoning because they underweight 'prior beliefs', ie the propositions by which an argument advances.

Then I considered whether the universe of imagery is simply smaller than the universe of ideas—which must, itself, include the universe of images, too.

But finally, I'm thinking maybe images are just harder to benchmark.

Or maybe they're _not_ actually worth 1,000 words.

by ThrowawayTestr1 hours ago|

prev|

[-]

They can make good images but they all look the same. Guess it's not hard to replicate a small section of what exists.