undefined

points

by jandrewrogers9 hours ago |

comments

by chubot9 hours ago|

[-]

Yesterday I was trying to add a patio to some real estate photos, i.e. "virtual staging".

I uploaded 4 different photos of the same house, from different angles.

I asked Gemini, ChatGPT, and Grok add the same patio to each photo. The results were laughably incoherent (not to mention ugly).

I repeatedly asked them to make the 4 photos consistent in 3D, and it's obvious that they have no way to do this -- no understanding of the scene.

They also could not understand the concept of "back patio" and "side patio" -- relative to the house, when the house was being photographed from different angles. Even when I pointed out which one was the side patio, they would get it confused in a different image.

(I should not be surprised by this, and yeah I know everyone says LLMs have "jagged" capabilities. But to me this shows how bad all the extrapolation of AI I read on say Hacker News has been)

by mootothemax8 hours ago|

parent|

[-]

I reckon this type of task is crossing capabilities in a way that the LLMs (very quietly, mind) don't support.

An LLM can describe a photo fine, but it cannot tell you the exact coordinates of what it's describing. When I last tested this, you could sometimes get it to lie about the coordinates, but asking it to draw an approximate bounding box around a given feature - total fail, sometimes laughably so.

by jazzyjackson1 hours ago|

parent|

[-]

This seems like a more pertinent alignment issue than guardrails and preventing bioterror etc: does the bot even know what it can’t do? Because the eagerness with which they’ll be wrong about drawing a bounding box concerns me much more than whether they’re willing to talk me through synthesizing LSD

by post-it8 hours ago|

parent|

prev|

[-]

Yeah, I tried to have AI arrange some furniture-shaped rectangles on a black-and-white floor plan. Impossible. I could painstakingly guide it through writing a floodfill to find the corners of a room and parse out the dimensions listed in that room, but at that point it wasn't using its image-seeing abilities at all, and it still couldn't do things like putting a particular rectangle against the bottom left corner.

by physicsguy7 hours ago|

prev|

[-]

I think they do but only when it's applied to learn some specific spatial task.

We can see for e.g. that frontier models have got really good at CAD sorts of tasks recently. That suggests benchmaxxing RL to a degree but it does mean they've suddenly got quite good at forming simple geometries in a step by step way.