I’ve been renovating a spatial/topology validation system for an OpenLayers driven map editor for robot fleets. I set up a clean interface for spatial indexes and predicates as well as your basic geotoolbox functions.
I was most surprised by a moment where Opus detected that using a union approach to ensure that robot footprints always sat within valid polygons (possibly straddling multiple valid polygons) was slow in testing. So it decided to find the difference between footprint and each polygon within the AABB of it. And if the remaining polygon is zero size, it’s valid. If not, the remaining shape is what is invalid.
I was also surprised just how good it was at recognizing, “in the test data there’s a lot of this kind of feature so we should explore what’s more common: hits or misses?
Before robotics I spent a lot of time building spatial models for data processing and analysis. I find that I can usually just explain the spatial question in natural language and it will figure out the relevant inputs from the dataset and what the process can be. I’m not obsolete quite yet though because maybe 10-20% of the time it goes on a pretty fantastical journey through unnecessary intermediate data products.
The worrying bit is if you're not keeping a close eye on it, very easy to think you're getting a solid system, but thanks to the internal madness it's used, your entire grid has in effect shifted 10 metres right, and any spatial measurements are far more approximate than it's letting on.
That was the bad old days for me. Nowadays I keep it locked down to specific systems for projections or actual spatial calculations, and that seems to keep it honest.
And the surprise is awesome when it happens! My favourite is all the QA steps you wish you'd had and that it reasons out on the raw data alone:
"oh hey, this one city has no data but it has a high population, oh right, that's because of this obscure lookup had a row chopped off, there you go, sorted."
I get to stand back and applaud it, honestly.
One that has saved me many hours: given a small subset A and a disjoint sample B of a larger set of, say, complicated JSON objects in a consistent but unknown schema, what do the objects in A have in common that isn't true of any objects in B?
Example result from Claude: The field foo contains dates between August 3 and August 7, 2016 in every record in A, and every record in B contains later dates.
Claude even correctly hypothesized that it was dealing with metadata from a document management system, that field foo was the creation date of the document, and that nearby field bar, identically 10 in all records, was the document retention period in years, and thus A represented documents that would have been purged between August 3 and August 7, 2026.
One thing that helped me a ton was making an adapter between my domain specific datasets/systems and QGIS. So I can watch and examine the intermediate steps and leverage that my brain is still pretty darn good at visual pattern matching.
For what it's worth, I use duckdb and its spatial stuff as my adapter. Works for my brain and if you force e.g. SQL files for each step, gets there in the same kind of way. Still some sharp edges, mind, but overall working well. And then Claude comes for my maths and I sob.
I uploaded 4 different photos of the same house, from different angles.
I asked Gemini, ChatGPT, and Grok add the same patio to each photo. The results were laughably incoherent (not to mention ugly).
I repeatedly asked them to make the 4 photos consistent in 3D, and it's obvious that they have no way to do this -- no understanding of the scene.
They also could not understand the concept of "back patio" and "side patio" -- relative to the house, when the house was being photographed from different angles. Even when I pointed out which one was the side patio, they would get it confused in a different image.
(I should not be surprised by this, and yeah I know everyone says LLMs have "jagged" capabilities. But to me this shows how bad all the extrapolation of AI I read on say Hacker News has been)
An LLM can describe a photo fine, but it cannot tell you the exact coordinates of what it's describing. When I last tested this, you could sometimes get it to lie about the coordinates, but asking it to draw an approximate bounding box around a given feature - total fail, sometimes laughably so.
We can see for e.g. that frontier models have got really good at CAD sorts of tasks recently. That suggests benchmaxxing RL to a degree but it does mean they've suddenly got quite good at forming simple geometries in a step by step way.
so i made a simple ab testing eval on fixed prompts: https://threejseval.com
There is a 7 degree o freedom robot arm that is really telling of the improvements in coordinates reasoning: