LLMs are very good at conceptually simple but tedious classification problems in general.

One that has saved me many hours: given a small subset A and a disjoint sample B of a larger set of, say, complicated JSON objects in a consistent but unknown schema, what do the objects in A have in common that isn't true of any objects in B?

Example result from Claude: The field foo contains dates between August 3 and August 7, 2016 in every record in A, and every record in B contains later dates.

Claude even correctly hypothesized that it was dealing with metadata from a document management system, that field foo was the creation date of the document, and that nearby field bar, identically 10 in all records, was the document retention period in years, and thus A represented documents that would have been purged between August 3 and August 7, 2026.

Oh absolutely. I just had flashbacks to getting outputs where a raster is completely wacko because the input coordinate systems were not the same and so the grids didn’t line up. It absolutely still has to be supervised closely, much more than a lot of more vanilla programming tasks I’ve experienced.

One thing that helped me a ton was making an adapter between my domain specific datasets/systems and QGIS. So I can watch and examine the intermediate steps and leverage that my brain is still pretty darn good at visual pattern matching.