upvote
> That is an irrationally cynical take. Everything described in the post is clearly within the abilities of a frontier LLM.

No, the burden of proof that it's actually novel sits on the author of this post, and we assume everything is cheated/shortcutted by default when it comes to agents and LLMs, unless we can prove it actually "found" this, not just looked up stuff that wasn't properly in the DB yet but already written about.

> The reasonable interpretation is that the model stumbled upon a previous discovery in the manner described

That'd be reasonable indeed. Instead the author chose the headline "My AI Agents Found an Unmapped Maya Site" and the first sentence is:

> By analyzing NASA lidar data, my AI agents discovered a Maya site in Mexico that, as far as anyone can tell, no one had reported before

Which without any evidence of what the agent actually did, I don't think we should just assume what the author says is true because again, we know that agents takes as many shortcuts as they can.

Plenty of times the people pushing LLMs are over-exaggerating more than in other ecosystems, and the community should rightly push back on these things, when they're not presented properly with proof and evidence. Otherwise we'd be doing previous actual archeologists a huge disservice.

Imagine being the archeologist who actually discovered this site, and waking up today reading some random person taking credit for your discovery! Kind of feels like people here often forget there are other people involved on the other side of things.

reply
He checked with the Mexican government before claiming it was new, which is reasonable due diligence surely?!
reply
No, you don't just do the simplest possible thing and see if it's in some DB because it happens to be more accessible to LLMs, then call it a day and a call it a "discovery". More from the Bluesky thread:

> any archaeologist knows to triple check before saying something is "new"

There is lots of due diligence needed before proclaiming stuff like that. Not just in archeology, but in sciences in general. There is an expectation that you'll review what you put out carefully, not rush to publish as soon as possible, and review prior art. Not least because it's embarrassing as hell to accidentally claim to have invented/discovered something, but in reality someone else was clearly way earlier than you, you just didn't look closely/broadly enough before.

As a second thing, once you find out you're wrong, you generally want to retract something to save face, if you're a good enough human to be able to admit when you're wrong. The author's "What I've learned" remains "You can just do things" even though the top of the article now has a correction that the agents maybe "missed it while going through the literature".

reply
>> any archaeologist knows to triple check before saying something is "new"

difficulty: not an archaeologist but a SWE

cut the guy/gal some slack

reply
> difficulty: not an archaeologist but a SWE cut the guy/gal some slack

Maybe don't publish archaeology "findings" and "discoveries" if you're merely a software developer. I don't go around publishing things about nuclear reactors, because it's not my area, I couldn't even review myself if it's right or not.

reply