Could you rephrase this or explain it more thoroughly? I don’t follow. What does it mean to categorize a written form by systems built with Claude?
The original prompts aren't provided, nor is the original context; even then, you can't really treat a stochastic system like an LLM as a major component in reproducibility.
If you had the other things, being "stochastic" is not even remotely a show-stopper. Stochastic processes abound and are the reason the mathematics of statistics was developed in the first place, ultimately allowing us to create such things as LLMs.
When all the relevant steps gets published, I absolutely expect a lot of people to (attempt to) reproduce this work even though LLMs are stochastic.
On the prompt formulation; prompts with very similar formulations (in terms of both semantics, hamming distance, or both) can lead to _wildly divergent_ outputs in my experience. It's not rigourous, and when that divergence happens, it's extremely difficult (arguably impossible, by nature of the architecture of transformers) to identify why the divergence happened and where.
It's not about being able to throw claude or codex at a loop and having it evaluate it for halting, it's about being able to do this for arbitrary code. Computer science rigourously defines the halting problem as not computable and undecidable. within the framework of using something akin to static analysis using any deterministic Turing machine.
There's not really a question of "solving" the halting problem like there's some as-yet unknown way of generally figuring out if arbitraty code halts. Turing proposed a proof in 1937 in favour of undecidability of what we now know as the halting problem, building on ideas first articulated by Church a few years prior.
Frankly, if anything, it's reasonable to say that the halting problem's been solved, just in the direction of undecidability rather than decidability.
Anyway, back to LLMs; as code gets more complex, the robot will need a bigger context window, more hardware resources, and more time, all of which will be variable due to the noise inherent in the system. It'll be difficult to put a useful upper and lower bound on how much computing power and time it'll take to figure out if a program ever halts. Which is all a bit moot, frankly, in the context of halting, but useful to keep in mind in the more general context of using these things as analysis tools.
Every day when you lower your butt onto your chair, you trust a stochastic system enough to assume you'll rest on the chair safely and not spontaneously phase through, which would lead to rather gory and painful terminal experience.
Physics at macro scale is stochastic, which is a good reminder that stochastic != uniformly random. Expected distributions matter.
IMO a better example would be the stochastic nature of quality control in manufacturing.
https://gist.github.com/fragmede/bbf277d36a2398065f109484f34...
Cynical read would be you’re stealing his thunder a bit by prematurely announcing this before it’s fully confirmed
One of the things I find weird with AI is how the dismissals of work that involve AI splits into two camps: like yours, saying the AI did the work while the human played no role and deserves no credit; and those saying the AI rips off its training data while the human using it played no role and deserves no credit.
No human, individually or as a team, has been able to solve this to date.
To the extent this was Claude solving it itself and thus denying Di Mino any thunder, there was nobody to have stolen anything from. To the extent he has thunder to be stolen, it wasn't ever in Claude's possession.
Either all information is stolen, or none is. Can't have it both ways.