Your post literally describes your fascination with trying to figure out the pattern of a "random" reward that you get, and trying to maximize the value you get out of it.
I put "random" in scare quotes because I strongly believe that—just as slot machine payouts are carefully structured to keep you playing—these LLM resets are structured to keep heavy users like you coming back to max out their usage, and to progressively upgrade it.
Several other commenters have also stated this same suspicion about the pattern of resets you're describing.
> "Not wanting to waste money" is the polar opposite of gambling.
From everything I've read about gambling addiction, particularly Jay Caspian Kang, that seems wrong.
The desire to "not waste money" and "get back to even" seems like a huge part of what motivates gamblers to keep gambling.
As someone who had family members go through gambling addiction this is the primary mechanism behind it.
Addicts don't see it as "cool fun dopamine kicks" but instead find it the only way they can get back to normal/where they are supposed to be
Like the LLM getting the solution right?
- 80% of prompts get everything correct and are confirmed correct with manual validation
- 19% of prompts make a minor mistake based on an ambiguity of the original prompt (user error not LLM error), but then reliably fixed in a followup prompt
- 1% of prompts causes more problems than it solves and is more pragmatic to just revert
For 99% good output, there isn't much of a dopamine rush when there is good output. The dopamine rushes are for the <1% odds.
From the other replies on this post, I suspect no one believes me, but I am offering these numbers in good faith.
I think many people who don't believe you just haven't built-up the kind of prompt history & MCP / CLI tooling etc that lets you get to the point where things work at that level of accuracy.
Hope it helps to know that at least some of us here understand and are seeing the same thing. And if it's anything like my experience with Fable, "always be more ambitious". The capabilities of the models are often limited only by what you're brave enough to ask for. I keep finding I'm not ambitious enough.
This right here. Any gambler would recognize that statement.
I've been researching LLM prompt optimization for longer than ChatGPT has existed; I was successfully optimizing the output of GPT-2 back in 2019.
Some of these things are only possible to really see in hindsight. Yes, you've been working on these things for a while, but these systems are notably different in their capacity and strings they pull on us.
Be well, please.
Every single prompt worked without issue, and it got most of the way on the first try with the initial prompt (+ a couple visibility bugs due to the agent not having Computer Vision to see said menu bar app) such as:
> Create a SwiftUI menu bar app named `swiftmote` using theto create the most user friendly app following Apple's HID guidelines for creating a remote that can operate a Apple TV on a local network. Instead of reimplementing the protocols needs to interface with an Apple TV, use the Python package `pyatv` and host it within the SwiftUI app as a sidecar along with a Python installation.
I have my own Apple TV I can manually verify that it worked as expected, which is notable because the agent can't test or lie about this pipeline because it does not have access to the Apple TV.
That is not hallucination or psychosis. If you want, I can release all the prompts I used. (EDIT: Sure, why not, here are the prompts. If I don't complain about something in a followup prompt, assume it worked correctly: https://gist.github.com/minimaxir/30fa820daa1392da13026ec6aa... )