It's very difficult to answer those two questions, through all the marketing and fluff text.
> No special harness, or access to models that weren't generally available to everyone were used in the initial bug discovery. I focused on something I had a background in, knew its vulnerability history well, and held a theory that specific bugs lurk within the codebase. This shows that "know your target" is still as applicable as ever, even in the LLM dominated VR age. No model is simply capable of unearthing "all bugs" with simple prompts.
They used a public model as a research assistant and a code assistant. They guided it with expert deep knowledge of the project. OpenAI did not lead this, ChatGPT was not instrumental in this.
It is perhaps at best to say OpenAI made it possible to find the bug more quickly. And on the double edged sword, also made it easier for it to be exploited days ahead of release. So I agree with the original commentator, did they on the balance actually do anything good here?
totally. I spent yesterday trying to get an AI to make a test case for a very specific bug I had. It kept telling me that it had it. When I could clearly trace thru the code and see it did not. After about 5-10 rounds of that I just wrote the test myself. But at least AI helped with all of the dumb cruft that went around it.
But if I did not know the domain I may have easily just said 'yep looks good'. AI on the other hand saved me at least 3-4 hours of writing all of the scaffolding that made it easier for me to check. In the end the bug had nothing to do with the test. AI did not find that. I went back point by point of what changed in the code and found where I had messed something 2 weeks ago.
A few months ago it seemed like it would not do this. Now in the past week I have come across 3 different ones that did something similar. Just 'stuck' in a local maxima and getting the AI to go around it is a pain. This is probably me and the way I am using it.
Is this better than I had a couple of years ago? I am not sure. Seems different at least.
Again:
> as part of our Patch the Planet initiative
To elaborate:
https://openai.com/index/patch-the-planet/
> We are introducing Patch the Planet
> [...]
> we will be partnering with HackerOne and Calif who are helping us take our efforts further with vulnerability triage, coordinated disclosure, and additional focused vulnerability discovery efforts
Calif found the vulnerability while working under an initiative lead by OpenAI, using publicly available AI tools developed by OpenAI.