Okay for feature work but the amount of bugs I can find with Claude in hour, with deterministic reproducers is staggering.
replyIf you are reporting a number of bugs per hour then you can't be properly reviewing each to make sure it isn't a complete hallucination, a behaviour-as-designed reaction to silly configuration, etc, nor are you reviewing the documentation to make sure it both actually makes sense and is appropriately concise to not waste time in the review process.
replyThis sort of thing, in the minds of not just you but many other people, is exactly the problem. On behalf of the projects that you have communicated with this way: thanks a bundle.
I said "in an hour" to mean, the time it takes to identify the issue.
replyThat fairly obviously doesn't turn into "bugs per hour" to verify, but if you find 10 candidates in an hour and then get reproducers of them out over the next few days, they are real bugs.
Like sure, "not properly reviewing" could be a thing but if I'm looking at a hard crash on my console, that's not behavior as designed.
But you and hundreds of others find these bugs and send them to review.
Lots of duplicates plus lots of hallucinations.
replyBug reports also need review because not every bug reported is a real bug, could be hallucination and the "fix" is useless or even dangerous.
replyYou don’t know the model and prompt they used to generate the report so you can’t even take the model quality into account