I actually have a pretty simple set of instructions right now and Claude still regularly messes them up. Like, my first instructions are:
- Never commit to git without permission.
- Never sign commit messages
Then it gets even more fun, because if I politely correct it in the session, it will understand, but then there's a decent chance it will be completely unable to commit anything in the session going forward. Phd level intelligence!
Anyway, my point is, if you're asking it to review a list of 200 items I guarantee it's quietly messing up on a lot of that list.
Negative prompting is very unreliable. Giving exact instructions on how you want commits made will give you better results.
if what you're suggesting is true, then more important instructions like "Don't delete the production database" are a problem. I shouldn't need to consider how to phrase "Don't delete the production database" in a positive manner. Isn't the point of an AI agent that it understands my intent and I don't need to hold its hand? I'm not saying your suggestion is wrong, I just think that suggests a limit to what these tools should be used for if that's the case.
My guess though is that it probably ignores some of the positive instructions too, and the hardcore AI users mostly don't notice because they probably aren't reviewing the work.
These are still stochastic machines, guardrails must be inserted at the system level.
They are getting better every day about managing their own guardrails, so we will get there eventually.
This is a gripe I've had with AI tools for a while now. Though it's gotten somewhat better in time, but we don't really know what to expect from the tool in terms of quality. Ex. I'd expect a human engineer to probably not use a brand new assertion library for a new test when there are 200 tests using an existing one. But Claude has done this to me multiple times. So I have to add yet another item to the list, like you have, and tell it to look for testing conventions before writing. But, there is plenty we don't have to tell it, like what a function is or a test should probably cover the change in the diff. But we don't really have a list of what things are on each side so we're just left to sort of hunt and peck to build a viable solution.
/init will make a project-wide one, or you can instruct it to "Create CLAUDE.md in any sub-directory that is sufficiently complex" then modify from there.
* https://github.com/alibaba/open-code-review
** https://layandreas.github.io/personal-blog/posts/beyond-vide...
I am curious what does it contain, for me a lot of times its a back and forth with agent until it "looks good to my eyes and taste", but haven't written any such list yet, because it is context dependant, in some projects I forgive minor issues, or allow magical numbers, but in other projects I force agent to use constants with meaningful names `SECONDS_IN_A_DAY = 24 * 60 * 60`
https://en.wikipedia.org/wiki/Code_smell
*edit: that wikipedia page ^ itself is a pretty answer to your request for a list of things to avoid when writing maintainable code.
EDIT: reading the WardsWiki reference from that Wikipedia page, there's also the point made by early users of the term that smells are something you have to check out, but don't always mean something needs fixing - e.g. a bad smell may be a gas leak, or it may just be a rubbish bin.
If that's true, we're left with a question equivalent to "does everyone see the same red?". As far as I know, the pure version of that question cannot be answered because subjective experiences of sensation cannot be transferred. And at that point, I'd say the manner in which they're experienced differently is equivalent.
For a Rust project, I created macros that output compiler errors when documentation and tests are not in a shape I want them to be, like missing function invocations or assertions, which forces the agent to address them, where otherwise they would've just worked around them by adding stupid trivial assertions like `assert_eq!(true, true)`.
That still isn't fool-proof either, but it helps minimise those instances. I'm bullish on the idea of integrating formal methods and model-checking with AI. I think that combo feels like a promising avenue for constraining the stochastic side of AI-generated code with something closer to deterministic verification. Provided you can write correct specs of course!
Failing that they default to the most common style they were trained on. Which, at this point, is mostly code they wrote...
Yes, yes, there has been a library of information on HN by now about how to use agents effectively. (And I'm grateful for that, because I can keep current and in the loop without feeling enslaved to the new style of development.)
None of that is a reason not to do what the title of TFA says. If your review process is doing the right thing, you should observe that it results in your agent moving the code in the "human-maintainable" direction. If you, for whatever reason, actually directly make commits yourself any more (read this ironically; I genuinely can't understand why anyone would want to give up on that, no matter how good the generated code gets, because "the LLM could do better" is not the point), then of course you should write it to be human-maintainable.
The reason humans find "human-maintainable" code to be maintainable is because maintainability is one of the precious few worthwhile at-least-vaguely-objective metrics of code quality we have.
Every time I see someone try to make a point about the fact that some code actually is just better than other code, only to be met with more of this sort of advice, I start to wonder whether I was alone in ever actually enjoying programming.
Good way to double your token use though, if you’re concerned about that.
What ever happened to communicating through code?
Agents can follow examples and infer patterns, and they can read commit history and diffs. Real-world commit logs for human-only projects are dominated by short commits (well, at least the ones where the humans are skilled, appreciate version control, care about the project, etc.) with thoughtful commit messages.
Will try your approach to distill the code to bullet points.
WTF - I need to implement a review command to guide to do its job properly.
Can you imagine any other industry charging people money for a product like this ?
When you are charging people money - scratching the surface cannot be an excuse.