You ask it to do something, then tell it to do something in a different way, then it assumes it needs to do the refactor in a backward compatible way, or creates migrations for it etc.
It contains (Among other things) stuff like "this is a single user personal project, I'm the only user, this will never be open to the public internet" etc.
It kinda-sorta tones down the proclivity to worry about backwards compatibility and slight edge cases where if someone has edited some template and the new code doesn't support it.
1) use the edit in-place functionality; or
2) ask it to rewrite to remove something
It'll write the prompt as if the agent (codex) knew about the conversation and add "don't do X" etc. At first that bothered but I realized it doesn't really change the output so I stopped caring.
Still, no experience is unique I guess.
Now people think I'm just parroting what Claude said. It sucks. I want my catch phrases back, I guess this is how em dash users felt
They're only a LinkedIn-ism when used to create an unduly dramatic juxtaposition for an otherwise mundane idea. But now they set off people's AI radar when used for any reason at all.
I asked Sol to reduce the length of some documentation we had by making it more concise. It came back after 20 minutes of work, did a line count and was aghast that the line count had somehow increased...
Even then it is conservative. For the love of God, compact the comments.
Comments become a huge maintenance burden, especially in the age of AI. They just grow and grow, and then mislead the AI later on.
That is a dramatic shortcoming, but I fix it with permalinks to other files. https://replicated.live/blog/link
I also love how it'll build local plans with phases, tasks, or decisions, then reference those numbers in those same useless comments.
I wish they trained the bots to be a notch more relaxed and less hysterical. Less is more.
But maybe that's just a consequence of the RL training being essentially AI torture to make them do what we want.
1) Understands the problem ("what are comments?")
2) Accepts that it is a problem ("how can more comments be a bad thing?")
3) Cares enough to solve the problem ("is this issue really a priority to solve right now? just accept the PR and we'll go back and fix if needed some other time cough never cough")
4) Believes you (this can take many forms, but the most common is, subconsciously, "this other engineer says it's not a problem, so I'll just assume it's not since that's easier")
All of these logistical, political and social factors are "the ocean"
To be fair, I would not be surprised.
"My peers are all bad", "Management has no idea what's going on", "My boss is out of touch", "My staff keep screwing up".
But if you're consistently seeing the problems while everyone else becomes defensive, you (that think that way) need to learn how to communicate effectively in a way that doesn't result in being 'the baddie' and solve the interpersonal dynamic along with the technical problems. "Crucial Conversations" is a good place to start.
If you really want to save your career, learning how to have real conversations is a vital skill.
Asking because when I was at capital one, certain comments or questions about topics like this, would actually get you noticed negatively by your manager and being disruptive to what leadership wants hurts your career.
In fact being the one to ask and point out questions like yours ultimately got me PIP'ed and removed from the company. So, like, yeah, being vulnerable and the first one to tell the truth is risky. That's why the bystander effect can happen and in politics or risky situations silence is a common response, and why we don't have more courageous people doing the right thing - because the risk is higher on the person speaking up and the rewards aren't disproportionately in their favor (but are evenly distributed though, so its not favorable for you as I learned from a game theory perspective to voice up / defend certain stances).
I started my first post-Capital One job a few months ago, and I'm still not used to the fact that this company expects everybody working here to still be working here in two years. Most employers are not going through everybody with a fine tooth comb every 6 months hunting for any hint of an excuse to PIP them.
Besides, GP was not talking about corporate, but about the inability to even suggest an improved usage of LLMs among their developer peers, as if every line of code generated by AI is now gospel and questioning it is treated as heresy. This level of defeatism is beyond cynical and approaching childish.
There's a reason 3rd places exists. There's a social construct that off-work discussion (even when on-work) stays off work.
If your coworker is professional and cares about keeping channels open, they too will respect the unspoken rules that govern 3rd place neutral locations.
Blame it on the alcohol later on if it actually pisses someone off.
Not anymore these days.
And, really, it works. You can copy and paste between tickets and Claude, and then do manual testing. Then you tell Claude to self-review for clarity and minimalism, and stop worrying. Sure, today it's not as good as a human, but for almost all the code out there, it gets the job done. There's no skill needed any more, and if the boss doesn't care about quality, I don't see why I should.
There may not be a career left in software development with these monstrosities, but if there is, it's in using decades of experience to get the abominations to produce something other than unreadable dreck.
If the code itself truly doesn't matter any more, programmers will no longer be paid.
All the execs know that they can refresh the screen faster and see the features for marketing and sales.
When has engineering ever had a voice anyway? The bean counters have been looking to cut us out since the inception of our industry.
In this fucked up job market, it is easier said than done.
I feel like it's all just perception and how companies can sell their stories to investors or potential acquirers, and everything else can be punted and dealt with later when we get acquired or when share prices are a zillion dollars etc.
It's a race to the bottom, for sure.
I don’t have time to go through and flag everything or even read the thousands of lines of code changes that are happening.
Corporate Capitalism's main innovation was virtualizing feudalism. Now anyone with a bit of cash flow can be a feudal lord with peasants to do his will. No need to maintain land or fight over it with other feudal lords. There's still fighting, naturally, but now the game isn't zero sum.
The serfs don't get any real say in this model. At best you can bend the lord's ear if you're sufficiently trusted an advisor. But part of the reason you're trusted is to be trusted not to treat the lord like he's too much of an idiot.
A huge problem is that capitalism rewards cash flows and accumulated capital so powerfully that lots of other things just don't matter, potentially for a very long time. Look at GitHub. This leads to immensely extended fuck around phases and "find out" looks like IBM (fossilization and bizarre holdings corporation / dead company parking lot) just as often as it looks like Enron.
I am a programmer but also in management, and if I knew my team thought like that I would be horrified. We absolutely care about what the team members want , like, hate etc. I for once would love someone to bring up stuff like this to me. As long as they are as open minded about things as they would like me to be, there is no issue. But it’s a serious issue to pretend everything is fine while thinking like you say.
Do the people above you in management care as much? It's the top levels of management that decide hiring, firing and budgets.
Reframe your context, pretend you're in a communist society, and you could be making the exact same point about speaking up against the grain being bad for you. It feels as though people are brainwashed by anti-capitalism sentiment at the moment, unable to see outside of that context.
Still I always find it funny when I give it code to review that it itself wrote and it immediately says how good the comments are, it's like that obama giving medal to obama meme.
1) Document what's there, not the diff. Documentation of how code was removed or changed to fix a bug or add a feature is not useful and difficult to maintain; documentation should explain how code works now.
2) Documentation should live close to the source as possible. Prefer line based comments and standardized function documentation. Top-level sweeping architectural essays are not maintainable for every change.
The last will depend on your codebase. It CAN be very useful to have a human-readable spec documented for the entire program and have it updated when anything changes. But the key is again, you're CHANGING it every time. If you add a whole new disconnected documentation file it should set off alarm bells; nothing in one system is truly disconnected.
We recently added a similar thing to our style guide, It’s astonishing to me that we have to spell this out, that something as obvious as this needs to be explained to LLM’s at all. They’re supposed to be exceeding human intelligence, at least at things like programming, but can’t understand basic things like what code comments are.
When vibe coding the content of user prompts is ground truth and the only way any human thought affect the code base. So if the vibe coder says "do X not Y", recording int comments that "we shouldn't do Y" is important. It ensures that the agent doesn't accidentally decide to do Y tomorrow, which would frustrate the vibe coder who'd feel that the agent doesn't "remember" what it was told yesterday.
So for people who look at the code the comments are obvious and completely superfluous, but for the vibe coder it's a way to ensure that their tiny (relative to the size of the code base) input is not forgotten.
Hehe. Yeah, that tendency of LLMs to document "the story" of the code instead of its current purpose (or non-obvious implementation details) is a pet peeve of mine too. I've added a slew of guidelines to try to sway Claude to not do this, but it still does it often.
At the same time, it feels like something to be expected to have this "failure mode". The model has its context to work on, and what is on its context if not the conversation you've been having (and its internal monologue) and the files it has read? It makes sense that it references the story on its text generations, because that behavior is usually a good thing for an LLM to do. Otherwise, what would it generate? If it generated things that had nothing to do with the conversation in its context, in many cases those things would be seen as "hallucinations", and they'd tend to be RLHF'ed out. So the models that we end up having are the ones that have been reinforced to be most "contextually relevant" and less "hallucinatory".
I might be completely wrong on that of course. It's just my intuitive reasoning of why this seems to be such a prevalent behavior.
This perception is a good part of why this market is irrational. LLM's aren't "intelligent". They do not reason, they are a very fancy kitbash of whatever it trains on.
Ad yeah, I'm not surprised that a lot of documentation on every bit of readable code online is awful. "Document the diff" sounds like an anti-pattern learned from people with an incentive to get as many PR's submmitted as possible, not make the most friendly documentation for people maintaining a project.
In this day and age, https://hypothesis.works/articles/claude-code-plugin/ might be useful.
I think enforcing black box testing is the best way to get useful tests out of both humans and robots. They must not know the internals, or it will lead them to do bad things.
I wonder if I can get a MacArthur grant for this epiphany...
How do you enforce that?
I'm not saying that's good or bad because I don't know, but I think that's the idea of dumping all that junk into PR descriptions.
However, annoyingly, we still need to review those descriptions very closely, because the robots are trained to put a lot of weight into things they read in the documentation. And they tend tospresent loose speculation as fact. They often end up documenting some assumption that isn't true, then end up writing code as if it were.
I've created myself a pre-commit harness hook to explicitly discard superfluous or too lengthy comments. Within code-review I also added comment review as blockers
They totally gotta be doing that at OpenAI. Meeting invitees: You, co-workers, GPT 5.6.
Give feedback that about the docs and block merging till the issue is resolved.
I've denied for poor branch names and commits from AI. I've denied for too verbose of comments from AI. I've denied for parts of the code base being touched that are not relevant to the case they are working on (login isn't broken, your case is to add a check box in the settings pane, remove the changes made to login).
Pre-AI I wasn't fine with PRs with multiple features and touching irrelevant areas of the code base. Why would I be fine with it because my co-workers got new toys? You want AI to refactor a part of the codebase? Fine. Separate branch, new PR, and in the description, present an argument for it. Don't shoe horn it into something else. Also, I'm not obligated to hit approved on shit.
I may also be a bit privileged because I can be a pain in the ass to whole team. I may be slower, but I've got the numbers, my code is creating way less bugs then my "fast more efficient" co-workers.
Turns out not every team has the idea of "progress" being yet another iteration of "more code submitted = better code". Some people actually desire or need to make quality products.
Actually, rejecting them is precisely what will make them easy.
"Sorry, the comments are so bad I'm stopping here. Please fix them and then I'll resume the code review."
You're giving everybody (including yourself) more work by:
1. Reviewing the code (even if you skip the documentation).
2. Letting too many abstruse comments in which everyone in the team will have to read.
3. Allowing the behavior to continue.
Become the bottleneck so the team can talk about it. If they decide this shouldn't be a blocker, just declare you won't review the comments going forward.
This is also how I do code review of AI work on my projects. If the work is offensive to the point I can't complete the review, I simply reject the code and tell the AI why. Then it goes off and fixes it. This repeats until the issues are either gone or are small enough that I can just fix them myself and move on.
There is no need to be upset. Just iterate until it's right. If it's cheap to write, it's equally cheap to rewrite.
Not to mention that the response to each review will come with its own set of new comments and new documentation. The ability of people to write things has exploded tenfold. You can’t out review the slop.
Every PR can be rewritten and re-architected on a whim.
I'm currently not working in tech, but I will again sometime within the next year or so, and I've been reflecting on my career in light of the recent AI madness. I think the biggest mistakes I've made over the last decade+ in tech as a software engineer have been caring. I've worked at small, medium, and large companies. Famous big names you've definitely heard of, less famous ones you probably haven't. In every case, in retrospect, as an individual contributor (non founder, non board member, etc) being personally invested in some outcome--quality, efficiency, cost, ux, customer value--is a mistake.
If you aren't actually in a position to change something, caring about it is futile. [edit] And I don't mean in the sense of "feeling empowered" or some such, but actually having the clout to steer the ship and make it happen. Very few people in a tech organization actually have that power, by design.
The industry wants to do away with code review, meaningful testing, computer security, and reliable services. Why swim upstream? You can't actually do anything about it so why try?
When I go back to work I'm going to do my 9-5 40hr/wk, smile, nod, punch the clock, and make the bosses smile or whatever. But I'll be damned if I'll ever give a shit again.
Because habits beget habits, and in form imprint into your identity. I'm not the biggest fan of GPA fixation, but it does have the side effect of imprinting work habits (I won't say "ethics", because the extremes I saw peers go to to get A's is anything but). It doesn't necessarily imply mastery of your class, but your ability to adjust to someone's standards and meet them. Which is something that will follow you into the workforce, and roughly correlate with people who will meet the standards of their company.
If I don't give a damn about code quality in a place I spend a third of my life, why do I expect to give a damn when I want to work on my own project in the future? Even if I try those habits will wear into something I can call my own. This might be fine if your overall goals have nothing to do with your work; that you are fine just going through the motions and getting a paycheck to empower your non-tech hobbies or focus on supporting your family. But that's not the path I've chosen.
As I said here and elsewhere: If management is concerned code review is a bottleneck, let management be aware that the process has to change. Either the human is not as thorough, or he uses some combination of his brain + LLM to conduct the review.
It's a basic fact: They have a machine that produces a lot of code, but don't have a machine to review that code. The bottleneck is obvious. I'd love to build planes for $1 too, but physics applies.
I wouldn’t say “post reading code” but it’s definitely trending in that direction.
I’d rather the agents put jumbo verbose descriptions in the PR description than in code comments TBH.
It might be touching one line of actual code in a file, and take advantage of it to remove 20+ lines of actual useful comments.
Everybody is talking about the opposite, so I'm wondering if this is rare.
Sometimes I have luck interrogating Claude on why it did something. It'll either point to a skill or agent file with the culprit, or it'll respond with some vapid nonsense and apologize.
I ask because when I started informing it I made personal edits, it stopped doing this kind of thing so often and let me work _with_ it more.
My saved prompt now says never to assume a file has not been edited since the last time it was read between prompts.
I don’t trust it with code anyway - every feature needs comprehensive test, and then a live deploy on a real working test system before it is approved - I mostly measure success with - after deployment is it doing what it’s supposed to be doing. It’s like “helping another team managing their work stream” experience rather than coding yourself.
Funny enough models seem to have personalities and the dis on each other - when I had an opus orchestrator dispatching fable workers, they would comment on how “unreliable” it was and it had “evidence to prove it” and fable thinks opus is too rigid and needs more hand holding… it really starts to feel like managing team egos and verifying work. And I code scan mostly to just spot check if it’s not doing anything super stupid. But my goal is to make sure anything shipped is easy to change and fix, and every mistake has a test behind it so it doesn’t happen again.
I ship more problems, but they get discovered and fixed quicker. Before they reach prod of course. And from time to time you do reorganisation and refactoring passes where I brainstorm how things could have been better with the help of evidence- chat sessions, tests, bugs etc.
It feels less like rigorous engineering and more like gentle gardening. Or like “project management” not “coding”.
Honestly given my age now I’m fine with that. Have enough “hard” projects under my belt (ORMs, sql parsers, etc) that I don’t feel I need to prove anything to anybody, but I don’t think that’s even relevant- the velocity change is … I guess around 5-10x for me - with provable metrics, so I try not to lent the good old days but figure out how I can now live in this brave new world and be happy with my work.
If you don’t trust it with code, surely you need the code to be readable so you can understand what it is writing?
My worry is that after several passes this compounds and starts introducing errors or biases, a bit like in the "telephone game" children play.
They're just helping you understand the whole picture!
> Features are coming out and metrics are improving, but we’re basically in a post readability code base, with the occasional performative comment about a variable name.
You futilely grasp for control and it eludes you. The Way is to ride the tides of life, move with the forces that shape you. Your code base is in the hands of the Machines now.
still get walls of text sometimes
Write REVIEW.md.
I have CC check itself pretty well.
I also put into agent/claude/review instructions to write using simple English skill and humanizer skill. Then not to write redundant comments.
It’s not perfect but definitely catches lots of slop.
Yes, that's the entire point. And it is extremely useful. Why wouldn't I want this?