Though a lot of the time this is more an inefficiency of the documentation and Google rather than something only LLMs could do.
* Ask someone to come over and look
* Come back the next day, work on something else
* Add comment # KNOWN-ISSUE: ...., and move on and forget about it.
But year spent days on a bug at work before ha ha!
This is a tried and true way of working on puzzles and other hard problems.
I generally have 2-4 important things in flight, so I find myself doing this a lot when I get stuck.
Just a note that, for chronic procrastinators, having 2 to 4 important things going on is a trigger & they'd rather not complete anything.
I wonder, for such folks, if SoTA LLMs help with procrastination?
If anyone remembers middleschool mathematics this is the coding example of the teacher making you write out the equations in their longest form instead of shortcutting. It's done this way because it shows you your exact train of thought and where you went wrong. That sticks in your head. You understand the problem by understanding yourself. Giving up after twenty minutes instead of stopping, clearing your active cognitive load, and then coming back erases your ability to understand that train of thought.
For a comparison it's like being in first person view in a videogame, and the only thing you have is the ability to look behind you, versus being able to bring up a map that has an overhead view. In first person you're likely to lose where exactly you went to get where you are, while with the overhead view map you can orient your traveled route according to landmarks and distance.
So, the short of it is that this is a great insightful comment that I can back up with my own experience in making a game from scratch over the last 4+ years.
If you want to solve the problem quickly then just use the resources you have, if you want to become someone who can solve problems quickly then you need to spend hundreds of hours banging your head against a wall.
2) There are different levels of debugging. Are your eyes going to glaze over searching volumes of logs for the needle in a haystack with awk/grep/find? Fire up the LLM immediately; don't wait at all. Do the fixes seem to just be bouncing the bugs around your codebase? There is probably a conceptual fault and you should be thinking and talking to other people rather than an AI.
3) Debugging requires you to do a brain inload of a model of what you are trying to fix and then correct that model gradually with experiments until you isolate the bug. That takes time, discipline and practice. If you never practice, you won't be able to fix the problem when the LLM can't.
4) The LLM will often give you a very, very suboptimal solution when a really good one is right around the corner. However, you have to have the technical knowledge to identify that what the LLM handed you was suboptimal AND know the right magic technical words to push it down the right path. "Bad AI. No biscuit." on every response is NOT enough to make an LLM correct itself properly; it will always try to "correct" itself even if it makes things worse.
This can be set as far as 1h of being stuck. Can also be 5 minutes. But by default it is 30 seconds.
My inner kid was screaming "that's cheating!" :-D but on second thought it is a very cool feature for us busy adults, however it's sad the extremes that gamedevs have to go in order to appease the short-term mindless consumers of today's tik-toks.
But more seriously, where's the joy of generating long-standing memories of being stuck for a while on a puzzle that will make you remember that scene for 30 years? An iconic experience that separates this genre from just being an animated movie with more steps.
I couldn't imagine "Monkey Island II but every 30 seconds we push you forward". Gimme that monkey wrench.
TFA and this comment just made me have this thought about today's pace of consumption, work, and even gaming.
Having a tool that instantly searches through the first 50 pages of google and comes up with a reasonable solution is just speeding up what I would have done manually anyways.
Would I have learned more about (and around) the system I‘m building? Absolutely. I just prefer making my system work over anything else, so I don’t mind losing that.
Just so many confusing things go wrong in real-world software, and it is asinine to think that Mythos finding a ton of convoluted memory errors in legacy native code means we've solved debugging. People should pay more attention to the conclusion of "Claude builds a C compiler" - eventually it wasn't able to make further progress, the code was too convoluted and the AI wasn't smart enough. What if that happens at your company in 2027, and all the devs are too atrophied to solve the problem themselves?
I don't think we're "doomed" like some anti-AI folks. But I think a lot of companies - potentially even Anthropic! - are going to collapse very quickly under LLM-assisted technical debt.
The euphoria I felt after fixing bugs that I stayed up late working on is like nothing else.
If you cant fix the bug just slop some code over it so its more hidden.
This is all gonna be fascinating in 5-10 years.
But for juniors, it's invaluable experience. And as a field we're already seeing problems resulting from the new generations of juniors being taught with modern web development, whose complexity is very obstructing of debugging.
I worked on a project that depended on an open source but deprecated/unmaintained Linux kernel module that we used for customers running RHEL[1]. There were a number of serious bugs causing panics that we encountered, but only for certain customers with high VFS workloads. I spent days to a week+ on each one, reading kernel code, writing userland utilities to repro the problem, and finally committing fixes to the module. I was the only one on the team up to the task.
We couldn't tell the customers to upgrade, we couldn't write an alternative module in a reasonable timeframe, and they paid us a lot of money, so I did what I had to do.
I'm sure there are lots of other examples like this out there.
[1] Known for its use of ancient kernels with 10000 patches hand-picked by Red Hat. At least at the time (5-10 years ago).
The time wasted thinking our craft matters more than solving real world problems?
The amount of ceremony we're giving bugs here is insane.
Paraphrasing some of y'all,
> "I don't have to spend a day stepping through with a debugger hoping to repro"
THAT IS NOT A PROBLEM!
We're turning sand into magic, making the universe come alive. It's as if we just got electricity and the internet and some of us are still reminiscing about whale blubber smells and chemical extraction of kerosene.
The job is to deliver value. Not miss how hard it used to be and how much time we wasted finding obscure cache invalidation bugs.
Only algorithms and data structures are pure. Your business logic does not deserve the same reverence. It will not live forever - it's ephemeral, to solve a problem for now. In a hundred years, we'll have all new code. So stop worrying and embrace the tools and the speed up.
This is both a strawman and a false dichotomy.
Too many of our engineering conversations are dominated by veneration of the old. Let me be hyperbolic so that I can interrupt your train of thought and say this:
We're starting to live in the future.
Let go of your old assumptions. Maybe they still matter, but it's also likely some of them will change.
The old ways of doing things should be put under scrutiny.
In ten years we might be writing in new languages that are better suited for LLMs to manipulate. Frameworks and libraries and languages we use today might get tossed out the door.
All energy devoted to the old way of doing things is perhaps malinvested into a temporary state of affairs. Don't over-index on that.
But just today a bug was reported by a customer (we are still in testing not a production bug). I implemented this project myself from an empty git repo and an empty AWS account including 3 weeks of pre implementation discovery.
I reproduced the issue and through the problem at Claude with nothing but two pieces of information - the ID of the event showing the bug and the description.
It worked backwards looking at the event stream in the database, looking at the code that stored the event stream, looking at the code that generated the event stream (separate Lambda), looking at the actual config table and found the root cause in 3 minutes.
After looking at the code locally, it even looked at the cached artifacts of my build and verified that what was deployed was the same thing that I had locally (same lambda deployment version in AWS as my artifacts). I had it document the debug steps it took in an md file.
Why make life harder on myself? Even if it were something I was doing as a hobby, I have a wife who I want to spend time with, I’m a gym rat and I’m learning Spanish. Why would I waste 6 hours doing something that a computer could do for me in 5 minutes?
Assuming he has a day job and gets off at 6, he would be spending all of his off time chasing down a bug that he could be using doing something else.
If you’re experienced as you are, you’re not learning the same way a junior assigned this might learn from it.
I also used Codex and asked questions about how the codebase worked to refresh my own memory. Why wouldn’t a junior developer do the same?
I mentioned that I had Codex describe in detail how it debugged it. It walked through each query it did, the lines of code it looked at and the IAC. It jogged my memory about code I wrote a year ago and after being on other projects
Just because it worked this time doesn’t mean it always will.
If you need further explanation of why you might want to spend more time resolving a bug to learn about the systems you’re tasked with maintaining then I’m at a loss sorry.
But he was doing this for education, not for work.