They're amazing at it, provided you keep asking the right questions.

Trust me when I say that in the hands of someone who doesn't have your experience, the LLMs would not be getting the results you get.

You might think what you're doing is trivial, it may be sessions that flow roughly, "Instrument this, okay this part is slow, profile this part, OK read the profile output and suggest a better approach".

But your experience will be steering it in the right direction, and you're probably unaware of just how much your experience is doing that guiding, as the LLM shoots off at 100mph, you feel like it's taking you with it, but you will be guiding it a lot more than you realise, and that's where learning and experience comes in, even if you're no longer operating at the lowest depth, your knowledge of that layer will be helping.

If nothing else, the experience to know when something is actually slow is a skill in itself. If a function takes 200ms, sometimes that's as quick as it can realistically go, and sometimes that's literally a million times slower than it could be, and there's actual skill and experience wrapped up in knowing what "slow" looks like.

Generally speaking, the pattern is that people are overestimating how much "work replacement" will happen, and underestimating how much "work shifting" will happen.

What is fascinating is how you can witness it at so many levels of organization. One example: Employer executive get enamored with moving from labor to capital. They believe that by using LLMs, they can replace a lot of workers. At my place of employment, we have people that are surprised they can't file a Jira ticket describing a product ask, and have it kick off an implementation. You can build the skill to attempt that, but invariably you'll get back questions like "what do you mean by <x>" and "what do you want to do in this case, a, b, or c?"; questions that a product person or an exec are not well suited to answer.

In the past, programmers did that kind of interpretation and judgment call. So then you're in a quandary; who should do that work? Work that previously, you never imagined was an inherent part of what the replaceable code monkeys do at your beck and call?

And then, how do you hire for that? How do you find the training for the people that are experienced enough with... something... to know what a cohesive error response is, or what kind of telemetry strategy is best for that particular product and organization, what collection of product asks are incredibly complicated for what they're asking and can deliver 95% of the benefits at 5% of the work if we just do this instead, and whether you want to aim more towards thick or thin clients?

Who are those people? Wait, those are programmers? Wait, there's this whole collection of inherently human skills that we devalued, by not appreciating they were always quietly doing that for us in the past?

That's just one example. There's a repeating pattern of discovering where the work truly is, work that was embedded in manual patterns we might not have to involve ourselves with anymore, but is yet still essential. So the nature of our jobs changes massively, but the overall level of employment does not.

At least, not in the medium to long term. There is a lot of painful churn we have to suffer through first.

I can only speak for myself, so I hope this resonates with you.

I wasn’t even really concerned with optimizing low level code before LLMs and that wasn’t why I was hired either.

However following that low level thread: We can look at the reasonable options and immediately know if they’re reasonable or nonsense. Why? We know the code. Now zoom a level out, where I think our expertise really lies.

Building a complex system isn’t easy. There are customers with requirements, there are budgets, SLAs etc. Sometimes one customer needs X and one needs Y. Our expertise is taking all of this in, and producing something that balances all the different variables. It’s knowing that we’ll expect X events a second so we’ll need Y to ensure we can tolerate failure.

Is it possible LLMs will be able to do all of that too? Maybe. But then why would our customers need the enterprises they pay for?

> I still learn new stuff, but I’m afraid it won’t have any value in a year or so.

I have stuff to do now, the value of the knowledge in a year or two isn't important if it solves the issues I have today.

I sometimes get vague ideas for solving maths/science problems. They never pan out but i can talk in detail on group theory and advanced maths and science topics due to investigating such vague ideas over the years. These days the LLM shoots the ideas down instantly and honestly correctly, i know enough to know "yeah that's right, oh well" and move on. Which actually takes away a huge avenue of learning. I'm pretty torn on the outcome of this honestly.

I'm not 'wasting time' but I'm also not really learning.

There's a big gap between being able to ask questions about something and understanding something.

The more things you understand, the higher the chance you'll spot a situation to use them in the future.

I think the best innovations come from times when someone is uniquely able to combine two of their previous experiences together. The more experiences you have in your back pocket the more combinations you have access to and the more likely you'll have a unique combination when the right problem comes along.

Not good at subjects you have no previously learnt knowledge.

They tell you have "hit the nail on the head" when you really haven't.

They tell you have had a "great insight" when you are really haven't.

They give you the illusion of learning and progress but essentially give you faulty preconceptions will trip you up further down the road.

You can ask the LLM to be more critical and less sycophantic but that only gets you so far:

They want you to continue using, being dependent on and feeding data into the LLM--your independence isn't a priority.

There’s still an immense value in training the brain to learn and be able to approach new problems with the sort of procedural thinking that LLMs enable. We can explore topics that we are curious about and develop that sort of “muscle” to continue asking questions when we have them. I have no fear that when those bigger (and existential) problems arise we’ll be well equipped to keep asking questions and figuring out ways to solve them.

I'm choosing to (mostly) switch off the news and continue learning things I find interesting anyway. Maybe the world will punish me for it at some point, but I guess I'll have to deal with that when it happens. The alternative is too depressing otherwise.

One possible value in learning new things is developing a habit of learning.

In particular it might be valuable to be in the habit of learning things that one is bad at doing.

Or not.

So you think building a warp drive is pointless because 99% of work, per your judgement, will be done by AI? Is your contribution meaningless and artifact useless? I wouldn't think so.

It depends strongly on the prompt. Like recently I was also doing some performance optimizations, and if I did not mention profiling none of the LLMs even profiled the code; they merely read the code and assumed, based on their own analysis of big-O time complexity. It is after I explicitly asked for profiling that the LLM started actually profiling.

> I still learn new stuff, but I’m afraid it won’t have any value in a year or so.

This is silly. This would be like arguing that encyclopedias made knowing things pointless. I learn new stuff for me.

Professionally, it's important to know enough to know if you're going in the correct direction. Practically, tokens are going to continue to cost money and knowledge can save you tokens.

I mean, part of the reason the LLM can do that is because you know enough to direct the LLM to do so and verify the results to some degree, right? It's good to learn new things because:

1. It satisfies you curiosity (and curiosity is always valuable)

2. You can better utilize the LLM to expedite something you now have knowledge about

3. You still improve as an engineer/programmer/prompter/whatever

I still think it's very important not to outsource everything to AI because there is a lot of value in learning and doing things yourself which is an important part of life.

I don’t agree with the implication that there has to be a practical reason to learn new things. I enjoy learning new technologies because it’s fun.

The way learning new stuff rewires your brain cannot really be predicted, but the effects are positive.

I decided to learn watchmaking instead

I have staff ranging from 10 years of IR experience to right out of college.

I can tell you that there is an enormous gap in ability between them despite them both using LLMs for daily IR work.

The reasons aren’t complicated. The senior responders have tacit knowledge of how breaches evolve and what to look for which gives them a much better framework for where to employ the LLM.

The juniors will normally start from “here are some logs, look for weird” which is fine but leads to tunnel vision and a lack of confidence in their reporting.

I don’t mandate that anyone do work with or without an LLM. I hire seniors based on experience and juniors based on interest. But my experience has so far been that our best up and comers focusing more on learning the technologies instead of leaving those details to the LLM are developing their intuition and understanding faster and in a more robust manner.