In my experience, LLMs are really good for taking up your time and feeling like you're learning, in the same way that many of the popular educational videos on YouTube are fun to watch and don't really teach you anything.
If you ask an LLM to give you a 500-word summary of quantum physics, it'll give you an oversimplification that probably leans on a hodgepodge of pop-sci metaphors. And if you start drilling down, you risk drilling down on these ELI5 metaphors, which can get you farther away from truth.
Dunno, I've been learning a lot of Rust in the past few days. Just dove right into a project and asked AI to teach me stuff on a need to know basis. I'm actually getting used to Rust by now.
And then I begin to think to myself that I should just read a book on the topic written by a trusted source who put a lot of effort into teaching the topic properly and presenting the information in a thoughtful way. So, I am back to books and mostly try to use LLMs to clarify certain questions or ideas I have.
human conceptual thinking is very much a multi-dimensional graph, which relies on light "approximate" concepts that are "good enough". LLM AR token generation is extremely one dimensional and doesnt care about the "weight" of the concept behind a token.
LLMs hold billions of parameters in "mind" at once. humans hold like four "concepts".
This is the essential mismatch and the primary reason LLM conversation can be so painful and exhausting.
Explaining this and limiting "concepts" to four at a time tops is one of the very few AGENTS.md / system prompts I always use, and it has proven invaluable time and again.
Thinking traces show how effective this is at forcing the LLM to simplify its thinking.
[edit] Also, myself and nearly all of my peers are struggling to choke down the flaws of LLM tooling along with the benefits. the speed at which LLM adoption is being forced, without truly crafting them into quality tools first, is not ok, and not normal.
LLMs have stirred an inhumane hunger and fear. the tech is fine, but the way tech companies (creators and consumers) are behaving should be deeply questioned.
it's NOT normal. it's not ok.
I just want to understand more.
Also, would you be willing to share the actual text of it that you put in AGENTS.md?
The full PDF is worth a read (Figure 1 may be of interest to many here): https://www.cambridge.org/core/services/aop-cambridge-core/c...
If "attention is all you need" then it's something we do indeed lack, in comparison to LLMs! But it's an interesting question: might machine cognition benefit from similar bottlenecks in an attention algorithm? Advancements like Kimi Linear seem to indicate that we're far from the finish line: https://arxiv.org/abs/2510.26692
For example having an LLM summarize a dense topic and to find books so that you can filter faster and spend time reading those books works way better than having the LLM summarize the books or the topic (or even relying on second hand information). Another one is having the LLM quiz you on your topics of interest. With questions tailored to attack specific areas that you struggle with. Its wonderful at this, nothing I've used comes close to what an LLM can do here.
You define for yourself what your goals are, slowly refining them as you learn more, and use LLM as a tool. This ,I find works best for learning.
It's long been the case that the best way to learn something is to teach something.
Which is pretty unfortunate for those that want to learn. I used to enjoy writing documentation at work, it was my favorite part of the job. And it did feel like it benefited me more than it benefited all the people that were (or weren't) reading my documentation. Now I can't really justify spending much time on docmentation when LLM's can do it in a fraction of the time and it's "good enough"
And you notice when it's a topic you know well or something like software where you can immediately tell the options it's giving you don't exist on the page. Leading to the amusing statement "LLMs are bad at what I do but great at everything else".
The question: what's the net positive gain of turning people who know nothing in a given field into sub-novices, while weighing actual experts down with work slop and marginal returns?
And I wonder what the true cost is of arming so many novices with that level of dangerous knowledge.
Tangentially, but related: I'm old enough to remember when the spirit of your comment was pervasive on HN.
I’m old enough that I worked my first IT summer job the same year slashdot was founded. I’ve seen a lot of tech tribalism form and dissipate, and this one didn’t feel organic. My gut says a lot of the us-vs-them tension originated in a deliberate campaign to cast AI boosters as the tech industry in-crowd, and ‘other’ the people not on-board. Who knows.
Just hoping folks don’t get hurt due to people not understanding what they’re doing with these things but believing they’re competent.
The body of literature on learning theory, and beyond that on specific types of learning and specific mediums such as learning from text is so rich there are way more useful models to draw from. Believe it or not, prellm, researchers in the textual learning field had already demonstrated you can achieve performance equal or better than novice tutors using pretty basic computer aids that follow specific hint/pump interaction structures. Guiding an LLM to use these findings has evidence backing it and is way better than telling it "i guess be like socrates". The problem is, to realize there might be richer more effective and highly researched ways of tackling the problem beyond the first fart of a thought you had one afternoon requires the deep respect for expertise and specialization that precisely basically everyone in the AI space right now fundamentally lacks.
Trying to diagrams/animations didn't yield good results even with frontier models. But pure text, any model does a decent job.
So it may be very slow or become unavailable, back end can't handle that, no caching whatsoever.
I'd imagine an application that uses LLMs will be created that better manages learning. It's just not clear what that UX is yet- it's obviously not just a chatbot
i run into context window limits, or practical limitations of digitizing the book
I even wonder if this behavior is due to next-token prediction architectures, somehow.
I know you probably don't consider it dense but wondering if someone can shed insight.
I find them like empty calories, like programming youtube tutorials. They maximize for feeling learnt instead of steady progress
Recently switched to OpenAI and I've gotta say Sol is so much better at writing than Claude. Opus has a distinctive sentence structure and Fable somehow manages to be even more obtuse. The personality of these models really does come through...
Personally, I find that its generated prose tends to have an undue weight to it, almost as if every topic I ask about somehow bears a heavy burden, or is otherwise load-bearing, to use its parlance.
Quite puzzling, really.
I have a personal theory: LLMs are *fundamentally* handicapped at perceiving what's going on in the mind of the human (this can't be "innovated away") and that's at the root of what makes them suck at conversation.
Next time you're chatting with someone, notice how much understanding is shared without anything being said. E.g. the other person might share something deeply disappointing, and they can tell without you even saying anything whether you get what they're going through. This unspoken-yet-communicated information guides the conversation. Or as another example: humans can read the room -- you walk into a room and immediately adjust your demeanor based on what you see and sense.
LLMs are totally blind to things like this, and this adds an inescapable awkwardness to interacting with them. I don't believe they'll ever grow out of this. Which thankfully implies more long term demand for humans instead of robots. :)
Does anyone have a read on if this is primarily a Claude issue, or if all LLMs do this?
I think this is one reason why LLM text is pretty exhausting to read for long stretches.
It's possible that this quality you describe stems from the extensive training corpora utilized by the major AI labs. These almost certainly include work from the esteemed economist Jacob Silj:
> I get exhausted reading LLM prose
So much this! If I see one more sentence with the words "genuinely" juxtaposed with "load bearing" my head is going to explode!
btw, I am building the tutorial here for anybody interested in this topic: https://github.com/avilay/learn-probml
it also researched vision correcting displays for me and i can finally put that idea to bed - i was never really going to pick up an optometry textbook tbh. plus it was able to pull together a bunch of geometric and physical context about light and the eye plugging exactly my personal knowledge gaps.
in general i suspect these materials might not be that interesting to others because they are so custom to my learning style and personal needs and preferences.
these are usually not one shot documents but rather many prompts deep before i get something I’m willing to sit down and read or study. but dramatically quicker than assembling it myself from primary sources. i wouldn’t say it matches master expositors but then they’re not available to write on any topic i happen to need right now.
plus I’ll just have a live voice discussion with the system when i go for a walk and there are still things bothering me on a topic. it takes a little patience but if i’m in the mood it’s amazing.
i generally find that it can help track down specific references if i suspect hallucinations. but especially on factual topics my experience so far has been extremely encouraging.
I view LLMs in education similarly to office hours. Some people abuse it to get homework answers without grappling with the material, but the optimal amount is not zero.
LLM certainly not a replacement for a book, where you get someone’s extended personal approach to a topic, thoughtfully organized, reviewed and edited, often times actual courses taught based on it, with answers checked and errata available online.
Perhaps the best example has been a native macOS app that is a completely custom text editor with built-in debugger, lsp support, fuzzy finder, etc stuff you'd expect. Inside the same app is a library of books i can read within the app completely formatted and for every chapter/section of each book that is a quiz to take (LLM generated of course), a "recitation" tab where i am asked a question and say outloud my response to the AI to evaluate me on and then finally practice problems to do within the custom text editor (these are usually programming books). The reader also has ai re-write built in.
As neat as this is, and i worked through K&R like this, i have ultimately fallen back on "just read the damn book and go to the AI when you've got questions."
But even with Claude, it's it's really the prose getting in the way you can install the caveman plugin or tell it to use that "standard technical English" thing.
Any more detail you can share? Do the others feel more "human"? Are there any that are particularly digestible/human-friendly?
I've been wondering for a while if this is just Claude because I mostly use Claude, so this is very telling.
I tried using a new agent service recently and could tell immediately that it's powered by Claude due to the way it writes.
So I have the LLM offer a very short explanation of something, and from there's it's just me asking questions. Anything that feels fuzzy or not fully internalized is something I poke at until I'm satisfied.
It really has helped me develop a sensitivity to what I understand vs what I don't, and the ability to drill into any part of it is amazing.
And yes, it is not that it is just presenting the facts. By me taking control of the direction the questions and answers go, I can flesh out my mental model. I won't retain every little thing it tells me. But I am much farther ahead than before.
I will say, opus 5 is an egregiously bad case of this, but other LLMs have this too, just less bad.
Agreed.
I find Opus 5, and even Fable, to be overly wordy in eg PR descriptions and code comments.
However, I suspect that's more to do with what they are trained to do by default than LLMs in general. I have a little setup where I tell Claude to work together with Codex to tighten up prose and comments, and for me that produces much more palatable text that needs less human editing afterwards.
That's speculative, isn't it
While my advice is specific to learning about codebases, the way I do it is to have it generate mock data and put it in the local development environment, and give me some exploratory commands, and then ask away. It's a machine after all, so I don't have to read its preceding prose to understand whether it did tell me something, it can just repeat it however many times I ask it, and the hands on commands etc. give me something to actually try and implement.
I'm not sure I'm better off with humans though -- I'm not qualified to judge whether a source is a proper authority, not an I qualified to judge whether someone knows enough to point me to a reliable source.
It seems this is a fundamental epistemological problem to which there may never be an answer.
This is my biggest gripe with reading AI-generated text as well (ignoring the meta issue of whether it's worth taking the time to read something that an author didn't think was worth the time to write). It's gotten to the point that weird AI-style analogies just take me completely out of the text and kill my interest.
And I can usually tolerate a lot of purple prose.
It's a loop that uses adversarial review to check several dimensions of the writing:
https://github.com/Vibecodelicious/llm-conductor/blob/main/w...
I’ve found the tone of Kimi K3 to be less obnoxious. Unfortunately it doesn’t wholly solve the issue, I don’t think any LLMs out there have a truly pleasant writing style, but at least not every assumption is “load bearing”.
Like I said, I'm essentially continually prompting to refine the material. LLMs certainly continue to append, and never cut back. It just keeps spitting out additional content at me. So that's a bit annoying too. But I can basically get figure out what's going on with a few extra promps.
If youre curious what i've got so far... just be warned it is quite literally AI slop plus me continually prompting for clarification/cleanup etc. : https://github.com/cmoscardi/ai-for-ai
It generates tutorials for you, and serves a webpage that lets you complete them. It does a remarkable job.
It still has a bit of the LLM prose problem, but it does help you fine tune the ‘voice’ it uses.
I've stopped using CC because of it. I find it insufferable.
This problem doesn't get talked about enough and is second only to the hallucination problem IMO.
AI produces so much noise to wade through in order to find signal, and the more expertise you have in a field the more that costs. That noise directly subtracts signifcantly from productivity gains.
And, I think the problem is directly related to the hallucination problem. It feels very much like an effort to kitchen sink the response in order to provide some value among possible hallucinations.
It also seems to be a byproduct of Gen AI operation. It just fundamentally doesn't understand what it's outputting, so doesn't know how to narrow down to the most salient bits.
It's just long. It just doesn't shut up. It's overly verbose. And you can't tell it to be concise or you degrade its quality.
If I ask what an integral is, the correct answer is that it is the continuos analog of a sum, generally used to calculate areas and volumes.
It should really be a single sentence, and then let me ask more about the terms I don't understand, and here's the beauty, in the previous one there can be only 5 terms I cannot know.
An LLM will vomit an entire page or more of explanation which isn't bad per se, but is an answer to something different: "give me a short introductory explanation to integrals". And that's not what I asked.
Try it out, fairly sure that if you out in 100 random words for 30 of them it will just refuse to translate them (it will copy paste the original word into the target language) or it will do silly things like use the target 4th dictionary definition instead of the primary one).
I'm actually going to make a prediction here as well. I think you will soon realize that using LLMs to clarify certain questions or ideas you have will turn out to have frustrations as well. And that you will soon direct those questions to either peers you know in real life or internet forums which are very likely to have a non-AI policy.
Much more likely that people will believe themselves to be an expert in a subject after having had a conversation with Claude about it.
Often "be concise, to the point." is enough, but you can also paste it some stuff you like as an example text and ask to do style transfer.
Literally saying "one sentence response" solves most of this problem.
I'm not sure I follow how this is actually guaranteed? The fact-checking process mentioned just seems to involve asking AI to review its own work.
And even if they are related - if Opus 4.8 always has a 1:100 chance of a specific hallucination - then running the same model twice does indeed dramatically reduce the odds of an error in the final output.
Personally I think this is a bad characterization of using LLMs to fix up LLMs because while you can never guarantee results this way (as the quoted line claims here, which is worthy of criticism), it is, in practice, useful to use LLMs on top of LLMs. And there's no infinite regress. Auto-mode in Claude Code, for example, seems to me like it's been successful at making the system more safe than --dangerously-bypass-permissions without prompting the user for permissions constantly.
What triggered my response was the “just review the output with another LLM and it’s perfectly correct”
Full story in the book
People don’t even have to be lying to be wrong about this stuff. Someone can learn enough about a topic to be halfway up Mt. Stupid in no time flat, and in doing so, think they not only truly understand the topic at hand, but might be particularly adept because they were such quick studies. People that know less are impressed, because why wouldn’t they be? Anybody that knows more than them sounds like an expert. And people that know what they’re talking about cringe at the overconfidence, and probably try not to engage: who wants to have to prove that someone’s boundless confidence is entirely baseless? Most of the time, they think the actual expert is full of shit because they think they’re the expert. It’s incredible how many times I’ve had people in tech confidently, even smugly “explain” design concepts and strategies to me that they did not actually understand, knowing I was an experienced, degree-holding designer… and they didn’t even have a chatbot’s lips on their ass telling them how smart and insightful they were.
Opus 5 first built me a detailed plan, but a couple important details were either obviously wrong or felt unnecessary. I went back and forth asking for sources and more information probably like 4 times and every time it did the "in looking at things in more detail it appears my previous advice was incorrect" spiel. It just became exhausting at some point because it feels like it really lays bare how LLMs are just minimizing that loss function but don't actually "understand" anything. It was really useful as a search engine (it correlated some highly relevant source docs), but I just couldn't trust it to believe it was actually done at any step.
I know it sounds silly but 1 layer ends uo being way worse than 2.
i can't even get agents to remember core instructions like "use jq instead of writing a python script to parse some json"..
I'm researching causal inference right now, and my main goal was to make sure I understand how to test estimation on synthetic data.
Basically, it's the same way it works with people. If you delegate a task that you don't understand, and you can't have a credibility proof (i.e. doctors, lawyers), then you research a topic well enough to be able to (1) define the task and (2) verify the end result.
In particular:
- I limit it/encourage it to give me single sentence questions
- I sometimes will ask it to tell me a motivating, human-grounded story, when we're starting a new concept: claude responds "Maya is a bond portfolio manager, and her boss has asked her to quickly price in what happened if yields go down. She knows her bond's average duration, a measure in time, but she doesn't have a percentage, which is what her manager wants. How can she give him a percentage number with just a duration figure and the proposed new yield?"
- I'll often ask claude to let me work through it, to derive the thing myself, often resulting in a string of thoughts with "yes/no" trailers, to get the LLM to reply yes or no only, and avoid derailing my train of thought. If yes, my train of thought keeps going. If no, I've got something wrong.
- I'll sometimes stop and have it craft an artifact. I typically say "build me a Brilliant.org-style interactive demo of the topic", especially when we get into the realm of looking at the actual maths of a thing (for which prose and dialog is not optimal by itself AFAICT)
- I'll do this while I'm traveling, while I'm walking, while I'm doing chores.
It's so much fun.
""" In this project, I require a socratically delivered line of conversation. Here's the typical structure to the conversation. I ask some question. You need to factor and reason about how to conduct and deliver a conversation. Best practices would be to limit terminology, or assess with the user whether they have a firm grasp on terminology before you use it. You must be very strict about this, it's unacceptable to just introduce a new concept, actor, phrase or other complication into the conversation without first labeling who what or why it exists for the conversation.
Conversation structure needs to be front-loaded with a brief interview for the user, "you understand X?", "whats your understanding of Y?".
Conversation structure then needs to proceed with single-sentence questions from the agent. User replies with an answer. Sometimes the agent needs to correct the user, but only ever do so with yet another question. """
^^ these are the instructions I have installed at the root of a "project".
Keep in mind, this is claude opus 5 low effort we're talking about, in the "projects" area of the mobile app. Here's the process I use to set up its knowledge:
1. I take screenshots of the textbook on my iPhone, and upload a chapter at a time.
2. I have it summarize the chapter into markdown by analyzing screenshots. You could probably achieve this simpler, if you just had the textbook in PDF.
3. I walk with my boy Clau-crates.
I've done this for a couple of weeks and haven't seen it revert back into its typical context-dumping behavior.
On second read, there's probably some clean up I could do. Thanks for making me pull it out and look at it. Things that could probably be improved:
- tell it to cross-check resources online to further ground itself
- use simple, short sentences (long sentences make the brain blur a bit)
- not be sycophantic (it seems like project mode has discarded with my root-level anti-sycophancy prompt)
I still learn new stuff, but I’m afraid it won’t have any value in a year or so.
For example, I’m pretty good at optimizing low level stuff, but right now you can just ask LLMs to do so and they are pretty good at it. They will profile the code and suggest reasonable options like 90% of the time.
Trust me when I say that in the hands of someone who doesn't have your experience, the LLMs would not be getting the results you get.
You might think what you're doing is trivial, it may be sessions that flow roughly, "Instrument this, okay this part is slow, profile this part, OK read the profile output and suggest a better approach".
But your experience will be steering it in the right direction, and you're probably unaware of just how much your experience is doing that guiding, as the LLM shoots off at 100mph, you feel like it's taking you with it, but you will be guiding it a lot more than you realise, and that's where learning and experience comes in, even if you're no longer operating at the lowest depth, your knowledge of that layer will be helping.
If nothing else, the experience to know when something is actually slow is a skill in itself. If a function takes 200ms, sometimes that's as quick as it can realistically go, and sometimes that's literally a million times slower than it could be, and there's actual skill and experience wrapped up in knowing what "slow" looks like.
And the craft is loose term, it can mean anything you like to get better at.
“asking the right questions” is also a moving target with each model release
People simply underestimate the value of doing the work and think that the end result is all that matters
https://en.wiktionary.org/wiki/eat_one%27s_seed_corn#English
What is fascinating is how you can witness it at so many levels of organization. One example: Employer executive get enamored with moving from labor to capital. They believe that by using LLMs, they can replace a lot of workers. At my place of employment, we have people that are surprised they can't file a Jira ticket describing a product ask, and have it kick off an implementation. You can build the skill to attempt that, but invariably you'll get back questions like "what do you mean by <x>" and "what do you want to do in this case, a, b, or c?"; questions that a product person or an exec are not well suited to answer.
In the past, programmers did that kind of interpretation and judgment call. So then you're in a quandary; who should do that work? Work that previously, you never imagined was an inherent part of what the replaceable code monkeys do at your beck and call?
And then, how do you hire for that? How do you find the training for the people that are experienced enough with... something... to know what a cohesive error response is, or what kind of telemetry strategy is best for that particular product and organization, what collection of product asks are incredibly complicated for what they're asking and can deliver 95% of the benefits at 5% of the work if we just do this instead, and whether you want to aim more towards thick or thin clients?
Who are those people? Wait, those are programmers? Wait, there's this whole collection of inherently human skills that we devalued, by not appreciating they were always quietly doing that for us in the past?
That's just one example. There's a repeating pattern of discovering where the work truly is, work that was embedded in manual patterns we might not have to involve ourselves with anymore, but is yet still essential. So the nature of our jobs changes massively, but the overall level of employment does not.
At least, not in the medium to long term. There is a lot of painful churn we have to suffer through first.
I wasn’t even really concerned with optimizing low level code before LLMs and that wasn’t why I was hired either.
However following that low level thread: We can look at the reasonable options and immediately know if they’re reasonable or nonsense. Why? We know the code. Now zoom a level out, where I think our expertise really lies.
Building a complex system isn’t easy. There are customers with requirements, there are budgets, SLAs etc. Sometimes one customer needs X and one needs Y. Our expertise is taking all of this in, and producing something that balances all the different variables. It’s knowing that we’ll expect X events a second so we’ll need Y to ensure we can tolerate failure.
Is it possible LLMs will be able to do all of that too? Maybe. But then why would our customers need the enterprises they pay for?
They tell you have "hit the nail on the head" when you really haven't.
They tell you have had a "great insight" when you are really haven't.
They give you the illusion of learning and progress but essentially give you faulty preconceptions will trip you up further down the road.
You can ask the LLM to be more critical and less sycophantic but that only gets you so far:
They want you to continue using, being dependent on and feeding data into the LLM--your independence isn't a priority.
I have stuff to do now, the value of the knowledge in a year or two isn't important if it solves the issues I have today.
I'm not 'wasting time' but I'm also not really learning.
The more things you understand, the higher the chance you'll spot a situation to use them in the future.
I think the best innovations come from times when someone is uniquely able to combine two of their previous experiences together. The more experiences you have in your back pocket the more combinations you have access to and the more likely you'll have a unique combination when the right problem comes along.
In particular it might be valuable to be in the habit of learning things that one is bad at doing.
Or not.
This is silly. This would be like arguing that encyclopedias made knowing things pointless. I learn new stuff for me.
Professionally, it's important to know enough to know if you're going in the correct direction. Practically, tokens are going to continue to cost money and knowledge can save you tokens.
1. It satisfies you curiosity (and curiosity is always valuable)
2. You can better utilize the LLM to expedite something you now have knowledge about
3. You still improve as an engineer/programmer/prompter/whatever
I still think it's very important not to outsource everything to AI because there is a lot of value in learning and doing things yourself which is an important part of life.
I can tell you that there is an enormous gap in ability between them despite them both using LLMs for daily IR work.
The reasons aren’t complicated. The senior responders have tacit knowledge of how breaches evolve and what to look for which gives them a much better framework for where to employ the LLM.
The juniors will normally start from “here are some logs, look for weird” which is fine but leads to tunnel vision and a lack of confidence in their reporting.
I don’t mandate that anyone do work with or without an LLM. I hire seniors based on experience and juniors based on interest. But my experience has so far been that our best up and comers focusing more on learning the technologies instead of leaving those details to the LLM are developing their intuition and understanding faster and in a more robust manner.
One thing I wonder is, do you mentally 'fight back' monotonicity of your interactive tool? All seem to be in 3D space, with low-poly, like in a factory moving through the belt and giving you an information + textual description to read more
But sometimes you want to visualize the charts, or graph of simulations, or maybe even the parts of an item in the rocket.
Makes sense.
> I ask it to review the accuracy of the knowledge base it built in the previous step.
Ooookay that sounds good.
> I proceed asking it to build a simulation of that topic in a low-poly, Rollercoaster Tycoon-like animation.
wat.
The main idea is you can do any style you want or like to learn.
In my experience it's infinitely easier and faster to learn deep, "boring" things when you understand how they relate to your shallow and wide understanding of all of the related components.
The LLM is merely a tool. And you can use it for domain discovery that enables efficient deep learning at an unprecedented rate or you can develop a cursory understanding of a topic and think yourself an expert.
It’s true of most things. Running, dieting, weightlifting being uncomfortable is a sign of progress.
This is the only path to mastery, or understanding if one prefers. There are no shortcuts to a person achieving deep understanding (a.k.a. "Aha!" moments).
Can a tool such as GenAI be beneficial to someone who already has done the work to understand? Absolutely. But it cannot infuse mastery into a person simply by its use.
Only the time and effort a person devotes can do that.
Another useful approach has been asking Codex to implement complex things, like a Kademlia DHT or BitTorrent client in a literate style with the explicit purpose to increase understanding by reviewing the source code.
Examples: https://rickcarlino.com/notes/note-dump-and-ai-summaries/ind...
That's actually a fun way to learn processes!
Totally agree, unfortunately careful simulation games are very rare
Otherwise you probably get more confused as you have mentioned.
On the other side, Peter Diamandis describes a situation where a bunch of kids were given a internet-connected computer and they had no teacher. Instead of it there was a “grandma” that checked kids from time to time.
After that there was a knowledge test that revealed “no teacher” approach was more efficient.
But it was a group, not an individual activity…
For example, “explain how the code in this file works,” I am familiar with the overall codebase, I know the purpose of the file, and I can read it or write tests to verify if I suspect what it’s telling me isn’t correct. Or, if it’s really important, I can overcome my introvertedness and ask the team member who wrote it…but that’s a last resort nowadays, which I am very thankful for. In 99% of cases since at least Claude 4.2 days, Claude and Codex have been very accurate. Gemini on the other hand messes up more frequently and sometimes does weird things like try to delete files it’s not familiar with, at least the 3.6 flash model I’ve been using lately does this. But, code explanations are still good for the most part.
You can ask the LLM how to do this. Start with a topic you know well to get the mechanism working and trust it well.
I assume this will become less of an issue in the future as there is more trust between the AI tools and me.
How does he know?
> Time and time again, when talking to people who rely on ChatGPT, Claude, Perplexity, and other general AI tools, I hear them say, “AI is incredible. It handles nearly everything I throw at them.”
> “What does it fumble with?” I’ll ask.
> “Well, it still gets things wrong when it comes to my line of work.”
https://www.dbreunig.com/2025/04/08/on-ai-observational-comi...
To be clear, I say "inaccurate" rather than "wrong" in this case because even if the information it returns is factually correct to the question being asked, students don't have an understanding of the complexity of the interdependent tectonic, regulatory, and spatial / experiential factors of a building sophisticated enough to ask their questions of the specificity and nuance necessary to get a good output that addresses the entire problem.
Anyway - with the students still learning to ask questions the right way, and the conditionally-incorrect facts making their learning more complicated rather than less, I hit on a strategy for them to use LLM's that seemed to help much better.
I suggested that instead of ask the LLM for the factual answer, or even better for the facts and an explanation, that they ask it to direct them to the proper place in the source material to find the answer themselves. Then, to treat it like a lab partner. IE:
Hey Claude I'm looking for "x."
Claude: "look at foo, bar."
Thank you - chapter (foo) part (bar) table (goo) says "car." However I notice that footnote (hoo) says there's an exception if "dar." Which is what I have. Walk me through this exception...
It seemed to have good results as a guide to understanding the disparate bodies of knowledge that they will eventually have to keep together in their heads and work synthetically and non-linearly through, rather than just as an external source of blindly trusted authority.
Hallucinations bring to question what you think you've learned. That's going to cost long-term if you labor under mis-apprehensions until you maybe figure out you learned something wrong.
Yes, sometimes its wrong, most times its right, cross checking is fairly easy, not using it because of the possibility its wrong seems a baby with the bath water thing.
I learnt a lot of functional programming from it, stuff I've always wanted to learn, but just didn't have the time and really the sources can be difficult, it really explained things well, and as someone else said in this thread, you can ask questions over and over until you understand, asking a person that (if you can get an expert) would drive them nuts. Maybe my experience isn't typical, its hard to tell, everyone reports something different.
Colleagues often suggest podcasts and videos - I very, very rarely listen to them or see them.
The bandwidth is too low. It's not efficient and ultimately I'm bored.
This is a nice project, it looks cute. I watched some of the pages But I want more than that, more information, and faster - still a Wiki fan.
Also, step number 2 in the flow: have the LLM check itself... Naah, I don't believe that.
But you're not the only using gen ai like that. Take care.
Indeed it's often a waste of time to just focus on talking people fully if you want to learn fast, reading and especially deliberate practice are better for that. But if you don't have the time, energy or focus, then listening to interviews in the background can be useful supplementally
having things defined/have correct solution to compare for review is useful, and keeps llm on track. Don't think i would trust llm if it were reviewing it all on its own
OTOH, I am curious if there's a "practical value" to this exercise? If the LLM already contains the information and implementation knowledge to implement the networking stack inside an FPGA by itself, what value do I gain by learning about HDL, TCP, the bespoke Xillinx tooling, reading the documentation, reading papers on the implementation and going through every bit of details and theory. I feel like there's a meta skill that is more worthwhile for "practical value".
The skill then riffs with me, judging my ideas and suggesting alternatives. We go back and forth until something useful comes out of it. This process isn’t unlike how I do normal development.
However, once agreed it breaks the work into “steps”. It then creates a tutorial for me, for those steps, explaining each line, why each change happens etc. I can then ask questions, muse about an alternative idea etc. Then I do the steps, and I’ve learned and gotten what I wanted to get done.
This has been how I’ve been learning Godot and making a game for the past month or so. I didn’t go in blind, I started with a course from GDQuest so I could feel confident guiding the tutorials. I will say though, having a tutor to bounce ideas off of has been really useful.
I still try to figure it out myself, consult the docs, discord etc. But if I’m stumped I’ll run my tutor skill and have some fun.
I have done this for all my work this week and it works quite well.
For one it lets you actually query the LLM as to why, their plans give a high level not every single change and it allows you to correct it as you go and the plan will change.
As for how useful it is to understand thins, I believe it's still useful and hope it will continue to be.
Does anyone else use Claude like this?
It's sped up my learning by 10x. I struggled with 'just reading a book.' Take kubernetes. I hemmed and hawed and spent years periodically reading some dry book or blog or official doc, falling asleep, and forgetting while I got busy. Now I'm aggressively working with it, almost like I'm addicted to a gamification, of getting through our learning timeline, and I'm excited to move forward as quickly as possible and pass its tests.
It's like a fake teacher, because I can also ask it to drill into a topic or re-explain itself if it made no sense.
The only thing that worries me is, sometimes I'll say something like, "Um, are you sure about that?", and it'll apologize and correct itself. I barely challenged it!
Cat also has an awesome podcast with her wife, Ashley Juavinett, Phd, called Change, Technically.[4]
I encourage everyone to check out her work! She’s dedicated her life to helping software developers get the support they need inside organizations to be seen as humans, not just robots.
1: https://www.drcathicks.com#book 2: https://github.com/DrCatHicks/learning-opportunities 3: https://github.com/DrCatHicks/learning-goal 4: https://www.changetechnically.fyi
I'm using LLMs right now to build a terminal browser, a GUI browser, and a PyTorch/LibTorch replacement. It's really fun to be able to learn and make progress this way. It's like reading multiple interactive books, where every concept can be explained again and again until I understand it.
I had a similar realization a few months back and am working on a tool that generates "mermaid walkthroughs". It is 1000% less pretty but it is fast and is pretty good at explaining how services work or what a code review does or just as a way for your agent to explain some decision to you.
LLMs and systems intersection - https://kernelspace.naigap.com
Distributed systems - https://byzantine.play.naigap.com
What is your process for creating these resources?
I don't think we are even that far when the complexity AI can handle surpasses 99.999% of what humans can handle, where AI make e.g. physics discoveries beyond the grasp of most humans and it will have to "dumb it down" when talking with humans -even physicists- but not with other AIs
I find LLM's do great for learning when I ask what are the principles, how the main applications work, what are the key drawbacks, where are the growth plates in the field, etc. - the kind of thing a good advisor points to. Sometimes I have to ask it explicitly to use topological order of topics and show relations, which often highlights the gradient changes in the learning curve. For pruning, it's surprisingly good applying philosophical heuristics - Occam's razor, or Derrida's differance (the difference that makes a difference), etc.
And finally, no learning is effective without problem sets, and for those LLM's at times get me over blocking issues.
The degenerate case is memorizing the glib phrases regurgitated back to me; they're helpful and functional enough to get me into real trouble!
> What you get is a beautiful animation that is 100% accurate and free of hallucinations.
How do you make that leap?
I think that would be a way more natural way to explore than being stuck on the classic linear output of a LLM.
But
> What you get is a beautiful animation that is 100% accurate and free of hallucinations
100% free of hallucinations when you're not an expert that can check it is impossible. LLM hallucinations are an unsolved problem.
It's not perfect, but I'm optimistic this will be a useful way to teach/learn in the future.
And, to be clear, I think this will be best utilized within a group/community setting. I don't think it will replace teachers or classrooms.
Also worth mentioning that Matt Pocock has a /teach skill that creates interactive, learning sites for learning a new skill.
> [I ask it to build an interactive thing]
> I then push it to a new repo and enable GitHub Pages for it.
Congratulations. You are an echo chamber for LLMs. Use it to create, and verify, and post to then be scraped and trained on again.
I've been working on a similar process of pushing to github pages, but focused more on having "practice sessions" with coding blocks to test content. Using webassembly and mock servers to mock backend endpoints Here's one I built to build a full stack llm chat system in the browser.
https://model-systems-labs.github.io/latent/llm-systems/less...
How do you know if you're learning this for the first time? Very risky to learn from LLMs. I've done it, but you have to keep your wits about you. Lots of "oh of course you're right - what I just told you was completely wrong".
Last month, I read The Prince and had it make a text adventure campaign for me.
For a lot of other topics, I often just ask it to create a simple python example that I can run.
It reduces friction a ton, but at the end of the day I’m not skipping anything.
This is my biggest societal issue with LLMs: they allow you to think that you’ve “learned” a topic because you read a lot of technical terms. I don’t think it bodes well for the future.
0: https://en.wikipedia.org/wiki/Semiconductor_device_fabricati...
Inthink this was always the best way to learn. But it used to require immense work for a teacher.
My compiler course is a great example - program in plug in a stage of a compiler.
These exercises can be made on demand and incredibly easy now.
God does everything have to be productized and glorified as if you’ve invented a new way of learning. Read some books!
Surprisingly effective.
(Yes I confirmed it was ok to use AI with the material)
You're just memorizing a nonsensical recipe. What are the constraints? Why do we do X rather than Y? How does a particular thing scale? etc.
All you're doing is fooling yourself into thinking that you've acquired some knowledge. When in reality you haven't even learned the basic mental model to reason about this stuff.
You've learned something when you have a mental model that makes correct predictions. Until then you've memorized it at best, and as with most memorized things it will decay exponentially and will be gone from your memory soon enough.
I also struggle with LLMs explaining things, but for the opposite reason.
I consistently have problems to get short, precise but plain/simple answers.
Instead I'm overwhelmed with walls of texts, often filled with jargon that is a mixture of imprecise and unneeded.
The style at which I learn better is by asking about stuff interactively. I ask you what something is, you give me a 3-4 sentences top answer. Then I explore and dig into the topic from your answer on the things I want to know better.
Then I went back to a book written by humans (Eric Nikitin's "Realm of Oberon").
Do you know it is free of hallucinations because you crossed checked it with the source material or because you told the LLM "don't hallucinate"
Perhaps basic special relativity could be done this way, or simple derivatives, but definitely not general relativity or integrals, much less PDEs. I guess I was hoping for a 3B1B type output. Oh well.