Incidentally, the skills for the lowest levels of LLMs aren't that far removed from those needed for mobile telephony, in that both are based on maths, computation and information theory.
But it was generally seen as a gimmick instead of desired before Apple made it look good. Even when the iPhone came out, one of the jokes was how the grid of icons looks like how a Windows user's desktop would look like when they didn't understand the filesystem.
What most people want from mobile technology is for it to work, not too expensively, and for it to get out of their way.
What most people want out of AI is for no leader to emerge and wield supremacy against the rest of us. People are afraid of it in ways they weren't afraid of mobile, so they're more willing to work together against whoever is in the lead.
Its more like an arms race and less like a utility. The disadvantage I face when my competition has better mobile coverage and bandwidth is minor. The disadvantage I face when my competition has better intelligence on tap is much more significant.
We don't really have the demand for as many telecom companies as actually exist in the world. There's a reason we just have one Whatsapp and one Instagram, not three or four almost-but-not-quite clones in every single country that mostly differ in branding. The reason for the current situation has mostly to do with regulation and traditional, enterprise, "obviously every country needs a separate local branch, because that's what mcDonalds does" thinking. Technology has very little to do with it.
This is why the telecom world now consist of equipment manufacturers, who do most of the hard tech stuff, and actual telecom companies, who operate the equipment, rig towers in their local country, and maybe write some glue code to integrate a core from vendor A, a billing system from vendor B and a CRM / corporate invoicing system from government-approved local vendor C.
Banking also works similarly, though modern Neobanks / Fintechs and bank consolidation are slowly dissolving the concept of national bank branches.
It is viable as a toy project, but there are vanishingly few career opportunities.
Building a browser in the early web was actually a very achievable goal for exactly the reasons it isn’t now. There was not JS. No CSS. No SVG. In fact very few widely supported image formats (and graphical browsers weren’t around in the earliest days of the web anyway). TLS didn’t exist. HTML only had a subset of methods. And even POST was usually just managed by CGI/BIN calling an external process, often written in C++ or Perl.
It was a simpler time.
That’s like saying someone who fabricates cars doesn’t have the skills to drive them. Perhaps not, but they’re very well placed to pick it up quickly. They’ve also shown they can do something far more challenging, which is actually better than hiring for narrow immediate skills.
In other words: I’d hire that candidate in a heartbeat.
If anyone reading this has this un-marketable skill, Carnegie robotics in Pittsburgh is hiring
carnegie-robotics.breezy.hr/p/2d85f5321cc7-software-engineer
I recently switched roles, and among the seven places I interviewed, none of them seemed to see my then-current browser job as a problem, even though they were not related to browsers. (The closest one was a company implementing a HTTP reverse proxy, and I did not work on the browser's HTTP stack.)
People keep talking about a browser in the modern context but the GP specifically said “early days of the internet” (which, in fairness, would mean pre-web. But I think it’s safe to assume they meant “web” not Internet).
In those days, it was actually a much simpler exercise to write a browser than it is today. I even wrote one! And writing a browser absolutely teaches you how HTTP and HTML worked. Plus a lot of backed development was forms data sent to CGI and thus written in languages we wouldn’t even dream of using for web development nowadays, including C++.
So in the early web, writing a browser absolutely was a transferable skill. It might not be now, but in the context defined by the GP, it was.
[Blake Ross] worked as an intern at Netscape at the age of 16 ... Ross became disenchanted with the browser he was working on and the direction given to it by America Online, which had recently purchased Netscape. Ross and Hyatt envisioned a smaller, easy-to-use browser that could have mass appeal, and Firefox was born from that idea ... in 2003 all of Mozilla's resources were devoted to the Firefox and Thunderbird projects. Released in November 2004, when Ross was 19, Firefox quickly grabbed market share ... with 100 million downloads in less than a year
Oh hey thank you for that. It really helps. Hope you have your rug pulled from under you today too.
— signed, a career changer trying his best.
Not a lot of demand, but also probably not a lot of supply.
FWIW I built browsers from 1999 for a long time. (But I was never a wunderkind, just somewhat tenaciously curious).
And I guess I am doing fine, but not amazingly rich or so.
Browsers were always a project closer to research/charity. I think Marc Andreessen said something similar - that he would never do that again. B2B is where you can make money.
You don't need your impressive product to succeed to land a good career.
If Alice is doing LLM-from-scratch work today, and Bob is doing agent harness work today, Bob's project is far more likely than Alice's to become useful/popular/profitable.
But if neither project survives, in 5 years, Alice will be more employable/at a higher market rate than Bob.
In general C++ work and similar, if not in Chrome development.
> I’d imagine those teenagers ended up with pretty good careers in technology?
The guy you mentioned doesn't seem to have ended up with a bad career in technology?
Is this the guy you’re talking about?
17 is an interesting age. There are way too many comments here saying things like, 17 year olds should just do whatever seems interesting or bum around the world or focus on getting into university. But historically most kids were expected to be productive adults at 16 or 18. 17 is about the right time to be thinking seriously about what kind of work you'll do, how you'll make a living. University won't help and will just delay this decision.
I contributed to a browser engine around that age (KHTML, which later became WebKit and Blink), and while I don't work in browsers right now, much of that knowledge, mindset and of course the professional network have done much to shape my life. And a fairly successful career, for that matter.
If teenagers could make small contributions to LLMs via open source then sure, go for it. Optimizing llama.cpp or similar would be a good learning project that might later get you good work via social networks. Contributing to open source is how I got started too.
Unfortunately, training LLMs isn't something that fits well to open source open collaboration. Inferencing codebases are better.
In that sense it's more a "seek out the open source community and real projects when young" rather than "do web browsers", with a bit of "look for ambitious types of projects few get to work on".
A lot of people will make better decisions with a few yeas more maturity, and spending a few years developing themselves.
University will help a lot of people, and for some it will help.
There is a lot more to life than making a living.
The early Internet (which the GP mentioned) didn’t even have the web. But that’s nitpicking.
Paradoxically I coded way more between ages 12-14, I regret my wasted late teens.
ages 8-15: lots of great 8-bit computer fun.
ages 15-20: girls, booze, motorcycles.
20 onwards: get a PC, back to computers, realize how much I've been missing.
To be honest, judging by my own kids and their friends, late teens seem to generally be an era of hard to avoid stupidity.
Which is paulg's point really
Jeff Bezos did similar, but he did his own fulfillment and hired out the coding.
Good luck with that approach when trying to toy around with models and their training/inference.
There is also a lot of math basics missing that a 12 year old may be able to grasp, but I would bet they are at least 13 by the time the knowledge is deep enough to understand what operations are happening.
The same is not so different for AI. A few people design novel AI, but there are a lot of people training AI (especially if you include fine tuning) and implementing AI, even as a hobby.
It seems to me like he’s saying that doing this thing would be 1) fun and 2) a great way to become employable in the future. I don’t believe he’s saying that this project would be some kind of job training exercise.
I "guess" you could put all your effort into moving up to the next level in computer science, or you could put the same progressive effort into reaching new levels in video games on the same hardware.
Alternatively you could even put all your effort into social activities and leave the technology to other people entirely.
It might even be possible to find a balance between things that are widely "understood" "socially", and those that are not ;)
But that was way back in the early 1970's and all I had to work with was a mainframe.
Well the mainframe itself wasn't bad, the real show-stopper was that I didn't own the computer outright, no strings attached, no debt, etc.
>I'd probably try to make an LLM that I could use on some specific problem.
I thought so too back then, still do so I guess this is one of those things that could stand the test of time. I always wanted to start with something a lot simpler than a Moon mission myself. At 17 I already had a significant breakthrough in the chem labs and it was from alternatives to a single processing step plus everything that descended from that, rather than trying to tackle a much more complex detailed multi-step synthesis. I was only 17 but I was not trying to be a slouch, I don't think pg was either at that age but his advice is not for just anybody. I couldn't have done it if I hadn't made major progress since being 16, and it really emphasized at the time how much maturity can make a difference. My imagination ran wild as I extrapolated :)
In a reply from LeCun to pg:
>>I'll figure out a set of methods and architectures beyond LLMs that can quickly learn to perform physical tasks as efficiently as humans and animals. That last item is also what I would if I were 30, 40, 50, or 66 years old
I see no reason to stop at 66 either ;)
But I figured that people owning more computer power than I could ever afford were going to be doing something like this as soon as they could, without having to wait for something like an LLM to arrive before getting peoples' attention.
It did seem like things were going to take longer than you expect, so it's pretty good to have a lifetime of concentrating on the specialized natural science domain expertise, focused now for 50 full years on how it would combine if AI ever got good enough.
Both the natural science and the AI need to be a major cut above, I still see dramatic room for improvement in my own work. If I'm going to have to rely on "other peoples' AI" then that natural science component is going to have to pull a lot of weight to keep up with the kind of computers that only rich-as-hell high-rollers have access to.
For anyone who wants to dork around there is https://github.com/rasbt/LLMs-from-scratch which is something amazing that I think anyone who wants to engineer things around LLMs should at least blast through and read.
When I was a pre-teen I stumbled upon CD-rom hacking guide to bypass disc requirements on games, I remember opening up the file and the screen being filled with HEX code. I was so overwhelmed I just closed it and never touched programming after that for 15 years. My life would have been totally different if I had embraced the unknown instead of retreating.
And the job postings are often ridiculous. I recently was an AMD job advert in Germany for an ML Kernel Engineer, not Senior mind you. The requirements went something like
> Masters Degree required with strong preference for a PhD with peer reviewed articles in {journals_list} > 10+ years of experience in C/C++ > GPU programming experience required > 10 more ridiculous lines
No idea how a teenager self teaching himself LLMs is supposed to even get a shot...
1. I can make turn a stone and water into a delicious soup
"17 year olds, learn to build an LLM from scratch"
2. This soup would be more delicious if we add a few carrots. Does anyone have carrots
"Increase your chance of success by getting a Masters degree"
3. How about potatoes?
"And get a PHD"
4. What about some salt?
"And publish some peer reviewed articles in {journals_list}
5. We should also add beef
"Now work in the industry for 10 years"
6. See, this soup is delicious, and I made it all with a stone
"See, you're rich, and it's all because you learned LLMs as a 17 year old"
There are high school students competing in contests that cover parts of the (Math) theory behind AI. A lot of high school research programs are integrating AI with other things and complex mathematical models…
To me, this is bizarre as Calculus is barely taught in high schools (and likely poorly).
Don’t get me wrong, these kids certainly aren’t the usual lot.
Yet, I really wonder if they know the fundamentals. Do they even understand derivatives or just memorized the rule for polynomials? Can they even explain what a transistor is?
Normal curriculum takes 5 years to go from Algebra I to Calculus. Real Analysis, Linear Systems, etc. are fundamentals taught only in college…
Feels like too many are trying sprint before even learning to walk.
Learning should not be done only as a direct path to getting paid.
Learn to create pattern matching and intuition to solve future problems.
When you are 17 it is a good time to understand how the world works so you can build on top of it in the future. If we assume most tech is going to have an LLM as part the stack, a solid basis in how LLMs work is likely to help you in future endeavors the same way a solid basis in how the web works helps you today.
Maybe a 17 year old should learn both. As a small anecdote when I was 17 I learned a lot about load balancers, failover, and building self-healing systems running small hosting company that had to be fault tolerant when I was attending high school. This wasn't at state of the art levels (e.g. I wasn't configuring gigabit routers or global CDNs -- but it was useful pattern matching for future problems)
I currently don't touch any of that tech, but I have working knowledge that still serves me today.
Think long term.
Sure you can gradually climb the ladder by demonstrating your skills bit by bit and getting access to more resources. It has very good prospects if you do manage to push through. But it's a hard and risky path, and you will not be able to get any interesting results for the longest time.
For a young middle-class student, it just doesn't make much sense. You can do much more impressive and impactful things with your time without getting into that black hole.
I know how to build an LLM, I know plenty of fellow young engineers that do too. It's really not that complex. But they can't do much with it without capital or access.
Good engineering has never been a bottleneck in this field, it's been all about having access to capital and taking smart but dangerous risks burning it on compute, without much idea of how long you need to keep burning for. There's still no end in sight, some are still managing to convince investors and keep burning, and we are seeing progress, but the business case is still unclear. If you want to get in that game, go ahead, but it's not something I would advice the average young engineer.
I’m 40, and I don’t.I took that abstraction for granted and “left it to the big labs”. However I want to build my own LLM for learning purposes.
On needing big expensive hardware.. necessity is the mother of great innovation. Perhaps 18year olds trying to build their own LLMs in constrained resources environments will result in ground breaking ideas of achieving better intelligence than the one we currently have….
The world needs pragmatic folks who work at a higher abstraction and make LLMs useful, AND also folks who think why not “this other way”? And build newer ways to do fundamental things.
Given the usefulness of current LLMs, I would certainly encourage anybody to try and build their own LLMs, and see what they come up with…
Heck if they build a rack full of old laptops and run something with it that could be done “better” with modern servers, I’d still appreciate the learning running things on those little machines bring.
Maybe with a decent consumer GPU like a 4090, you could do experiments like distilling and fine tuning a small image model for edge deployment for specific tasks.
Even there, many use cases might require renting compute for $10/hour and investing a few hundred.
A LLM from scratch? Forget it. You can do theoretical experiments, but not build anything remotely useful with that kind of budget.
If you're talented enough to come up with revolutionary methods, maybe an university or AI lab would be the place to be.
10000%.
Even more so when things are not just expensive by nature, but truly overpriced beyond that point.
>10000%
Once in a while you do get somebody who only spends a dollar and gets more out of it than a seasoned high-roller spending $10000. Most of the time the waste is borne by those who can afford to throw away $10000 more easily than an economizer can afford to lose one dollar, so nobody is crying about it.
With how ridiculously large the language models have gotten though, a 10000x improvement in actual intelligence does seem like it could be lurking unrecognized at a different point on the compass.
You can however learn everything you need to know to get on the career ladder as a software engineer on a regular home PC.
Do you take the first step or rule it out because you don’t yet see the complete picture.
As a teenager I never hesitated to try things out. As a young adult I wanted the whole picture. Now I’m back to playing / trying things out. I kinda wish I’d not given it up. PG being a bit older and reminiscing - I bet he’s in that bucket too, whereas someone trying to establish themselves professionally probably (aka me early 20s) wants to see the path.
The “large” qualifier dates back to pre-transformer language models, where even training a multi-million model was hard due to how poorly it scaled. GPT-2 was a large language model, despite being only 124 millions parameters.
Due to how much high quality data is readily available, anyone can now train a sub-billion (L?)LM on commodity hardware.
And I'm personally convinced that pretty much any enterprise use-case of an LLM (except coding) is better served by a fine-tuned small (<2B) model that is trained specifically on the task, rather than a generalist frontier model, so learning the engineering around fine-tuning is a key skill that companies will realize they need sooner than later.
I saw my first language model in action in 2014, I was writing blog posts about them back in 2016. In recent years, like many of us, I spent some time learning ML frameworks to see if it'd be a fun career pivot.
But:
1. It doesn't seem especially creative. All the frontier labs have nearly identical model personalities, capabilities and even app designs. We're not seeing them differentiate from each other, implying that the design space might not be that large. In which case the opinions and unique approaches of specific engineers aren't that important, they are interchangeable at the right level of skill, and what to do next is usually obvious to everyone.
2. It doesn't seem like a big job market. A lot of ML jobs were wiped out in recent years by the rise of LLMs. Lots of NLP specialists etc were suddenly replaceable with a cheap API. The jobs that remain have compacted into a small number of companies. It's a small community which greatly increases career risk, especially as so many are unprofitable and/or have strong ideological requirements.
3. It's unclear how much demand for better models there actually is. Do we actually need smarter models? In robotics clearly yes and robotics is interesting and high potential, but for pure LLMs/image models, most users are already incapable of setting tasks that stress the best models and are happy with the cheaper smaller ones.
Using the models on the other hand is a very large design space, and has a lot of scope for creativity. I see use cases for AI everywhere, but most companies seem to stop at putting a chatbot on their website or asking Copilot to rewrite an email before they send it. A lot of companies have hollowed out their IT departments over the past twenty years. It feels like a new golden age of consulting work could be upon us.
I really don't think I'd tell a 17 year old to learn how to train LLMs. Learn how they work and how to use them, sure, absolutely.
There is the more likely reason they are not differentiating. They use almost exactly the same class of model. Everything is linear, parallelizable. It's incredible path dependence that's now invisible enough we think it's a natural law. Nature is not linear.
Good luck relying on in-context learning for a 600M LLM.
> The actual adapter training is automated and put behind simple APIs.
That's like saying it's worthless to learn infra because you can use serverless instead…
> All the frontier labs have nearly identical model personalities, capabilities and even app designs. We're not seeing them differentiate from each other, implying that the design space might not be that large
The design space for a generalist model isn't large, by definition. But the design space for specialized smaller models is much larger. If you can train a 200M model that, for your use-case, is competitive with a frontier one, then you'll make your company save a lot of money in tokens.
> 2. It doesn't seem like a big job market. A lot of ML jobs were wiped out in recent years by the rise of LLMs. Lots of NLP specialists etc were suddenly replaceable with a cheap API. The jobs that remain have compacted into a small number of companies.
We are in a strange place where a few companies are collectively burning a hundreds of billions a year to sell things a few pennies for the dollar. Of course it's going to be cheap and concentrated. How is it supposed to end though?
> 3. It's unclear how much demand for better models there actually is. Do we actually need smarter models?
That's the thing actually: I don't think we need better models this much, and if we don't need better models we need the cheapest possible model for a given use-case.
But it feels like it sort of backs up my point about there being good models at every size class. Fine tuning Needle looks automated. Yes, you need to know basics like what validation loss means and how to use Python, but otherwise it's all about creating the dataset.
If you look at model training jobs a lot of the work at this point is creating RL gyms (normal programming work), but most people still think the work is all neural architecture research. Doing the former is fine but won't teach you much about how to build LLMs, whatever that means now. Doing the latter is a very hard market to get into: not many jobs and requirements are often like, "you must have published at one of the following conferences". Prior experience is assumed. Most of them seem to treat Google as ML university and source of new recruits. It's understandable given the cost of training runs.
I'm not sure why it's like this. If you look at the real world, you have stuff like ggml, which is about as hardcore as it gets in the LLM space, and it was made buy just a guy. Same for this like ComfyUI
If you get enough academics in a place, they tend to close rank, and not let anyone in without the same credentials. Data science used to be like this, they were constantly on about how you need a Math Phd to even apply, yet when I met these guys IRL, most of them were just running Python math libraries.
These previous examples show that if you understand at least a part of the problem space, you can 100% contribute without academic credentials.
The question is more one of opportunity cost. At 17 you need to start finding your way in the world. It's best to learn skills lots of people need.
-- 3Blue 1Brown's Neural Network Series: https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_6700...
-- Karpathy on LLMs: https://www.youtube.com/watch?v=7xTGNNLPyMI
-- Stanford CS336: https://www.youtube.com/watch?v=JuoVZkPBiKk
Then do this hands-on:
-- Karpathy's zero-to-hero: https://karpathy.ai/zero-to-hero.html
what kinds of startups ?
So I'd say that understanding LLMs to the level that you get to by training your own baby one would be a solid foundation for pretty much anything built on top of the "real" ones.
Paul likely assumes there will be a sequence of additional papers with the same impact as Attention is all you need, which will spawn a lot of opportunity for a larger group of experts who are conversant enough to advance the field even if they do not themselves create such a major innovation. Not only is this deeply exciting, it is also highly meritocratic as there is still scarcity of the kind of intellect and creativity necessary to swim there.
Machine intelligence might soon surpass it, though, and Deepseek is 100% Chinese mainland educated. Paul's description of building an LLM from scratch is meant as a vague starting point for being an innovator of the highest value aspect of modern AI innovation, not as a specific prescription.
And so I think the idea is more to understand tomorrow ... from first principles.
In the late 80's, as a teenager, I learned x86 assembly and C because that was the only way to squeeze out enough juice from my shitty CGA (and later VGA) card to programm the games/graphics that interested me.
I haven't written assembly in years.
But whatever I did in my career: it helped me and gave me an edge over my peers to have a foundation that is very close to the metal.
We don't know this. So many people are simply claiming this confidently, and a lot of them are betting their careers on it, but nobody has a crystal ball. Whenever someone tells you confidently, and without any doubt or qualifications, that something "is the future," be skeptical.
I remember when the Segway was definitely going to change urban planning worldwide.
Current AI can automate significant amounts of grunt work in programming and math. It's good at running web searches and writing summaries. There are a few other niches where it is currently successful. But other than that, many corporate AI projects are spectacular failures.
So just given what we have in hand, assuming no further breakthroughs, then we're maybe looking at AI being somewhat bigger than the Internet. Which would make it a revolutionary technology, sure.
But to get from "a revolutionary technology" to "the substrate the future runs on", then you need to assume more breakthroughs: long-context operation over weeks or months, displacing human workers 100% instead of 75%, and the ability to directly economically compete with actual humans. And people are investing literal trillions of dollars to make that future come true, without really thinking through what truly competitive-with-human AI would actually mean. We might be looking at massive job loss, centralization of power, fully automated "companies" with no humans dominating markets, and other dystopian scenarios.
And in those worlds, it's unclear that being good at CUDA and matrix math will be all that helpful, careerwise. The AIs are already pretty good at that stuff. Data scientists get paid OK when they actually get hired, but it's not everything college students were promised in the 2010s, either.
We can't yet build a fully-general competitor for the human mind. But we're getting closer. And if we ever do build one, the consequences will be really weird in any number of ways. So I worry about visions of the future that assume AI keeps improving significantly, but that also assume it still somehow remains a "normal" technology that doesn't, for example, render most humans fundamentally uncompetitive.
At 75% replacement of a worker we would already have huge job losses as each individual would be doing what several before did.
The only alleviation would be the creation of new equivalently paid jobs, which is no better than a hypothesis right now.
Citation needed
AI is a great solution in niche areas but generally doesn't make much money. All the large companies are in the negative.
The steam engine was less of a bubble and was much more revolutionary and had a greater impact.
Finetuning model is cheap and incredibly useful for deployment. You don't need to pre-train a frontier llm from scratch to make useful models.
There is tons of domains where you and fine-tune llms and deploy them for value in companies and for your own entrepreneurship ambitions. I have made this a big part of my career for the last few years and now I'm working on finetuning models for starting my own companies.
End of the day they're all customized data stores and protocols to interact with them. May as well stick to a uniform toolkit with fine-tunes.
Not that other tools aren't useful. But reaching straight for a bunch of infrastructure reliant services is like jumping in with k8s when you're still at a stage where basic mocks in code are sufficient.
I won't roll my own encryption or UI lib but want to stay focused on the incompleteness of the project I have to ship not all the buttons and knobs of some dependency or framework. Same old manage context switch problem.
But there are also a lot of prerequisites, namely does the enterprise have its sh*t together on a technical level. Does it have the processes and data pipelines available to train and benefit from these models? Probably not!
Applied ML is at the crown of a tech pyramid whereas most enterprises are still struggling at ground level. Being able to build from be ground is likely a safer skillset than only knowing how to work at the (non-existent) apex.
We had scores of students study how microprocessors work and compilers work over decades, yet we have 3 or 4 major processor companies and a handful of programming languages. Yet, what they learned was probably crucial in their development as engineers.
We are also so early right now that even 2-3 years from now who knows how many LLMs and model firms survive (esp. given the "snake eating its tail" venture/investor funding situation)
But LLMs are tools. Does a great engineer need to know how vscode works? Might be helpful to understand how extensions work, LSPs, and project configurations.
Usually when working with any tools, you need to understand how to get the most out of your tool for your needs and that's about it. Core fundamentals about how software and hardware works in general seems like it would be MUCH more useful than LLM core knowledge.
There are plenty of areas were we need people to do this for insurances, banks etc.
AI/ML exists on many levels.
I think this is disingenuous. One could say that drones are nowhere the efficiency limit either: a bee can fly for hours on the energy contained in just a few milligrams of honey, while our best battery-powered drones can't stay airborne for more than 30 minutes. But comparing energy efficiency of electric/mechanical devices to their biological counterparts is not an apples-to-apples comparison. There's a world of difference between the energy storage and delivery mechanisms.
And as many have pointed out already in the siblings, it's not just about the compute but the access to petabytes of training data.
A 17yo who trains their own LLM will have a much richer understanding of what AI is, how it works, what its potential capabilities and pitfalls are, versus someone who spends the same time doing something else.
I didn't write a LLM from scratch but it is on my "wishlist" so to speak. From what it seems, a GPT-1 class LLM can be done from scratch in a few days and tens of dollars of cloud compute or a high-end gaming GPU.
It is an exercise not unlike building a compiler, a school classic. You are unlikely to ever work on a compiler, but at least, now, you know your tools a little better. It is not about becoming an expert, that takes years, it is about knowing what you are doing.
If you intend to make software engineering your career, you will want more than surface knowledge. And that part is entirely on you, or on your school if you are a student. Companies will not pay for you to learn the fundamentals, they want short term returns, because you may leave at any time. But you as a software engineer may have 40+ years left, so it is worth thinking long term. Claude code may become obsolete a few years, but linear algebra is not going anywhere.
However, I do learn stuff about models that takes it from “magic” to “useful tool I understand the limitations of.”
Do I do it for that reason? No not really, I’ve never had luck learning something because it would be good for my career. I do it because at my core I’m a bored teenager who wants to make the computer do cool shit.
Not that expect to make it big as a LLM researcher but building something from scratch gives a much deeper understanding than what you can get from simply using something.
Much in the same way as implementing and designing your own programming language makes you a much better programmer.
Probably this also is too cynical and simplistic, but: really what’s special is the ability to get this kind of capital, with the freedom to burn it on mad moonshots, with long enough leeway to actually get to see a few of the moonshots come true.
No wonder that the head of YC made this happen, this is exactly what they are world-leading at.
I would think not, but when I started to look into OCR options recently - assuming that obviously a dedicated tool would do a better job than an LLM - I was wrong (apparently).
It similar to understanding how a very basic CPU works. Just because I'm not going to work at intel or nvidia or whatever optimizing the hell out of a chip, it doesn't mean I just throw my hands up and think "magic" - the basic architecture isn't that difficult, and the value of knowing it is astronomical for anyone writing software.
The only jobs that he found he was highly over qualified or paid very little.
In any case, it doesn't look like there's this crazy rush to hire all ML talent, even the one that understand the math and technology deeply.
Maybe people simply don't want math PhDs but something else? Since 1-2 years ago I started doing consulting/freelancing in the ML space, but more on the infrastructure, deployments and similar stuff, as a general purpose developer, and I have a waiting list of clients interested in more work, some of them even trying to recruit me to work for them full-time as well. I'm based in continental Europe, fwiw.
Also bunch of past workplaces who've adopted AI in various ways who reach out once they find out what my current focus lies, but that's harder for others to replicate unless you've already had a career as a developer.
I don't think the "incredibly small minority of companies in the world do any real training or optimisation" part is necessarily as true, as some parts of the work I do get is about helping them optimize training and infrastructure around training. Mind you, none of this is for building LLMs from scratch, it's 99% fine-tuning existing checkpoints.
I'd also agree with "paulg is in somewhat of a bubble" regardless of this, which is worth remembering whenever you read his content. Same goes for any person living in SF, and dare I say the US. But also, YMMV, I live and work in Europe, probably why I have this perspective.
Worst of both worlds - no casual gossip feed in the Bay Area, no big dog meetings in the UK. (Which mostly has no idea he exists.)
As for the question - what are the odds LLMs will be anywhere near the top of the tech tree five years from now?
The trend seems pretty clear to me - local/offshore models are snapping at the heels of the big names in the US, and the current investment arc is insane.
I wouldn't bet on Anthropic or OpenAI being leaders five years from now. Longer term, I especially wouldn't bet on the US build-yourself-a-monopoly corporate model surviving AI at all.
Is it possible to see some of your old works? Personal research?
Citation needed. I'm guessing you're not counting children.
Now with LLMs, people write native apps in Rust, and I'd like to think some of them found that there isn't such a huge jump in difficulty they assumed there would be.
That's the same case finding companies that will actually pay for hand made LLM instead of using something from big providers will be hard because most companies won't be able to afford it.
Yeah if you find a company that will do that stuff directly, good for you, but you will have to be very lucky and you will have to compete with other people who followed PG advice.
So I would rather learn all there is about properly using LLMs and integrating them with existing systems, that will most likely by useful for 90% of companies out there.
Building business niche harnesses is in my opinion much better direction. Knowing what will work best in specific cases is it FTS or vector search, optimisation of usage, getting best results while using cheaper models, knowing how to use tools to run models on the servers, and all the tooling around that like various MCP or just tooling that will be provided to models.
That is what I am currently busy with and I already have customers for that knowledge.
But that’s the tip of the iceberg. If you have any ambitions of doing this professionally, it quickly becomes clear that all it’s all about knowing how to deal with problems that are only present at massive scale, when an LLM is actually L and becomes AI.
The mundane details about how to build a tiny autocomplete model and the maths behind it you can learn in a couple weeks easily. It’s not black magic, there are much harder areas of computers science.
I feel like the "ALWAYS HAS BEEN" meme is apropos here.
I think a lot of people assume that only the big AI labs can do cutting edge research, but there's a strong argument you can do it as part of little tech as well.
I just finished fine tuning Gemma e2b for local code completion on my local machine.
This comment just reinforces what the post actually means. We need people that are LLM natives, computing solves itself with time and with scale adjustments
2. That's why I have an agentic agent as well installed, Qwen 27B, outrageously good, better than sonnet 2 years ago. And it's mine, I can give it confidential info to work with since I own the whole chain. See where I'm going with this?
1. Both training and optimisation will get significantly cheaper and easier quickly.
2. Politics will probably get even more insane before a potential reprieve on the 20th of Jan 2029.
3. The big AI firms will become part of the surveillance capitalism network, if they're not already.
So I think for self-protection a lot of companies will be looking near to medium term AI independence.
For the time being, unless you truly have millions, the outcome from training will be very net negative, while focusing on building on top of existing AI will yield amazing things if you apply the same talent and effort.
When it does get cheaper, then it will be easier to acquire the skills and experience too, and the struggle you went through by trying to do it now will be somewhat wasted.
Besides, I am well versed in this field, and it is not rocket science. There are plenty of software engineering domains that are a lot more challenging, like high-end graphics, large-scale data engineering or kernel programming. People will learn to train LLMs when people want them to.
In reality, enterprises are happy to offload even risky tasks to others as long as they get some contractual guarantees about their data. Would they like more choice in who to buy from? Yes, but not enough to in-house such a specific discipline.
Fine-tuning a model or LoRA based on the companies data set is more feasible but you're likely going to need several runs as you test/try out different base models, parameters, etc. This is why there are a lot of fine-tuned models on huggingface based on base or instruction-trained models from the larger AI companies that have released open weight models (Microsoft, Google, IBM, Mistral, DeepSeek, Qwen, etc.).
Training is limited on memory first (storing training data and weights) and computation second. Realistically you need to own or rent 2-8 H100/B100 devices or Google's TPUs.
The majority of workflows for a company providing AI capabilities are likely best solved by tailoring a system prompt for the chosen model, evaluating the prompt and model with tools like promptfoo, and then running it on a compute cloud provider (including AWS Bedrock). If the company is big/financially well off enough they could look at buying the hardware needed to run it on their own servers.
For other uses like agentic software development you'd need to spin up a suitable model on a compute cloud provider (or local hardware if the model is small enough) and then tell your IDE/editor to use that model. You would need some way of benchmarking and evaluating the models to see if they are capable of doing the tasks you need. -- There have been some tests done by people on YouTube that suggests that Qwen 3.8 27B is a decent model, but your needs may vary.
Paul, I think, is talking about achieving outsized outcomes in relatively shorter timeframes (as the timing is just right to be investing in learning this tech) for high agency folks who can also afford the ordeal in wanting to maximize for impact & ambition. Of course, there's real risk one may get no where, but even in failure, given you were building the LLM yourself, you might end up with other adjacent, high reward opportunities.
At the scale you are probably imagining, this is true - but take the hype out of the OP and what you have is just someone saying the field of data science exists and is growing.
I can see the point. It's unlikely that a 2.4T LLM will be integrated into, say, a pesticide drone. You'll still need some kind of LLMs to achieve maneuvers that "normal" programmings can't achieve.
But what if everything basically turn into that? Essentially, instead of build me a web app to solve X and do Y, build me an LLM to serve X and do Y. (unless the current LLMs are able to do it end-to-end but then they can hardly write coherent software/personal opinion).
You’re missing the point. Understanding how Unity works fundamentally makes you a better Unity dev.
Writing your own game engine makes you realize that the Unity engine is not really that well written....
In reality the problem is that it gets blasted out of the water by a much worse architecture trained on 10000x the infrastructure. And while I'm sure the freshly brought in ML student came up with a 10%, even 30% better architecture, it just doesn't matter. (and never mind that even OpenAI hasn't really solved a voice model yet. Try it. It can probably match 2026-quality call centers, but it's no substitute for an actually empowered human)
... and yet, if you look at what hyperscalers are getting paid for ... comfortably more than half the income is training. Which makes no sense on so many levels.
e.g. https://valueaddvc.com/blog/inference-chips-vs-training-chip... (I get it, not great first source, but st
For some very niche cases I think this is probably the case but for the vast majority, the company's data isn't as useful as they think it is or anywhere near the size needed.
I don't know where you are located, but in EU, in China, and yes even in Silicon Valley, the vast majority of companies do not do any real AI engineering. There's nothing wrong with it, it's just not a smart path for most purposes. You can do amazing things without training, and if you try to train, you cannot get anything amazing unless you burn millions.
Very few people can afford to play the long game and cross that dessert. And, sure, you will not get far without good engineering, but good engineering is definitely not sufficient and is not the primary bottleneck.