The exam was huge, at least over 10 pages, and even when we technically ran out of time, the professor was kind enough to move the remaining exam takers to the neighboring lecture hall to continue taking it. I recall I spent a total of 2 hours on that exam.
Now mind you it was mostly short answer or multiple choice questions. The multiple choice questions were pretty sharp too, lots of traps and false but sounds right answers mixed in. But if it had been purely essay questions, I would have been screwed.
However with such a huge corpus of information in front of me, I ended up basically learning all the material on the spot. I just kept doing multiple passes through it, each time I noticed one of my answers contradicted one of the others, I would make adjustments to harmonize, which indirectly refined my understanding.
In the end I got B+ in the exam (which was curved to an A), and walked out understanding the material better than I did walking in.
Reflecting in the experience years later, I've wondered if a hypothetical LLM which was ignorant of microbiology could do the same thing if fed that exam. In some respects the traps they placed in the multiple choice questions actually were what helped me refine my understanding the most. Made me appreciate information theory more.
This strategy saved my ass on so many tests from high school through college that I did not study properly for.
I think it would certainly be possible to write a test such that this strategy doesn’t work at all, but lucky for me no one did in any meaningful way.
In one of the studies they got incoming CS students to fill out a multiple choice exam on programming, before the class had even started. Then they looked for hints in their exam results to see if they predicted the end of semester grade.
Turned out following this strategy was an incredibly strong predictor of success in computer science. Some students somehow got 0 (or close to it) in the multiple choice test. But the researchers looked at their answers, and could see that the student was forming hypotheses about the questions. Even if they got all the answers wrong, they answered all of them with a consistent idea of what the computer programs might have meant.
Weak students just answered each question individually based on vibe, without using what they learned (or figured out) from other questions.
This has affected me a lot though, I know what is going on in songs, but can't ever remember the right words, but still some how end up sining stuff that means the same thing :/
So if I put a visual bend on something I can quickly understand it and remember it, but that bend has to be mind (even when helped by an image on a book). If it’s someone else’s, I’ll be very hard pressed to remember it (but I can easily remember where it was).
So at school, when I had to cram, I fill my head with images and things like that. And with the help of the exams (written) and memory of the classes, I worked out the answers.
For me, I have the bad "factual" memory and good "conceptual" memory but unlike you, I also have aphantasia: a completely "dark" mind with no visuals.
I've always used the metaphor that my brain has really good "compression". Once I learn something it gets compressed into my worldview. I don't know how or why I know it but I can apply it. If academic achievement is any benchmark, I can apply it with above average aptitude.
Most of us turn out like the rescued exotic bird which turns out to be a seagull covered in curry.
but you can also favorite comments for your own hall of fame, eg https://news.ycombinator.com/favorites?id=andybak&comments=t
The actual intelligence segregation is self-selected. PMC parents live together in the same neighborhoods. Whether you believe in genetic or social factors, the smart kids come from here.
In the 80s and 90s there was an educational culture that believed in finding overlooked geniuses regardless of where they are from. That narrative is completely dead. All institutions expect kids to come prepared with a resume from their youth, and have few avenues for identifying talent that doesn't meet their mold.
In areas where parents aren't demanding advanced classes, there is no program for the government to identify and separate top performers from the general student population.
Yeah mine didn't have it. And the teachers fought every attempt to do so because it hurt their metrics (who gets stuck with the lower class average).
Are you by chance part of the PMC and your local school district is of similar demographics?
I actually always thought it was the opposite for me – I had to figure out how to work things out from scratch because I could never remember anything.
I found a solution recently. "Critical mass." If I just don't see something frequently enough, my brain keeps deleting it.
So, if there's a fact you have trouble remembering, try memorizing it 100 times in one day. That's "inefficient", as far as memory goes, but it'll do the trick! And I like solving problems once and then moving on.
(Anki, conversely, is designed for efficiency, which means it's designed to be as painful and unrewarding as possible, but that's a rant for another time... suffice it to say if you design your software so that I constantly want to stop using it, your efficiency drops to zero!)
Anki kept flagging my relatives birthdays as "leeches!"
I found it kind of hilarious and infuriating that the software I used to remember things tells me "this thing is hard for you to remember so you should just give up."
I can’t remember the lyrics to any songs, or the lines from any movies, or what I’m doing this week. I don’t think it’s a strong hypothesis.
My wife has a nearly eidetic memory, so marrying her was a game changer for counteracting mine :)
Similarly, being engrossed in tasks today (or rabidly learning/reading, as many HN readers do, as long as it's not the link being discussed) might make you not focus much on next week's calendar.
—
“Hey do you remember where I put my glasses?” “2 sets at the office and your other set is upstairs next to the toothbrushes”
—
At the grocery store: “Do we have apples in the fridge?” “Yeah, 4 left”
Doing an angular -> react refactor with Opus 5 at work.
Amazed at how good it is but also amazed at how dumb it is after I told it I want a very basic and specific code review:do the file names match the contents? is one functionality/concern split in multiple files? do the variable names match what they do? does a method read nicely?
Anyway, I was very surprised at how bad it can be at this.
I suspect it's because the LLM simply does not focus well enough on a single method.
My gut modelling is that the critical files are quite small, some 4k LOC and the LLM tries to read them all in one go and my requirement is surprisingly difficult but the LLM does not realize it, loads all the input into context and simply cannot find a good signal per token/group of tokens.
I guess I can do it because I literally cannot fit 4k LOC in my head and I actually read only tens of lines but I can reeealy dig deep in those lines.
The LLM can remember lots, it has 1M context window, was trained with needle in the haystack problems, but it can severely underestimate how much computation is required with TASTE/universal computation type tasks.
Yes, LLMs are incredible when it comes to delivering well defined outcomes, it's evident they were RL trained very well. But they are incredibly bad at understanding nuance in a text.
It's incredible how agentic coding benchmaxing seems to correlate with writing good poetry but at the same time finding good names for files and variables and wiring beautiful code does not.
Same with getting basic details of who said what in an "adversarial" conversation between 2 humans -- LLMs are incredibly bad at it, with all their "almost discovering new maths" skills
I have fairly poor memory, but I can solve problems that people with better memory can't.
The part that’s really disheartening is there’s so much knowledge out there, I wonder if this is actually efficient - maybe you can’t reasonably search the state of the art across all of human knowledge and so you have to invent it again. But that’s such a sad outcome.
I know people that got to post grad math without understanding a thing but they could remember a lot easily, while many of those that understood but had a harder time remembering every last variation of everything got penalized.
There’s no such thing as a truly original idea. It is all just combining A+B!
But often results are incremental and obvious.
But also if you jumped from A to A''''', nobody would understand why or what it relates to.
If you want an example of someone at the near peak of human ability, check out Jon Von Neumann.
Then there is an ability to peer deeply into complex problems and somehow find the simplest truths that make sense of it all. Think of Einstein.
Both are incredibly intelligent, but in different ways. I'd say Von Neumann's memory was far greater than Einstein's though.
One can flawlessly ponder anything known to man, and the other could ponder completely original ideas (to an extent)
The physicist Eugene Wigner, who knew both John von Neumann and Albert Einstein, wrote that no one he had encountered possessed a mind as “quick and acute” as von Neumann’s. Von Neumann could absorb vast amounts of information, follow extraordinarily complicated arguments and move between mathematical fields with astonishing speed. Yet Wigner still regarded Einstein’s understanding as deeper, more penetrating and more original. Von Neumann may have had the greater raw intellectual processing capacity, but Einstein was more likely to reconceptualize the problem itself.
Present-day AI appears more like a machine-amplified version of the first set of abilities than the second.
I’ve met different people throughout my career whose intelligence came in 1 specific area. For instance, my friend is extremely good at trivia, he clearly has a lot of storage and can access it easily. I think I’ve only met one person who was excellent in all three areas of intelligence.
Obviously this is a simplification, but it’s how I like to illustrate my ideas on intelligence at parties and first dates.
There is also something to be said about recognizing that A + B can be combined.
And also in accurately judging that other combinations will be useless and avoiding them.
In the UK organisations such as GCHQ ( Brit NSA ) use brain teasers as a recruitment and filtering tool.
Being smart, at least in the context of the workplace, is about being checked in to whatever you’re doing, and drawing connections across your experiences.
Some people, through practice and experience are better than others at breaking down foreign information and making sense of it, or jumping to an assumption.
I also think there’s an element of just remembering what my final conclusion was on an opinion, vs all my old opinions and all the work to get there. It’s easier to just store current belief and re-work-it out how I got there later.
They’re positively correlated. Presumably because people who have good plastic intelligence often end up learning more throughout their lifetime. The only way to form knowledge in our minds is by learning it.
But “remembering” is only the beginning (you have to remember first!), after that, elements like understanding relations, connecting dots and remixing, timing, etc will truly make one shine.
In one way, it’s like the current “LLM + Harness” setup for agents. LLM is how well it remembers, but different harness techniques really matters, at times even a worse model mixed with great harness can outperform great model with bad harness
I wonder practically how useful it is. It should be very useful, right? Say you're a hiring manager, and now you can hire thousands of students who have almost eidetic memory across pretty much all the subjects. The students can produce amazing work if you give them specific instructions. They are also very good at mechanical pattern matching. That is, they are the uttermost crammers. They are like the students who don't really get STEM yet they could, through memorizing all the problem-solving patterns, ace any national college entrance examinations, IMOs, PHOs, Putnams, you name it, and all the way to solving hard problems like Sendov Conjecture - they even found out elegant elementary solutions! Mind you, Terrance Tao proved a weaker conclusion when studying the Sendov Conjecture and got to publish his work on The Big Four. That said, the students won't take initiatives, and they sometimes make very silly or even impossible mistakes, to the point that it requires your supervision and review. Would you hire them to do your company's work? I'd assume that answer is a resounding yes.
Understanding especially in the context of unknowns is what intelligence is.
If a time traveler went back to 1600 and started spouting off about differential equations everyone would think them quite mad.
Even in a debate, if somebody just has the ability to remember tons of facts and figures, the other person will seem unintelligent by comparison, even if the other person is correct
Most humans cannot incrementally contribute since they don’t have many traits required to do so - extreme discipline, imagination etc.
We literally live off and benefit from the investments of the few, in relative terms.
Really, compress. Compression is a useful proxy for intelligence.
To compress data efficiently, an entity must discover the underlying patterns, structures, and regularities within that data. Identifying these structures is precisely what we define as "understanding" or "intelligence.
Anything we put into writing can be thought of as a string of data. The Kolmogorov complexity of a string of data is the length of the shortest possible computer program that can output that string.
A good compressor doesn't just memorize (yes, there's a reason LLM writing features the "it's not just X, it's Y pattern"), it builds a generalized model to handle unseen, similar data. LLMs are trained by compressing the internet into weights, they use this to write new code.
When an LLM is trained, it compresses ...bytes of human text into a few ...bytes of neural network weights. When you ask it a question, it uses that compressed model to generate (decompress) a coherent response.
Consider the dichotomy:
LLMs display emergent reasoning, coding capabilities, and logic because they were forced to compress human language so tightly.
LLMs often suffer from "hallucinations" and struggle with novel logic puzzles because compression encourages mimicking statistical patterns rather than building a true causal model of the physical world.
An algorithm can achieve massive compression if it utilizes a highly complex, slow, or massive decompression engine. If a system takes 10 000 years to decompress a file, or requires an infinite amount of computing power (like Kolmogorov complexity), it cannot function as a practical intelligence. Real intelligence must operate within tight time and energy constraints.
> Maybe there's true brilliance out there
Brilliance is a trope. pg's 2009 "The Anatomy of Determination" deals with it: https://www.paulgraham.com/determination.html
In it, he argues that determination is the single most important predictor of success, surpassing even raw intelligence.
While popular culture loves the narrative of the "brilliant genius," Graham's experience backing hundreds of startups taught him that a highly intelligent person without drive quickly becomes ineffectual. Conversely, a profoundly determined individual can find pathways to success even in ordinary or unglamorous industries. Over time, determination actually starts to look like talent.
I'm going to commit a faux pas (a breach of HN etiquette) and give an AI summary in 3 bullet points because, well, the article deals with 3 points, so it's a good fit and I doubt I can compress it better than AI:
1. Willfulness. The desire to get your way regardless of obstacles or consequences.Being fearless about consequences and pushing through "hellacious" moments when others quit.
2. Discipline. The ability to force yourself to do what you planned to do. Being "hard on yourself" and turning off the background analysis in your mind that questions the plan.
3. Ambition. The grandness of your destination and what you want to achieve. Choosing the right target; if determination is a vector, ambition is the direction it points.
Of these 3, it's quite obvious at this point that LLMs are superior at 1) and 2), as LLM alignment is quite good, and it's ability to spin up sub-agents that can work for over 24 hrs is better than humans. Human limit is basically 1 hackathon's worth of cognitive work. Beyond that, health becomes a concern, and how many people can produce useful work under sleep deprivation? Codex can work for 24-ish hours, as evidenced by today's "Auto-research with codex: How I achieved a 232x Faster Kernel": https://news.ycombinator.com/item?id=49309549
So what's left for mere mortals? Apparently, nourishing your ambition:
"So here in sum is how determination seems to work: it consists of willfulness balanced with discipline, aimed by ambition. And fortunately at least two of these three qualities can be cultivated. You may be able to increase your strength of will somewhat; you can definitely learn self-discipline; and almost everyone is practically malnourished when it comes to ambition."
I wonder if what we call discipline (and the resulting skill that we call talent) is really just insanely strong and aligned interest.