That is, you can apply for grants and write papers in the transformational grammar paradigm endlessly. You can pose and answer questions that make sense within in the system.
Chomskyism was disappointing in two respects: (1) was that it was an inadequate basis for machine understanding of text, (2) was that structuralists were hoping everything in culture could be understood in terms of "... having the structure of language" but syntactic structures weren't all adequate for that task.
(1) and (2) both pointed to Chomskyism being hermetic and not something really useful or inspiring to other fields and maybe just a little less "true".
Also, I'll mention that there is a lot of empirical knowledge about natural language syntax in generative grammar, even if the details aren't quite right about their theoretical underpinnings. And a lot of them were "low-hanging fruit" waiting for someone to point out how to find them and characterize them. That Chomsky did that is the best argument for his intellectual value - permitting all that "normal science" to happen.
Science doesn't guarantee technological applications, whether or not that science is correct.
Finally, just saying: use of a word like "Chomskyism" makes you sound like one of those creationists who call evolution "Darwinism".
Yes this is one of my critiques of the academic system (I have finished a PhD and will hopefully never work at a university ever again). Many papers in the introduction start with explaining how the study builds on plenty of previous research, but that alone doesn’t prove that something is a good direction to investigate. If academics were completely honest, many papers would write “We researched topic X because we got funding for it. But since we actually don’t care about X, we just wrote a proposal about our favorite niche topic that only one of the co-authors cares about, namely topic Y, and wrote it such that it sounds like X. Next we show that Y is actually very cool in the hope to convince others that it is cool plus to convince others of the conclusion that our most senior professor already found during the PhD which was 30 years ago. Apart from the professor, we don’t believe in the main argument of the paper but since he got the grant we’ll just go with it in the hope to one day be able to push our own ideas forward”.
in college, i was somewhat involved in cogsci wrt semantic mapping of linguistic concepts based on human surveys (this was well before LLM/AI-word2vec), and it surprises me today how the mathematical properties of relatedness/distance seem now to drive machine intelligence, something which we did not consider at the time at all.
maybe i'm getting in my metaphysical sentimental stage, a turn a lot of adults make after middle age, but i wouldn't fully dismiss the fact that there may be an equivalence.
There's pretty interesting (I think) research in how effectively it's possible to scale down LLMs. If you're interested, take a look at the BabyLM conferences:
(I have no connection to them.) The work there demonstrates that gap between what we can do now with human-relevant amounts of training text, and what remains to be discovered. Quite a bit, it looks like.
Can you use a human language well enough and it not sound intelligent? And no, I don't mean just syntactically correct because semantics is a part of human languages too. I think the astounding thing is that they managed to model a language this well. It's like trying to learn English without a functional understanding of the word "Please".
My take is that a lot of the intelligence we natively ascribe to humans exist in human languages and just like the nutrients in your food, by harnessing a language that possesses latent intelligence, you too can behave like an intelligent person. And how did the nutrients of intellect got in there in the first place? Just like languages evolve nowadays: via contributions from speakers over time. Not unlike soil fertilisation.
but from all models, it's the closest, "all models are wrong, but some are useful"
Care to back this up? What evidence do we have to suggest that back propagation, self attention, and byte pair encoding are mechanistically similar to human cognition? Similarity of the outputs says nothing about similarity of the machinery underneath. Turboprop engines tell us nothing about how birds fly - doesn't matter that they successfully transport millions of people around the world.
I would say that when it comes to higher level cognition, logic engines are more human like even if they only have a tiny fraction of the practical utility of LLMs.
That is not true.
Turboprop engines and flying birds are different things, but there is a set of common properties.
A blade of a screw propeller functions exactly in the same way as a flapping wing of a bird, except that the wing cannot rotate without limits around the body of the bird, so it must stop, change angles and revert the motion, while the blade can rotate continuously around the axis, functioning on one side like a falling wing and on the other side like a raising wing. The similarities between propeller blades and insect wings are even greater, because insect wings cannot change their shapes so much as those of birds.
For now, our understanding of the networks composed of the neurons of animals is much less advanced than our understanding of wings and propellers, which is why there is a much greater uncertainty about whether there are any similar mechanisms in biological neural networks and in LLMs.
"Nothing" may be a little too strong -- physics is physics, after all -- but (small) birds and airplanes have Reynolds numbers about 1000x different and different underlying material science.
Airplanes are engineered (with a very small number of exceptions) to be stable during flight, whereas bird flight is inherently unstable, requiring a lot of specialized cognition so birds can fly.
I would think a lot of cognitive ornithologists (who I really hope exist!) would be interested in finding out about this.
At this point, I'm at the complete end of my knowledge of bird flight!
And I'd be hard pressed to identify many cognitive characteristics of LLMs during execution that resemble human thinking, aside from those that are readily parrotable.
As a model of human cognition, LLMs deep suck.
We considered the hypothesis that humans are a general intelligence that figures language out by exposure. The evidence has consistently come back that that is not how humans actually work. A model where humans are hardwired with a lot of knowledge about language, then fill in the gaps based on exposure still appears to be far more accurate.
The only thing LLMs show is that it is possible to learn language de-novo.
Regarding "hard-wired"; it's not if you consider that "hard-wiring" of an individual is the result of millions of years of evolution.
I agree, but would add, "But not under conditions that humans learn language."