upvote

points

by qarl7 hours ago |
comments
upvoteby willy_k7 hours ago|
[-]
Language is absolutely pattern matching, and the whole thing is that the more patterns you put in, the more the transformer can pull out what’s useful. Agents are this effective because people figured out that training on tasks being completed gets the models to complete tasks - that’s pattern matching.

And it can only go so far, where those patterns are feasibly captured, eg internet discussions and coding and mathematics. It won’t measure up to the scaling laws of the physical world.

reply
upvoteby qarl6 hours ago|
parent|
[-]
> Language is absolutely pattern matching

OK - so long as you also agree that language is thought.

Until the advent of LLMs - language was considered the pinnacle of human intelligence. But humans have some need for mysticism at the root of their beliefs - so they chase the gaps in our knowledge. Like "God of the gaps."

Sure, world models will help navigate physical space; they are effectively the animal mind. Will they help unwind the laws of the universe? Probably not at all. Language is sufficient. Because it is thought.

reply
upvoteby willy_k6 hours ago|
parent|
[-]
Language reflects thought. LLMs are a search function over text, text that reflects previous thought; they cannot create.
reply
upvoteby qarl6 hours ago|
parent|
[-]
> they cannot create

Every single day we have new and wondrous examples of the things LLMs have created.

Clearly we are using very different definitions of the word.

reply
upvoteby willy_k5 hours ago|
parent|
[-]
Synthesize != create. A new combination of existing things is different from a new thing
reply
upvoteby qarl5 hours ago|
parent|
[-]
Music is combined notes. Stories are combined words. Newton combined existing mathematics to produce calculus.

I'm wondering if you can name anything that counts as a "creation" under your definition?

reply
upvoteby willy_k2 hours ago|
parent|
[-]
Stories and music, their character, comes from combined motifs and themes. These are what gives them meaning. Creation is tied to meaning; one can’t separate human meaning out of the conversation. Without human meaning - new perspectives gained, beauty found - all things are equivalently pointless, and all an LLM is doing is shifting (very high-order, having gone through the steps to end up in training data) entropy around.
reply
upvoteby qarl1 hours ago|
parent|
[-]
No - I mean my question genuinely. I'd like you to name one specific creation that isn't a combination of what has come before it.

I'd rather get specific than talk about hypotheticals.

reply