undefined

points

[-]

I like the illustration that the models are working on a convex hull of known information. Filling gaps with linear combinations of known facts and results.

They can't exit the hull until the "intuition" starts spawning points outside the convex hull.

by bee_rider5 minutes ago|

parent|

[-]

Is that actually true though? I think it is an analogy, and as an analogy it seems quite risky because “convex hull” and “linear combination” are technical terms that might give the recipient the impression that it is a technical argument.

by jiggawatts15 minutes ago|

parent|

prev|

[-]

Neural nets can extrapolate past their training data, and there is no reason to think LLMs don’t inherit this capability.

The extent to which they are able to do this is the more interesting question!

by tuvix1 hours ago|

prev|

[-]

All arguments like this boil down to semantics at a certain point, but yes large language models can “intuit” because they can generalize between examples. The issue then becomes how you pack new examples into context.

Humans can “intuit” based on a much larger, if not unlimited, context. Also I just want to say that human cognition is something so insanely complex and deep that we will not understand it at all in my lifetime. To attribute all, or really any, aspects of human cognition to a machine at this point is silly to me.

by michaelmrose1 hours ago|

parent|

[-]

Define insanely complex and deep in a way that isn't illiterate hand waving.

Most humans are dumber than a box of rocks. Here in Seattle we had one of many light rail-related fuckups where they had to replace part of the line with buses. People piled into the front of one when it was full. When people got out they never moved back. As the driver struggled to close the door and people struggled to get in the wad of people never moved back to fill the ample space.

Chatgpt was smarter than the average person a while ago

by zahlman1 hours ago|

parent|

[-]

> People piled into the front of one when it was full. When people got out they never moved back. As the driver struggled to close the door and people struggled to get in the wad of people never moved back to fill the ample space.

This does not demonstrate a lack of intelligence. It demonstrates laziness and a lack of interest in spreading apart. Or just lack of consideration (or even malice) on the part of those at the back of the wad.

> Chatgpt was smarter than the average person a while ago

This is an absurd claim that fundamentally misunderstands what it means to be "smart". Reasoning that would get you to this conclusion would equally well apply to Google's search engine over a decade ago.

by tuvix1 hours ago|

parent|

prev|

[-]

I’m not talking about the actions we take or how we might perform at certain tasks, I’m talking about how our brains actually work. My point is that we have no idea how I’m able to imagine an apple and see it in my mind’s eye. It’s basically biological magic to us at this point.

There are processes at work there that we don’t even have the language to describe.

by zahlman58 minutes ago|

parent|

[-]

Not only that, but we do it with a processor that is basically required to operate in a narrow temperature band below 40C, using a mere 86 billion neurons (although the equivalence with either machine-learning "neurons" or LLM parameters is not at all clear) operating on a few dozen watts; and with this we operate many other systems besides language processing. It's not clear that our reasoning process requires language, either.

(86 billion is the number ChatGPT, ironically enough, has given me a couple of times. I remember hearing for a long time that it was estimated to be somewhere in the ballpark of 100 billion. This is not my field of study.)

by claytongulick3 hours ago|

prev|

[-]

> People keep saying this. Why?

For the same reason that you can't draw a 15 of Diamonds from a regular card deck.

by treis3 hours ago|

parent|

[-]

Of course you can. Tape a 7 and 8 of diamonds together and boom 15 of diamonds

by 2 hours ago|

parent|

[-]

deleted