undefined

points

[-]

  > "Second, there is no guarantee that a given neural network can be approximated by DISCOVER"

Page 7.

They train what appears as embeddings for outer product of roles and fillers. The role for language model can be a position in text, the filler can be an embedding of a word at that position. Then that matrix of a sum of these outer products is linearly mapped into NN encodings and then decoded by NN decoder.

The embeddings learned by this process are not necessarily smaller than original ones. Given that they participate in an outer product computation gives me impression that the resulting sum is much bigger than actual NN encoding, that is why it needs to be linearly mapped into NN encoding.

So, this paper will not necessarily lead to any computation savings.

But I am at page 6. ;)

by sigpwned18 hours ago|

parent|

[-]

Good find! But they stop short of saying it cannot be distilled to symbolic algebra. Regardless, your point stands. I scanned the paper in the middle of the night instead of sleeping. Clearly I didn’t do a close read! Thank you for pointing that out.

by yorwba20 hours ago|

prev|

[-]

Their representation is a key-value store that they embed via a tensor product to reconstruct the vectors the model produces. The catch is that it works best if the key-value mapping is task-specific. So if you already know how to solve the task the model is performing and can transform the input into a data structure that makes it easy to just read off the answer, you can probably also recover the model output.

That in itself is of course not super useful for tasks that we don't already know to do symbolically, but it's nice to see that they can manipulate the keys and values of their representation somewhat independently. So at least some of what LLMs do internally seems to involve a bit more structure than just linear combinations of some overcomplete dictionary.

The interesting question is whether this can be used to interpret the operation of a single layer by lifting it to operate on key-value stores instead, replacing matrix-vector products with reading from some keys and then writing to others. That could potentially move the balance of power more towards CPU inference instead of GPUs.

by eli_gottlieb17 hours ago|

parent|

[-]

> So at least some of what LLMs do internally seems to involve a bit more structure than just linear combinations of some overcomplete dictionary.

Well that's good to hear. Would have been rather exasperating to find out we could have somehow gotten Fable or Sol out of sufficiently advanced sparse-coding ages ago.

by riceflippa20 hours ago|

prev|

[-]

> Is evaluating these closed-form representations more computationally efficient?

No it is way worse since computationally they are same things with symbolic having overhead of attending to the symbols.

by sigpwned18 hours ago|

parent|

[-]

My naive middle-of-the-night Claude question said the same thing. At least for this approach. I have not read the paper closely enough to refute you. But the concept of a lower-dimensional closed-form solution — which the paper seeks to discuss, please correct me if you read differently — is tantalizing, if only because it opens the path to different math, which can lead to optimization.

by rwmj20 hours ago|

parent|

prev|

[-]

Is it though? Brute force searching generally scales like 2^n. LLMs on the other hand can obviously be trained, and even though training is massively resource intensive, it's not 2^n.

by Oarch21 hours ago|

prev|

[-]

It's a fascinating emergent behaviour but also one that could have been predicted?

I'd imagine that our brains have become similarly biased towards generating such a generalised symbolic structure over hundreds of millions of years of evolution?

I'm thinking how certain regions of the brain reliably represent things like motor skills, speech, abstract thought.

by sigpwned18 hours ago|

parent|

[-]

I agree. In retrospect, this seems almost inevitable. And our own minds at least to do some form of symbolic reasoning — literally language, which you are apparently capable of, dear reader, as a lower bound. There might be more symbolic reasoning in the conscious, unconscious, and subliminal parts of the mental workspace. I always hesitate to infer similarities between the brain/mind and LLMs, but I certainly track the comparison here.

by noduerme22 hours ago|

prev|

[-]

Inference is just tokens transformed through a fixed crystalline structure, no? You already could put that on a chip. There's no particular reason it couldn't be represented as some set of symbolic equations instead of a layered process... it's just another kind of quantization.

When symbolic algorithms are that large, they're equally hard to reason with most of the time. The upshot would be a lot more storage required in exchange for more generalized computing, lessening the need for so much GPU in a lot of cases.

I don't see why a model couldn't be represented that way. After all, if you just polled the output of a model, you could evolve genetic algorithms to predict it with fairly high accuracy in a limited domain. Take that out to the Nth degree and you're basically just unspooling the model into a giant set of equations.

by sigpwned17 hours ago|

parent|

[-]

Yes, you certainly could put that on a chip. And people are doing it, for smaller models. The question is how big that chip would be for something like Fable, which is generally estimated in the trillions+ of parameters, and if, given the amount of memory, computation, and bandwidth required - at least with current methods, i.e., very high dimensional matrix math — a chip is the right way to go. Can you put a LM (just a smaller language model) on a chip? Empirically, yes, I believe it is done. Small enough ones probably even fit on a FPGA. Can you put a LLM on a chip? Depends on how L it is! My intuition says that some open smaller models might fit, like Haiku, but not Fable. But that’s just intuition talking.

The hope would be that this unlocks some substantially more efficient or parsimonious math that would fit better on a chip. And that’s clearly my words, not the authors’, per the comment above.

by virgil_disgr4ce15 hours ago|

prev|

[-]

Maybe someone can clear this up for me. I always thought it was already obvious that neural networks, artificial or otherwise, are symbol systems. Some input pattern reliably associated with some output pattern is, nearly by definition, a symbol. No? Am I just using the word symbol differently than researchers in these fields?