upvote
I'm curious how you think "word predictor" meaningfully describes an instruct model that has developed novel mathematical proofs that have eluded mathematicians for decades?

edit:

You cannot predict all the actions or words of someone smarter than you. If I could always predict Magnus Carlsen's next chess move, I'd be at least as good at chess as Magnus - and that would have to involve a deep understanding of chess, even if I can't explain my understanding.

I can't predict the next token in a novel mathematical proof unless I've already understood the solution.

reply
I think that's more of a limitation in how people think about word predictors

If you can predict the words a bright person will say about X... Isn't that some truly astounding tool? That could be used in myriad useful ways if one is a little creative with it

Since it's also "alien" it can also detect and explore paths that we simply haven't noticed since their biases aren't quite the same as ours

reply
deleted
reply
What's the difference between "smart" and "next word prediction", at this point? Back when they first came out, sure, but now they can write code and create art.

What would it take for you to concede a future model was smart?

reply
My personal take would always be that it produces something that isn't in the training set, ie: Demonstrable Creativity, or innovation.

For example, it's training set it purely engineering and code with general language data set, would be "aware" what art is, but has never seen an artistic image, aware what colours are and able to create something it never saw before.

Like a child with a paintbrush, there is an intuitive behavior that happens.

reply
Can you name any examples of a human doing this? I learned about colors, color theory, and so forth in school. I've definitely seen artistic images before.

They can already create something they've never seen - you can prompt ChatGPT to generate images, and there's a few dedicated models for it: https://chatgpt.com/images/

Terence Tao feels like they've done innovative work on mathematics: https://www.scientificamerican.com/article/amateur-armed-wit...

reply