For the completion to work, the source text needs to be ‘good’. That’s a basic kernel of how the thing works. Even with a perfect oracle autocomplete if the source text is ‘bullshit’, the output is too.
replyOr, slightly changing this. The source text needs to speak the correct vocabulary and language to produce a good completion. See the chat where Terry Tao is doing maths with an LLM. There’s _no way in hell_ I could get to his output because I just have no idea, and can’t speak the language.
Same with any field.
fucking is it? AI labs certainly want you to think so but they're running out of time and money. they've been telling me that the advancements of the last three months have made this a reality for the last five years. it's starting to feel a lot like Elon promising fully automated taxi services by next year... 11 years ago
replynext year I PROMISE!
replyCome on bro, PhD level intelligence!
replySo far, the LLMs I've used need problems to be fairly specifically scoped, or they don't produce the correct solution. Scoping problems correctly is a different skill-set than implementation, but it's still a technical skill that is expected from mid-level and higher engineers.
replyYep and I just ran a simple enterprisey "ambiguity" bench on the big three (US) model providers: same ambiguous initial-prompt with same clarifications and pushback prompt sequence afterwards.
replyThe edge of correct/better when facing ambiguity is very fuzzy, all models from the past 6 month or so have similar random ways of spinning between too-literal avenues and oddly misplaced misled fixations. Taking the right initiatives in face of uncertainty is definitely AGI, and its not there, and perceptrons + attention layers just ain't got what it takes no matter how hard you push.
The best I've come up with is to ask an agent to come up with questions to help get to a focused design. But it still needs me in the loop for that.
reply