> Personally, I'm really interested to see if we're headed towards a spot where the model is entirely distinct from the knowledge store.

This is what I've been trying to focus on with local AI for now. I've been trying to build all new documentation so it's more AI friendly. It's been pretty interesting. Qwen-35BA3B with a small prompt does a good job of surfacing what I'd consider institutional knowledge.

I've been trying to silo the docs I write from the model with a prompt that tells it not to use general knowledge unless asked to. From the anecdotal testing I did, Qwen-35BA3B is great for it. It does a really good job of following the prompt and calling tools, so I've been able to play around a lot to see what seems to work best.

Ultimately, I think one of the most effective uses of AI will be having a distinct knowledge store combined with an opinionated agent (and sub-agent) setup along with different models for each task.

Who owns the knowledge store is going to be the big caveat. Right now I think the big online models are trying for generic, persistent memory and I'd be very hesitant to let that happen. Think of having someone with a perfect memory following you around forever, but someone else has the ability to make them disappear. That's not a good situation.

One of the consequences of encountering a lot of LLM generated text which includes things the model vaguely remembers from its training is that honestly I have grown less tolerant even of human comments and documents that are based on mostly ‘I seem to recall that…’ level sourcing.

In a discussion on economic history, say, someone will opine that Alexander Hamilton had some particular opinion about tariff policy… based on their having a vague memory of a blog post where someone quoted a passage in support of some point. But wait - you can search the federalist papers, the text’s right there to be read, before you commit to saying online ‘Hamilton thought tariffs were a great idea’ you could take your internal ‘I seem to recall reading something about hamilton’s opinion on tariffs’ thought and turn it into a little RAG query where you pull up a source and check before you put another factoid out onto the internet.

And so I feel absolutely the same way about LLMs. I don’t care how much factual information was in the training data, when the LLM wants to rely on something it vaguely recalls having been trained on, it owes it to me to dig up a source and vet it.

There are limits to this, of course. I don’t want it to be thinking ‘but wait, maybe my memory of Python syntax is faulty. Is = used for assignment? <web search>…’.

But in general some caution about repeating vaguely recalled easily checked facts is warranted.