That way the tiny model isn't acting as a potentially incorrect interpreter but you still get semantic search to help you find the right documents quickly.
I suppose there is something to be said for a good summarization, I just worry that these tiny models aren't really GOOD summarizers.
I'm all up for using the better/simpler task for the job. That's been a thing lately also with the Jev/decision models hype, most of those things can either already be done by a pretrained classifier[0], or you can start with a bigger model and then train your smaller/adhoc one when you have some data (it doesn't even take much for most applications, you can get about 95% accuracy on email classification with 50-100 examples)
0: http://jeffyclassify.com/blog/when-llms-fail-at-classificati...
The bigger issue in this context would be canning, such as incorrectly explaining the mechanisms to avoid botulism (acid and/or pressure canning), or perhaps incorrectly wiring in repairs. However, again, it’s best to practice with these prior to emergencies anyway.
It should be noted though, that the model does source its claims, so you can fact check it and should review this information regularly.
Consider they won't have access to the plentiful practice opportunities we have now, most resources would have to be spent with a practical goal.
One of these situations prohibits the other.
A field guide to foraging for edible mushrooms will probably be more practical than naïve use of an LLM: “can I eat this mushroom?” is an exercise in trust that I will maybe never be comfortable with.
But “how do I effectively forage for edible mushrooms” is a great stepping stone.
Kiwix is great, but I'd also like a language model to be able to deeply interrogate all of that information much faster than I can read.
Just arrange its harness so that its responses are grounded in the source material and require source citations for everything. Then read the citations to confirm it's not a hallucination.
Except the build process there requires a specific recent version of swift. Hardly a good definition of "anything"
You can run bonsai2 27b on basically any gaming GPU from the last 15 years including on game consoles.
Everyone has access to this stuff now if they bothered to look beyond the marketing from big labs trying to collect subscription revenue.
I work on compilers, kernels, and operating systems using exclusively consumer GPUs.
If things are so bad that you need to factor in the non-operability of any electronic devices, how many indestructible paper libraries are you planning to build and manage in advance, and at what budget?
Either way the comparisons here are unfair. An LLM that runs off a backpack-sized solar panel and a laptop might be a substitute for a broad, low quality, surface-level encyclopedia but it's many orders of magnitude worse than a library of books.
I would argue that in a survival situation the accuracy of information becomes even more important. If an LLM to misidentifies a plant (or a medical condition, its treatment, etc.) it could put you in an ER, but in a survival situation that could kill you.
If there's any place to get an answer it's here.
Bravo my friend. And I mean that in a good way. Love me some diversity.
That seems to have been mostly solved through scale. I was asking Opus 5.5 about some obscure stuff and it kept hedging "this is from memory since I don't have internet access" (it's sandboxed), "I'm reasonably sure about this but not about the numbers" etc.
Even though it was obscure enough that Google couldn't easily find the paper that Opus remembered (I couldn't find it at all with the traditional search results; the AI summary found it on my third try), it actually didn't hallucinate it.
In fact I can't remember Opus 5.5 ever hallucinating. They've clearly done some work on that.
Obviously this local LLM is going to hallucinate all over you though.
Undoubtedly useful in the right contexts, particularly for local repair or homesteading, but “survival” does imply that this would be more useful in an SHTF situation.