But OK, let's assume that: The power is out, but you have a generator with so much fuel you can run a desktop just fine; Your neighborhood will somehow make a mesh network; Your neighbors need some already stored information and the best solution for that is texting a chatbot rather than a survival/emergency handbook or Wikipedia; Your mesh-network will also be good enough to match the time-sensitivity of the questions.
Under those assumption, which of these sounds better?
1. Buying an "LLM-in-a-box for emergency supply kits", which you deploy so that your neighbors can ask questions (text over the mesh) of the offline chatbot.
2. Buying a satellite internet transciever for your emergency supply kit, so that your neighbors can ask questions of a much better chatbot and communicate with human experts, their worried relatives, and coordinate with rescue/relief efforts...
I’m only out the cost of the drive, which is like $40 and doesn’t require anybody on the other side cooperate with me.
- - -
More broadly…
You call it unlikely mixes, but we see it all the time:
- people already have a computer for gaming or work
- people (ie, “preppers” like we’re discussing) buy a generator for emergencies
- local emergency response sets up mesh networking during disasters, both official and unofficial
Have you ever tried to use a handbook you’re not intimately familiar with during an emergency? It’s rough.
For personal preparedness, nothing replaces familiarity and practice — eg, weekend survival trips and reading your manual ahead of time.
But for providing information in a random lookup manner to unpracticed people who weren’t prepared? Yes, I think an LLM/chatbot is the practical way to operationalize all that information which you stored (eg, survival guides or machine manuals).
Also, it’s unlikely a general purpose chatbot would be superior at survival advice to one specialized for that purpose — and indeed, is likely to refuse your questions as “unsafe” or “criminal”.
At current prices you are also out about $4k for a Spark to actually run the inference on, if you want a full LLM in a low-power package.
In general, I'm not sure why one would want to pin your survival to an expensive, hallucination-prone data source, when an offline copy of wikipedia with a little vector search attached to a Raspberry Pi can fulfil the same role...
Sounds like the absolute worst time to rely on a crappy little model that will inevitably hallucinate.