So the full solution would be models trained in an open verifiable way and running locally.
You can trigger the the service's ToS violation or worse, get tipped off to law enforcement for something you didn't even write.
In HN circles perhaps. Average Joes don’t care.
anthropic, google, openai etc, decided that their consumer ai plans would not be private. partly to collect training data, the other half to employ moderators to review user activity for safety.
we trust that human moderators will not review and flag our icloud docs, onedrive or gmail, or aggregate such documents into training data for llms. it became the norm that an llm is somehow not private. it became a norm that you can't opt out of training, even on paid plans (see meta and google); or if you can opt out of training, you can't opt out of moderation.
cloud models with a zero retention privacy policy are private enough for almost everyone, the subscriptions, google search, ai search engines are either 'buying' your digital life or covering themselves for legal reasons.
you can and should have private cloud services, and if legal agreement is not enough, cryptographic attestation is already used in compute, with AWS nitro enclaves and other providers.
I personally think everyone should default to using local resources. Cloud resources should only be used for expansion and be relatively bursty rather than the default.
As an enthusiastic reader of books like Privacy is Power and Surveillance Capitalism, it feels good to have a private tool that is ready at hand.
I saw a service named Phala, which claims to be actually no-knowledge to server side (I think). It was significantly more expensive, but interesting to see it's out there. My thought was escaping the data-collection-hungry consumer models was a big win.
That's two halves of "why", sure.
Another interesting half would be that those companies have US military officers on their boards, and LLMs are the ultimate voluntary data collection platform, even better trojan horses than smartphones.
Yet another "half" could be how much enterprise value might be found by datamining for a minute or two... may I suggest reading a couple of Martha Wells books.
cryptographic confirmation of zero knowledge: yes.
the latter, based on trust in the hardware manufacturer and their root ca. so, encrypted if you trust intel/nvidia to sign it.
there are a few services, phala, tinfoil, near ai, redpill is an aggregator of those
if you are happy with off-prem then the llm is ok too, if you need on-prem this is when you will need local.
The private thing is the prompt.
But also, a local LLM opens up the possibility of agentic workflows that don't have to touch the Internet.