upvote
Not an expert, I consult with medium size firms that so far have seen nice things from the AI. However, the lightly-informed C-Suite hears about the hacks and the data threats on CNBC/Bloomberg which has made them cautious. Even with their cursory understanding they are saying "can we look at these open models". N=2
reply
Isn't that the problem themselves, in that those issues only exist if you go for open models, but outsource inference to third parties? It seems to me that there are only two safe options: either go full local and do inference on-prem/in your own cloud, or just deal with the big labs directly. Everything between these two extremes involves third parties with sketchy reputations.
reply
I think there's definitely a market, on prem feels expensive and intimidating for many no matter what it actually costs or takes
reply
"On prem" here just means renting a server with a GPU. (Possibly "in the cloud".)
reply
So even more expensive.
reply
Multiple companies are already advertising TEE-based private inference.
reply