This is Apple's bet, among others.
replyTraining purpose-specific miniature models lets you have a lot of tasks you can run with high confidence on consumer hardware.
Or on a commodity EC2 instance with a relatively cheap inference sidecar.
replyTraining purpose-specific miniature models lets you have a lot of tasks you can run with high confidence on consumer hardware.