upvote
I think that's what they are trying to avoid. If you need on-device intelligence, their pitch was "The model the device already has is best", and if you need something more specific an adapter (aka, a fine-tune/lora) is best.

They were wrong when their on-device model was way behind. They still might be right in the long term.

While multiple app I use might need Gemma 4 E4B, I use dozens of apps and app devs can choose from hundreds of models. A shared cache might reduce size a little when there's overlap, but the core problem still exists. If each app chooses a model disk and memory-swapping explode.

Its probably be better for device manufacturers to bake in a default. I'm not proposing they limit you from using others, but one shared default might be best developer/user experience for 99% of apps.

- Being warm in memory is the single biggest perf speedup you can get, and a default is much more likely to be warm.

- "Best model" is usually "best model for this device" given both RAM and compute. A developer can't test every device but Apple can/will.

- Each model needs to be optimized for the hardware (what's running on ANE, what's running on Metal, what's running on CPU). The default gets optimized.

- If you need custom model, a Lora is probably best (30MB, benefits from all of the above)

You could say the default should be swappable, but that's more a linux ideal than an Apple one so I doubt we ever see that. Plus there are real downsides: intentional or not, prompts end up optimized to the model they are developed for, so swapping the default system model would degrade every app.

reply
That's a great opportunity for Apple to provide a universal unique model ID protocol and some shared storage space to allow devs to register models.
reply
Check out “Bring an LLM provider to the Foundation Models framework” - https://developer.apple.com/videos/play/wwdc2026/339
reply
I see an id based ability suggesting `modelId`. but in current docs I cannot find any context to it. The other limit is that it suggests Swift Packages. but I'm not seeing any model management hints similar to Docker/Ollama/etc where:

- Application can ask for specific model, if available use it. if not, ask to download it (or try some fallback / alternative)

- User can manage models. So as a user I can clean unused models (and for non-techie have something similar to offloading apps when unused for some period of time).

reply
Do you guys not have phones (with at least 1TB of storage)?
reply
Who’s “you guys” a developer from Bay Area? A student with a MacBook Neo? Or John Appleseed who bought basic iPhone 17e?
reply
I have a Mac with 4TB of storage but it’s still annoying when every new AI app I try installs its own virtual environment with a fresh copy of Python, PyTorch, other duplicate libraries, and then models on top of that.
reply
As an occasional python user I'm always amazed and frustrated that it seems that the only way to be able to use/build anything is to create a whole separate environment.

And now given everybody now does this I guess the incentive to stop breaking stuff reduces even further.

Might as well have static binaries.

reply
The meme phrase “it’s fractally wrong” applies to the entire python ecosystem, IMHO. Virtual environments are just another layer of this fractal wrongness in the layer cake of ecosystem awfulness.

It’s a nice language though.

reply
I have a couple small apps that have a (non-LLM) model, and originally the models and code were in PyTorch, built by Python devs.

The original plan was to ship Python. However I found out I can migrate them to CoreML, and now it's a model file + Swift code. I got some massive performance improvements as well.

Of course, this doesn't work at all for non-Mac environments, but it was nice to be able to do it. (Also doesn't solve the duplicate large models problem)

reply
No? iPhones don't come standard with that much storage.
reply
The apps can use the system provided on-device model using the same framework and APIs; but there's no affordances to deduplicate custom models between apps.
reply
Sounds ripe for block-level deduplication. :D Or an API that lets you request a model and handles caching.
reply