Minions is an internal enterprise system already in use at Apple. It pools idle Mac GPUs into a shared compute cluster for ML inference.
replyThere is no distributed execution of a single LLM across multiple GPUs using tensor parallelism here. Instead, it relies on task-level parallelism-breaking down a large workload into independent subtasks that run across separate machines. For example, generating embeddings for a million documents.
This pattern has been implemented across hundreds if not thousands of different organizations (and between organizations). I have a patent on data distribution strategies from my time at Google running a large (>1M cores 24/7 for 3 years) idle cycle harvesting project.
replyit's a good idea but you often end up with huge tail latencies, slight incompatibilities due to fleet heterogeneity, and a database/scheduler management problem.
My university ran its render farm across idle workstations using the screensaver back in early 2000. Same concept.
replydeleted
replyAnd Apple has had simple distributed video encoding since 2005, so yes it is not new in computing, but it is still useful for the gap before a given task can fit comfortably on one machine
replyPsst, it is "innovative". /s
replyIt is if it’s GPU compute.
replyDistributed GPU compute has existed for quite some time,
replyhttps://developer.nvidia.com/gpudirect
And this is because I can't be bothered to dive into older papers, so recent NVIDIA stuff will do.
So basically works the same way as Folding@Home?
replyEmbedding@Work
replyOnly on the same local network I guess.
replyor distributed.net
reply