That sounds like the way. Everyone trains their own small problems to maximally compressed weights and then merges.