I think I first saw Flash-MoE (https://github.com/danveloper/flash-moe) in April. Huge respect to them, it was a big inspiration for this project!