I figure that the factory is exactly the adversary in the whole threat model? So why give them the keys to the castle so they can moonlight their own genuine batch?
You probably aren't going to do this whole rigmarole for small runs of devices like in the OP.
Serious question: why?
A local-only device with no telemetry can probably skip some of those