It seems implementing transformers is the new "hello world" of AI assisted coding.
Zero surprise it did it
Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...
(unfortunately for a decent model it will also be price competitive with a big nvidia card)