> Google is now on their 8th generation TPU
replyRemarkable that the TPU pre-dates the attention paper. Was a solid bet on energy efficient dense matrix multiplication and has stood the test of time.
Yes, they'd been using machine learning internally since c.2011 with their DistBelief system, which was then replaced with TensorFlow in 2015.
reply