Python is often used as (very useful!) glue for calling CUDA, C, C++, Fortran, etc… codes.

So, if you are thinking about the sort of “business logic” that’s often Python, but the performance comes from the parts that are usually not.

Yes, but the numerical backend of the python libraries are written in lower-level languages. For example, pytorch uses a C++ backend.

You heard correctly. This discussion is about C++ though.