As long as you can express everything you need on the library's terms. As soon as you write a Python loop, your performance plummets.

Only if you doing something thousands of people has done before. Anything new, even very simple and you are on your own and Python is 100x slower than naive C implementation on many tasks.

Last little project I remember is writing a solver for a puzzle game my friend published. Python just doesn't work at all for such tasks.

I think you are wrong about speed of those libraries as well. In my experience naive code designed for a specific task beats highly sophisticated general code and it doesn't take a rocket scientist to get huge speed-ups over some well established fast library.