This is exactly what subinterpreters are for! Basically isolated copies of Python in the same process.
replyhttps://docs.python.org/3/library/concurrent.interpreters.ht...
If you want a higher-level interface, there is InterpreterPoolExecutor:
https://docs.python.org/3/library/concurrent.futures.html#co...
How would this really help python though? This doesn't solve the difficult problem, which is that python objects don't support parallel access by multiple threads/processes, no? Concurrent threads, yes, but only one thread can be operating on a python object at a time (I'm simplifying here for brevity).
replyThere are already means of passing around bulk data with zero copy characteristics in python, but there's a lot of bureaucracy around it. A true solution must work with the GIL (or remove it altogether), no?
I'm not familiar with CPython GC internals, but I there there are mechanisms for Python objects to be safely handed to C,C++ libraries and used there in parallel? Perhaps one could implement a handoff mechanism that uses those same mechanisms? Interesting idea!
reply