Why three.js? Your LLM will do fine without it and be able to optimize further for your particular use case without being stopped by three.js architecture that may or may not fit your project.
Ask your LLM, "Can we remove three.js so that we can re-architect the graphics system to be more optimized for our specific use case?" and see what it comes back with.