This wasn't obvious a year ago, but today CAD literally reduces to Simon Wilson's pelican test, since CAD is largely a matter of functional CSG, and CSG is really not that different from SVG. It's just one more dimension, which it turns out is not a problem.
LLMs consistently one-shot CSG based video game levels with interesting physics puzzles (citing myself). Given this I'm willing to conclude that the frontier models are good at automated CAD if given the correct harness. But I guess a lot of people don't know this yet.
Show me the blog posts where people talk about the results they got "vibe coding" and those arXiv papers look great in comparison!
There is the insidious thing with LLMs is that they can get the general shape of something right but that thing will not be useful if the last 1% is wrong. It might be that the operator sees the problem and fixes it, but it may also be that the LLM hypnotizes the operator into not seeing the errors and gaps.
I know there are many sorts of problems where I've had good experiences with LLMs but I know other people have had bad experiences and some of it might be my skill but some of it is just plain luck.
Neck: https://pastebin.com/Sg3LmmUq Body: https://pastebin.com/FE9nikYB
edit: screenshot, too https://i.imgur.com/FZGyyVO.png
Injecting a natural language layer into the workflow is just not optimal. CAD itself is not a difficult tool to learn and use effectively. There are essentially no layers of abstraction that an LLM can assist in cutting through, and no obfuscated rules or languages to learn.
I think of it this way. If there was someone sitting at my computer, and I had to do all of my CAD design by explaining what I wanted them to do verbally, I'd rip out my hair.
LLMs are doing for programmers what virtual CAD did to the drafter 35+ years ago, optimizing the effort expanded to create the thing already in your brain
Modifying the prompt and then trying it again did not lead to that self-verification loop and the output was unusable garbage.