They can remove all of the literal code matches from the codebase, but if it was written by having an LLM study the original code and then write something similar, it's hard to argue it's not a derivative work.
There is a lot of legal history of doing new designs with influence from reverse engineered sources. The basic rule is that you can't have someone familiar with the original code involved with writing the new code.
LLMs are a new area, but I would expect that feeding the original source into an LLM context and saying "Write this, but different" wouldn't qualify as a clean-room design. There isn't enough general removal from the original subject matter.
The difference is that if humans do it, there's plausible deniability that it is derived.
I mean, even when humans do it, we still do clean room engineering.
It goes further though: LLMs are just a tool, and tools don't get rights, humans do.
Humans having the right to learn from a few dozen books and projects is not the same as a tool that infers output using all copyrighted works know to man.