The author dismissed that option saying "I value that docker build can have a clean environment every time", so this is self-inflicted.

Many docker users develop on arm64-darwin and deploy to x86_64 (g)libc, so I don't think that'll work generally.