Author of LFortran here. The historical answer is that both LFortran and Flang started the same year, possibly the very same month (November 2017), and for a while we didn't know about each other. After that both teams looked at the other compiler and didn't think it could do what they wanted, so continued on their current endeavor. We tried to collaborate on several fronts, but it's hard in practice, because the compiler internals are different.

I can only talk about my own motivation to continue developing and delivering LFortran. Flang is great, but on its own I do not think it will be enough to fix Fortran. What I want as a user is a compiler that is fast to compile itself (under 30s for LFortran on my Apple M4, and even that is at least 10x too long for me, but we would need to switch from C++ to C, which we might later), that is very easy to contribute to, that can compile Fortran codes as fast as possible (LLVM is unfortunately the bottleneck here, so we are also developing a custom backend that does not use LLVM that is 10x faster), that has good runtime performance (LLVM is great here), that can be interactive (runs in Jupyter notebooks), that creates lean (small) binaries, that fully runs in the browser (both the compiler and the generated code), that has various extensions that users have been asking for, etc. The list is long.

Finally, I have not seen Fortran users complaining that there is more than one compiler. On the contrary, everybody seems very excited that they will soon have several independent high-quality open source compilers. I think it is essential for a healthy language ecosystem to have many good compilers.

For extra fun, "flang" actually refers to three different projects, one of which also completely rebuilt its frontend, so there are four separate "flang" frontends to LLVM. And I know of at least one proprietary LLVM-based Fortran compiler that's not based on any of the flags, and I suspect there are a few more lurking about.

The short answer as to why there are so many different LLVM Fortran frontends is that open source community took a long time to find someone willing to commit to a Fortran frontend to LLVM, and even when that happened, the initial results weren't particularly usable--it's not until I want to say late 2020 or early 2021 that there's an in-LLVM-tree flang project, and the exe isn't renamed from flang-new to flang until 2024.