Technically usable but with bad/broken code. I found 3 different bugs with 1 feature, found a duplicate feature (their vibe coding missed the fact that the feature was already implemented), and the docs were wrong. Other features were ridiculously badly implemented. Reported them all, submitted multiple changes. None were accepted. Their repo was a hellscape of AI-generated issues and AI-generated PRs; I think mine was the only one written by a human. This was a month and a half ago.
Google is one of the most valuable corporations in the world, yet even they shipped a turd of an app to real customers and can't even take a bug fix. I think AI coding might be cooked.
One simple example is you can use @ to reference filenames - but the file list is cached and never updates. Ask Gemini to split a file into two files, then type @ and the new files will never appear. Those kind of extremely basic bugs.
But hey, the text has gradient colours...
Then a few weeks back, I gave it another try and I was pleasantly surprised.
It was insanely good!
A colleague and I have been on-and-off trying to build a C++ binary against specific Google libraries for months without success. Then, Gemini CLI was able to build the binary after 2-3 days iterating and refining prompts
Even with pro, I have caught it going off the rails a few times. The most frustrating was when I asked it to do translations, and it decided there were too many to do so it wrote a python script that ran locally and used some terrible library to do literal translations, and some of them were downright offensive and sexual in nature. For translations though, Gemini is the best but you have to have it do a sentence or two at a time. If you provide the context around the text, it really knocks it out of the park
note that it will sometimes fall back to flash 2, which sucks
Pro is expensive, but good. However they've decreased the pitiful stipend they used to include in even the ultra plan to the point were it's barely usable. I pivoted back to ChatGPT Pro after the recent downgrade they gave Ultra users. Googles Ultra plan cost 2.5x as much and delivers about half the usage.
Thanks for the laugh. :)
But last month I picked it up again and it has crushed everything I've thrown at it. As Codex limits tighten on the Plus plan it's been my main fallback and doesn't even feel like a downgrade when I switch over. Haven't hit a single loop so far using it nearly every day for several weeks so that problem seems solved finally, thank god.
I've been using it in the auto router mode and haven't felt the need to manually lock in the bigger model yet. It's incredibly snappy which I realized I really appreciate vs. waiting around endlessly for minutes each turn, but I've read other people's experiences needing to manually select the Pro model so YMMV.
I do not use super broad prompts, though. None of this "build me a webapp" stuff. It's more like, "adjust this part of this class to do Y instead of X."