I was recently working on an AI harness too, but I wasn't using AI to code it. It's really easy and requires little code. I wasn't even using langchain - that would require even less code.

UI is harder for sure, but it's not that bad. You need to think though where to catch which exceptions.

LLMs might opt for langchain which has had multiple breaking changes after the knowledge cutoff, making it hard for the LLM to work with it. This is probably going to lead to the LLM having to make many changes to it's code, making it messy and leading to further code being less maintainable.

Yeah, I've had similar experiences, also starting out with dynamic languages and migrating to Rust. If the LLM will write a lot of the code for me, why not choose something (1) super fast, and (2) which has types I can use to understand and specify the code I want without having to read all the output?

I've been trying out Lean for related reasons, to good effect. It's really interesting there since it can crank out proofs that would've been completely infeasible for a dedicated team of PhDs before, whereas I haven't seen any LLM projects written in Python that I couldn't have slung out in a few months myself. I personally think it's a lot more interesting to focus on the new things you can now do with LLMs that weren't possible before, as opposed to doing the same old stuff at moderately higher velocity.