upvote
Purely anecdotal, but my experience has been that LLMs generate low quality python code. It's spaghetti code on par with what I've seen when companies I worked at tried offshoring development. It's basically what you get when you give bad or incomplete specs to a team of inexperienced programmers with poor development practices. It's usually good in small chunks, but it gets extremely sloppy as the scope of work increases and more decisions are introduced. Interestingly, I've seen LLMs generate good clojure code.

My guess is it comes down the the training data more than anything else, although I suspect functional languages will fare a little better. At least that's been my experience. There's undoubtedly a ton of python code in the training corpus and portions of it are of dubious quality. Niche functional languages likely have a smaller training corpus where a larger portion of it is better quality.

reply
I think it's far from being this simple. What you're describing is productivity on a greenfield project, but what's really interesting is productivity when working on an existing large codebase, with existing conventions, architecture decisions (or lack of)... How easy is it to do a product pivot, to rearchitect for performance, etc etc etc.
reply
It is unclear to me though how much of your expectation might be set by the training dataset.

For example, Python and Typescript have the most amount of codebases and training being done on. So I feel as if that plays a part into the overall thing.

Languages which are more niche have genuinely hard times (Try arturo lang for example), so it depends on a lot of things/nuance, or well that has been my experience trying something recently.

My personal opinion is that if each language has the same amount of training. Golang comes close but the first might be Elixir. I have seen Elixir language perform really well with LLM's with magnitudes less training dataset. There have been some studies which had Elixir as the number one language for such tests iirc.

Gleam is a new addition as well and I feel as if it could be good and its another interesting option as well with more type-safety and an interesting language overall.

reply
I've been in elixir for nigh a decade now and the one thing that you can try to pry from my cold dead hands is the BEAM. Elixir and Gleam are my go to languages right now and damn are they fun to write and reason in, but the part that has left me never wanting to leave the ecosystem is BEAM + OTP.

- BEAM makes monoliths sexy. You don't have to worry about a bunch of microservices, just focus on using proper process division for modeling your problem. - Debugging on the BEAM is first class. Drop into an interactive shell, pull up telemetry, or recon and hammer down on where your live app is slowing down if your metrics have a blindspot.

I could go on and on. I'm constantly blown away every day by the amount of time and effort and all of the sage learnings in distributed computing problems that came out of Ericsson that became the foundation of erlang + OTP + BEAM and in turn elixir + Gleam.

reply
Never tried Elixir myself but came here to say the same thing. Tencent put out this study showing that Elixir seems to reign supreme: https://autocodebench.github.io/
reply
How do you propose to quantify the terms "elegant" and "easy to reason about"? What unit of measurement do you use for these?

This sounds like your personal feelings, not quantification.

reply
deleted
reply