You can talk to a bunch of designers who will say the opposite. Claude Design Studio generated this garbage UI, that I fixed manually, but it created great code j never could have that made it work.
findfantasyxviii.com
These systems should allow rapid iteration on discovery and thinking. One can now make a prototype a day that would have taken a week. That means that we should be able to converge on a much better design in the same amount of time it would have taken to make a v0 that turns how to have systemic flaws.
AI should scale our understanding of systems, not just shovel out half baked features and apps.
Where I’m at when building personal applications for my home / life is: does the code execute and perform the desired task?
If so, what do I care how shitty it is? I’m not publishing these projects (for the most part… I have one joke application up at songshift.reachnick.co) so efficient, clean, secure code are not really a priority for me.
Well when you put it that way ... monetizing the Dunning-Kruger effect does actually sound like a very good business idea.
https://en.wikipedia.org/wiki/Michael_Crichton#:~:text=%5B14...
People are never perfectly even in intelligence across all possible disciplines.
Gell-Mann's observation was a sincere and thoughtful caution about the way we transmit information about complicated ideas. Crichton's "amnesia effect" is an excuse to ignore media you dislike.
You're suggesting that (a) their UI skills are lacking (based on what? isn't UI exactly what they were iterating on and trying to improve?), and (b) that a real UI expert would've somehow felt the UI they were working on was consistently garbage, despite how many times they iterate on it?
Which means you're saying you don't believe anyone can actually produce high quality (to an expert) output with AI on the same target they're working on, and if they think they are, that just means they don't have a good sense of quality?
the llm produced something the operator thought was garbage for the design too, and the operator iterated it from garbage to good.
they could also have the llm iterate the underlying code from garbage to good, if they wanted.
most likely a specialist would say its neither good nor bad, since its not considering the right things, and hasnt collected the right useability feedback, but making straightforward designs isnt that hard, and counting clicks and interactions, and avoiding hidden functionality is all measureable stuff
It's only confusing because you don't know the field. Which is kind of the point.
Tell me about it… I was forced to use a program called Farmer’s Wife for a time. What a fucking nightmare of a UX.
AI pixel art looks particularly bad because most users don’t even go through the effort of downscaling and then upscaling it using something as simple as nearest-neighbor scaling, which by itself will squash out a lot of high-frequency noise that manifests in the form of terrible looking "fringing". Proper grid alignment also makes a big difference. It’s not perfect by a long shot, but it helps.
basically the AI-slop version of food, yet still they thrive
In the Tailwind thread the other day I was explicitly told that the intended experience of many frameworks is "write-only code" so maybe this is just the way of the future that we have to learn to embrace. Don't worry how it's all hooked up, if it works it works and if it stops working tell the AI to fix it.
It's kind of liberating I guess. I'm not sure if I've reached AI nirvana on accepting this yet, but I do think that moment is close.
There is an enormous untapped market for crappy low-effort apps which previously weren't worth the time - but with the effort so low put together a simple dashboard or one-off tool it becomes much more attractive.
Which is probably why so many random buttons in microsoft/apple/spotify just stop working once you get off the beaten path or load the app in some state which is slightly off base
The number of edge cases in a software is not fixed at all. One of the largest markers of competence in software development is being able to keep them at minimum, and LLMs tend to make that number higher than humanely possible.
The people pushing AI _over_ humans never thought they were. They just don't care about 'good' or 'bad', only 'time-to-market'. A bad app making money is better than a good one that isn't deployed yet. And who cares about anything past the end of the quarter? That's the next guy's problem.
In terms of "junior dev following" it would be the model trying to think and write it as a Senior or Staff Level engineer would.
Humorously, this could be the result of LLMs vacuuming up all the sentiment on the web that the code that LLMs produce is trash-tier.
Maybe a more idealized training set could improve things, but at least for today’s SOTA, you have to get the shitty first draft out and then improve it.
Harnessing makes a difference, but it’s only shuffling around when and where the tokens get generated. It can trade being slower by doing a hidden first draft and only showing the output after doing a self review. But the models still need to generate it all explicitly.
I did an experiment on this a few weekends ago and Codex for example was a lot more adversarial and thorough in its review when given Claude-authored code compared to when given the same code with "I wrote this, can you review it?"
It's not the default, because the training data is full of unmaintainable code done wrong with mistakes. People literally complain that LLMs write too many tests or add comments.
If instead of "do it right", you give it specific actionable advice of how to right code, it does surprisingly well. Newer frontier models also do a great job of mimicking the style and rigor of the surrounding codebase without prompting, if you're working in an established codebase, for better or worse.
You never wrote quick exploratory code? One off scripts? How is the Ai suppsed to know unless you tell it.
If you tell another person to write some code, how are they suppsed to know? If you have your boss come to you and ask you to write some code to do some data analysis are you going to spend weeks writing units tests and perfect abstractions? Or do it quick and get the data and result?
For example, I built up a programming language from scratch with Claude, it knows nuances about my languages syntax, and can write code in my language effectively. I did it mostly as a test. It definitely helped that my language is heavily mostly Python based.
Granted, the load bearing thing here is whether we’re actually getting good at rebuilding up to any sort of standard of quality. Or if the tooling is even structurally capable of doing that rather than just introducing new baskets of problems with each build.
At the moment, we understand the basic tech, could reasonably DIY, but choose not to knowing full well there's a mess of understandable code somewhere we could go clean up but dont want to. We accept fast iterations because we know roughly the shape of how it "should be" and can guide an automated framework towards that. This is especially true on our own projects or something we built originally! Stark/Iron man knew/moved, the suit assisted by adding momentum.
We're riding our "knowledge momentum".
If companies can hold out long enough, that knowledge completely fades, and the tool is all you have. At that point, they are locked in. Then it's not Iron man, it's an Iron lung (couldn't resist!)
I love the Iron lung reference. Perfect.
Prototypes are practically free now. You can ask the AI try each architectural or stylistic option and just see which code you like better.
To your point, another interesting note is that rewriting and rearchitecting are also very good.
One pattern I like is to vibe code a set of solutions, pick the approach, then backfill tests and do major refactors to make it maintainable.
Here the skill is knowing what good architecture looks like, and knowing how to prompt and validate (eg what level of tests will speed up the feedback cycle or enable me to make the LLM’s changes legible).
To be fair the “ready, fire, aim” approach of rapid prototyping has been known for a long time, but you need to be quite quick at coding in old world for it to work well IMO.
- first I've created a skill how the architecture of the system should look like
- I'll tell the LLM to follow the guidelines; it will not do that 100%, but it will be good enough
- I'll go through what it produced, align to the template; if I like something (either I've not thought about the problem in that way, or simply forgot) I add that to the skill template
- rinse and repeat
This is not only for architecture of the system, but also when (and how to) write backend, frontend, e2e tests, docs. I know what I want to achieve = I know how the code should be organized and how it should work, I know how tests should be written. LLMs allow me to eliminate the tediousness of following the same template every time. Without these guardrails it switches patterns so often, creating unmaintainable crap
Bear in mind - the output requires constant supervision = LLM will touch something I told it not to touch, or not follow what I told it to do. The amount of the output can also sometimes be overwhelming (so, peer review is still needed), but at this point I can iterate over what LLM produces with it, with another LLM, then give to a human if it together makes sense
It doesn't really make sense to suggest AI can work on something any make it now and work correctly, and at the same time say it's unmaintainable. It is maintainable with AI.
The real question is whether or not you're happy to ship AI-generated code that you can't modify to production unless you use AI. Few developers are there yet, plenty of non-tech people are there already. I don't know which group is actually wrong.
I'm not a designer either, but I've been around designers long enough to recognize when something is bad but just not know what is needed to make it better/good. I've taken time to find sites that are designed well and then recreated them by hand coding the html/css to the point that I consider myself pretty decent at css now. I don't need libraries or frameworks. My css/html is so much lighter than what's found in those frameworks as well. I still would not call myself a designer, but pages look like they were designed by a mediocre designer rather than an engineer :shrug:
AI is great at creating slop that almost works.
But, my god, it is terrible at following clear as day instructions on how to cleanup slop.
It wrote 150k lines of code that almost works in 2 months. It's taken 1 month to delete about 2000 lines of broken architecture and fix it, and it still hasn't gotten it done, despite nonstop repeated efforts to do something not that hard.
I definitely could've fixed it less time then I've spent prompting at this point (but no way I'd have gotten the other 150k lines). But doing it myself is not the point. It's to see if it can actually scale.
The answer is yes... But my god is it agonizing.
The creating garbage part that almost works is fun.
The inevitable cleanup is not.
And unfortunately I don't see this aspect materially improving in the short term.
If you want it to code you something about 5-10k lines of code that's already been done 1000 times before or only slightly different, it's great.
Most people want more than that.
Which I think is perfectly worthy of exploration. Some people want to check in the prompts. Or even better, check in a plan.md or evenest betterest: some set of very well-defined specifications.
I'm not sure what the answer will be. Probably some mix of things. But today it is absolutely imperative that the code I write for the case I wrote it in is good quality and can be maintained by more than just me.
It's tempting to move out a layer and try making prompts and plan.md the "source code", and then the generated actual-source-code becomes just another ephemeral form of "intermediate representation" in the toolchain while building the final executable product. But then how are you versioning the toolchain and maintaining any reasonable sense of "stability" (in terms of features/bugs/etc) in the final output?
Example: last week, someone ran our "LLM inputs" source code through AgentCo SuperModel-7-39b, and produced a product output that users loved and it seemed to work well. Next week, management asks for a new feature. The "developer" adds the new feature to the prompting with a few trial iterations, but the resulting new product now has 339 new subtle bugs in areas that were working fine in last week's build owing the fact that, in the meantime, AgentCo has tweaked some weights in SuperModel-7-39b under the hood because of some concern about CSAM results or whatever and this had subtle unrelated effects. Or better yet: next month, management has learned that OtherCo MegaModel-42.7c seems to be the new hotness and tells everyone to switch models. Re-building from our "source" with the new model fixes 72 known bugs filed by users, fixes another 337 bugs nobody had even noticed yet, and causes 111 new bugs to be created that are yet-unknown.
If you treat the output source code as a write-only messy artifact, and you don't have stable, repeatable models, and don't treat model updates/changes as carefully as switching compiler vendors and build environments, this kind of methodology can only lead to chaos.
And don't even get me started on the parallel excuses of "Your specifications should be more-perfect" (perfection is impossible), or "An expansive testsuite should catch and correct all new bugs" (also impossible. testing is only as good as the imperfect specification, and then layers in its own finite capabilities to boot).
I never tried spec driven development for myself, but if I review other's MRs I am typically exhausted after the first 10 lines.
And there are hundreds of lines, nearly always with major inaccuracies.
For myself I always found the plan mode to work well. Once the implementation is done, the code is the source of truth. If it works, it works.
When I want to add more functionality or change it, I just tell the agent what I want changed.
I doubt walls of semi-accurate existing specs are going to be beneficial there, but maybe my work differs from yours.
I've been using LLMs daily and I spun up a few spec driven flows once or twice but like the person above I think the code is the source of truth.
Also why wouldn't you use TDD to enforce the 'spec' then?
I mitigate this by few things: 1. Checkpoints every few days to thoroughly review and flag issues. Asking the LLM to impersonate (Linus Torvalds is my favorite) yields different results. 2. Frequent refactors. LLMs don't get discouraged from throwing things out like humans do. So I ask for a refactor when enough stuff accumulates. 3. Use verbose, typed languages. C# on the backend, TypeScript on the frontend.
Does it produce quality code? Locally yes, architecturally I don't know - it works so far, I guess. Anyway, my alternative is not to make this software I'm writing better but not making it at all for the lack of time, so even if it's subpar it still brings business value.
I suppose you could solve that in two ways. Manually rewrite it as you did. Or formalize an architecture and let the AI rewrite it with that in mind. I suspect that either works.
the power comes from creating the machine you can steer. Treat AI like an over eager college intern who you need to hand hold, but do tasks.
I gave up on this recently. It achieved the goal now, and in a year or two, when you actually want to add whatever feature, the SOTA AI will probably be able to clean it up as it does so. What does "maintain" even mean anymore?
If you don't agree, how many years into the future do we need until you would agree?
And people have been saying this exact thing for years now. Someone said this very thing two years ago. And we're still at the "maintenance dead end" stage. So let me flip it back on you: how many years are we going to pour an obscene amount of resources into this thing that is always going to be able to clean up its own messes "in a year or two" before we realize its a dead end (at best) and we need to be using those resources elsewhere? And, similarly, what happens to you when the SOTA AI in two years can't clean up the code it wrote for you two years ago, but people are depending on it and your still on the hook for maintaining it?
Respectfully, I asked first. ;)
> before we realize its a dead end (at best)
You've declared the future, which doesn't leave much room for a conversation. So, cheers!
I feel what you write, but then again: every now and then i write small greasmonkey scripts to remove annoyances from webpages, and to do so i have to look at the html and the kind of trash you describe is already there.
This is talk and talk is cheap. Prove it, otherwise it's still a million dollar question... unanswered.
HN is notoriously mentally deficient when it comes to AI. They were wrong about self driving cars (I sit in AI cars daily), they were wrong about AI getting used for coding (I don't use an IDE or type code anymore as a SWE). So I have to say unless there's something evidence based or substantial here it's likely given HN track record that most people here will end up being another wrong, baseless and over confident answer.
I'm looking for legit answers not confidently biased statements with no evidence.
Iron Man created Jarvis whose capabilities are way beyond any models in the near future. So it wasn’t an Iron Man moment.
(And on a personal note, I'm glad we don't have a publicly released Jarvis before we get our act together about the use.)
> Iron Man created Jarvis whose capabilities are way beyond any models in the near future. So it wasn’t an Iron Man moment.
Like an LLM, you misunderstood the context. The voyeuristic experience doesn't require fiction to be reality.
I keep asking myself the same questions, and the conclusion I keep coming to is the clean modeled structure we want to see is for humans to maintain and extend, but the AI doesn't need this.
There's definitely an efficiency angle here where it's faster for AI to go from a clean modeled solution to the desired solution because it's likely been trained on cleaner code. Is this really going to matter though?
The best argument I can come up with is the clean modeled solution is better for existing development tools because it's less likely to get confused by the patch work of vibes throughout the code; but this feels like it ultimately becomes an efficiency concern as well.
This just might be the new reality, and we need to stop looking behind the curtain and accept what the wizard presents us.
This does not match my experience. I do a lot of AI-assisted coding at this point, and what I've seen is that when the AI is asked to extend or modify existing code, it does a much better job on clean, well-structured and well-abstracted code.
I think the reason is simple, and tracks for humans as well: well-structured code is simply easier to understand and reason about, and takes a smaller amount of working-set memory. Even as LLMs get better with coding, I expect that they would converge on the same conclusion, namely that good structure + good abstractions make for code that is more efficient to work with.
I think it’s all about the structure you use to work in and how you use the model. We are shipping better, more human friendly code, with less bugs, then we ever did before and doing it at 1/10 the cost before LLMs.
But we are definitely not vibe coding, and the key seems to be devs with years of experience managing teams, managing the LLM instead. Basically you create the same kind of formal specifications, conventions, and documentation that you would develop for a project with two or three teams, then use that to keep the project on the rails recursively looping back through the docs as you go along. I’ve only had to back out of a couple of issues over the last year, and even though that cost a couple of hours, it was still extremely cheap.
Meanwhile we are shipping at 4x speed with 1/4 the labor, and the code is better than it was because the “overhead” of writing maintainable, self documented code has inverted into the secret ingredient to shipping bug free code at unprecedented speed.
If you just explain the standards to which you want the code written, use a strict style guide, have a separate process that ensures test coverage (not in the same context) you can get example quality code all the way through. Turns out that’s also in the training data.
So yeah, it’s imminent. Let’s see how demand shifts in response in the future.
I think we eventually end up at the tool approach via vendors providing the tools to other companies, but it still feels like there's a long road ahead to get there.
That's not true. The LLM performance will degrade as the codebase gets messier as well. You get to a point where every fix breaks something else and you can't really make forward progress.
Yes, you might be able to get a bit further with a messy codebase just because the LLM won't complain and will just grind through fixing things, but eventually it will just start disabling failing tests instead of actually fixing things.
Of course that just leads to: what’s the best way to achieve that goal? Through elegant code or adding lots of tests? Which is a debate from long before LLMs existed.
LLMs have a limit to how deep they can understand and refactor architectural issues.
That limit is far, far lower than a human's.
This is how societies become shittier. People who are ostensibly responsible for doing their jobs not giving a damn about quality.