upvote

points

by jbjbjbjb3 hours ago |
comments
upvoteby nomel2 hours ago|
next
[-]
I would claim that LLMs desperately need proprietary code in their training, before we see any big gains in quality.

There's some incredible source available code out there. Statistically, I think there's a LOT more not so great source available code out there, because the majority of output of seasoned/high skill developers is proprietary.

To me, a surprising portion of Claude 4.5 output definitely looks like student homework answers, because I think that's closer to the mean of the code population.

reply
upvoteby bearjaws31 minutes ago|
parent|
next
[-]
I will say many closed source repos are probably equally as poor as open source ones.

Even worse in many cases because they are so over engineered nobody understands how they work.

reply
upvoteby bhadass2 hours ago|
parent|
prev|
next
[-]
yeah, but isn't the whole point of claude code to get people to provide preference data/telemetry data to anthropic (unless you opt out?). same w/ other providers.

i'm guessing most of the gains we've seen recently are post training rather than pretraining.

reply
upvoteby nomel1 hours ago|
parent|
[-]
Yes, but you have the problem that a good portion of that is going to be AI generated.

But, I naively assume most orgs would opt out. I know some orgs have a proxy in place that will prevent certain proprietary code from passing through!

This makes me curious if, in the allow case, Anthropic is recording generated output, to maybe down-weight it if it's seen in the training data (or something similar)?

reply
upvoteby typ39 minutes ago|
parent|
prev|
next
[-]
I'd bet, on average, the quality of proprietary code is worse than open-source code. There have been decades of accumulated slop generated by human agents with wildly varied skill levels, all vibe-coded by ruthless, incompetent corporate bosses.
reply
upvoteby Take843553 seconds ago|
parent|
next
[-]
Not to mention, a team member is (surprise!) fired or let go, and no knowledge transfer exists. Womp, womp.
reply
upvoteby Manouchehri15 minutes ago|
parent|
prev|
[-]
There's only very niche fields where closed-source code quality is often better than open-source code.

Exploits and HFT are the two examples I can think of. Both are usually closed source because of the financial incentives.

reply
upvoteby andai2 hours ago|
parent|
prev|
[-]
Let's start with the source code for the Flash IDE :)
reply
upvoteby wvenable2 hours ago|
prev|
next
[-]
This is cool and actually demonstrates real utility. Using AI to take something that already exists and create it for a different library / framework / platform is cool. I'm sure there's a lot of training data in there for just this case.

But I wonder how it would fare given a language specification for a non-existent non-trivial language and build a compiler for that instead?

reply
upvoteby nmstoker1 hours ago|
parent|
[-]
If you come up with a realistic language spec and wait maybe six months, by then it'll probably be approach being cheap enough that you could test the scenario yourself!
reply
upvoteby nlawalker1 hours ago|
prev|
next
[-]
I see that as the point that all this is proving - most people, most of the time, are essentially reinventing the wheel at some scope and scale or another, so we’d all benefit from being able to find and copy each others’ homework more efficiently.
reply
upvoteby luke54412 hours ago|
prev|
next
[-]
It looks like a much more progressed/complete version of https://github.com/kidoz/smdc-toolchain/tree/master/crates/s... . But that one is only a month old. So a bit confused there. Maybe that was also created via LLM?
reply
upvoteby madmax9112 hours ago|
parent|
[-]
[dead]
reply
upvoteby computerex40 minutes ago|
prev|
next
[-]
And the goal post shifts.
reply
upvoteby kreelman1 hours ago|
prev|
[-]
..A small thing, but it won't compile the RISCV version of hello.c if the source isn't installed on the machine it's running on.

It is standing on the shoulders of giants (all of the compilers of the past, built into it's training data... and the recent learnings about getting these agents to break up tasks) to get itself going. Still fairly impressive.

On a side-quest, I wonder where Anthropic is getting there power from. The whole energy debacle in the US at the moment probably means it made some CO2 in the process. Would be hard to avoid?

reply