Using regular Claude code I can generate 4k lines an hour. Using some bash I can generate 4k lines of code per second.
replyLines of code per hour is a bad metric.
Can it solve a problem using production quality code that doesn’t take four times as long to review? That sounds like something I would pay $100 for.
I think the point is that we are given no real idea at all, not even some ballpark figures, about how much actual money actually needs to be spent in order to run Gas Town. The original article had some handwavy stuff about how it cost An Awful Lot, but no actual numbers.
replyCost per line of code is not an amazing metric, but at least it's an attempt to come up with a figure.
(I would also be interested to find out how much it costs to run...)
Well, yes ideally we would eventually also have metrics about error rates and reject rates. Like ideally at some point someone could do a study of "for every 100 PRs Gas Town generates, how many are accepted after code review and how many are rejected" or "for every 100 lines of code Gas Town generates, how many coding errors are detected by human reviewers".
replyUnfortunately I think things are moving so fast that by the time such a study was done, we would already be on to newer models and newer versions of gas town.