upvote
They're probably embarrasing. Circuit design does not in the least work like code design.
reply
This is a very verifiable task with well defined metrics and verifiable results. Chip designers are going to have just a rough time of it as software engineers.

It's just so obvious that with access to a timing and power simulator they will be able to do this just as well, as let's say opus 5.5 is at blackwell optimization. Which is to say, effectively superhuman.

reply
> very verifiable task with well defined metrics

It's really not though. Both AMD and Intel have learnt bitter lessons about trusting simulators too much. Chips and antenna design are hard because it's a chaotic system, small changes in placement can whack the whole interference pattern.

reply
Again the vast major of performance/power issues are very much unveiled in simulators on digital designs. Yes you can have gaps, like di/dt issues in the power tree or whatever. Chips are full of inefficient RTL that is below the bar of where engineers should spend their time. LLMs are really good at that stuff.
reply
Good ASIC design is not like FPGA design. For FPGA design I think your analysis is spot on. But FPGA design is configuring someone else's ASIC. ASIC design is circuit design, even if you're describing the circuit with RTL, it's still circuit design. To differentiate that ASIC you need to transform the net lists into circuits that are actually interesting. If you just use the defaults you're unlikely to sell a single chip as your performance will almost certainly underwhelm. In think in most ASICs I've worked on I have more TCL code driving the tools to make them do what I want them to do than I have RTL. RTL is shockingly efficient at describing data flow.
reply
How do you think the architects that optimize the performance per watt of designs are working? they simulate at the circuit level. It's all stuff that llms can be fantastic at. There is a well defined goal, and metrics for being better that are very accurate. The only question is will anyone invest enough money building the datasets to be good at this. Given the hype around RSI, the answer is .. yes.

anyone who thinks llms can't get good at this is engaging in wishful thinking

reply
I agree with this, and I think most of the posters here aren’t following.

LLMs are very good at RTL (all I can consider is that the parallel posters arguing otherwise haven’t tried a modern one against significant RTL?). It’s easy to verify, the optimization constraints are relatively straightforward, and there’s adequate simulation to build a good gym for it.

LLMs are not good at layout and floor planning and because there is near zero public data available, I’m not sure how well they’ll iterate. For PCB design they’re improving because while it’s a similarly thorny problem, the physical constraints (trace properties on a given substrate) are highly public and can be simulated and turned into an RL gym. For ASIC, any public PDK is 10-20 years old so making an RL gym becomes outrageously difficult on top of the fundamental problem (highly multivariate/layered spatial optimization) being hard to solve.

reply
Sure, but it's not like humans are particularly good at this aspect either. I assume a lot of human-created designs need to be reworked once they become a physical reality
reply
...Until they encounter reality in the field. Chip designs are convoluted because of strange effects no one or no AI could realistically predict.
reply
> task with well defined metrics and verifiable results.

You have not had the joy of being introduced to the dark arts of Analog/RF chip design.

reply
I'm not really talking about analog design here.
reply
>verifiable results

"mean normalized reward" hahaha

reply
By “rough time of it” do you mean chip designers are going to be inundated with slop that “works” but is overly complex, slowly degrades the system over time, and which nobody understands and this can fix? Or was it that they’d have to speak to insufferable AI agents who pretend to do things they don’t and also write in the most obtuse language possible?
reply
Is the management of hardware companies much better than that of software companies? Because yeah your description seems accurate. Which points to the question of “why would software developers sign up for this willingly.” They largely don’t, the extent to which the system seems obviously stupid on the ground is the extent to which it is mandated from above.
reply
In many ways, I think so.

The big difference is in hardware it's much easier to measure cost/bug and days_of_schedule_slip/bug. Because we have those metrics, and people trust them, we can much more easily justify decisions that software teams never make. Like spending big on verification.

Why do you think we try to push everything into software. You guys are free, right? :)

reply
I think these designers will be forced to deal with this slop, when they all collectively realize llms can produce more efficient designs then they could before the project tape out date.

People who work on high performance code in software had a similar attitude, but the problem is that they cannot compete with the ability of llms at this point.

reply
I've heard this too, all the IC designers I know think there's some fundamental training missing that prevents the models from creating highly optimized circuits, which is what's needed for most tasks of interest.
reply
Am I missing something because DEC apparently cracked this via expert systems in the 80s?

> SID (Synthesis of Integral Design) was a logic synthesis program used to generate logic gates for the VAX 9000. From high-level behavioral and register-transfer level sources, approximately 93% of the CPU scalar and vector units, over 700,000 gates, were synthesized.

> SID was an artificial intelligence rule-based system and expert system with over 1000 hand-written rules. In addition to logic gate creation, SID took the design to the wiring level, allocating loads to nets and providing parameters for place and route CAD tools. As the program ran, it generated and expanded its own rule-base to 384,000 low-level rules.

> Initially it was somewhat controversial but was accepted in order to reduce the overall VAX 9000 project budget. Some engineers refused to use it. Others compared their own gate-level designs to those created by SID, eventually accepting SID for the gate-level design job. Since SID rules were written by expert logic designers and with input from the best designers on the team, excellent results were achieved. As the project progressed and new rules were written, SID-generated results became equal to or better than manual results for both area and timing. For example, SID produced a 64-bit adder that was faster than the manually-designed one. Manually-designed areas averaged 1 bug per 200 gates, whereas SID-generated logic averaged 1 bug per 20,000 gates. After finding a bug, SID rules were corrected, resulting in 0 bugs on subsequent runs.

https://en.wikipedia.org/wiki/VAX_9000#SID_Scalar_and_Vector...

reply
That sort of logic synthesis program has plenty of equivalent tools in use today, though I'm not sure if any expert systems are still used. CAD is essential in all phases.

But that's not really relevant at all to asking Claude to fix your counter, or figure out the logic and layout for a certain goal, or whatever.

reply
How is chip design more complicated than Navier-Stokes?
reply
I personally think part of what makes chip design harder is a good amount of the training data is proprietary, and of training data you can find in public, over half of it is poorly designed or only acceptable for FPGA deployment not ASIC.

So you have to figure out what training data is acceptable and also make deals to get access to proprietary data.

reply
That is probably what partnering with synopsis is effectively solving.
reply
I've posted a very similar research back in April (https://github.com/FeSens/auto-arch-tournament), did not make as much noise as openAI of course
reply
It's not that it is harder so much as it is that the training set is incredibly small.

Mathematics is generally done in the open. SOTA chip design is done by a handful of people all behind closed private doors.

reply
deleted
reply
With proprietary synopsis software.
reply
Navier-Stokes is ultimately a textual problem rather than reasoning about shapes in the physical world like laying out gates. There are, e.g., mazes small enough that LLMs can solve them but generally they're much worse at stuff like that than humans.
reply
Navier Stokes is about modeling fluid flow through space. It’s not any more textual than chip design. Just like you can reduce fluid flow to textual partial differential equations, you can reduce chip design to purely textual constraint systems.
reply
A lot trickier to do self supervised RL on chip design than Navier Stokes, or maybe they aren't trying hard enough. I mean even if you check their outputs with Altium and KiCad DRCs it won't capture everything that goes into a good board. Still need human training data which is slow and expensive.

I suspect before the end of next year they will have enough data to bootstrap a production level self supervised learning environment.

I wonder if they do training verification with real pick and place circuits too.

reply
This is simply a case of they haven't gotten to it yet. They definitely will be able to do RTL optimization.
reply
do you understand how large the training sets are for LLMs? there is not enough RTL in the world (let alone public RTL) to usefully contribute to the training of a O(T) model. and contrary to popular belief, HDLs are nothing like programming languages so nothing transfers.
reply
This isn't done using a transformer, they probably doing something else .
reply
If they can figure out a good classifier for good vs bad circuits then that should be enough to bootstrap a robust model. It's just extremely extremely expensive getting good EEs to do this sort of data annotation for them.
reply
you're just saying the same thing in different words (classifiers are trained on data)
reply
Not necessarily, for electronics it could be done using a shit ton of heuristics. It won't be a good classifier, but for the purposes of bootstrapping ML you can do it step by step until you get a Good Enough classifier.
reply
did you miss the part where they partnered with synopsis? you think they might have some reference RTL?
reply
reminder: LLMs aren't actually intelligent, they just have trillions of parameters fit on a training set.
reply
Give me definition of intelligence that excludes LLMs but includes below-average intelligence person.
reply
Give me any rigorous definition of human level intelligence.

So far it doesn't exist.

reply
did you read this thread? the answer is "ability to write RTL without reading/training on 20T RTL tokens".
reply
All it takes is to offer some retired IC designers a ton of cash to enjoy their retirement more and provide some solid training data and verify some outputs every week.
reply
'some solid training data' -> an example or two would be nice. Otherwise 'some' might turn out to be 'quite a lot more than expected'.
reply
But there are encoded heuristics in the design software for chips/IC, or formalizable constraints, are there not?
reply
If you want to make what is basically a "Hello World" chip, then yes. But if you want to make a chip that is not essentially identical to your competitors, then no.
reply
This is pre synthesis logic. It’s a lot like code and indeed Berkeley Boom is designed in Chisel which is based on Scala.
reply
Personally, I dont care how it looks as long as it works, and works well, and lasts long.
reply