upvote
> Going further backwards, the transistor (or a PNP junction) is the hardware level enabler of the IF statement. The action (switching) (..)

Back up a bit please! Analog computing is a thing. And it isn't even new - not by a long shot.

There are good reasons why practically all computing today is the digital kind. But electronic 'equivalents' of neural nets is one area where analog might make sense. Adding inputs can be as simple as a bunch of resistors + a transistor. Even on modern silicon nodes, that might be a more efficient setup than digital inputs, N-bit adders/multipliers etc. Not saying that's the case, and AI hardware should be based on analog circuitry. But it could be, and perhaps found to be practical.

reply
> But electronic 'equivalents' of neural nets is one area where analog might make sense.

That's an interesting idea, but could the weights be transferred to different hardware and still work? If not, that would be a significant limitation, even if it were preferable in some cases.

reply
An artificial neuron needs values to compare (the sum of weighted inputs). You can add values with transistors of course, but you need more than a dozen just to do simple addition. The activation function could be a simple binary comparison (e.g. between a weight and a threshold), but it’s usually more complicated.

Artificial neurons are significantly more complex that single transistors, and even a minimal hardwired circuit to implement just one neuron requires quite a number of transistors.

reply
Fundimentally, when you talk about a if statement, you are talking about the ability to do something different dependent upon some state.

It's the same thing as stimulus, response.

Unchanging in response to circumstances is static.

Changing in the absence of circumstances is randomness.

The conditional is all that remains. Changing in response to circumstances

(Arguably, unchanging in the absence of circumstances completes the truth table, but it's a whole lot of nothing)

reply
So, what if, we build a stack/set of transistors in same shape as a trained model? It would eliminate most of the software stack too and should run very fast. No memory/gpu required, the chip acts as both storage and processing device, purpose built to be physical model of a trained model.
reply
This is literally what talaas has done with chatjimmy.ai.

Try it, it's llama 3.1 8B at 16000 tokens per second.

chatjimmy.ai https://taalas.com/the-path-to-ubiquitous-ai/

reply
Wow that incredibly fast. I like this outcome more than centralized datacenters.
reply
But it can only run that model, so it will be outdated in a few years at best.
reply
There’s lots of things you can do in hardware that could be done in software but cost. FPGA should have solved this long ago, but apparently the guys who own the IP want to make it as hard as possible to use it …
reply
The alternative IF expression or statement, in the form introduced by John McCarthy in 1958 (which he used in early 1959 to define his version of AND, OR and NOT), is one of the possible primitives for computation.

There exist several equivalent sets of primitive operations. While the sets containing only NAND or only NOR, or both AND and NOT or both OR and NOT are more notorious, these logical operations are more abstract and they do not indicate precisely a hardware implementation, i.e. there are many distinct hardware methods to make such logical gates.

Other sets of primitive operations map directly to hardware devices, e.g. the sets of primitive operations composed of maximum and complement or of minimum and complement map directly to a hardware implementation using rectifier diodes and inverting amplifiers (which can be made with either semiconductor devices or with vacuum tubes, or also with pneumatic or hydraulic devices).

Other sets of primitive operations are obtained by replacing the maximum or minimum circuits with series or parallel connections of switches, like in the CMOS logic that is nowadays dominant.

The alternative IF expression corresponds in hardware to a 2-way multiplexer, which, together with the 2 constant functions "0" and "1" (a.k.a. "false" and "true" or "low voltage" and "high voltage"), is sufficient for a complete set of computational primitives.

Besides those mentioned above, the main remaining variant for a complete set of computational primitives consists of an analog (possibly weighted) adder and an analog comparator, which had been used in the so-called RTL circuits (resistor-transistor logic) and which also corresponds to perceptrons. RTL had been used in some early integrated circuits, before being replaced by DTL and TTL circuits (which are based on minimum and complement functions).

In hardware, e.g. in RTL circuits, a combined analog adder+comparator can be made with a single high-gain amplifying device, together with a set of weighting resistors and a bias resistor. RTL circuits can implement complex logic with fewer devices (e.g. they can implement a neural network in the analog domain), but they were replaced during the sixties of the past century with DTL, then TTL, because those were faster (in RTL, the resistors limit the charging currents for input capacitors and parasitic capacitors, which slows down the logical transitions) and the fact that they needed more devices was not important, due to the quick increase in circuit density.

reply
This reminds me of a book, whose title I forget, about creating a compact set of knowledge that will enable us to "rebuild civilization from scratch" in case of a future/fictional post-collapse scenario. I sometimes wonder, given the immense complexity and global supply-chain dependencies of our computing stack, whether we could bootstrap from first principles something of equivalent power and expressivity, but orders of magnitude simpler.

There's a wide variety of computational primitives, including lambda calculus, combinators, cellular automata, rewriting systems. Perhaps some are more practical to implement in hardware, particularly the kind of DIY electronics or analog machines that can also be put together from scratch. It might look like a whole building of mechanical switches, powered by a water wheel ("watermill"), for example.

reply
I think if civilization collapses they'll have other priorities, and very little benefit to get from the effort required for such simpler computing, for a good while...
reply
I suppose people will have more important things to work on, like growing food and fighting off roving bands of bandits, than building primitive analog calculators/computers.
reply
it not really an if statement here in a perceptron though. It is more akin a logic gate.

A transistor (driven to saturation) is a much better model.

reply
You might enjoy playing with Turing Tumble.
reply