I started going down the path of building a ripple carry adder already (which seems to work fine). Then I was going to try for a full on ALU, then some sort of ISA that sits on top of it all.
I have no idea what the end result will look like if it all comes together. Hopefully I'll find some weird primitives along the way. :D
It's very hand-wavy, but I'm kinda hoping I can somehow have a machine manually constructed out of neurons that can naturally interact with one built with looser hebbian learning rules.