I wouldn't expect companies not sharing their weights today to be any more likely to share them if they're on hardware, this doesn't sufficiently hide weights from a local user.
replyThe weights are very unlikely to be on the chip itself. That wouldn't work for SOTA models that are terabyte scale, even quantized. This is probably an accelerator for specific kernels in the model, but the weights are likely loaded from memory. The chip may have SRAM to store some of the weights temporarily during inference.
replyAt least in the case of Taalas the weights are physically encoded directly on the chip.
replyIt’s composed of 4-bit multiplier cells that compute all 16 possible results in parallel. The top metal wiring layer physically selects the one that corresponds to a multiplication with that cell’s constant weight, and routes it to the next layer.
Are you sure? Source? (does not seem to be https://taalas.com/the-path-to-ubiquitous-ai/ , for example)
replyI don’t get why this is an issue? You can run Claude/OpenAI SOTA models through Amazon bedrock. These weights have to live somewhere to run on Bedrock.
replysomewhere = an AWS data center with multiple layers of security and NDAs
replyThey won't sell/rent/license the weights to an end user at any price because they don't trust your security.
I work in embedded space. Just because it’s in hardware doesn’t mean you can’t “protect” it. Most modern software (regardless if it’s hardware or not) can be cryptophically signed.
reply[dead]
replyOne idea would be to use an open model.
reply