Interesting comment: https://news.ycombinator.com/item?id=49264602
reply> IMO the much more likely explanation is that the brain uses contrastive learning somewhere. [...] Instead of using calculus and outer products to calculate derivatives, you feed real data and false data forward through the network and each neuron tracks and trains on the difference.
Separately, do I understand the idea correctly:
reply- when learning on the real examples, just propagate forward
- when learning on the fake examples, first generate them by propagating white noise backward
That's a technique that was tried in machine learning and worked (although obviously not optimally or we'd all be using it by now) and then we hypothesised that animal brains use it because it would explain sleep.
replyI'm curious if there is an explanation for it not being more common to sleep like dolphins do, small parts of the brain at a time.
reply