Is it fair to define Reinforcement Learning (RL) as forcing a gradient onto a system / simulator?
replyThough, I suppose RL has non-gradient based methods too.
Orders-of-magnitude improvements in compute efficiency are needed to become a practical replacement for backprop… but those improvements are coming.
replyAs a mixture, could activation-space search produce useful teaching targets for backprop?
Zeroth-order search would discover candidates, first-order learning would consolidate them. The potentially valuable step is converting a sparse judgment into a reusable training target.
This also changes the relevance of convexity.
I'm not sure I follow why your argument changes the relevance of convexity? If the problems were convex, I don't think we'd be having this discussion -- first-order methods would tend to win.
replydeleted
reply