I've tried, casually, quite a bit of this. With generally poor results. Often this ends up similar what I have seen as classic RL problems with seeking reward in sparse environments. Unless you give the agent an a-priori goal outside the world it tends to not do anything interesting. Theoretically there ought to be ways around this such as herarchical approaches, but those tend to scale in complexity faster than my limited resources can keep up.

It's very neat though.

Edit: I really should watch the video before commenting. Looks like it goes into lots of these problems itself, and they got much further than me.