Deep Reinforcement Learning: Curiosity driven Super Mario

I have used Deep Reinforcement Learning with Curiosity-driven Exploration (see to train an agent playing Super Mario in the OpenAI GYM for Nintendo NES emulator. The untrained Mario is obviously the one on the left side. The input data for the agent are the raw pixels. The environmental rewards (i.e. the value which the agent tries to maximize) is the game score.
I ran the training in a Docker container based on the latest pytorch/pytorch image with some adaptations for the graphics output. My starting point was the example source code from the MEAP book “Deep Reinforcement Learning in Action” by Alexander Zai and Brandon Brown, which I highly recommend. See and click on “Source Code”. The training took less than an hour on a 4 core i5 @2,9 GHz, 16 GB memory, NO GPU involved. It is a little scary to realize, how far one can get with relatively modest computational resources in such a short training time.