Deep Reinforcement Learning for Bitcoin trading

It’s been more than a year, since the last entry regarding automated Bitcoin trading has been published here. The series was supposed to cover a project, in which we have used deep learning to predict Bitcoin exchange rates for fun and profit.

We have developed the system in 2014 and operated it all through the year 2015. It has performed very well during the first 3 quarters of 2015, … and terribly during the last quarter. At the end of the year we have stopped it. Despite serious losses during the last three months, it can still be considered a solid overall success.

I have never finished the series, but recently we have deployed a new version, which includes some major changes, that hopefully will turn out to be improvements:

  • We use Reinforcement Learning, following DeepMind’s basic recipe (Deep Q-learning with Experience Replay) from the iconic Atari article in Nature magazine. This eliminates the separation of prediction and trading as distinct processes. The inference component directly creates a buy/sell decision instead of just a prediction. Furthermore the new approach eliminates the separation of training and production (after an initial training phase). The neural network is trained continuously on the trading machine. No more downtime is needed for re-training once a week, and no separate compute hardware is lying idle with nothing to do for the other six days of the week.
  • We use Deeplearning4J (DL4J) instead of Matlab code for the training of the neural network. DL4J is a Java framework for defining, training and executing machine learning models. It integrates nicely with the trading code, which is written in Java.

This will change the course of this blog. Instead of finishing the report on what we have done in 2014, I am now planning to write about the new system. It turns out, that most of the code we have looked at so far, is also in the new system, so we can just continue where we left off a year ago.

Advertisements

2 thoughts on “Deep Reinforcement Learning for Bitcoin trading

  1. Hello
    I have just started working on a similar algorithm with rl4j (but I may switch to another library because the documentation is almost non-existent …) and I am curious to know if you have continued to develop this project and if you already have results to communicate. And if you are open to discuss the technical details of this project or even to collaborate to make it a common project do not hesitate to contact me, it would be with pleasure 😉

    Like

    • Actually I have used DL4J (not RL4J) and built the reinforcement learning framework myself. I had a lot of trouble with the native part of DL4J crashing after a while on the host system. And lately I lacked time to work on it. Maybe in the meantime the problem can be solved just by a DL4J update… I’ll continue reporting as soon as I get back to it. The concept of a Java library for deep learning / reinforcement learing is still very attractive for me, so for now I am willing to jump any hoop necessary to get this into production. If you’d like to stay in contact, please DM me at https://t.me/hhauschild

      Like

Comments are closed.