Start collecting data: the BitstampQuoteReader

img_20151220_145133.jpgThe QuoteReader implementations we need for a start, are luckily quite simple. They really do just one thing: loading information from an URL and returning it on request.

The most important data source for our project will be the Bitstamp API, so we will start with this.

It contains public functions that can be used without authentication and without an API key.  They have a throughput limitation in place though. If you send more then 600 requests in 10 minutes, your IP address will be banned.

For now, we just need one API function: ticker. The URL is

https://www.bitstamp.net/api/ticker/

When you open it in a web browser, you see, that the default response format is JSON. I have used org.json-20120521.jar for JSON parsing. Since the format in the ticker is fairly simple, any JSON java library will probably do the job.

We end up with a function that looks somewhat like this:

 

/**
* Read current data from the Bitstamp API and update fields.
*/
private void readNextAndUpdate() {
BufferedReader in = null;
StringBuffer sb = new StringBuffer();
try {
in = new BufferedReader(new InputStreamReader(
BITSTAMP_API_URL.openStream()));

String inputLine;
while ((inputLine = in.readLine()) != null)
sb.append(inputLine);
} catch (IOException e) {
log.severe(String.format(
"Error reading data from bitstamp api: %s",
e.getLocalizedMessage()));
// IO Exceptions will happen from time to time. If there is no systematic
// problem, the best way for us to deal with them is, to log and ignore
// them. As a consequence, the next time interval will be executed with
// outdated data. But this data is only a minute old, so it will not result
// in completely insane predictions.
return;
} finally {
try {
in.close();
} catch (IOException e) {
log.severe("Unable to close input stream from bitstamp api");
}
}
try {

JSONObject jo = new JSONObject(sb.toString());
currentQuote = jo.getDouble("last");
high24 = jo.getDouble("high");
low24 = jo.getDouble("low");
volume = jo.getDouble("volume");
bid = jo.getDouble("bid");
ask = jo.getDouble("ask");
vwap = jo.getDouble("vwap");
log.info("quote from remote service: " + sb + "\nquote-Time: "
+ new Date(1000 * jo.getLong("timestamp")));

} catch (Exception e) {
// catch-all because we don't have a surrounding framework (other
// then the
// JRE) to handle unexpected exceptions).
log.severe(e.getLocalizedMessage());
}
}

Note: the last trade price is not necessarily a good representation for the current value of a Bitcoin, because it can easily be be manipulated during low volume times. For the prediction, this is ok, because we have a neural network, which will either downgrade the importance of this value, if it turns out not to contribute to future prices, or it even extracts additional predictive power from recognizing the manipulation. Either way is fine for us.

A problem might arise from it though during trading. When you want to sell a Bitcoin at the exchange, and the last price is lower then the fair price would be, then you might be tempted to offer your Coin to a lower price then necessary.

When I have noticed this, my system ran stable for quite a while, so I refrained from changing it. But when you start from the scratch, you might want to keep this in mind.

For the sake of completeness, here is the rest of the class:

package de.hsec.datascience.btctrader;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.Date;
import java.util.logging.Logger;

import org.json.JSONObject;

/**
 * Reads a BTC quote from the Bitstamp REST api.
 * 
 * @author helmut hauschild
 *
 */
public class BitstampQuoteReader implements QuoteReader {
  /**
   * Constant API URL
   */
  private static final URL BITSTAMP_API_URL;

  // static initializer for API URL because we must check for
  // MalformatedURLException
  static {
    try {
      BITSTAMP_API_URL = new URL("https://www.bitstamp.net/api/ticker/");
    } catch (MalformedURLException e) {
      throw new RuntimeException(
          "Cannot initialize BitstampQuoteReader class.", e);
    }
  }

  /**
   * Logger
   */
  static Logger log = Logger.getLogger(BitstampQuoteReader.class.getName());

  // fields
  private double currentQuote;
  private double high24;
  private double low24;
  private double volume;
  private double bid;
  private double ask;
  private double vwap;

  /**
   * Accessor for current quote. Updates all fields, so it should be called
   * before reading all other fields. The textbook way to do this would be to
   * create a data object with all relevant fields and return that object. We
   * don't do that because we want to prevent object creation for performance
   * reasons.
   */
  public double getCurrentQuote() {
    readNextAndUpdate();
    return currentQuote;
  }

  //Other Accessors
  public double getBid() {
    return bid;
  }
  ...

  /**
   * Read current data from the Bitstamp API and update fields.
   */
  private void readNextAndUpdate() {
  ...
  }

  /**
   * Main method for a simple test run.
   */
  public final static void main(String[] args) {
    BitstampQuoteReader rqr = new BitstampQuoteReader();
    log.info("Next Quote: " + rqr.getCurrentQuote());
  }
}
Advertisements

One thought on “Start collecting data: the BitstampQuoteReader

  1. Pingback: Data Re-Coding 1 | notes on personal data science

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s