May 12

C++, WebSockets, and Mt.Gox

I’ve been reading quite a bit about websockets lately. To test my understanding of the concepts involved, I was trying to connect an application to Mt.Gox in order to collect bitcoin-related pricing information. I was having some difficulty getting all of the parameters aligned properly with all of the various websocket libraries, so I went through the process of reading other people’s code on github, sifting through stackoverflow-style-forums, and experimenting until things worked correctly. I thought I’d document some of the minor hurdles here, just to save some people out there in the wild some effort.

I tried a few approaches:

  • Python using autobahn
  • C++ using libwebsockets (a C library)

The Python implementation worked perfectly.  The only quirk was that I had to trick the application into inserting an “Origin:” field in the websocket handshake in order to get the application to connect.  (This had something to do with something called the Web Origin Concept.  It’s a safety mechanism for web browsers.)  After tweaking the code to force an ‘Origin’ using the autobahn library, the handshake with the server ended up looking something like this:

GET /mtgox HTTP/1.1
User-Agent: disease
Origin: websocket.mtgox.com:80
Host: websocket.mtgox.com:80
Upgrade: WebSocket
Connection: Upgrade
Pragma: no-cache
Cache-Control: no-cache
Sec-WebSocket-Key: JI2o83NARHJDps1XCN7eCQ==
Sec-WebSocket-Version: 13

After a handshake of this style, data starts streaming over the connection with no issue. With a working Python version, I had enough knowledge to craft an appropriate handshake in the C++ version of my bitcoin application.

With libwebsockets, it took some tinkering with the source code to get things to work right with Mt.Gox. The extensions field in the ‘lws_context_creation_info’ should not be filled in, and in the connection phase in libwebsocket_client_connect(), the specified protocol should be NULL. The origin and host strings fed into the libwebsocket_client_connect() function should mirror what you see above in the successful python handshake.

I didn’t bother to post code here, but the general idea to get things to start streaming correctly is to have a handshake that looks something like what I’ve copied above for the python implementation I used.

I hope this helps the random Googler out there searching for information.

Apr 15

jQuery, Websockets, and Bitcoin Time and Sales

If you haven’t seen Clark Moody’s Mt.Gox bitcoin market data website yet, you should.  It’s a really beautiful presentation of realtime Mt.Gox BitCoin orderbook depth, the trade tape, and charts at various time frames.

I was so impressed by Clark’s presentation of market information that I wanted to dig into how modern web sites present information from real-time data sources.  I don’t typically work on web applications, so I had to figure out what was involved.  There were three pieces of key knowledge:  1) Basic javascript, 2) knowing how to use a websocket API, and 3) using CSS and jquery to figure out how to manipulate documents in a web browser.  (I had very minimal knowledge of all three.  Fortunately, Javascript is easy to get a handle on for someone with previous programming experience.)

I first went to look at the Mt.Gox API web page, where I found some simple explanations on how to connect to their streaming market data.  After using a dozen console.log() statements to see what I was looking at, I finally tweaked my Javascript code to manipulate table rows in a tbody styled by CSS.  I ended up with this time and sales viewer.  (Please note that when loading the viewer, it will take a few seconds to establish a connection to the exchange.  Once the connection occurs, you should be able to see current trades happening in realtime.  Note that I’ve only tried the viewer in Chrome and Firefox.  I have no idea what browsers it works and doesn’t work in, as the project was just an experiment.)

The mini-project was a great deal of fun — websockets are very interesting.  I was impressed by how much could be accomplished with so little work.