This past week, I found another developer blog where a post caught my attention: Pythonic Perambulations. More specifically, it was this post regarding presenting data in an “xkcd” style. I figured I’d give this blog post some inbound link props because I found the content both useful and amusing. The author seems to have gotten quite a bit of well-deserved praise already in his blog’s comments section.
Readers of xkcd are often shown very interesting plots illustrating a humorous point or interesting concept. In any serious data analysis project, often there are many iterations of sifting and sorting through data until a some hypothesis is rejected or accepted. Rather than put a disclaimer to warn against drawing serious conclusions from a rough-draft grade presentation, I may just start xkcdifying things. If nothing else, xkcd-ifying plots may take the sting out of any data-driven negative implications. 🙂
I’ve been reading quite a bit about websockets lately. To test my understanding of the concepts involved, I was trying to connect an application to Mt.Gox in order to collect bitcoin-related pricing information. I was having some difficulty getting all of the parameters aligned properly with all of the various websocket libraries, so I went through the process of reading other people’s code on github, sifting through stackoverflow-style-forums, and experimenting until things worked correctly. I thought I’d document some of the minor hurdles here, just to save some people out there in the wild some effort.
I tried a few approaches:
- Python using autobahn
- C++ using libwebsockets (a C library)
The Python implementation worked perfectly. The only quirk was that I had to trick the application into inserting an “Origin:” field in the websocket handshake in order to get the application to connect. (This had something to do with something called the Web Origin Concept. It’s a safety mechanism for web browsers.) After tweaking the code to force an ‘Origin’ using the autobahn library, the handshake with the server ended up looking something like this:
GET /mtgox HTTP/1.1
After a handshake of this style, data starts streaming over the connection with no issue. With a working Python version, I had enough knowledge to craft an appropriate handshake in the C++ version of my bitcoin application.
With libwebsockets, it took some tinkering with the source code to get things to work right with Mt.Gox. The extensions field in the ‘lws_context_creation_info’ should not be filled in, and in the connection phase in libwebsocket_client_connect(), the specified protocol should be NULL. The origin and host strings fed into the libwebsocket_client_connect() function should mirror what you see above in the successful python handshake.
I didn’t bother to post code here, but the general idea to get things to start streaming correctly is to have a handshake that looks something like what I’ve copied above for the python implementation I used.
I hope this helps the random Googler out there searching for information.