Rory Fitzpatrick: Visualising the Edinburgh Festival

Published: February 13th 2013by Orde Saunders

Rory Fitzpatrick (@roryf) was speaking about Visualising the Edinburgh Festival using geotagged information on Twitter and Flickr at TechMeetup, these are my notes from his talk.

A story of Edinburgh during the festival.

Visualisations are available at: vimeo.com/thinkclear

Set it running recording geotagged tweets, Flickr photos and Gowala(?) data. Ended up with 1.3 million data points. Visualised as a night time view of Edinburgh.

Technology

Main component is Node.js
Twitter streaming API filtered on a bounding box of Edinburgh
CouchDB data store - five views to bring the data down to a manageable level (6.2GB ~2.8m rows)
PubSubHubbub
Flickr API
dotCloud - multi technology PAS
Iris Couch/Coludant

Node and the Twitter streaming API is really easy to get up and running. Gets a pain when running 24/7 on a remote connection as Twitter will silently drop connections. Custom code to handle rate limiting - restarted process every two hours.

setTimeout (function() {
  process.exit(1);
}, 2 * 60 * 60 * 1000);

Uses processing.org to produce images and videos from Java.

Twitter and Flickr data differ in concentrations at different locations.