Rory Fitzpatrick: Visualising the Edinburgh Festival

Orde Saunders' avatarPublished: by Orde Saunders

Rory Fitzpatrick (@roryf) was speaking about Visualising the Edinburgh Festival using geotagged information on Twitter and Flickr at TechMeetup, these are my notes from his talk.

A story of Edinburgh during the festival.

Visualisations are available at: vimeo.com/thinkclear

Set it running recording geotagged tweets, Flickr photos and Gowala(?) data.  Ended up with 1.3 million data points.  Visualised as a night time view of Edinburgh.

Technology

  • Main component is Node.js
  • Twitter streaming API filtered on a bounding box of Edinburgh
  • CouchDB data store - five views to bring the data down to a manageable level (6.2GB ~2.8m rows)
  • PubSubHubbub
  • Flickr API
  • dotCloud - multi technology PAS
  • Iris Couch/Coludant

Node and the Twitter streaming API is really easy to get up and running.  Gets a pain when running 24/7 on a remote connection as Twitter will silently drop connections.  Custom code to handle rate limiting - restarted process every two hours.

setTimeout (function() {
  process.exit(1);
}, 2 * 60 * 60 * 1000);

Uses processing.org to produce images and videos from Java. 

Twitter and Flickr data differ in concentrations at different locations.