Just over a week after the Toronto budget mini-hackathon, The Toronto Star launched their Lobby Watch mini-site, featuring some great “interactives.” The timing is uncanny: we were working with the same data at the hackathon, and had similar outputs in mind. The work was done by Marc Ellison (@marceellison, marcellison.com), and has received a great response on Twitter. A few observations:
- The infographics they posted use static data, which they explain is current as of July 20th. (Which suggests that Marc was working on this at the same time that we were!) It’s unclear if they intend to regularly update the data.
- In the great write-up about the project, they also wrote that they used a custom scraper written in Python to extract the data. On Twitter, Marc explained that the XML data available through the City’s open data portal is hard to parse (which we also discovered at the hackathon), and is out of date. I haven’t looked closely enough at the data to see how out-of-date it is, but the information page about it, it’s suggested that the data is current (“Real time refresh”). It’s very curious why two interfaces to the same database would return different data.
- The Star has shared an 8MB Excel file containing the cleaned-up data they used — this is really great, and suggests that the data should be offered in this format (or equivalent, like CSV) on the City’s Open Data portal.
- The Star also provide an interface to query the data by councillor or lobbyist.
- In (at least) Chrome, some of the pages are wonky, and many are slow. It turns out this is because many of the features are actually hosted on Marc’s personal website, and iframe-d in to The Star’s site. This seems disappointingly sloppy on the Star’s part. [As I write this, Marc’s site — and the Star’s lobbyist data query interface — are down entirely.]
- The graphs look slick, and were created with a range of slick tools: one with d3, one d3+slickGrid, and one Tableau. They’re a bit more slick than legible, but that’s rather forgivable since the data’s also been shared.
Congrats, Marc — great work!
