Place

Log and analyze the pixel stream from /r/place.

GitHub

Description

I wrote this to log all pixels from /r/place. At the time, I thought I could do some cool stuff with all the data, so I wrote this tiny scraper to get data from the WebSocket connection and log it to a sqlite3 database. However, I did not really expect the massive popularity of /r/place - in retrospect, sqlite didn't work out, and couldn't keep up with the load.

Even though the data was very incomplete, I was able to make some cool visualizations with it, like this heatmap on Tableau Public.

In addition, the export_png.js script can generate PNGs from the /r/place canvas at any time it has scraped, so I made a timelapse of the data I had:

Project Readme

place

Log pixels on /r/place

Setup

git clone https://github.com/wgoodall01/place.git
cd place
npm install
npm install -g pm2
pm2 start --name "place" index.js

It also works without pm2, but there are some fun metrics you would miss out on.

What it does

  1. Make a sqlite3 database with some tables and stuff
  2. Connect to the /r/place websocket, log everything to the place table
  3. Request the place bitmap every 60s, write that as a BLOB to the bitmap table

Other stuff

Run ./export_csv.sh, and assuming you have sqlite3 in your system path it will dump the whole place table into a export-${timestamp}.csv