ReactJS 3d Map DataViz with DeckGL: Flights of Berlin

Background

Back in December 2017 I saw a visualisation for flights from London made with Deck.GL - and I thought I'd like to see that for Berlin. For the impatient you can just open it here, https://makersden.io/berlin-flights

You can also just watch the video below:

Raw Flight Data​

First step was to find flight data, and OpenSky had me covered. It contained one state vector (position, direction and velocity) for every second of every flight on the planet Earth. You can find sample data here: https://opensky-network.org/datasets/states/

​One Avro file contained 1 hour of transponder recording from all commercial flights, and was on average a hundred megabytes. I wanted to analyze 7 hours (from 06am to 1pm), so well over half a gigabyte. Not something I would want to load in a webapp, so I had to massage it.​

Kneading the Data​

I needed a decoder for the Avro files. The path of least resistence was using Java libraries, which I used to generate Java classes and a deserializer from the Avro schema. With this I could filter & massage the data, and finally output as static json to be consumed by the frontend.​

Ladies and Gentlemen, we're about to land in Berlin​

I only wanted routes to and from Berlin, which meant that the flights had to have a point on their route within a few km of one of the Berlin airports, with a low enough altitude (a few meters/feet) so we don't get fly-overs.​

Reducing accuracy​

The accuracy of the positions was way more than what was needed for my visualisation, so I reduced each longitude and latitude to five decimals.​

Terminate glitchy data

​Turns out transponders sending out raw data can have wild anomalies in the data. To remove glitchy artifacts from routes I did some simple smoothing out of points. If a point deviated too wildly to their 4 closest neighbours I adjusted the longitude, latitude and altitude to be some sort of interpolation between its closest neighbours.​

Encoding for the frontend​

I encoded it all into a json array of flights, each represented by an array of longitude/latitude/altitude points.

[
  ["flightCode1",  2.12345, 8.56789, 1200,   2.12340, 8.56792, 1220,   2.1233, 8.56795, 1240],
  ["flightCode2",  2.12345, 8.56789, 1200,   2.12340, 8.56792, 1220,   2.1233, 8.56795, 1240]
]

​With this format the flight data was at about 2.8MB. After gzip, 1.1MB.

Not great but good enough for a demo.​

Visualizing with Deck.GL​

For visualisation I was already set on Deck.GL which is a React & WebGL powered visualisation library for large data sets.​

This is basically the easy part because Deck.GL does what we want out of the box. E.g. create a layer for the flights:

const layers = [
  new LineLayer({
    id: "flight-layer",
    data: flightData,
    getSourcePosition: (d) => d[1],
    getTargetPosition: (d) => d[2],
    getColor,
  }),
];

..and declare your DeckGL component and configure it as you wish:

<DeckGL layers={layers}>
  <ReactMapGL
    mapboxApiAccessToken={mapBoxToken}
    mapStyle="mapbox://styles/kallebertell/cjbhj191scvc02rmxgjbgtzju"
  />
</DeckGL>

Dark Mode Map Tiles

To make the map tiles extra cool I created a custom dark map tile theme on Mapbox.

Color me Badd

​It's rather boring with uniformly colored lines, so I recommend transitioning color based on altitude. Experimentation will get you where you want.

const groundColor = { red: 0, green: 255, blue: 255 };
const cruiseColor = { red: 0, green: 128, blue: 0 };

const getColor = (data: any): RGBAColor => {
  const altitudeInFeet = data[1][2];
  // scale is a 0 -> 1 float.
  const scale = Math.min(1, altitudeInFeet / 10000);

  const red = groundColor.red + scale * (cruiseColor.red - groundColor.red);
  const green =
    groundColor.green + scale * (cruiseColor.green - groundColor.green);
  const blue =
    groundColor.blue + scale * (cruiseColor.blue - groundColor.green);
  const alpha = 128 * (1 - scale / 1.5);

  return [red, green, blue, alpha];
};

Add a story​

To accompany the visualisation I wrote up some factoids about the Berlin airports and made a stepper/carousel to do animated fly-throughs between them.

We’ve just hit our cruising altitude of 11,000 feet.

You can experience the end result here: https://makersden.io/berlin-flights

On behalf of your cockpit and cabin crews, please, sit back and enjoy your trip.