Visualize world data automagically

Several years ago I started a project writing a plug-in for D3.js to combine the map projections with data markers. That project became the earth plug-in (which you can get in my d3-plugins repo – https://github.com/hrobertking/d3-plugins). Although it's been fully functional for a while, I've recently made a few changes to it that enable it to go a little further, so I thought this would be a good opportunity to write a (hopefully) brief post to explain a little better what it can do.

For this post, I'm going to describe hypothetical scenarios that will almost certainly never occur in real life, but scenarios that, hopefully, will be similar to real-life scenarios you can imagine. Also, since I have been a strong advocate – dare I say 'evangelist' – of Progressive Enhancement and a few other UI engineering topics for many years, this post will have a specific structure and make a few assumptions without going too deeply into the case for Progressive Enhancement or other topics, such as accessibility. Even though those topics are important and seldom, even in my own blog posts, get the attention they deserve, they're not the focus here.

That all being said, let's start with the task at hand – developing an interesting page that shows data points scattered across the globe. For our example, we'll use the world's busiest airports...but it could as easily be something else, like sales data or web traffic.

The most descriptive thing we can do with the data is to put it in a table, which also happens to be the best approach for accessibility issues. So we have a table something like this...

HTML for our example

<table class="earth-viz" id="busiest-airports">   <caption>10 of the world's busiest airports</caption>   <thead>     <tr>       <th>Country</th>       <th>City</th>       <th>Activity</th>       <th>Name</th>       <th>Latitude</th>       <th>Longitude</th>       <th>Size</th>     </tr>   </thead>   <tbody>     <tr>       <td>US</td>       <td>ATL</td>       <td>68343</td>       <td>Hartsfield Jackson Atlanta International</td>       <td>33.636719</td>       <td>-84.428067</td>       <td>16</td>     </tr>     <tr>       <td>US</td>       <td>ORD</td>       <td>59692</td>       <td>Chicago O'Hare International</td>       <td>41.978603</td>       <td>-87.904842</td>       <td>14</td>     </tr>     <tr>       <td>US</td>       <td>DFW</td>       <td>56496</td>       <td>Dallas Fort Worth International</td>       <td>32.896828</td>       <td>-97.037997</td>       <td>12</td>     </tr>     <tr>       <td>US</td>       <td>LAX</td>       <td>51396</td>       <td>Los Angeles International</td>       <td>33.942536</td>       <td>-118.408075</td>       <td>10</td>     </tr>     <tr>       <td>CN</td>       <td>PEK</td>       <td>48226</td>       <td>Capital International</td>       <td>40.080111</td>       <td>116.584556</td>       <td>10</td>     </tr>     <tr>       <td>US</td>       <td>CLT</td>       <td>44583</td>       <td>Charlotte Douglas International</td>       <td>35.214</td>       <td>-80.943139</td>       <td>8</td>     </tr>     <tr>       <td>US</td>       <td>DEN</td>       <td>44438</td>       <td>Denver International</td>       <td>39.861656</td>       <td>-104.673178</td>       <td>8</td>     </tr>     <tr>       <td>US</td>       <td>LAS</td>       <td>41164</td>       <td>McCarran International</td>       <td>36.080056</td>       <td>-115.15225</td>       <td>6</td>     </tr>     <tr>       <td>US</td>       <td>IAH</td>       <td>39808</td>       <td>George Bush Intercontinental</td>       <td>29.984433</td>       <td>-95.341442</td>       <td>6</td>     </tr>     <tr>       <td>GB</td>       <td>LHR</td>       <td>37680</td>       <td>London Heathrow</td>       <td>51.4775</td>       <td>-0.461389</td>       <td>4</td>     </tr>   </tbody> </table>

...and we can add all the standard accessibility information to describe it and we can make the 'relative size' column a simple rank or we can scale the number, as I've done here, or, for example, scale it so it reflects a percentage of total volume.

Note that the longitude and latitude columns here are important for the visualization even if they're not important to the reader. If you decide to hide them from the reader (and AT), they still need to be present in the code.

As you'll notice at this point, this is not unlike code you might push on a daily (or more frequently) basis – which is the point. At this juncture, you would, of course, add CSS to style the table...perhaps to hide the longitude and latitude columns. After getting the table styled as you want it, it's time to add the JavaScript.

Here you'll add the D3.js geo libraries – I use d3.v3.min.js for the core, d3.geo.projection.v0.min.js, and topojson.v1.min.js – and my earth plug-in library. With the enhancements made earlier this month, that's all you have to do. The plug-in will pull the marker information using the class earth-viz and generate the visualization. If you want the visualization rendered in another part of the page (rather than where the table is placed), you can still use the constructor to call the plug-in explicitly.

One other change is that you can call the constructor with either some or all four parameters – visualizationElement, projection, visualizationWidth, and markerDataTable – or use can use a config object { data:<marker-data-table>, element:<visualization-element>, style:<projection>, width:<visualization-width> }. This change is stylistic only – there is no functional difference, it's only purpose is to allow greater flexibility so that engineers using it can maintain consistent coding practices.

Of course, calling the constructor is entirely optional now as long as the markup contains the key class, so if you're displaying static information, that's really all there is to it – you need do nothing more. If your visualization needs to be updated, or if you're just interested to learn more, keep reading.

The second major change in the most recent version is that the visualization object is maintained in the window scope, and can be accessed at any point after it's been created. The name of the window property is the same as the visualization ID (i.e., the ID of the marker table with '-earth-viz' added to the end). This means that with our example,the visualization is accessible by referencing window['busiest-airports-earth-viz'].

When this is tied to the ability to get the marker table id directly from the visualization, this makes it remarkably easy to hook updates to the visualization to updates to the table, by simply calling the parseMarkerData function something like... window['busiest-airports-earth-viz'].parseMarkerData('busiest-airports');. If you already have a timed refresh of the data that is updating the table (along with all the accessibility hooks that announce the live data), simply add this into the callback and your visualization is updated on the data refresh.

I know this has been a quick jaunt through a fairly powerful plug-in, but hopefully it's explained a bit about the newest features. As always –

Happy coding.

Show Your Work: Markup and styles for showing code

It's been quite some time since I've posted. Everyone is busy, and has different priorities, and sometimes that leads to an erratic posting schedule. As another year ends, I thought I end it with a short post about something I do quite often...writing code samples online. Why this topic? Because it's one of the things that all UI engineers should be dealing with - writing about code is one of the ways we share knowledge, teaching and learning at the same time - and it's one of the ways we creatively express ourselves.

As for the more technical stuff about the topic - I've gone back and forth about how best to do it - how to mark it up so that it's still semantic but at the same time make the design look a little traditional and make the code example itself easy to reference, by saying something like "in Example 1, line 3 you'll see the reference to parseMarkerData". Typically, that means banding (each line an alternating color - like the old-style printer paper) and line numbers. The problem with line numbers is that either you typically use an ordered list (which has its semantic advantages) or somehow put the line numbers in the markup (which is not semantic). I don't like the ordered list, mostly because a list item is not necessarily an item - it may be more or less than an item - and the ordered list is typically very difficult to style because of all the built-in browser styles.

I, therefore, have been going with a slightly different approach...I've started using a BLOCKQUOTE tag (because I'm typically quoting a block of code) and a SPAN tag to identify portions of the quoted code. This allows me to use a 'header' to describe the code block as well as wrap the quoted code in a DIV tag to enable scrolling for large blocks and maintain semantics by using a CSS counter to show the line numbers. My code sample ends up looking something like this...

<blockquote class="code"><p class="header">Example 1</p>
  <div class="viewport">
    <span class="line">&lt;script&gt;</span>
    <span class="line">var terra = new d3.geo.earth(document.getElementById('map')), trans;</span>
    <span class="line"></span>
    <span class="line">terra.parseMarkerData('airport-data');</span>
    <span class="line">trans = document.getElementById('btnTransform');</span>
    <span class="line">trans.onclick = function(e) {</span>
    <span class="line">    var style = 'Equirectangular';
    <span class="line">    if (terra.style() === style) {</span>
    <span class="line">      style = '';</span>
    <span class="line">    }</span>
    <span class="line">    terra.style(style);</span>
    <span class="line'>  };</span>
    <span class="line">&lt;/script&gt;</span>
  </div>
</blockquote>

...and my CSS, with its use of pseudo classes (like nth-of-type and before) and a counter looks like this...

blockquote.code { counter-reset:line; border:2px solid rgb(51, 153, 187); border-radius:20px; box-shadow:4px 4px 4px rgba(51, 153, 187, 0.7); font-family:monospace; font-size:0.8em; margin: 0.5em; padding:10px; }
blockquote.code > p.header { border-bottom:1px solid rgb(192, 219, 229); font-size:1.2em; font-weight:bold; margin:0 0 0.2em 0; }
blockquote.code > div.viewport { max-height:20em;   overflow-y:auto; }
blockquote.code > div.viewport > span:nth-of-type(odd) { background-color:#efefef; }
blockquote.code > div.viewport > .line { display:block; }
blockquote.code > div.viewport > .line:before { display:inline-block; margin:0 1em 0 0; text-align:right; width:2em; }
blockquote.code > div.viewport > .line:before { counter-increment:line; content:counter(line); }

That's all there is to it. This combination ends up giving me something useful that looks decent. Something that looks like this...

If you want to see how this works in a real-world example, head on over to my D3 earth plugin example page where you'll find several uses of the code sample style.

Rambling

This post is content previously published on www.cathmhaol.com

I've a fine, felt hat
And a strong pair of brogues
I have rosin in my pocket for my bow
O my fiddle strings are new
And I've learned a tune or two
So, I'm well prepared to ramble and must go

Tommy Makem

Animated flight from LAX to LHR
with stops in LAS, DEN and ORD

In today's update, I'll describe how to draw routes on your cjl-earth.js map, so that you get something like this sample image, but first, I should explain the structure of the generated markup.

Within the map group (which is a 'g' tag - <g> - with an id attribute composed of the map object id and '-map'), are groups (using a 'g' tag - <g>) for the oceans, countries, markers, and routes. These 'route' groups each have the class 'route', and each group is created as it is read from the data, so you can style the individual 'routes' using the #map_object_id-routes g and the nth-of-type selectors as well as the g.route (an individual route) and path.travel-route (individual paths) selectors. Each route is composed of paths from the origin to the first destination and then from each subsequent 'destination' to the next, giving markup similar to the markup below. Because D3 is translating longitude and latitude to x and y coordinates in the SVG, all of this code is generated - you don't have to do any calculations.

Markup Example

<g id="cjl-globe-04748665224760771-routes">
 <g class="route">
  <path class="travel-route" d="M 88 203 L 93 200"></path>
  <path class="travel-route" d="M 93 200 L 107 195"></path>
  <path class="travel-route" d="M 107 195 L 130 192"></path>
  <path class="travel-route" d="M 130 192 L 250 179"></path>
 </g>
</g>

Now that we've covered what markup is generated, we're ready to begin.

Let's begin by deciding whether we want to use some sort of a marker - like an airplane or storm symbol - to animate the route(s) you're going to define or if you just want a line. If you want a simple line, the display will be controlled by a style - for example, <style type="text/css">.travel-route { fill:none; stroke:#ff0000; stroke-width:1px; }</style> will make the paths defined in your route appear as a thin, red line.

If you want to move a marker along the route, you'll need to provide the SVG equivalent of the image in a marker object. Marker objects have a 'd' property that contains the pen instructions ordinarily contained in the 'd' attribute of the path tag, and may optionally have an 'orient' property for those symbols that have an orientation (e.g. an airplane or other object that has a 'front'). If orient is set to true, the object is rotated so that the 'top' is facing in the direction of movement, like an airplane that 'flies' along a route. Markers are not scaled unless specifically requested by providing the 'scale' property, so the image that's provided should be the appropriate size when loaded; however, if the map is 'zoomed' by user interaction, the markers are 'zoomed' as well and will maintain their relative size.

Note: you may also choose to have the two types of animation combined, so that a travel path is revealed as your marker moves along the route; however, this is only possible if the marker is provided and the 'combineAnimation' flag is set - see the documentation regarding the travel method in the cjl-earth.js library for more information.

After you've decided which sort of animation you'd like, decide how long the animation should run and whether you want it to loop or run once.

Once all your design decisions are made, decide how you want to provide the data to the function. Here again, you have two choices: directly or via AJAX. With this final decision, you're ready to write the code to display the routes by calling the 'travel' method...something like the following.

Providing data using AJAX with no marker earth.travel('/ReST/api/routes?origin=LAX&amp;destination=LHR', null, 2000, true);

or

Providing data directly with a marker earth.travel( [
 {origin:[-118.408075, 33.942536],
  destination:[
    [-115.15225, 36.080056],
    [-104.673178, 39.861656],
    [-87.904842, 41.978603],
    [0.461389, 51.4775]
   ]
 }
], {d:"m25.21488,3.93375c-0.44355,0 -0.84275,0.18332 -1.17933,0.51592c-0.33397,0.33267 -0.61055,0.80884 -0.84275,1.40377c-0.45922,1.18911 -0.74362,2.85964 -0.89755,4.86085c-0.15655,1.99729 -0.18263,4.32223 -0.11741,6.81118c-5.51835,2.26427 -16.7116,6.93857 -17.60916,7.98223c-1.19759,1.38937 -0.81143,2.98095 -0.32874,4.03902l18.39971,-3.74549c0.38616,4.88048 0.94192,9.7138 1.42461,13.50099c-1.80032,0.52703 -5.1609,1.56679 -5.85232,2.21255c-0.95496,0.88711 -0.95496,3.75718 -0.95496,3.75718l7.53,-0.61316c0.17743,1.23545 0.28701,1.95767 0.28701,1.95767l0.01304,0.06557l0.06002,0l0.13829,0l0.0574,0l0.01043,-0.06557c0,0 0.11218,-0.72222 0.28961,-1.95767l7.53164,0.61316c0,0 0,-2.87006 -0.95496,-3.75718c-0.69044,-0.64577 -4.05363,-1.68813 -5.85133,-2.21516c0.48009,-3.77545 1.03061,-8.58921 1.42198,-13.45404l18.18207,3.70115c0.48009,-1.05806 0.86881,-2.64965 -0.32617,-4.03902c-0.88969,-1.03062 -11.81147,-5.60054 -17.39409,-7.89352c0.06524,-2.52287 0.04175,-4.88024 -0.1148,-6.89989l0,-0.00476c-0.15655,-1.99844 -0.44094,-3.6683 -0.90277,-4.8561c-0.22699,-0.59493 -0.50356,-1.07111 -0.83754,-1.40377c-0.33658,-0.3326 -0.73578,-0.51592 -1.18194,-0.51592l0,0l-0.00001,0l0,0l0.00002,0.00001z", orient:true, scale:0.3}, 2000, true, true);

One final note - the travel method does not play well with the transition method, which is covered in the last blog entry, A different point of view, nor does it work well with projections that are designed to rotate, such as the 'orthographic' projection.

Additional Resources:

• There are prototype pages showing implementations of cjl-earth at http://products.cathmhaol.com/products/prototypes/earth/
• You can find out all about the D3 libraries at http://d3js.org/
• You can download a minified version of cjl-earth as well as the other files you'll need to use it from http://download.cathmhaol.com/cjl-earth.zip - but if you just want the minified version of cjl-earth, you can get that at http://js.cathmhaol.com/cjl-earth.min.js

A different point of view

This post is content previously published on www.cathmhaol.com

In the post preceding this, Around the world in eighty days, I discussed how to add a spinning globe to show global data collections. In the time since that post went live, I've made a few changes - all of which will be in the library on github - as soon as I get that up.

First, the library has been hardened so that if the core functionality is not available, the library will not work, but it also will not throw an error that breaks other JavaScript you have on your page. This should have been done earlier, and I apologize that it was wasn't - that is simply poor writing.

Second, the library has been extended by a new feature that allows developers to transition between projection styles. This transition, rather than being a sharp, jarring redraw of the image, is an animation that makes use of the d3 tweening function. I've tried to make sure that I'm only supporting styles that transition well to other styles, but I suspect there are some projection pairs that do not transition well. As you experiment to see which styles work best for you, I'd appreciate a little 'heads up' about the styles you find don't work well together.

Overall, though, you should now be able to transition between a Orthographic (globe) shape and an Equirectangular shape, for example…

…with just one line of code along the lines of…

var earth = new Cathmhaol.Earth('map', '/prototypes/earth/world-110m.js');
document.getElementById('trn').onclick = function() { earth.transition('Equirectangular'); }

That's all there is to it - just call the transition method and pass in the name of the projection you want to transition to (and optionally, how long you want the transition to take in milliseconds - the default is 750).

As before, you can see how this works by visiting the prototype page, which now has a special 'transition' button that will run the animation. The transition button on the prototype page will toggle between the two styles - Orthographic and Equirectangular - but if you're feeling really adventurous you could build a drop down (select) using the array returned by a supportedTypes method call and allow the user to select the transitions - it just takes a little creativity.

Again, you will soon be able to find the unminified files used in this project in my cjl github repo, or you can get them all bundled up from the download tent at cathmhaol.com - which also contains the HTML for the prototype, or you can get the required files through the links on the prototype page.

Still to come for this library:

• a method that lets you set up and show animated routes between an origin and destination, e.g., a flight path
• the ability to use a different shape for location markers rather than a circle

As always, feel free to drop me an email or ping me on Twitter with your ideas and/or how you've used what you've seen here.

Around the world in eighty days

This post is content previously published on www.cathmhaol.com

One of the projects I've been working on is developing a map that will visually represent data. This seems easy enough, but when practice meets theory, it hasn't been.

My project is fairly straightforward. Take a collection of quantifiable global information - it could be nearly any quantifiable information - e.g., the amount of money moving in or out of world banks, kilowatt hours consumed, number of mobile users, or the number of take-off and landing events at world airports - and tie it to a location like a city. Then represent the relative activity with a marker on a map. So basically, translating data that we might find in a table into a visual format.

Country	City	Activity
US	ATL	68343
US	ORD	59692
US	DFW	56496
US	LAX	51396
CN	PEK	48226
US	CLT	44583
US	DEN	44438
US	LAS	41164
US	IAH	39808
GB	LHR	37680

Beyond the basic requirement - to visually represent the data - there were a few other requirements I had set for myself. I didn't want the map to just be a two-dimensional projection - like a Mercator - I wanted it be a globe that rotates.

One option was a WebGL map - something like Akamai's GNET globe - but that doesn't work in Safari, practically eliminating iOS devices. Another option was planetary.js - which works most everywhere but didn't have the functionality I needed, mainly because it uses an HTML canvas instead of an SVG. I was left to build what I needed.

Even though the planetary.js library was slightly inadequate, it gave clues that led me to the library published as D3.js - which made building the exact tool I needed easy. Additionally, there are several online examples and tutorials that explain how to use D3 and topoJSON to create maps, making it a simple matter to generate the map. Add in the ability to zoom, for which there are hooks in the D3 library and a little finesse in JavaScript to convince the library that touch events should be handled like mouse events, and I had a nearly fully-functional globe with countries in a short amount of time.

Next came the markers...

First, we need some way of identifying the latitude and longitude of cities...or in our example data, airports. To accomplish this, I built a massive JSON file that contains nearly every airport, the country it is in, and the coordinates (which is airports.json in one of my GitHub repos, along with coordinates of most major cities). With this, it's a simple matter of mapping an airport to a specific point on the grid.

Once I had the coordinates, I needed some way to represent the relative size...which is really just a matter of creating a scale based on the smallest and largest numbers in the dataset. Here's where it gets a little tricky...because we want the markers to do something - be animated in some way. Transitions and animations are pretty time consuming, but luckily D3 again fills the gap. By using two somewhat clever transitions and fiddling with the timing, there are two slightly different animations - pulse and ping - that give the markers extra visual appeal.

Of course, the visual appeal of the rotating globe doesn't really solve all the problems, even with the ability to stop the globe and drag it to turn it or even zoom in or out. People see the animated markers and can see that traffic is heavier in ATL than in LHR, but the actual data is now hidden. To expose the data, I added a scrollable, sortable table to the library.

By specifying the columns or properties of the data elements that you want to display, a table is generated and attached to the same DOM node that contains the SVG with the map. By again using D3, we quickly make the table sortable - meaning users can click on the headers to sort by that property, e.g., clicking on the Country header to sort by country. Since there is likely little real estate available when a globe is present, I limit the table to around 10 rows and make it scrollable - the CSS to accomplish this is generated by the library and appended to the DOM when the table is generated.

You can find prototypes for both the marked map and the scrollable table at products.cathmhaol.com. You can find the D3 plugin that will do all this in my GitHub D3 plugins repo. Feel free to fork the repo, or if you'd rather, let me know what you'd like to see changed - maybe your requests will make it into the official repo.