Data Mystification Techniques

I am a fan of studies in Data Visualisation. It is a creative and dynamic field with a lot of room for experiment. I am considering report and dashboard design, and within this frame Data Visualisation, as a form of practical art. Well designed and built reports are critical for solution adoption and usability. However, in this post I will concentrate on exactly the opposite topic – intentionally mystifying reports, obscuring the data and making it hard, for the report consumers to reach informed conclusions. I will also show how we can make the data visualisations as misleading as possible. This post is not as abstract as one may think, as it draws its examples from a very real project, for which I had to build a report under heavy pressure.

Initially, I was asked to build a report based on a small data set (~450 rows) stored in an Excel workbook. The data was perfectly suitable for building a Pivot Table on top of it, so I did so and then I decided to use the pivot table as a source for my report. The users have one measure – Spending Amount and a few dimensions – Category and Focus for that spending. The request came in the form: “We want to see the Amount by Category and Focus in both graphical and numeric form“. So, I sat down and produced this prototype (with their colour theme):

As you can see from the screenshot, the report is fairly simple – the bar graphs on the top clearly show how the Amount is distributed per Category and Focus. Also, because of an explicit request, I build the bar graph on the second row to show category and focus amounts combined, and in order to clarify the whole picture, I added the table at the bottom of the report. The report works for colour-blind people too, as the details for the Focus per Category Expenditure bar graph are shown directly below in the data table.

I sent the report prototype for review, and the response was:

1. Remove the table.
2. Change the bar graphs to make them more “flat”. Meaning: there is too much difference between the bars.

The reasoning – it is “too obvious” that the data is unevenly distributed. As the report is supposed to be presented to higher level management, discrepancies in the amount allocation would “look bad” on the people who requested me to create the report at the first place.

Adding more fake data got rejected, so I was advised to prepare a new prototype report with the new requirements. It follows:

Now, Stephen Few would spew if presented with such a report. I did everything I could to obscure the report – added 3D effects, presented the amount in 100% stacked graphs when absolutely not necessary and, of course, I had to add a Pie Chart. A 3D Pie Chart. The whole report is totally useless. It is impossible to deduct the actual numbers of the various category/focus amounts. Furthermore, the Pie Chart combines the focus and category and uses virtually indistinguishable colours for the different slices. The 3D effect distorts the proportions of these slices and if printed in Black and White, or if viewed by a colour-blind person, the report looks like this:
Since my goal of total mystification of the report was achieved, I sent the second prototype back.
The response was: “It looks much better, and we really like the top right bar graph and the Pie Chart. Would it be possible to leave just those two and change the Pie Chart to show data broken down just by Category?

So, the users did not like my combined series. A point for them. Then I decided to remove the 3D cone graph, to remove all 3D effects, to make it more readable and to create the following 3rd prototype:

Here, we can see that the pie has the actual percentages displayed, and each category amount can be calculated from there. The stacked bar graph is again almost useless.

The users liked it, but still thought that it was too revealing, and were particularly concerned with the fact that there are a couple of “invisible” categories (the ones with small percentages) on the Pie Chart. They had a new suggestion – change the pie chart back to a bar graph and play with the format, so that even the smallest amount is visible. I offered an option, made another prototype and it finally got approved. The exact words were: “The graphs are exactly what we want“. There it is:

The Y Axis in the first graph is interesting. Not only that there is a “scale break”, but in fact the scale is totally useless, as it is manually adjusted, because of the big gaps between the amounts. I needed two scale breaks, which for a series with 6 values is a bit too much. You can compare the normal linear scale of the first Prototype I created and this one. However, it hit the goal – my users were satisfied.

I consider this effort to be contrary to be an exercise in “Data Mystification”. It is very easy to draw the wrong conclusions from the report, and it achieves the opposite to the goals of Business Intelligence, as instead of empowering users, it could actually confuse them and lead them to making wrong decisions.

Advertisements

9 thoughts on “Data Mystification Techniques”

  1. This situation sounds strangely familiar. Well done for fighting the good fight and giving the client what is best for them (even when they don't know it in the beginning)Cheers,Nick

    Like

  2. Nick, I am still not happy with the result. yes, there are no pie charts, but it is still very hard for report users to see what's going on at a glance. The scale breaks on the first bar graph make it very deceiving and there is no point in making such a graph, because you cannot visually compare the data points…But I can't actually do much more – they are VERY strict on getting it like that to "make them look better"…Also, the second bar graph is somewhat pointless – since the users of the report cannot see how the expenditure amouints are distributed per focus. They can just see how focus expenditure is distributed per category..But everything looks even – therefore, again, they look good, because the allocations look quite even, while in reality there are massive gaps..

    Like

  3. Boyan, Have you considered presenting the data from the pie chart on a logarithmic scale rather than a scale with discontinuities in the axis? Most moderately mathematically literate users can read a log chart effectively, and it gives you a clearer view of relative scale than a discontinuous axis. The only downside is that log charts really need gridlines in the background.

    Like

  4. Hi Ozziemedes,Yes, I considered it, tried it and failed. Unfortunately, Logarithmic scale works quite bad for values under 1 (say 0.1) – they show as a negative. Especially annoying when you have to present percentages and some of them are really small.

    Like

  5. Hmm… there's always a way to skin that cat… Consider an axis mapping function Y(y) = log^10(100 * y) / 2. This will effectively lift the scale of the logarithm above your fractional values that are less than 1, then provide a linear correction to compensate for the "scale up".

    Like

  6. The formula works great.I am, though, doubtful about the value of a logarithmic scale in this case. I do not think that a bar graph should use this sort of a scale unless it is intended as a really rough visualisation technique.I guess that a simple table with a list of categories in acs or desc order would server a similar purpose, use less real-estate and possibly be less confusing…Would be interesting to hear other people's thoughts about the value of logarithmic scales, so please comment 🙂

    Like

Comments are closed.