I am a fan of studies in Data Visualisation. It is a creative and dynamic field with a lot of room for experiment. I am considering report and dashboard design, and within this frame Data Visualisation, as a form of practical art. Well designed and built reports are critical for solution adoption and usability. However, in this post I will concentrate on exactly the opposite topic – intentionally mystifying reports, obscuring the data and making it hard, for the report consumers to reach informed conclusions. I will also show how we can make the data visualisations as misleading as possible. This post is not as abstract as one may think, as it draws its examples from a very real project, for which I had to build a report under heavy pressure.
Initially, I was asked to build a report based on a small data set (~450 rows) stored in an Excel workbook. The data was perfectly suitable for building a Pivot Table on top of it, so I did so and then I decided to use the pivot table as a source for my report. The users have one measure – Spending Amount and a few dimensions – Category and Focus for that spending. The request came in the form: “We want to see the Amount by Category and Focus in both graphical and numeric form“. So, I sat down and produced this prototype (with their colour theme):
I sent the report prototype for review, and the response was:
1. Remove the table.
2. Change the bar graphs to make them more “flat”. Meaning: there is too much difference between the bars.
The reasoning – it is “too obvious” that the data is unevenly distributed. As the report is supposed to be presented to higher level management, discrepancies in the amount allocation would “look bad” on the people who requested me to create the report at the first place.
Adding more fake data got rejected, so I was advised to prepare a new prototype report with the new requirements. It follows:
The response was: “It looks much better, and we really like the top right bar graph and the Pie Chart. Would it be possible to leave just those two and change the Pie Chart to show data broken down just by Category?”
So, the users did not like my combined series. A point for them. Then I decided to remove the 3D cone graph, to remove all 3D effects, to make it more readable and to create the following 3rd prototype:
The users liked it, but still thought that it was too revealing, and were particularly concerned with the fact that there are a couple of “invisible” categories (the ones with small percentages) on the Pie Chart. They had a new suggestion – change the pie chart back to a bar graph and play with the format, so that even the smallest amount is visible. I offered an option, made another prototype and it finally got approved. The exact words were: “The graphs are exactly what we want“. There it is:
I consider this effort to be contrary to be an exercise in “Data Mystification”. It is very easy to draw the wrong conclusions from the report, and it achieves the opposite to the goals of Business Intelligence, as instead of empowering users, it could actually confuse them and lead them to making wrong decisions.
9 thoughts on “Data Mystification Techniques”
This situation sounds strangely familiar. Well done for fighting the good fight and giving the client what is best for them (even when they don't know it in the beginning)Cheers,Nick
Nick, I am still not happy with the result. yes, there are no pie charts, but it is still very hard for report users to see what's going on at a glance. The scale breaks on the first bar graph make it very deceiving and there is no point in making such a graph, because you cannot visually compare the data points…But I can't actually do much more – they are VERY strict on getting it like that to "make them look better"…Also, the second bar graph is somewhat pointless – since the users of the report cannot see how the expenditure amouints are distributed per focus. They can just see how focus expenditure is distributed per category..But everything looks even – therefore, again, they look good, because the allocations look quite even, while in reality there are massive gaps..
May I suggest throwing in a radar chart?Sadly your story rings a bell… 😦
Boyan, Have you considered presenting the data from the pie chart on a logarithmic scale rather than a scale with discontinuities in the axis? Most moderately mathematically literate users can read a log chart effectively, and it gives you a clearer view of relative scale than a discontinuous axis. The only downside is that log charts really need gridlines in the background.
Hi Ozziemedes,Yes, I considered it, tried it and failed. Unfortunately, Logarithmic scale works quite bad for values under 1 (say 0.1) – they show as a negative. Especially annoying when you have to present percentages and some of them are really small.
Hmm… there's always a way to skin that cat… Consider an axis mapping function Y(y) = log^10(100 * y) / 2. This will effectively lift the scale of the logarithm above your fractional values that are less than 1, then provide a linear correction to compensate for the "scale up".
The formula works great.I am, though, doubtful about the value of a logarithmic scale in this case. I do not think that a bar graph should use this sort of a scale unless it is intended as a really rough visualisation technique.I guess that a simple table with a list of categories in acs or desc order would server a similar purpose, use less real-estate and possibly be less confusing…Would be interesting to hear other people's thoughts about the value of logarithmic scales, so please comment 🙂
Thanks for this – it was hysterical and made my day. I especially liked “Stephan Few would spew” !
I think we work for the same boss… what, no XY scatter plot?
Comments are closed.