The Data Literacy Triad

by | May 15, 2018

Data literacy” has become a hot topic. At least, in the world of business data it has. There are multiple good reasons for that.

There is a data explo­sion:

And, there is an explo­sion of tools for access­ing and analyz­ing the data:

With more data and more ways to work with that data, it is no surprise that orga­ni­za­tions are increas­ingly embrac­ing the need to be knowl­edge­able and compe­tent — liter­ate — when it comes to the appli­ca­tion of data.

Do a little Googling and you will find any number of defi­n­i­tions of data liter­acy, like this one (the one Google bubbles up and summa­rizes) and this one (which includes a nice little 2-minute video) and this one (which has tips for closing the “data liter­acy divide”) and even this one (because Wikipedia — it’s a surpris­ingly brief, yet broad, entry).

When it comes to a prac­ti­cal discus­sion about data liter­acy within an orga­ni­za­tion, it seems like the term is often defined situ­a­tion­ally (and narrowly). While that may meet an imme­di­ate need, it can be short-sighted. Data liter­acy is a mult-faceted thing, and it’s impor­tant to think through, plan, and educate along all of those facets:

  • Metrics Liter­acy — knowing what differ­ent data means
  • Tool Liter­acy — being able to self-service data needs effi­ciently and to an appro­pri­ate extent
  • Concep­tual Liter­acy — approach­ing and apply­ing data with clarity and sophis­ti­ca­tion

The first two of the above are, by far, the most common defi­n­i­tions used. The third, though, is equally crit­i­cal and is where many — if not most — orga­ni­za­tions fall short.

Metrics Literacy

Metrics liter­acy is, at its core, having a strong under­stand­ing of the data actu­ally being used within an area of the busi­ness.

There are, essen­tially, two levels of metrics liter­acy:

  • A general under­stand­ing of a partic­u­lar dimen­sion or metric — even if an employee does not know the exact mechan­ics and defi­n­i­tion of a value that appears in a report — lead, oppor­tu­nity, page view, session, bounce rate, channel, open rate, conver­sion rate — he may still have a fairly accu­rate concep­tual idea of what the data means. This can be danger­ous. In the world of website analyt­ics, for instance, many busi­ness users think they have a strong concep­tual idea of what the metrics “time on site” and “time on page” mean, when, in reality, those are two consis­tently misun­der­stood metrics.
  • A true and accu­rate under­stand­ing of a partic­u­lar dimen­sion or metric – this is where true metrics liter­acy comes from. It actu­ally requires having an under­stand­ing of the under­ly­ing process that created the data. As an example, a sales­per­son who uses Salesforce.com as a CRM on a daily basis likely regu­larly creates and main­tains leads, contacts, accounts and oppor­tu­ni­ties. So, when she sees data from the CRM — regard­less of whether that report is within the CRM itself, in a spread­sheet, in a dash­board within a BI tool, or even simply refer­enced in an email — she likely will intu­itively and correctly inter­pret any of that data.

The chal­lenges of achiev­ing metrics liter­acy are many:

  • A single employee may be highly liter­ate with some data and not liter­ate at all with other data
  • It is human nature to assume one has achieved a true and accu­rate under­stand­ing of the data when, in reality, one only has a high-level, intu­itive under­stand­ing of the metric, or, much worse, one’s under­stand­ing is actu­ally incor­rect.
  • The best way to become deeply liter­ate regard­ing a data set is to work — oper­a­tionally — with the under­ly­ing system that gener­ates the data (sales­peo­ple when it comes to CRM data, digital analysts when it comes to web analyt­ics data, book­keep­ers when it comes to cost account­ing data, etc.), and this isn’t prac­ti­cal in an envi­ron­ment where collab­o­ra­tion occurs (as it should) across many differ­ent groups.

Recog­niz­ing that metrics liter­acy is actu­ally more involved than it seems on the surface means data defi­n­i­tions need to be readily avail­able (embed­ded in reports and analy­ses to the extent possi­ble), and they need to high­light data that is most prone to being misin­ter­preted.

Tool Literacy

Unlike metrics liter­acy, tool liter­acy is not, defi­n­i­tion­ally, some­thing that varies based on the type of data. Every company has that person that every­one knows is “really good with Excel.” That person is highly liter­ate with Excel (or, at least, is suffi­ciently more liter­ate than others in the orga­ni­za­tion to be perceived as such).

Tool liter­acy is a measure of how effi­ciently, intu­itively, and effec­tively an employee uses the tools that are needed to access, explore, and present the data.

Which tool(s) an employee will benefit the most from a high degree of tools liter­acy varies based on the orga­ni­za­tion and the employee’s role. Microsoft Excel (or, these days, Google Sheets) is still, often, a core tool in an orga­ni­za­tion, but many orga­ni­za­tions have started adopt­ing second gener­a­tion busi­ness intel­li­gence (BI) plat­forms such as Tableau, Domo, Qlik, PowerBI, and the like. Busi­ness users do not neces­sar­ily need to be adept at the ins and outs of getting enter­prise data into these plat­forms, but the ability to quickly filter, drill­down, combine, and visu­al­ize data within the plat­form is a sign of a high degree of tools liter­acy with a BI plat­form. Simply knowing how to log in and create reports or dash­boards created by someone else is only the barest level of tool liter­acy with BI tools.

Histor­i­cally, employ­ees with “analyst” in their titles tend to be fairly tools-liter­ate. That makes sense, as they spend more time with the analyt­ics tools, and they will be inef­fec­tive in their roles if they are using those tools inef­fi­ciently (although, sadly, there are many analysts out there who, while “way better with the tool than anyone else in the orga­ni­za­tion,” are simply bene­fit­ting by an excru­ci­at­ingly low level of tool liter­acy in the orga­ni­za­tion, overall). Increas­ingly, though, tool liter­acy is being expected of a broader set of employ­ees:

  • The analyt­i­cal tools built into the oper­a­tional systems they use are much richer (see Einstein Analyt­ics from Salesforce.com, Analy­sis Work­space from Adobe Analyt­ics, Bizible Discover from Bizible, etc.)
  • The orga­ni­za­tion has invested in a BI plat­form that was sold as being a means for “democ­ra­tiz­ing the data” — the exec­u­tive who made the deci­sion to invest in the plat­form bought into the platform’s assur­ances that all employ­ees would quickly be digging deeply into the data using their intu­itive user inter­face.

The chal­lenge is that more power­ful data analy­sis tools are, inher­ently, more complex. Certainly, the user expe­ri­ence varies dras­ti­cally from tool to tool, but no amount of UX work can change the fact that complex systems make for complex data, and offer­ing a high degree of flex­i­bil­ity adds complex­ity in its own right.

And, to add to the messi­ness, many analysts are finding that their toolsets are expand­ing to include the use of open source program­ming languages like Python or R.

On the one hand, tools liter­acy is the “easiest” aspect of data liter­acy to manage, as most tools have a wealth of train­ing resources avail­able. And, as long as users are regu­larly using the tools, they’ll inher­ently get better with them through repe­ti­tion. But, on the other hand, tools liter­acy requires a degree of dili­gence on the part of the employee to ensure that his use of the tool does not stop at the “good enough to get by” point and, instead, is a steady and contin­u­ous progres­sion.

Conceptual Literacy

Some orga­ni­za­tions focus almost exclu­sively on metrics liter­acy and tools liter­acy without tack­ling the murkier — but crit­i­cal — area of concep­tual liter­acy. Concep­tual liter­acy is the ability to think about data in ways that are appro­pri­ate, mean­ing­ful, and action­able.  

At a basic level, this aspect of data liter­acy means:

  • Truly under­stand­ing the purpose of KPIs, and being well-versed in approaches for estab­lish­ing them
  • Being able to clearly artic­u­late and prior­i­tize hypothe­ses
  • Having a reason­able degree of comfort with differ­ent approaches for using data to vali­date hypothe­ses and where each is most appro­pri­ate: histor­i­cal data analy­sis, qual­i­ta­tive research, quan­ti­ta­tive research, split testing, etc.

Beyond the basic level of concep­tual liter­acy is a more advanced level that goes to inter­nal­iz­ing the rela­tion­ship between cost and uncer­tainty, as well as the inher­ent vari­abil­ity in the data.

This is a tricky set of ideas, but they are impor­tant. With all of the buzz and excite­ment about machine learn­ing and arti­fi­cial intel­li­gence, it’s clear that many busi­ness users are hoping that they can short­cut the need for concep­tual liter­acy and “just throw the data into the machine.” This is danger­ous terri­tory and, every time I hear a busi­ness user make a version of this claim, I cringe. Despite the increas­ingly power­ful analyt­ics plat­forms, there remains a crit­i­cal need under­stand where the machines fit — and where they cannot or should not — in an organization’s overall analyt­ics ecosys­tem. And, that requires a high level of concep­tual liter­acy.

The seed of this post were some inter­nal discus­sions we’ve been having at Search Discovery about the differ­ent ways we help compa­nies develop and acti­vate strate­gies for effec­tively using their data to drive mean­ing­ful busi­ness outcomes. Ensur­ing that the stake­hold­ers have a suffi­cient degree of data liter­acy across all three of these aspects is a crit­i­cal compo­nent of that work. If you’d like to discuss how we can help you assess your data strat­egy, we’d be happy to chat with you.