Every business will have metrics they use to define success. Revenue, profits, days since the last safety incident, lines of code written, on time performance, etc. The hope when identifying KPIs is that by watching a few numbers, we as a company will be able to notice problems and resolve them in a timely way. This is a good first step to using data to make decisions, but there are a lot of ways KPIs can give an incomplete and sometimes downright inaccurate picture.
The first key question when looking at a KPI is to ask if the data used to make it is likely to be accurate or not. One example from manufacturing is that companies may rate their suppliers based on both timeliness of deliveries and defect rates. However, what happens when that process involves a person manually entering the defect rate from their favorite supplier? Even without intentional data errors, if your ordering and receiving systems are not linked it is easy to have inaccurate data.
But suppose all your data are trustworthy: Are we safe then? We can hope that the KPIs we are using are reflective of company goals, but goals other than revenue and profit are often hard to quantify. You could judge developers by the number of lines of code written (some do), but that is a poor measure of how much value they are creating. We could be thrilled to have employees not taking many sick days, but have in aggregate much less work done because every time one person gets a cold, the whole office does.
Now imagine you have navigated the difficulties of both having high quality data and choosing the right metrics. But even then, by their very nature any KPI will involve aggregating data into a single number. If you let your analysis end when you have good data and KPIs that reflect the priorities of your business, you still miss out on a lot of opportunities by digging deeper. One salesperson may be exceptional at handling a certain kind of customer, and terrible at another kind. Unless you have the ability to test those kinds of ideas with your data you are missing out on the full potential of understanding your data.
So what is the punchline? Those steps to make sure your KPIs "work" are the same first steps for any analytics project. It is a question of what you do next that distinguishes a company with high quality metrics from one that counts as a "data driven" business.
For a nice walkthrough of what good KPIs look like in different industries, read more here: http://firstround.com/review/im-sorry-but-those-are-vanity-metrics/
At Mashey, we know defining metrics can be difficult and it’s a moving target. We have helped several companies conquer this by developing and delivering data analytics solutions for their organization.
Contact us to find out how we can help you too.