Are you Measuring the Right Things or is Amelia Bedilia Defining Your KPIs?

Amelia Bedelia measuring 2 cups of rice
One extremely competent CFO was looking for help with performance management. His team defined 5 elements of success:

1.       Identify KPIs and prototype cascading dashboards
2.       Define metrics we should track for 5 business units, 40 departments
3.       Create a metrics-driven culture
4.       Be sure to include leading indicators
5.       Set thresholds so we know when KPIs are outside targeted ranges

If you were the management consultant on the project, where would you begin? 

One leading expert in the performance measurement space notes:
"In the absence of a clearly articulated and up-to-date strategy or set of objectives, a KPI team must spend significant time interviewing departmental executives and corporate executives to understand the mission and direction of their group before they can commence with the actual work of defining KPIs.”  
-Wayne Eckerson, Performance Dashboards: Measuring, Monitoring, and Managing your Business

At the time I knew that companies including Intel, Oracle, LinkedIn, Google, and Twitter began performance management systems with Objectives and Key Results (OKRs) with seemingly great success. Everything they measured, they measured for a reason.** And they didn't hire Amelia Bedilia to define what to measure!

What I did

I began by asking what KPIs are already being measured to get a baseline on where we are before trying to figure out how to achieve the 5 elements of success. The temptation to brainstorm KPIs and start reporting the data we already have at our fingertips was strong! After all, we could quickly create some cascading dashboards with available data and get a feeling of making progress. We could even purchase BI software and visualize the data we already have. But this process would be akin to looking at the data and saying “now what” as opposed to agreeing on our vision and then clarifying it with metrics that we can use to monitor and track our progress. 

One very highly respected leader within the organization told me the classic sufi story about a man looking for his keys in a dark parking lot under a lamp post even though he dropped them in the dark simply because the light is better. So clearly I was warned not to make the mistake of analyzing data simply because it’s available. Ultimately, leadership wanted me to help ensure we measured the right things.

So, I took the guidance in the above quote to “understand the mission and direction before defining KPIs.” The MOKRs approach ensured that the KPIs emerged organically in the context of confirming strategic direction.

I’m pleased to share some example sets of OKRs from a marketing team to illustrate how KPIs emerged during a 45-minute OKRs drafting session with the marketing VP:


Marketing Team OKRs

O: Achieve lead targets provided by sales team
 KR 1.1: Deliver 100 leads in Q1, 200/Q2, 250/Q3, 300/Q4

O: Deliver quality leads cost effectively
 KR 2.1: Identify best and worst marketing event by analyzing ROI of at least 5 conferences
 KR 2.2: Achieve an overall cost of marketing per lead below $65 in Q1
 KR 2.3: 30% of leads convert to opportunity within 6 weeks of creation

First notice that this marketing team has a very focused set of OKRs with only 2 Os and 4 KRs in total.

Next, obeserve that while we identified 4 KRs, the following 3 KPIs emerged as well:

1.       Leads (KPI from KR 1.1)
2.       Overall cost of marketing per lead (KPI from KR 2.2)
3.       Lead to opportunity conversion rate within 6 weeks (KPI from KR 2.3)

Notice that KR 2.1 does not really map to a KPI as “ROI of a conference” does not fit the definition of a KPI. However, by identifying the best and worst marketing event by analyzing the ROI of at least 5 conferences, we may be more likely to achieve KR 2.2 to keep overall cost per lead below $65. The report from KR 2.1 is very likely to be actionable in that we can use the results of the report to inform decisions that will optimize our marketing spend. That is, “deliver quality leads cost effectively.” So, clearly a KR is not the same as a KPI.

How do you measure quality?
KR 2.3 illustrates how “quality” can and should be measurable. At first, the marketing VP wanted to use a “lead score” as a quality metric. However, the sales VP did not feel like lead score was meaningful. Since the VP sales in this case does not feel lead score is the right thing to measure, it's not the right thing to measure!

Turns out, the sales VP cared only about the conversion rate to opportunity. We knew that this averaged about 25% in the past and the sales team was not feeling like the leads they received from marketing were high enough quality. Ultimately, we set a target of 30% to represent an improvement over the 25%. Simply by monitoring this percentage and knowing it was a key result, marketing worked more closely with sales to ensure the right leads were provided and that they did everything within their power to help sales reps convert those leads into opportunities within 6 weeks.

How did we know we measured the right thing? 
We got better results, people worked together across departments. Most importantly, when we saw the numbers at the end of the quarter, people cheered and wanted to move the right things in the right direction even more to drive even better results going forward.

So, are you measuring the right things?

Request a 30-minute OKRs drafting session


**By the way if you put “Mission” in front of OKRs, you get MOKRs. I first learned about OKRs as MOKRs, and I think the above quote would suggest that we leave the M in there. For more on MOKRs at Oracle check out Never-before-disclosed Oracle Planning Techniques.