A story about correlation and causation
Have you heard the claims people make about their successes? Things like, increased the budget by “x” dollars? Or, increased sales by “x” percent? Always examine these claims critically. This is true specially in the context of sales leaders managing to activity metrics. Consider the following story.
During a recent client onboarding meeting, a cold shiver went up my spine when reviewing an account manager’s call notes. Deep in the notes is a quote from a competent head of sales operations at a large global software company. “At the start of last year, we instituted a strict policy of managing to clearly defined activity metrics. As a result, we experienced a 15% sales increase.” Otherwise stated this means, I took an action. The team had success. Therefore, this specific action caused success.
As a proud data geek, I’m horrified. This conclusion implies a lack of understanding of the meanings of correlation and causation by someone who should know better. (For a brilliantly simple explanation of the two visit twitter.com/KirkDBorne.) More importantly, I have had a front row seat to this same movie more times than I can count. It always has a sad ending.
What is truly behind success?
It’s not uncommon for business leaders to point to their innovations as “the” drivers for team success. A little self-promotion is necessary in some organizations to navigate around sticky political situations or climb the corporate ladder.
In this case, however, the SVP of Sales hired our company to help his team achieve greater success through the application of advanced analytics. This was not self-promotion. He believed his story and encouraged us to believe that this change was the only factor in the year-over-year improvement. Nothing else had changed, he claimed. Further, he pointed to an incredibly powerful statistic. His own data analysis showed that tracked activities (phone calls, opened opportunities, meetings, emails) in aggregate did increase substantially. Fifteen percent to be precise – the exact level that sales increased by.
This statistic is powerful, but ultimately misleading.
Does managing to activity metrics produce results?
It took a data scientist on our team less than an hour to poke holes in the sales leader’s theory that the company’s new management style caused the increased sales. Many companies have an over-reliance on activity metrics. We have seen many companies fall into this trap. That being the case, we knew how to test the hypothesis that the 15% increase in activities directly led to the 15% uptick in sales.
As a first step we divided the sales reps into four equally sized quartiles based on sales. We also divided the sales reps into quartiles based on win rates. Not surprisingly, most reps landed in the same quartile in either trial (reps with the highest gross sales tended to be the reps with the highest win rates).
When we look at activities and results in these more granular buckets it quickly becomes interesting. There was only a negligible increase in activities for the top two quartiles (less than 2%). The bulk of the activity increase came from the bottom half of sales performers. The bottom quartile witnessed a whopping 34% increase in activity levels. However, when we look deeper (gross sales and win rates), the results flip upside down. In other words, the top half performers with little change in their activity levels saw the greatest increase in win rates and gross sales.
Clearly, something other than the new activity based management approach was driving the sales increase.
Shifting rep focus from quality to quantity
In this fairly standard case, we see evidence that the bottom half of the sales organization pumping the pipeline full of questionable opportunities. They log more calls and emails for these questionable opportunities in order to avoid negative consequences. The reps have done as instructed, but there is not a corresponding focus on the quality. Nor is there focus on validity of the opportunity or the interaction. Moreover, when evaluated on their implementation of the strategy, managers focus on the activity tally for their reps. In doing so, they are less likely to focus on the viability of the opportunity or the quality of the interaction.
How to interpret these data
This sales leader made a common mistake in interpreting data. More importantly, companies who focus their sales strategies exclusively on activity metrics they are in for disappointment. Here’s why.
We expect to see lower performing reps entering more unqualified opportunities and closing at lower win rates. Fudging bad data wastes time. Expanding skills sets is a better use of time.
Higher performing sales reps have a laser focus on achievement: hitting sales targets, getting the next sale, achieving the next rung in the commission ladder. They understand how to sell and loathe being micromanaged. Focusing on activities fixes a problem that doesn’t exist.
Even the best reps have weaknesses. If sales management makes the investment in coaching reps, they should include some efforts to identify and coach to those weaknesses. The visibility of these weaknesses might only be microscopic. This is especially true when viewing them in combination with other predictive variables. Such variables may include: a certain type of account, size of the prospect, deal size, type of meeting, funnel stage, deals in which a certain competitor is involved, industry of the prospect, or product(s) being sold.
The data might show that a rep’s aggregate sales activities are high, but frequency, kind or quality of these interactions are predictive of the final outcome. Some combination of several of these individual weaknesses may be a blind spot for the rep and the manager. Coaching to activity metrics typically involves coaching to metrics designed from a whole team instead of coaching to the individual rep’s most impactful activities or behaviors.
Coaching to average metrics takes significant management focus and prevents a search for all the other predictors of success to which managers can coach.
In context, managing to activity metrics is important
None of this should suggest that we don’t value activity metrics as an important part of the sales process. However, activities, like anything else need to be considered in the context of all the other sales data when implementing sales strategies and interpreting results. They may sometimes correlate to success, but do not confuse that with causing success alone. Organizations that measure the quality of interactions between reps and prospects, as opposed to just the quantity of those interactions, are much better positioned to understand the drivers of sales success and failure.
Postscript: After this article was first published in an industry blog several readers wanted to know what led to the 15% growth rate that the sales leader quoted. Despite the sales leader’s initial claims that nothing else had changed, there were more subtle moves that appear to have had some impact. These moves involved changes to sales territories and comp plans. This was also the company’s 4th consecutive year with a double digit growth rate.
