PMs & Metrics: Counter Metrics
While north star metrics are incredibly valuable for creating focus and alignment, they provide even more value when paired with a counter metric.
What’s a counter metric? A counter metric is something that you measure to ensure that you haven’t over-optimized your north star metric to the detriment of your customers and your business.
Below, I’ll discuss why you need a counter metric, how to select a compelling counter metric, and how to set a threshold for your counter metric.
Afterwards, we’ll pair together the north star metric and the counter metric to drive alignment across stakeholders and to push business outcomes forward.
So let’s dive into why you need counter metrics to succeed as a product manager.
Why You Need a Counter Metric
Why do you need a counter metric? Well, human beings aren’t perfect, and human beings tend to enjoy increasing numbers that they’re measuring.
If you only measure the north star metric, you won’t have any visibility into other key areas of the business that you need to maintain. You won’t be able to capture any negative downstream effects that could come with changes to your north star metric.
For example, let’s say that you’re the product manager for a website that sells Spanish language courses.
You’ve previously decided your north star metric is “number of users who visit your site every week,” because your business hypothesis is that if more users visit the site, you’ll be able to convert more of them into customers.
You currently have 10,000 weekly visitors, and you convert at a 1% rate, so that means you’re selling 10,000 * 1% = 100 Spanish courses every week.
You learn that running a giveaway for prizes is a fantastic way to bring new users to any website, so you decide to give away an Oculus Rift.
To get even more attention, you make the Oculus Rift giveaway the primary call to action (CTA) of your landing page. You also run social media campaigns on Snapchat.
Your north star metric shoots upwards by 30%, so now you have 13,000 visitors every week! That means you’re doing well, right?
But then, you find out that you’re now only selling 50 Spanish courses per week. How is that possible?
First, your giveaway visitors may not actually about the core product that you’re selling on your site. So, all they do is register for the giveaway, then they leave. They never convert.
Second, your typical visitor base may have been unable to access the content that they would typically access because the giveaway takes up so much space and attention.
Your typical visitor base is looking to learn Spanish. They’re likely not interested in winning an Oculus Rift, since an Oculus Rift has nothing to do with their core pain of learning Spanish.
Your typical visitors likely became confused and left your site because they couldn’t find what they were looking for.
In other words, not only did you fail to convert the giveaway visitors to your site, you also drove away the original audience that would have converted.
So, to ensure that you’re focusing on true success and not decent-looking stats that never positively convert, you need to have a counter metric.
In our example above, a good counter metric would have been conversion rates. If we saw conversion rates dropping even though the traffic was increasing, we could have stopped what we were doing to investigate the problem.
On top of that, you need a counter metric not just for your own sake, but for the sake of your stakeholders as well.
Even if you’re already aware that you shouldn’t run unrelated giveaways to increase your north star metric, you can’t guarantee that your other stakeholders have that knowledge.
Because product managers have so much influence across so many stakeholders, you need to ensure that you’ve aligned stakeholders to do what’s right for the business – and that means ensuring that they’re not over-optimizing on just a single metric. That’s what every product manager in the PMHQ community will tell you.
To enable all of your cross-functional teammates to do what’s best for the business, you need to provide them with both a north star metric as well as a counter metric.
Now that we understand the importance of counter metrics, let’s discuss how to select a compelling counter metric.
How to Select a Compelling Counter Metric
You use counter metrics to provide alternative hypotheses for why your north star metric increased.
That is, if your north star metric is increasing at the expense of your counter metric, then you didn’t actually move the north star metric forward in the way that you wanted it to.
Similar to north star metrics, you should select counter metrics for critical, measurable, actionable business outcomes.
Counter metrics need to be critical to the business. If you’re using a counter metric that isn’t critical, then you’re not proactively protecting the business.
For example, say you use a vanity counter metric like “total of registered accounts.” There’s literally no way for this to drop because totals can only grow over time. Using a vanity metric as a counter metric doesn’t provide any value to the business.
Counter metrics need to be measurable. If you have no way to measure the counter metric, then you have no way to monitor whether you’re protecting the business.
Counter metrics need to be actionable. If you select a counter metric that you can’t act on, you’ll make your team feel entirely helpless.
For example, a counter metric of “number of deals closed per quarter” is a poor choice if you’re a platform product manager, because you and your team have no way of directly influencing deal velocity. No matter what you ship, you aren’t going to accelerate deal flow within the next quarter.
Below are some examples of compelling counter metrics.
If your north star metric is an engagement metric, such as “the average of times a user clicks within your site”, you should use a counter metric of “a total of users.”
Why is that? Well, say that you tried to boost the number of times users click by turning key content pages into bite-sized “slideshows.” Now, users have to click the “next” button 10 times to be able to get all of the information that they were looking for.
You might wind up increasing times per click but you might drive away so many users that your user numbers go down, even though it seems like you’re increasing engagement!
So, while you’re driving up the average number of clicks, you need to ensure that total users don’t fall dramatically as part of your experimentation.
If your north star metric is a revenue metric, such as “ad revenue generated per week”, you should use a counter metric of “net promoter score.”
As an example, one easy way to drive lots of ad revenue is to add many more ads to the layout of your pages. But, all of those ads may drive away visitors because they find it too distracting for them to focus on their task. And, if those ads are untargeted, your visitors will feel that the ads aren’t providing them value, which therefore makes your site less valuable to them.
In other words, select a counter metric that protects a key part of your business economics. Just about every north star metric has a variety of compelling counter metrics available – it all depends on what your business absolutely needs to protect while you’re on the offense in driving up your north star metric.
I’ve provided more examples of counter metrics below, paired with their north star metric:
One last thing to keep in mind: the goal is not to optimize for the counter metric. Counter metrics are not north star metrics, and your job should not be to drive the counter metric forward.
Counter metrics are meant as a sanity check. Therefore, when you implement a counter metric, your goal is to keep them above some predetermined threshold.
So how might you set a threshold that makes sense?
How to Set a Threshold for Your Counter Metric
Honestly, there are no hard and fast rules that you can use to set counter metric thresholds. That’s because the goal of a counter metric is to serve as a sanity check.
It really depends on the risk appetite for your business. In other words, how much risk are they willing to take to be able to drive the north star metric upwards?
Based on your understanding of the business, what would be an alarming change in your counter metric? Use that threshold to set up alerts, so that if the threshold is crossed, you and your team can act immediately.
I’ve generally been using a threshold of about 3-5% for my counter metrics. Keep in mind that you can’t set the threshold to 0% because it’s simply not reasonable to expect that the counter metric will invariably stay positive.
Even in the absence of any initiatives that you might kick off to increase your north star metric, the counter metric might decline just due to random noise or seasonality.
You don’t want to randomly start a fire alarm when it’s not actually actionable!
Therefore, be judicious in selecting a counter metric threshold that is lenient enough to allow for noise and experimentation, but also one that is strict enough that it provides value.
After all, if you only tripped the alarm once the counter metric fell by 50%, your business is probably in a really bad spot!
Combining Counter Metrics with North Star Metrics
Once you’ve determined what your north star metric is and what your counter metric is, ensure that you’ve implemented them in your tracking.
You’ll want to report out on these in product review meetings or in project retrospectives. You want to ensure that you’re being accountable and transparent for these metrics.
You don’t just want to report them at the aggregate level. You’ll want to demonstrate how each of your initiatives has changed both the north star metric and the counter metric.
After all, if your initiative isn’t helping the north star metric, you’ve likely deviated from your strategy, because your north star metric is supposed to be measuring the most critical component of success.
And, if you’re not looking at how each of your initiatives is changing the counter metric, then you don’t really have a sanity check in place at all!
Closing Thoughts
Part of the joy and the curse of product management is that you need to find a balance between art and science.
While you can be highly scientific in measuring particular metrics, you still need to make subjective decisions on what thresholds to create and how to pair different metrics together.
Use north star metrics to ensure that you have focus. Use counter metrics to ensure that you’re still taking care of what matters and that you’re not taking detrimental shortcuts.
Have thoughts that you’d like to contribute around counter metrics? Chat with other product managers around the world in our PMHQ Community!
