Fooled By SaaS Metrics: Churn

Metrics are data formatted to deliver meaningful information.

Metrics shouldn’t lie, they are meant to be an objective view of the business. Metrics are backward as well as forward-looking. They help with monitoring the current health of the business as well as enabling accurate forecasts.

Yet, it is easy to fall into traps that can lead to misleading conclusions from metrics.

Founders need to understand what the purpose of a particular metric is – why they are measuring it and what is it that they propose to do with the information. Here we will look at one such metric – Churn. Churn is arguably the cornerstone of all SaaS businesses. Measured right it can inform impactful product decisions and done wrong can lead to a minefield of product mishaps.

Churn refers to any customer who is retained in the current period as compared to a comparable previous period. So if you’re measuring month over month churn then all customers from last month who are no more customers this month contribute to churn. Similarly, quarter over quarter churn is measured against customers that were paying for the product in the previous quarter vs current quarter. Churn calculations should not include any new customers acquired in the current quarter.

Pretty straightforward. But a number of SaaS companies have annual or quarterly contracts where the customers pay upfront for the service. How do you account for churn in those situations? Say you signed up 100 customers on an annual contract the past month. Monthly churn, in this case, will be zero for the next 11 months at a minimum as customers continue to be ‘paid users’ on the service month over month. Is that the right number to both measure and report? No.

The objective of churn is to identify if the business is losing customers and at what rate. It should answer the question “Is my product delivering on the value proposition that the customer expects ?”

Churn is a direct indication of level of satisfaction with the service. Churn indicates if the business is a leaky bucket where customers sign up and leave quickly or a sticky service where paying customers sign-up and continue to stay because they are happy with the value proposition.

High churn means there is something wrong with the value proposition leading to a lot of customers deciding to leave the service. What if the customers have signed up for longer term contracts where they have no option to leave the service, even if dissatisfied? This is like paying for an annual subscription for a magazine that you realize doesn’t meet up to your expectations after a couple of issues.

You can solve this by measuring churn only against customers who are “up for renewal“.

This means the denominator for a monthly churn calculation should only customers who have a renewal decision to make. This will include all customers on a monthly plan and those annual customers whose plans are expiring in the previous month. The numerator is the number of customers who have decided not to renew. So at 10% churn in this case means that 10% of the customers, who could have renewed the plan, decided not to renew.

Churn = (# of users who did not renew) / (Total users ‘up for renewal’)

Another key question that churn can answer is “Is my service improving over time?”

Measuring churn for an ‘average user’ significantly reduces the usefulness of the metric. Here, different types of users – users who joined at different points of time, or who belong to different customer segments- are combined into an ‘average user’ for measurement. To understand if the service is improving, you should measure churn against cohorts of users acquired over a period of time. It can be monthly cohorts – all users acquired in a particular month constituting one cohort.

If the service is improving its value proposition, subsequent cohorts should have lower churn than earlier cohorts. This means every month the product is retaining increasing percentage of users, a very healthy sign of improving product value.

Finally, calculate both user-churn and revenue-churn. User-churn refers to unique customers or logos that a business is losing in a particular period. And revenue-churn is the total $$ value of revenue that is churned. A single large value account can skew results by leading to a low revenue-churn percentage even if a large number of customers are leaving the service.

Expansion revenue in a few large value accounts can also lead to low or negative revenue churn, where current period revenues from existing customers are higher than the previous period due to greater usage of the product within existing customers. The business could also be moving to a different customer mix – from large number of small accounts to a few large value accounts. This is not bad if that is the conscious strategy of the business. If not, it can hide a dangerous underlying trend.

In a nutshell, to get the most out of your churn metrics, watch out for three traps:

• Only count churn against users who are “up for renewal”. Else you will understate churn and it will fluctuate as annual contracts come up for renewal
• Segment users to calculate cohort wise churn. Cohorts can be based on time of signup, type of plan, geography, etc. This will enable you to determine if the service is improving over time or how different segments are viewing the value proposition
• Calculate both user-churn and revenue-churn and evaluate the relative trajectories

[The article was first published on LinkedIn and has been reproduced with permission.]