Audience Engagement Digital Publishing
6 mins read

Understanding Loyalty: How to measure, understand and fix the retention rates of your readers

padlocks on bridge with the word thanks on it

“Gone are the days of chasing traffic,” said Stat’s Rick Berke at Nieman Lab recently. To which we say,“Amen.”

Now, while there will still be publications for whom traffic is an important indicator of ‘success’, there are other, more nuanced, ways to discern whether or not what you’re producing represents a good return on investment.

In this climate, failing to keep a weather eye on how content is performing – and genuinely performing at that – is daft at best, reckless at worst (and though we’re an optimistic bunch, we’d lean towards the latter here).

It’s hardly revelatory to say that quality content and successfully implementing reader revenue models go hand in hand. At this juncture, there can’t be a publisher on the planet who doesn’t understand that. Content must deliver value to its readership, or that readership will scatter to the four winds faster than you can say ‘fake news’.

So what should we be paying attention to, if not traffic and volume? Well, as Julia B. Chan noted last year: “It was right in front of us all along — it just wasn’t pageviews.”

She’s right: it’s loyalty.

How do you measure loyalty, then?

Well, while there’s no industry standard, that hasn’t stopped publishers gravitating towards certain methodologies.

Recency, frequency, and returning visitors are the commonly used methods of assessing reader loyalty – and it’s easy to see why. If a reader has visited your site this morning and tends to visit daily, those two measures combined represent a pretty good indication that they’ve formed a habit around your content. ‘Habitual’ is, after all, a word with exceedingly positive connotations for publishers when they’re studying reader behavior.

As with all metrics, however, looking at each individually isn’t likely to be particularly insightful. Yes, it’s a solution, but it isn’t optimal, and that’s the thing.

Loyalty is a human behavior, requiring a different algorithmic approach

Loyalty as a concept spreads itself in several ways: it may be felt for the brand as a whole, but it may also express itself in a section (think the hugely successful crossword subscription at the New York Times); in individual authors (Renan Borelli’s excellent piece for Nieman Lab argues exactly this: that personal brands carry increasing weight); or for an overriding political, geographical or ideological standpoint (The Washington Post; The Bristol CableThe Local).

It’s not as simple as plotting the course between A and B. There are complexities involved in getting to the bottom of what ‘loyal’ means because, at its core, it’s an emotive reaction to something, and emotional responses can be tricky to map through algorithms.

A definitive definition

Deep in the data bunker at Content Insights, the team spent the summer grappling with the definition of loyalty, the lack of consensus about what it was and – critically – how it could be measured.

“When you want to solve something, you first need to define your problem,” says our Head of Labs, Ognjen Zelenbabić. “Why? Well, if you’re just trying to calculate something without a clear definition, then you’ll get results that you can’t explain.”

Recency and frequency are certainly steps in the right direction, but, within those two metrics, there are multiple ways to gather the information and process it.

“If you calculate frequency, do you calculate it on session cookies or user-lifetime cookies and, if you do, what does it mean?” says Ognjen. “How do you treat users who come several times during the day? Do you calculate them once, or every time they come? You have a bunch of different approaches just for frequency and also for recency. How do you combine them? What does it mean for you? It’s really difficult to explain and you need to know a lot about how it’s implemented and how it’s calculated to be able to know if that number works for you.”

That’s a whole bunch of questions right there. What’s the solution?

“Sequentially highly engaged”

The definition the team settled on and have pinned their colors to is that loyal users are ‘sequentially highly engaged’. If it sounds quite maths-y, you’d be right: it is, after all, a technical explanation of the issue. Bojan Popic, our Head of Customer Relations, puts it this way: “in a non-mathematical sense what we mean is ‘routinely highly engaged users’”.

What both of them mean is this: a loyal user doesn’t just have to have visited the site recently. They don’t just come back time and time again. They don’t even just consume higher volumes of content than your average, random user. While all three of these things might be true, the behavior must be shown to be habitual, not just current or frequent. To be specific, over the course of a 15-day period a CI-defined ‘loyal user’ will have visited a site at least eight days out of those 15. This is a rolling parameter: look back over the past 15 days at any point and this must still hold true.

“Think about your own experiences,” says Bojan. “If you’ve genuinely formed a habit around something, you’ll be doing it regularly. You won’t have two-week gaps between reading one article and the next – and if you’re paying for those articles, going that long between reads is exactly the kind of behavior that sounds like you’d be contributing to the churn rate in the not-so-distant future…”

Of course he’s right: loyalty is as much about sustaining a habit as it is about forming one.

From loyal users to loyal behavior: you can’t have one without the other

It’s critical, you see, that those loyal users have been identified first, before you even start thinking about assessing the content itself for signs of loyalty.

After all, loyalty is a human behavior, not a browser event.

So, when people talk about single articles triggering ‘loyal behavior’ it makes no sense. Articles by themselves can’t be indicators of loyalty. How loyal-defined users interact with a given piece of content, though? Well, that’s something that’s possible to ascertain.

The distinction may be subtle, but it’s vital: it’s not a single step process. Once you know who your loyal segment are – say a certain 10 percent – then you’re in a position to start looking at how those users behave across the website. From here, you can drill down into articles or sections of the website, and start to ask the kinds of questions any sane human would ask: what were those loyal users doing in the article? How highly engaged were they with the piece? What was their read depth?

Always, always you’re asking those questions through the lens of your loyal user base, not through individual articles or sections.

What you’re asking, therefore, is how to know loyal users interact with content on your website. Once you know this, it becomes easier to understand how to nurture this user base so they continue to stick around, and how it may be possible to emulate that kind of success elsewhere across your platform.

Actionable information

All of this information can feed back into discussions about growing this vital user base, ensuring that readers remain engaged with the product they’re presumably financially – and personally – invested in.

Ronan Borelli talks of a lifelong devotion to a particular sports writer, whom he and his father have followed from news outlet to news outlet over a 25-year span. That’s the kind of loyal following you can only dream of. “A great writer’s name carries weight, and their most loyal readers will follow them anywhere.”

Perhaps we all have writers who inspire that kind of devotion – and it would be nice to think those writers emanate from our own publications. What is true is that loyalty is going to be something that only grows in importance this year as it becomes increasingly important to understand how and why readers start paying for news (and continue doing so).

We’re not aware of a standardized definition across the industry yet, which is why we’re taking a deep breath and offering up ours to you. With the parameters defined, real questions can now be asked – and should be.

by Em Kuntze

Republished with kind permission of Content Insights, the next generation content analytics solution that translates complex editorial data into actionable insights.