|
Getting your Trinity Audio player ready...
|
A/B testing can yield valuable insights for publishers to improve their offerings incrementally. A new report guides them on how to effectively implement A/B testing in their organization.
Publishers are increasingly using A/B testing to boost engagement and revenue. Inc42, India’s largest tech media platform with 1.5M monthly users, used insights from A/B testing to increase its average monthly sales volume by 100% and customer retention rate by 43%.
Spanish digital publication El Confidencial (EC) increased subscriptions by 60% after using A/B testing for various offers to identify the ones that resonated most with its readers. These included identifying the most effective ways to feature premium articles on its homepage, more engaging call-to-action buttons, and revenue benefits of discounted prices.
“Growth is achieved by changes”
“The incremental change A/B testing offers is one of the biggest drivers of publisher growth,” according to a new report, “The publisher’s guide to A/B testing,” from Digiday and Piano. It presents an overview of the critical aspects of A/B testing and suggests tactics for success.
“Growth is achieved by changes — in setup, in pricing, UX, etc.,” says Alexander Kreybig, Director of Strategic Services, Piano. “But every change represents a risk. A/B testing mitigates the risk and enables you to test that change and see if it brings statistically conclusive results.”
For publishers with existing subscription programs in places, looking for ways to optimize and incrementally improve conversions is essential. Understanding how an audience is responding to content and offers, and then injecting those insights into your subscription strategy will allow for continued growth.
The publisher’s guide to A/B testing
The report covers:
- How publishers can identify KPIs for A/B testing
- How they can set up and run an A/B test on their publishing platform
- How to interpret the metrics and statistics behind A/B testing
- How to identify A/B testing partners and resources
“Think simply. Test a single thing”
There are usually multiple variables at play in the achievement of any objective, publishers need to focus on the factors that are critical to their overarching goals. For example, a publisher focused on growing subscriptions may refrain from increasing the price as that could impact growth even if it indicates potential for greater revenue.
“Imagine you test the price and at the same time the variant has a different price and color, and then you get better results,” says Kreybig. “Did you get the results because of the price change or the color change? There are plenty of examples like this where you don’t understand the results at the end.”
Think simply. Test a single thing — what you truly think will make a difference. Not five, one.
Alexander Kreybig, Director of Strategic Services, Piano
Timing is a critical factor in A/B testing. The report emphasizes the need to understand when to stop testing and use insights to optimize the campaign and then move on to the next test.
“Metrics that can reflect growth”
Coming to testing approaches, while there are multiple options, “there’s no single ‘right’ way to conduct A/B tests,” according to the report. However, “publishers aren’t required to know the ins and outs of testing methodologies with modern A/B testing platforms. They simply need to identify the metrics that can reflect growth.”
Which leads to the tools, technologies, and partnerships that will enable publishers to conduct A/B testing. Kreybig recommends large partners or technology solutions that have industry benchmark data available.
“When publishers are looking at their performance, many are asking themselves if [the numbers] are good or bad. But they have no place to check because nobody’s publishing it. When you have a large enough partner who has a few hundred sites, that partner can help you understand your results in a broader context.
Alexander Kreybig, Director of Strategic Services, Piano
The full report is available here:
The publisher’s guide to A/B testing
