|
Getting your Trinity Audio player ready...
|
More and more publishers are asking us the same question: just how do you compare analytics tools? How do you know you’ve got the most important things covered? What is good, and what is good for us?
So, based on the experience we’ve gained in the last decade, we thought we’d write a small guide.
What not to do
When comparing tools, there are a few things that can often go wrong.
When people find they’ve made a bad decision, it’s usually because only one department made the decision – and they didn’t have a good checklist in place for what the company as a whole actually needs.
Going for the cheap option (or the one everyone else has) can often be an expensive mistake too: often the buying process can be too one-dimensional and off-the-shelf, while looking simple, actually doesn’t address the issues you need sorted.
Keep in mind that although analytics are an important practical asset, they also have a deep strategic impact on an organisation as well. In this day and age, data is critical, and if it’s a disaster it can have serious everyday and long-term consequences. And we don’t mean the funny ones.
Know yourself-categories
To choose the right data analytics tool, you have to look in the mirror first. That means, first know thyself! Connect your buying plans to your workflow, technical depth, business strategy and use cases. Let’s run through the most important self-reflection categories:
– Strategy and ambitions
We’re always a bit surprised that this point needs to be made, but believe us, it does: it’s extremely hard to pick data software when the overall strategy is unclear. So ask yourself: what is the business model? What are the shared success metrics? Do your people and your company show ambition in using insights, or is it a hygiene decision for now? Also, because you really don’t want to be switching your analytics every year, where does the company stand in 3 years’ time? Will you have outgrown the tool you’re looking at then? Can it grow with you?
– Use cases
Based on the strategy, what are the data and insights being used for? It might be the case that your editorial team isn’t ready to create these yet, but it’s worth getting a couple of use cases ready so you can ask the analytics company how they would support these situations, issues and challenges. Don’t be afraid to ask questions about whether they have experience of these things with other clients. They should know – or be prepared to find out.
– People and culture
We all know that culture eats strategy for breakfast, and this is especially true for media and marketing. So selecting a nice tool does not necessarily mean that it will be well used. It needs to be a good fit.
Do you have data-hungry people in the newsroom, or are they implicitly telling you that they will not spend a lot of time on it because they ‘like their creativity’? Is it part of the culture to use data to improve, to justify actions, or to judge? And, while we’re on the topic, who will be the users: management, editors, marketers, analysts? All of them? The answers to these questions mean a lot on the level of data detail, or the simplicity of the interface that might suit you best.
– Your tech stack
Modern analytics tools don’t usually work in splendid isolation anymore – and that is a good thing. They will sometimes get numbers from your workflow systems, but they can also enrich your data. Know what data you have, what the quality is, and more importantly: what you are missing. Are the systems that are enabling the connection old and rigid, or modern and flexible? Tip: buying a good analytics tool can often save you money by replacing parts of that.
– Budget
You have that secured right? And it is a simple and clear decision-making process? Everybody understands the business case, right? Thought so. Just checking.
Ask the tool-categories
Now, powered by all that self-knowledge, you are ready to ask the right questions to the analytics company you’re interested in partnering with. We will give you a more detailed checklist later on, but they mostly fall into these main categories:
– Company vision and culture
First of all, ask yourself: is the analytics company an organisational fit? Are you both funky, innovative, creative companies, or more serious? Do you have the feeling that the people behind the tool understand your use cases, your mission? Do they come from your industry? Do they have the values and the do-good philosophy that speak to your heart? During implementations, issues or advice sessions you will be in contact and it is important that it just feels right.
Pay also attention to the mission of the product company. It is quite telling to see if they are just optimising and selling the tool, or whether they have an ambitious roadmap. What is the real difference between the diverse analytics companies in approach, and do you like that?
– Data and channels
Of course, this is pretty basic stuff: what exactly do they measure? Which channel or story metrics will be displayed and do you really need that? It is important to avoid the ‘black dog syndrome’ here. While you might think you’re both talking about the same animal, one could be thinking of a pitbull, and the other of a poodle. You don’t want to discover this when it is too late. If it is crucial data, be as precise as possible in your definitions. And ask how the data is measured and collected.
– Data flow and delivery
If you know your metrics, it is worthwhile spending some time on the actual delivery of data. This, in our experience, is where silly mistakes can be made.
How realtime is realtime? Does the data refresh every 10 seconds or 5 minutes? Speed in essence is good, but can lead to its own inaccuracies and increased data costs. Does the system have an API to get data in your data lake, and when are the daily reports really ready to be received in your mailbox? Can the data be automatically placed in your internal communication channels or CMS? In the end, these variables will play an important part in your experiences with the analytics tool.
– User interface, features and use cases
The way the data is served to users is the most underestimated aspect of analytics. While all the data is there, a boring or simply clumsy user interface can turn any newsroom off. Interfaces – especially if the newsroom uses it – should be tailored towards use cases, towards concrete decision making. Actionability is the key word here – graphs, numbers, lists, they should all point to action. In the right place, to the right person, at the right time.
Do you need data to be realtime, or rich and historical? Do you believe that notifications can make a big difference in activating storytellers? To make predictions? It can be all kinds of things. But remember: the system should not only give you data or even insights, it should energize and empower your team to make things better.
– Customisation, growth model
Maybe you have seen a really nice analytics tool, but it turns out to be really static. Chances are, it’s a one-size-fits-all, and has no possibilities to align itself to your workflow or technology or culture. Think hard about which customisations are necessary, and if the system allows you to be in the driver’s seat. On that note, it can be a great asset if the tool can slowly grow with your organisation; start with simple features and gradually build up possibilities into a more advanced model? Can you take your team by the hand and save budget at the same time that way?
– Security, privacy and safety measures
A SaaS solution can be funky but it always needs to be safe. Ask about security measures, hashed data and compliance with GDPR. Maybe your data cannot go out of Europe? Data will be the blood of your organization, don’t cut costs on this aspect.
And also: if there’s a disturbance, does the analytics company make the effort to recover the ‘lost’ data? The answer can be quite telling.
– Implementation and onboarding
Media companies and marketing companies usually ask a lot about the implementation efforts. They are usually short on time, understaffed, and they don’t want to depend too much on the goodwill of their tech departments. The technical implementation shouldn’t be too complicated. Totally understandable, the code speaks first. But don’t forget to ask questions about people. For example: do you get a lot of implementation assistance and are user training sessions included? You shouldn’t feel like you’re being left to get on with it alone.
– Assistance
To some of you it might sound counterintuitive, but in the end, story data is all about newsrooms, strategy, and daily tactics. It is the business of people. And it can make all the difference if you know that you can count on each other. Is there extra advice or data science available, to make a more complex diagnosis on your performance? Can you call them in the middle of the night, if something is really off? Do they have e-learning? We always tend to think you can judge that by the amount of questions the analytics company asks you, to really bring that value.
– Additional benefits
Next to these basic factors, some other benefits can be at play. Maybe the provider speaks your specific language or lets you cooperate with another strategic client of theirs. Sometimes it might be possible to be part of a lab setting, where you can try out stuff and give more feedback in return for a lower price.
– Roadmap
And if some of the stuff that you want isn’t there, it might already be on the roadmap, so ask if it’s part of the ideas they are working on – and if it isn’t see if they’ll consider adding it! Make a list of your need-to-haves and nice-to-haves and compare that to their roadmap. Please feel free to ask what’s in store for you. It might be an estimate, but in any case it will give you an interesting insight in the future of analytics.
– Price model
This isn’t just about what your budget will allow; it’s also about the price model in general. Do you pay a flat fee license price or do the costs go up when you grow? What happens if your organisation wants to measure more brands or channels? Are more users automatically welcome? That could be a vital point as you are trying to include the whole newsroom or marketing department in the data strategy.
– Pilot
After you have convinced yourself that this or that tool is the right one for you, a lot of digital publishers insist on doing a (paid) pilot or at least a trial period in the contract. Analytics companies that are confident about their product will not have an objection to that, so it can be a good idea to minimise some risks in adaptation, whether they’re technical or cultural. But be mindful of the effort and money that goes into this. A couple of tips to be as efficient and effective as possible:
- Be precise in what you are testing. A pilot is usually to take away last doubts, but what are they? Are you able to define success KPIs for a trial?
- If you are still pretty unsure if what you are testing will lead to a license, it can be wise to implement the easiest setup that will give you the necessary results, and adapt or refine that later into a more ambitious version. It saves time and money for both.
- Don’t hook up every possible user for a pilot, because they can be grumpy if you decide not to go with it. Involve the key-users for the most important use cases and force them to really make time for it. The most disappointing pilot is always the one that just doesn’t get enough attention. If done well, it will always pay off, because you learn in any case.
- Oh, and be sure to arrange good solid contracts for a pilot as well. It is still confidential data and you need the ownership for that. If you are more sure about taking on a continuing license, consider signing a contract with a trial period as part of that. It saves everybody from implementing and doing the legal work twice.
At the end of the trial consider the license period as well. You don’t want to be locked in forever with something you don’t like. But a longer period will be cheaper and will save you from considering your analytics every year. And trust us, you don’t want that. It will be a hassle and the historical data will make less sense. On top of that, your editorial team needs to embrace the data tooling that you choose and include it in their daily workflow.
We hope this is a useful first step in thinking about asking the right questions when you compare tools. If you really think it through and involve the right people, getting the right software will give you a lot more value and fun than you might imagine.
by Erik van Heeswijk
Republished with kind permission of smartocto, the world’s most actionable editorial analytics system offering a bird’s-eye view on The Story Life Cycle©.
