Why your analytics initiatives are failing and how to fix it

You’d be hard-pressed to find a business executive who doesn’t see the value of decision-making based on insight derived from reliable data. But, in the COVID-19 era, businesses are extremely cost-conscious, and most are reluctant to invest in data science and analytics because the value is untested and the costs unknown. So, how do you strike the balance?

Start with an analytics value case

Wait. A what? A value case is a rapid proof-of-value exercise that:

1. Addresses a specific business problem or need
2. Uses a small multidisciplinary team, including data scientists, data engineers and business domain specialists
3. Delivers tangible business benefit

By leveraging analytics value cases, businesses can incrementally move towards intelligent operations, while managing costs. This helps create essential buy-in across leadership by demonstrating return on investment (ROI).

By using an agile, sprint-style approach, analytics value cases are able to rapidly understand the business problem and its impacts. The approach comprises of 3 ‘layers’, each building on its predecessor, and typically takes 10-16 weeks, depending on the complexity of the problem. After every sprint, a review – or playback – session is conducted with business to ensure clear and continuous delivery of value into the business

#1 No clear business problem

If you’re vague about the problem you’re trying to solve, no amount of technical genius or analytical knowledge will magically solve it. It’s important to spend the time upfront understanding the challenge and the pain points you’re hoping to address. One of the ways to do this is by defining a problem statement: The problem of [statement of problem] affects [business areas / stakeholders affected], the impact of which is [statement of issues / costs / other impacts]. Once you have the problem statement clearly defined, you can effectively prioritise it against other problems.

#2 No business champion or product owner

A value case team without a product owner is like a ship’s crew without a captain. The product owner helps the team decide which features are critical to achieve the desired business outcome. They need a strong understanding of business needs, and preferably some experience with technology, digital and analytics teams.

Just like the ship’s captain, the product owner sets the course, defining the value case roadmap, which gets the team to the minimum viable product (MVP). By working closely with the team, guiding the MVP development, the product owner is able to validate the work done through testing and measuring outcomes against business objectives.

#4 Infrequent or irregular business feedback

To understand what someone wants you must talk to them. Not just once. You need to talk to them often to ensure you still understand what they want. By following a two-week sprint cycle, with playbacks at the end of every sprint, you ensure each iteration meets the key needs of the business. This also gives the business an opportunity to adjust the focus, instead of waiting until the end of the project to realise something wasn’t 100% aligned to their needs. By co-creating the final product in this way, you also ensure buy-in from key business stakeholders.

#5 Not enough time upfront to understand the problem and the data

Upfront understanding of the business context and the data you are working with is critical to success. Going back to our layered approach, layer 1 is about validating the business problem, data exploration and the initial surfacing of insight. From here, build a simple proof of concept (POC) to showcase the vision of the solution, and define the backlog of work to build the MVP. 

Our layered, iterative approach keeps you on track, ensuring you’re solving the problem and not finding excuses to build a solution that may not address the problem appropriately, or uses excessive resources to get there. 

A layered, iterative approach can help ensure you’re solving the right problem

In layer 2, if it is required, you expand on the POC and build the MVP in two-week iterations, each one building on the previous in terms of functionality and model outputs (and incorporating valuable stakeholder feedback from the playback sessions). Once you have an MVP that meets the business need, you finalise and operationalise the solution for deployment in layer 3.

Think about it like making a cake – you can’t put the icing on until you’ve made the batter and baked the cake. But, you can decide to bake cupcakes rather than a cake once you’ve made the batter. That’s the benefit of layer 2 – if necessary, you can pivot your solution based on the feedback you get from the business playback sessions.

#6 No clear benefit framework agreed upfront

A value case must be supported by clearly defined metrics that will shift quantifiable business outcomes, like revenue, cost or risk. You may have a clear business problem and a good understanding of the solution, but are the expected benefits defined with business, and do you know how they will be measured and over what period of time? 

You can understand the potential benefits and metrics by asking some key questions:

1. Which area of the business will be impacted by the solution? 
2. What are the quantitative and qualitative benefits expected?
3. How do they relate to the company’s bottom line (cost and/or revenue)?
4. What’s the baseline calculation against which to compare the new solutions effectiveness?
5. Has the product owner (or business sponsor) signed off the benefit framework? 
6. What are the costs of delivery and support, post-go-live? 
7. What is the expected actual ROI? 

Keep the metrics simple. Ensure they are SMART (specific, measurable, attainable, relevant and time-based) to help create clarity and alignment with all stakeholders involved. 

#7 Poor data quality

Analytics value cases are driven by data. Garbage in, garbage out. Start by assessing the state of the data upfront. Engage with data governance structures (if they exist in the business) to understand what data is available and how it will be used. Be mindful of and address inherent biases in your data, as this will impact the quality of the output. 

Having a focus on data quality is not only important for the piece of work undertaken, it is also a great opportunity to improve the maturity of data within the organisation. Value cases serve a business purpose with a clear business owner, and it’s therefore critical the data supporting it is well managed and curated, and readily available.

The quality of your data will impact the quality of what you get out

#8 Lack of a cohesive architecture to ensure a sustainable, maintainable solution

Plan and architect your solution upfront, then discuss it with IT infrastructure and operations teams to ensure it will work in the technology landscape. Giving them a surprise at the end isn’t going to win you any friends for future initiatives. A data platform that can manage the development, testing, deployment, data quality and governance, and maintenance is great, but it’s also very expensive and few businesses have this luxury. Ensuring your development, data science and operations teams are aligned on what is being built and how it will fit into the existing landscape ensures risks and issues can be assessed and addressed early. 

#9 Long development cycles

Things change. And in a digital, post (during?) COVID-19 world, things change fast. If you agreed a business problem, then closeted yourself away for 3 months building a solution, you could very well find the problem you were solving isn’t even a problem anymore, or it’s changed so much that the solution doesn’t quite fit anymore.

By applying an agile, iterative Scrum approach, you can ensure:

• Regular delivery of value through tangible outputs
• Constant alignment of outputs to needs through regular playback sessions

DevOps combines development and operations to enable application and service delivery at high velocity. DataOps, which – as the name implies – combines data and operations to bring speed and agility to end-to-end data pipelines. And MLOps (Machine Learning and operations), is a practice for collaboration between data scientists and operations professionals to help manage the ML lifecycle. By combining DevOps, DataOps and MLOps, you can create a sustainable environment for development, testing, build, deployment and maintenance of all your value case solutions.

#10 Inability to operationalise the solution

Many businesses want to use machine learning, but few are ready – or equipped – to integrate it into their day-to-day operations. There are a number of factors that can influence this, from a lack of understanding at leadership level, to cost and technology landscape constraints. But, the best solution in the world is completely worthless if it’s not embedded into day-to-day operations. 

Adoption and operationalisation of a solution are often the hardest parts of a value case, and it’s largely a people issue. People are averse to change, and getting them to adopt a new way of doing something, and then entrenching that new behaviour, requires a culture shift and a strong change management and support framework. 

Solutions require ongoing support and maintenance if they are to continue to deliver the benefit they were designed for. During the initial build phase, this is managed by the product owner, who guides and steers the process. But, when a solution is operationalised, it’s handed over to an operational support team and, if the handover is not effectively managed, it can fail. Ensure the operational support teams are part of the solution, that it’s intended benefits and inner workings are clear to help safeguard the solution, ensuring it achieves its potential value for the business. 

We’ve driven out some great value cases over the past few years, some achieving as much as 26x ROI over a 9 month period. Because their benefit and measurements are defined upfront, ROI of analytics value cases is extremely quantifiable. Take care to avoid the pitfalls of value cases and you’ll be sure to see immediate results to incrementally move you towards a future of intelligent business operations.

If you’d like more information about value cases we’ve supported our clients with, or would like to understand how we can help you with your analytics value cases, get in touch.