challenges embedding data science in a business

More and more businesses are looking to invest in data science capabilities and expand their knowledge of their customers with deeper data insights. Whilst the appetite for these 2 areas may be there, many businesses will invest without properly assessing their own capabilities to support data science and the additional roles it may need - below is my experience of some of these challenges.

,As big non-tech organizations start investing to develop machine learning and data analytics capabilities, failure to deliver to management’s expectations and frustration among data scientists are becoming commonplace. 

Data scientists and data engineers are among the best-paid professionals in the tech sector. They usually combine a strong background in mathematics, statistics, and computer science. They are extremely motivated: most have arrived at the field through self-learning as there are not many university degrees in “Data science”, and invest important amounts of time to keep up with the last developments in the area.

However, for big companies, integrating these capabilities into old legacy systems is becoming a great challenge - making implementation of growth models difficult between data scientists and business stakeholders, and a technological gap are among the main contributors to this challenge.

In the world or high service distribution - where businesses may be managing thousands or indeed millions of skus, not only does the apetite need to be there at a senior level but there needs to be a capable  infrastructure and ability to support the aspirations – my own experience of trying to bed in data science capabilities has been interesting – with failure to get senior buy in and indeed build out opportunity to improve the wider drive the opportunity leading to a lot of frustration.

The following challenges sum up my own experience of trying to bed in a Data Science team - and whilst not exhaustive, they give you a good picture of some of the challenges this key but much misunderstood area can experience.

Be wary of 3rd party recommendations

Be careful with external consultants advocating the embedding of data science capabilities when, on reflection, the infrastructure may be too challenging to deliver this. Having robust interrogation from data dumps which consultancies have extracted and modelled using their own tech and models may look great and provide key insights, however is more challenging when you have a new team come in try to do similar with limited tools and abilities.

Whilst your consultancies may come in and sell the dream, they (and you)  also need to understand if the actual structure of the business can support the aspiration. It is essential that part of any external recommendations includes a full review of the data infrastructure to support.

Raise Awareness in the business

Embedding any new team in a business can be challenging – but embedding a data science team can be especially so – the appetite for this shiny new team who work magic with data may sound great but you need to ensure the business understands exactly what it is, key stakeholder relationships and build on these stakeholders trust to ensure all know objectives (outwith technical ability to embed and deliver).


How does Data Science work with Analytics? How does Data Science improve on already existing but localized and poor data modelling? How does Data Science work with BI Team to ensure it has access to resources to deliver modelling?   Understand business tech capabilities

What are the tech capabilities to fully support embedding data science within the organization? Is their a suitable supporting architecture to ensure data is both easily accessible and manipulable to build outline Proof of Concept models? Does that data architecture enable engineering to schedule and pull data to support data science POC?

In my own experience, the business was under prepared to support Data Science for a number of reasons – not least of which was a gap in technical capabilities stemming from years of legacy underinvestment.

Understand business data strategy

Is there any Data Strategy in the business? Who owns the data strategy and is there full clarity re where data resides? Do you have any existing data access tech to support understanding the data infrastructure? What team in the business can support all of the above?

In my example the answer to all was a negative one – historical underinvestment in architecture had led to a legacy challenge to get any semblance of usable data – ultimately leading to frustration on behalf of the team and inability to develop more robust value driven POC models

Define stakeholders before embedding

Who are the stakeholders needed in the business to support data science and can they technically do this? Or in our case – is it one person who has no time to spend supporting data scientists on a day to day basis?

This caused until frustration – brick walls having to pulled down and this can be tough. Especially when Data Science can often be a team coming in to build on and improve on already existing but poorly managed, delivered and executed models. This can cause resentment, confusion and conflict when multiple teams may all believe they are doing the same thing.

It is also essential to get a business sponsor when embedding Data Science in any business – this is not only a new function but one which a lot of people my struggle to understand – let the team get on with outline POCs based on strategy roadmap without having to worry about managing the politics and conflict embedding team may result in.

Summary

Setting up and launching any new team in a business is challenging. Setting up and launching a Data Science team is especially challenging when legacy architecture, data access and overall lack of buy in from wider business can affect the ability to deliver valuable outline POCs. Data Science, when embedded correctly in the business can help drive Ecommerce KPI growth but only if the bedrock of the business is structured to support. 

Consider some of the above  - laying the groundwork almost - before even considering if Data Science can actually deliver value in your organization.