Everyone wants to get the most from artificial intelligence technologies. Everyone, understandably, wants to use their data more effectively and efficiently. However, that doesn’t mean you need to build a data science team.
Very often, it’s more effective to invest in an automation, analytics and visualisation tool. If you do need a data science team, it’s worth making sure they’re set up to deliver the most value.
A costly journey
One of our clients has a team of over 15 full-time Data Scientists. This team was full of talented, intelligent scientists, but they were slow to produce models and results. For instance, it took more than eight weeks to build a model that ranked customers who were most likely to buy a new product or participate in a loyalty program. Every time there was a new product added to the catalogue – maybe because of a seasonal event – they restarted their Data Science life cycle to retrain and push a new model to production systems.
The client believed that these models should only take a few days to create, and they were right.
We delved deeper into this and realised there were two main problems. The first problem was that new datasets were required for each new model. However, the datasets were messy and required significant ETL processing before they were ready to be consumed by the team. We knew that if the data scientists had clean datasets, they could get a model up and running in a matter of hours.
We were asking data scientists to do a job they weren’t good at. It isn’t the job of the data scientist to wrangle, clean and prepare data.
The second problem was that there was a lack of clarity about what was ‘good enough’. Data scientists are all about the quality of the model, tuning hyper parameters and getting two per cent more, or improving it by 50 basis points — that’s what gets them excited. However, it isn’t always necessary to build the best, most perfect model. For example, if you have a $100 million renewal opportunity, you may not need to wait until your renewal prediction model works at 99.88% accuracy. It might be enough to move the product into production when you hit 90% accuracy. Getting a ‘quite-good’ product into use quickly might deliver more revenue than waiting to deliver something that is near-perfect.
A path forward
Data scientists are expensive. If you need an agile data science team, it is important you are getting the most value from them. One way you can help is to ensure you have clear definitions of what ‘good’ looks like. The second way is to ensure that the initial work of cleaning, wrangling and processing the data is completed by a data platform. Here is where ProArch’s Dataware comes into play.
To learn more about Dataware and how it can improve your readiness to apply data science to make or save millions, click here. If you wish to talk with one of the team, get in touch using the form below.