Machine Learning - success starts with these 10 steps

“Machine Learning (ML) can take an organisation’s digital transformation to new heights” — It’s a statement we hear time and time again, but in practice, it doesn’t achieve that warm and fuzzy turn-key transformation feeling the statement asserts. By Santiago Giraldo, Director of Product Marketing at Cloudera.

  • 3 years ago Posted in

The truth is that the promise of ML can be difficult to attain, but it is ultimately there for the taking — for those ready to embrace a new way of thinking. While some deem ML as a pie-in-the-sky assertion that’s too good to be true, others are grabbing it by the horns and witnessing the true value it can bring to a business. In fact, according to Forbes research, the global machine learning market was valued at $1.58B in 2017 and is expected to reach $20.83B in 2024.

There’s no denying that in order to see the benefits of ML, businesses have to embark on a new kind of data journey — one that may seem difficult, or even uncomfortable. But once an organisation has full scale ML models in production, the benefits are endless. It can help to increase revenue, decrease costs, and even help teams work smarter and do things faster. It’s also sustainable, if a company is willing to work at it.

The thing is, ML is not always easy to implement. We often see teams running into the most issues when bridging the gap from simply dipping their toes in the ML waters to getting to grips with full scale ML production. Luckily for them, these barriers are easily overcome.

In order to achieve enterprise ML success, there are ten proven steps for an organisation to follow:

Taking a holistic approach

When it comes to embracing ML models in all their glory, leaders have to adopt the right mindset and take a holistic approach. Before it can become a driver for change, ML must first be treated as an integral part of an organisation's data strategy and be baked in from the very beginning. By integrating ML from the start of a project and running it alongside existing IT environments, processes, applications and workflows, organisations can drive better business results. This is because the ML will be continuously learning and developing from the very beginning, ensuring they are working to the best of their ability from the get go.

Evolve the organisation to embrace ML

For businesses that have already dabbled in ML, they will have noticed that there’s a wall between experimentation and large-scale adoption. This wall is there because an organisation may lack the knowledge and skills needed to weave ML development, production and maintenance into their existing processes, workflows, architecture and culture. That’s why embracing ML requires flexibility in the structure of a company and their approaches to how they manage their data. Data scientists and engineers should work closely with leaders, to lead them on the right path when it comes to managing data and use these insights to guide business strategy.

Building a multi-disciplined team

A crucial part of successfully implementing ML is recognising that people are just as important as the technology itself. To build a team that can support ML models in their day-to-day functions, collaboration and freedom from organisational restriction are key. A data scientist will want a platform and tools that give them practical access to data, compute resources and libraries, without feeling tied down by red tape or access barriers. Whereas leaders will want to see the ROI from adopting ML from the beginning. Bringing a team together from a range of disciplines ultimately means ML models can answer a range of organisational needs and power better business decisions.

A willingness to experiment, and fail

While there are many benefits that come with ML implementation, from automating processes to solving business issues, at its core, it’s about science. Proper science takes experimentation and observation, as well as a willingness to accept the failures alongside the successes. Fortunately, when it comes to ML, even failures can be viewed as victories; once an organisation finds that a specific business problem can’t be solved with ML, that knowledge frees up efforts to be focused elsewhere. Every experiment should be learned from, and these learnings should form the basis of data strategies moving forward.

Iterating quickly

A common mistake for any company wanting to jump on the ML bandwagon is a rush to create an ML model that’s flawless from the start. Instead, they should recognise that getting to grips with ML is a process and let teams experiment rapidly, fail early and often, continuously learn, and try new things. This way, they can ensure ML models are performing in the best way for the organisation, fuelled by the right data and insights to drive the business forward.

Choosing the right technology to optimise the data lifecycle

Another important aspect of creating ML models is having the right technology to optimise the lifecycle. Data engineering and data science teams need the ability to work across and control the entire journey an ML model goes on with a business. This lifecycle can be divided into two phases:

1. Holistic ML development and the building of ML models

2. Getting to production, scaling and ongoing operations

The right platform and tools will empower your teams to work seamlessly across both of these phases — to ensure ML models are built properly, are put into production at the right time and scaled with the organisation.

Maintaining integrity

Looking ahead to when an organisation has successfully deployed a few ML models at scale, they need to know that their work is far from over. This is because the underlying data driving those models shift over time, and the models need to react accordingly. Once an organisation has an effective model in place, keeping it fine-tuned takes ongoing effort to ensure it’s working accurately. The aforementioned involves continuous reviews of how ML models are performing, how they are reacting to changes and the impact this will have on the future of the algorithms and the business they serve.

Closing the skills gap

When it comes to choosing the right team of people to work and support ML models, organisations should try to build a team with experience, talents, and capabilities across a wide range of skill sets. Companies should think about including everything from data engineering and data science to DevOps and product development people into these teams. That’s because a range of people will bring different perspectives and knowledge levels to all the projects in hand, ensuring the best results. The more diverse the team is, the more its members can learn from one another and grow collectively.

Treating models in production like living software

While ML models must be maintained, they must also be protected. And that means having visibility into model lineage and monitoring who can access and make changes to them. Putting access restrictions in place ensures only those who need or should amend ML models are able to do so. Taking these steps will maintain their integrity and accuracy — two crucial aspects to any successful ML model.

Abiding by ethical obligations

Lastly, businesses have to take into account the ethical considerations when it comes to ML. For starters, organisations must have consent from customers and other stakeholders before applying the necessary data against an ML model. By establishing and adhering to a rigorous set of ethical ML standards early on, companies will save time and a huge headache trying to retrofit practices later down the line.

By following these ten steps, businesses can start their journey to seeing ML success. As ML spend continues to grow, organisations need to invest in their ML modes. IT and business leaders have to ensure the integrity of the models are maintained, access is only granted to those that need it and they have the right team in place to develop and grow models with the organisation. The benefits of ML are there for the taking, but only if enterprises are willing to commit to the process. Implementing ML can seem like a daunting task, but by following these ten steps, it doesn’t have to be.

By Danny Lopez, CEO of Glasswall.
By Oz Olivo, VP, Product Management at Inrupt.
By Jason Beckett, Head of Technical Sales, Hitachi Vantara.
By Thomas Kiessling, CTO Siemens Smart Infrastructure & Gerhard Kress, SVP Xcelerator Portfolio...
By Dael Williamson, Chief Technology Officer EMEA at Databricks.
By Ramzi Charif, VP Technical Operations, EMEA, VIRTUS Data Centres.
Companies are facing a Catch 22 when it comes to the need to invest in new forms of AI, whilst...