With the first industrial revolution, optimizing and scaling processes became urgent. Since then, technology has gained the spotlight by increasing productivity, optimizing earnings, and helping companies make intelligent decisions. At the beginning of the 21st century, DevOps simplified the production lifecycle and added a new component, Big Data (Lwakatare et al., 2015). The enormous amounts of available data created significant opportunities for data science, specifically machine learning (ML) models. However, ML models are only as valuable as the currency of data, iteration of results and trust of their users. These challenges can be addressed through an approach of constant model nurturing and deployment, called MLOps. In this article, we will explore the features of MLOps and why you need it.
What is MLOps?
MLOps accounts for retaining data to train, deploying ML models, and integrating the required resources to maintain and evaluate performance. The first step is to formulate the problem the model aims to solve. It is essential to ensure it is aligned with the business objectives. Then, review the data sources available and describe the architecture. At this point, it is crucial to determine how the data will be made available, how frequently incoming data arrives, and the tools required to achieve an efficient and effective pipeline. Once the data is available, a thorough process is needed to ensure quality. That step includes identifying and managing outliers, rebalancing, and determining which features contribute to the data model. An efficient pipeline must produce a clean dataset for training and optimization to fit the selected models for experimentation. Thus, testing different models will lead to choosing the most accurate ones. Completing this stage makes it possible to automate and deploy the pipeline. Ultimately, the models will need to be periodical evaluated and retrained to ensure adequate performance.
The figure below illustrates the MLOps process:
The Challenges of Machine Learning
With the increasing use of machine learning solutions in business, data architects and scientist teams have faced challenges in each step of the process (Tamburri, 2020). The shortage of data and its messiness is often stressed as a considerable difficulty (Mäkinen et al., 2021). Besides, companies lack qualified personnel to develop and implement the process end-to-end. Data scientists also face the challenge of reproducibility, which could be solved with version control. However, it is vital to notice that developing and comparing trustable models can be time-consuming due to the availability of proper tools. Finally, there are vague or unrealistic expectations, which can often lead to a lack of buy-in from the executive side.
The Advantages of MLOps
MLOps solutions increase cooperation within the different sectors of an IT department. As in DevOps, MLOps aims to improve the speed of processes such as model development, deployment, monitoring, and enhancement. Bringing hybrid teams together helps in the adequate management of the machine learning lifecycle. There are tools in the marketplace that support automation, autoscaling, and continuous integration/continuous deployment (CI/CD). Additionally, some platforms offer an advanced data-drift analysis to improve model performance over time.
On the inventory side, it reduces a tremendous amount of time by keeping a catalog of data assets and models available. A data registry encompasses business definitions, technical metadata, operational insights, and data lineage. In addition, curated datasets can be shared with other users who want to work on the same asset simultaneously. A data catalog can also offer control access over assets. It means that only certain users who have permission can see specific data and projects.
One of the most significant advantages of MLOps is comparing and contrasting. The process framework enables data scientists to retain version history and model steps to facilitate auditing. Automatic machine learning tools also accelerate the selection of features by testing all variables against several pre-defined models. It automatically handles missing values and outliers but also allows the user to include their preferred methods before incorporating the variable into the model. Once the models have been generated, the user can evaluate each model and determine what is most suitable to answer the business questions.
The Risks: White-box Models vs. Black-box Models
White-box models deliver prediction outcomes and their influencing variables. Therefore, their predictions are fully explainable. A keyword for this type of approach is traceability. It is possible to visualize and comprehend the data lineage through the white-box system. It must be transparent and accessible to clarify model-based decisions to other business stakeholders.
Black-box machine-learning projects deliver high precision; however, they offer minimal or nonactionable insights. Additionally, these models lack the required transparency in the data-driven decision-making process to input some accountability.
Designing interpretable models (i.e. white-box) can be an arduous task, if not impossible, in some cases. But even considering less complex business problems, black-box models generally outperform white-box solutions. That happens due to models’ capability to grasp high non-linearity and relations between features. AutoML adds value to companies that rely on users with a mixture of skill levels and backgrounds. For this reason, both approaches can be helpful but in different contexts.
Why You Need It
MLOps has gained momentum over the past few years, and it will grow to become part of daily operations soon. The primary purpose of implementing MLOps is to employ machine learning models more efficiently to support the business and solve problems. The solution allows models to be exhaustively tested and make it to production, instead of residing in the development stage. Additionally, it centralizes the deployment and dismisses the need to update and manage several systems.
Assessing the health of individual machine learning models consumes valuable time that could be used to develop new models. For this reason, often, models are deployed across departments with different or no maintenance procedures. It is also probable that the data has never been refreshed. Ultimately, the model’s performance is rarely or never evaluated due to work overload. MLOps offers a centralized platform that enables collaborative development, automated testing, accelerated deployment, constant lifecycle monitoring, and enhancement when available or needed.
Bianca Firtin is a Senior Consultant at CTI, Data & Analytics Practice.
References
Lwakatare, L. E., Kuvaja, P., & Oivo, M. (2015, May). Dimensions of DevOps. In International conference on agile software development (pp. 212-217). Springer, Cham.
Skogström, H., Laaksonen, E., & Mikkonen, T. (2021, May). Who Needs MLOps: What Data Scientists Seek to Accomplish and How Can MLOps Help?. In 2021 IEEE/ACM 1st Workshop on AI Engineering-Software Engineering for AI (WAIN) (pp. 109-112). IEEE.
Tamburri, D. A. (2020, September). Sustainable mlops: Trends and challenges. In 2020 22nd International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC) (pp. 17-23). IEEE.