Learn Blog, Governance Mark Nanning 21 April 2023

From Deployment to Monitoring: Deeploy’s Comprehensive MLOps Lifecycle

6 Steps to Understand Deeploy’s Comprehensive MLOps Lifecycle

MLOps is a combination of data engineering, DevOps, and Machine Learning with the objective to automate the end-to-end process of developing, deploying, and managing machine learning models in production. Deeploy’s platform provides a best-of-breed solution for deploying and managing machine learning models, including deployment, serving, prediction and explanation, human feedback loop, model management, and model monitoring. With Deeploy, organizations can ensure compliance with regulations and policies while maintaining control over their predictions.

The Premises of MLOps

Machine Learning Operations (MLOps) is a field that brings together three important domains: Data Engineering, DevOps, and Machine Learning. MLOps automates the end-to-end process of developing, deploying, and managing Machine Learning models in production, making it more scalable, repeatable, and manageable. By combining these domains, MLOps creates a streamlined and automated process for organizations to leverage the power of Machine Learning and generate value for their business.

Image showing the intersection between Machine Learning (ML), Development Operations (DevOps), and Data Engineering (DE) in the context of MLOps. The diagram features three overlapping circles, with ML, DevOps, and DE written in each circle, respectively. The overlapping areas are labeled MLOps and illustrate how the three fields come together in this practice

Deploying Machine Learning models brings them to life, adding value to your company. However, models should remain accurate, reliable, and explainable throughout the lifecycle. Therefore, Deeploy simplifies this process, allowing organizations to quickly bring their models to life and keep control of them.

With Deeploy, organizations enable interaction between humans and Machine Learning models. Additionally, Deeploy’s platform allows organizations to ensure compliance with regulations and policies, which is crucial for high-risk AI use cases.

How Deeploy fits into the MLOps process

Deeploy’s platform covers the deployment step and beyond, ensuring that your ML model predictions stay in control. The platform offers a best-of-breed solution rather than an end-to-end solution.

Image representing the ML lifecycle and coverage provided by Deeploy. The image features a block diagram with five stages of model development and training: data preparation, model building, model training, model testing, and model evaluation. The image also depicts the coverage provided by Deeploy for deploying machine learning models. The coverage includes six areas: 1) Model deployment, 2) Model serving, 3) Model prediction and explanation, 4) Human feedback loop, 5) Model management, and 6) Model monitoring. The image also indicates that model debugging is the last step of the lifecycle but not covered by Deeploy. Overall, the image illustrates how Deeploy's coverage fits within the larger context of the ML lifecycle.

Deeploy’s deployment process is customizable, offering a variety of options for organizations to seamlessly deploy their models on the cloud or on-premises.

Deeploy offers an integrated model-serving experience using KServe and Kubernetes, ensuring uninterrupted availability for end-users. The framework can scale models based on usage patterns and traffic volume, and even supports serverless deployments for batch prediction systems.

Deeploy’s platform offers explainable AI models with various techniques such as feature importance, SHAP, and LIME values, enabling users to comprehend model behavior and make informed decisions. We also provide Conversational XAI, a new innovation to enhance decision-making for all stakeholders in the organization.

Deeploy’s platform enables organizations to incorporate human feedback into their machine-learning models, ensuring continuous accuracy and reliability. Data Scientists or Risk & Compliance officers can provide feedback or evaluations, allowing for ongoing monitoring and training of models.

Deeploy’s model management capabilities automatically log and trace every model change and prediction, making it easy for users to manage and monitor the models’ performance.

Deeploy’s model monitoring capabilities enable real-time performance monitoring of models. The platform offers various monitoring metrics to identify issues and take prompt corrective actions.

After developing and training your models in your own infrastructure, you can easily bring them into production in just a few clicks.

What are the supported model frameworks:

1. Model Deployment

Firstly, the deployment process is straightforward and intuitive, allowing organizations to deploy their models seamlessly. The platform offers a wide range of deployment options, including cloud or on-premises. Deeploy’s deployment process is fully customizable, allowing organizations to choose the deployment options that best fit their specific use case.

What are the requirements to deploy my ML models in minutes?

- - Model code in the repository in our preferred format
  - Access to the repository – public/private
  - Knowing which commit to select
  - Knowing which model type/framework are using
  - An API endpoint/integration

Our default deployment flow uses KServe as a deployment backend. We also support deployments in AWS SageMaker.

2. Model Serving

Once the model is deployed, Deeploy provides an integrated model-serving experience, ensuring that the models are available to the end users without any downtime. The serving framework is based on KServe and runs on Kubernetes. This allows organizations to scale their models up or down based on their usage patterns and traffic volume. In addition, it even provides the option for serverless deployments which is ideal for batch prediction systems.

Resource consumption

For advanced configuration, “CPU request”, “CPU limit”, “Memory request” and “Memory limit” can be configured. In addition, the model and explainer can be deployed serverless (Once deployed, serverless apps respond to demand and automatically scale up and down as needed). Especially, if model predictions do not need to be continuously explained, it can be helpful to deploy the explainer serverless to save resources.

3. Model Prediction and Explanations

Deeploy’s platform provides explainable AI models that help organizations understand how the models make their predictions. The platform offers several explainability techniques, including feature importance, SHAP, and LIME values, to help users understand the models’ behavior and make informed decisions. With our latest innovation, we provide Conversational XAI to make even better-informed decisions for every stakeholder within your organization.

What is the supported explainer framework?

- - Anchor tabular
  - Anchor images
  - Anchor text
  - SHAP kernel
  - Counterfactual
  - Partial Dependance Plot
  - What-if
  - Conversational XAI

4. Human Feedback Loop

Deeploy’s platform allows organizations to incorporate human feedback into their Machine Learning models, ensuring that the models remain accurate and reliable over time. As a Data Scientist or Risk & Compliance officer, you can give feedback or evaluation, allowing organizations to continuously monitor your models’ performance and use the provided feedback for model training.

Where is the feedback stored?

The feedback on predictions is visible within Deeploy and in parallel stored in an SQL database that can be accessed and used by the Data Scientist. Within that database feedback can be labeled. For now, feedback is not automatically labeled as it is collected via a free text field.

5. Model Management

Deeploy’s model management capabilities automatically log and trace every model change and prediction, making it easy for users to manage and monitor the models’ performance.

6. Model Monitoring

Finally, Deeploy’s model monitoring capabilities allow organizations to monitor their models’ performance in real-time. The platform provides several monitoring metrics allowing users to identify any issues and take corrective actions quickly.

What are our monitoring features?

- - Traffic
  - Performance
  - Evaluation
  - Drift (coming soon)

Streamline Your ML Lifecycle and Ensure AI Governance

In conclusion, Deeploy offers a comprehensive MLOps platform that covers the in-production aspects of the machine learning lifecycle. This enables organizations to responsibly deploy and monitor their high-risk AI use cases. Furthermore, Deeploy provides opportunities to implement Explainable Machine Learning models and enables interaction between AI and humans in the loop.

As a Data Science and/or Risk and Compliance expert, interact with AI using Deeploy, to streamline your ML lifecycle while ensuring compliance with your AI governance.

Want to know more? Contact us!

Want to stay updated?

Please fill in your e-mail and we update you when we have new content!