Independer works with Deeploy to professionalize their AI infrastructure
Deeploy supported Independer in future-proofing their AI organization, focusing on the control and manageability of their AI models, and compliance with (new) AI regulations. Human feedback loops and explainability play a key role, both internally and to consumers.
About Independer
Independer is a prominent Dutch comparison platform that has been transforming the way consumers make decisions about insurance, financial products, and services since its establishment.
Launched in 1999 and with over 20 million yearly visitors, Independer has earned a reputation as a trusted and independent source of information for individuals seeking transparent and unbiased comparisons of insurance policies, mortgages, loans, and other financial offerings available in the Netherlands.
The platform’s user-friendly interface empowers users to easily compare various options, enabling them to find the most suitable and cost-effective solutions for their unique needs. With its commitment to providing accurate, up-to-date information and empowering consumers, Independer has become an essential tool for those looking to navigate the complex landscape of financial products with confidence.
Independer provides comparison services for a wide range of products, including:
- Insurances
- Mortgages
- Energy suppliers
- Mobile and Internet subscriptions
- Savings and loan products
Current MLOps process at Independer
Independer is growing at a significant pace, requiring the scalability of its ML models. Independer has around 15 AI use cases in production within different areas of use, including models powering part of the platform, models for internal reporting, for personalized advice, and also for marketing purposes. The number of use cases has grown organically over the years, which has resulted in case-by-case specific implementation of models. This has also resulted in a more dispersed MLOps infrastructure, ranging from deployments directly on Kubernetes to use cases hard-coded on the website’s backend.
Moreover, Independer now requires the establishment of comprehensive end-to-end traceability of their models, as well as extensive experiment tracking. Next to that, model prediction logs are needed in order to ensure reproducibility.
The commitment to progress is also evident in the strides taken to enhance model monitoring and explainability, catering to the needs of both valuable stakeholders and the platform’s users.
A collaborative solution
Considering the current ML landscape, the following principles are important for Independer:
- Keep control of models over time, with monitoring and alerts on model performance.
- Ensure explainability of models to internal and external stakeholders
- Ensure traceability and reproducibility of model activity
- Have a feedback loop system in place to continuously improve the models
- Comply with (upcoming) regulations
Instead of disperse deployments across the organization, Independer drafted out a unified MLOps infrastructure where Deeploy plays a central role.
With the help of Deeploy, data teams can easily keep track of deployed models, explain predictions to end users (stakeholders), and monitor performance and feedback to actively improve & update models. This leads to better maintainability, lower risks, and better performance. Moreover, the new MLOps infrastructure takes into account financial regulations and AI regulations (like the EU AI Act) by providing model registry and checklists to make sure models are following the best compliance practices.
Continuous model monitoring, continuous controllability
Continuous model monitoring allows to keep control of the ML models in production and get notified in case the performance of a model drops. For all monitoring capabilities, customized alerts can be set on metrics such as accuracy, errors, and drift.
Notifications for alerts are customizable as Deeploy seamlessly integrates with Slack and email providers (e.g. Gmail, Outlook), notifying model owners immediately when something has gone over or under the set parameters.
Ensuring compliant models through traceability & reproducibility
With the help of Deeploy, fintechs like Independer can use audit trails and signoff flows for model deployment and serving. This registry of model activity is not only important internally but also important for compliance with regulations.
With clear and simple signoff flows, the deployment decisions are traceable, and relevant stakeholders are involved in the process (e.g. compliance officer). Depending on the use case, different signoff flows and layers are defined and documented. Stakeholders are then notified via email or Slack when their signoff is required.
Model predictions made explainable to every stakeholder
Explainability techniques are imperative, as they explain model behavior and its decisions to every stakeholder. Which general trends are represented by the model? Which feature contributed most to the prediction? What input would lead to a different decision?
Based on the requirements for the explanation in each model, different explainers can be created with underlying explainability techniques. Finally, an explanation is presented to the stakeholder. In this case, the conceptual inner workings of explainability for the “Personalized Search Result” is displayed, showing the three most important variables that led to an insurance product in the Top 3.
Providing feedback for model improvement
The feedback loop refers to the possibility for the stakeholder “in the loop” to approve or overrule the model’s decision and provide comments on the reasoning behind this overruling. This data is then taken into account and utilized to better model performance, leading to better predictions.
