Banking: The Benefits of Explainability
Making Transaction Monitoring Explainable
Through the use of Deeploy, bunq meets the hard requirements of AI usage in banking:
- Explainability: leading to more control and higher performance by providing explainability to stakeholders, providing feedback, and actively improving the AI models.
- Compliance: being compliant with the constraints and requirements of the DNB.
- Manageability: control, monitor, alert, making sure the AI models are fair, sound, and used accountable, therefore trustworthy AI.
About bunq
bunq is the second largest neobank in the EU, founded in 2012 by Ali Niknam. From its inception, bunq set out to challenge what banking is and can do. Pioneering many things considered impossible, the innovations bunq brought to the banking industry fundamentally changed how millions of people spend, save and invest their money.
Safety of its users is at the heart of bunq. For years, the neobank has been using AI to monitor and detect fraudulent transactions. Thanks to the transaction monitoring system built in-house, bunq dramatically reduces false hits by a factor of 2.5, compared to a “rule-based” approach often used by traditional banks. Moreover, it’s completely scalable (which is important for a growing scale-up). As of recently, banks are allowed to monitor transactions by a combination of rule-based models and AI models. Every transaction goes through this system and if either one of the two components generates a hit, the transaction gets sent to the compliance department for manual checking.
Being at the forefront of using AI to make users’ lives easy, bunq manages to both improve the user experience as well as user safety. bunq was one of the first to use data science to streamline its onboarding, screening, and risk processes. Ultimately, this enables a broader collaboration between the financial sector and online players, as AML can only be tackled effectively if companies work together.
Challenges in transaction monitoring
Bank transactions are monitored by a combination of a rule-based systems and machine learning. The Machine Learning system assesses the probability of fraud for each transaction based on historical patterns, profile information, and transaction details. In case a transaction is flagged, an investigation by a Compliance Operations specialist (ComOps) follows. The investigation can lead to a hit, reporting the user to the Financial Intelligence Unit (FIU), a hit and not reporting the user to the FIU or clearing the transaction
To keep control of both implemented systems, it is key to put monitoring, explainability, traceability, and the human-feedback loop in place between the Data Scientists team, the machine learning system, and ComOps agents.
It’s crucial to explain why hits are raised, giving ComOps agents clues about where to start their investigation, enabling them to handle hits efficiently. Furthermore, explainability is required in order to control the risks and provide effective human oversight for every stakeholder.
Together with bunq and the ComOps agents, Deeploy developed a tailored transaction monitoring explainer to make the model interpretable to the stakeholders, and to provide a good starting point for investigations. Other important aspects include continuous monitoring and an effective feedback loop.
”It’s crucial to explain why hits are raised, in order to handle hits efficiently by our ComOps team, to give clues about where to start the investigations. Furthermore, explainability is required in order to control the risks and provide effective human oversight for every stakeholder.
Ali el Hassouni, Head of Data @ bunq
The Deeploy platform solution
Deeploy provides the platform that enables the Data Scientists of bunq to technically deploy their models while meeting all compliance constraints and requirements with respect to explainability, traceability, and monitoring of the AI models. Without such a platform, converting extensive data science modeling, time to production and compliance-ready deployments is simply not possible, meaning expensive data science time is thrown away.
The data team of bunq uses Deeploy to easily deploy their models, explain predictions to end users (stakeholders), monitor performance and feedback, and actively improve & update models more frequently. bunq and Deeploy collaborated on the development of a domain-specific explainer for transaction monitoring, making sure the model is transparent and explainable to everyone involved.
Tailored explainability
Deeploy and bunq collaborated on a tailored explainability method for transaction monitoring. The big challenge comes down to the fact that fraud might follow from a series of transactions, in combination with information about its users. This means that explaining just the most important variables in the model, won’t tell ComOps agents much. It’s crucial to give a full, understandable explanation of the Machine Learning Signals (ML Signals), such that they understand what to look for and double down on the investigation, or provide feedback in case the model is wrong.
The feedback loop is monitored through Deeploy’s platform by bunq’s data team. This means among others that the percentage of overruled predictions can be monitored and when there is an increase, this might be a reason to retrain models. The feedback loop enables bunq to retrain the models and explainers at the moments needed.
A technical insight: high-level architecture blueprint
How Deeploy is integrated within the tech stack of bunq, running on AWS.
- Jupyter + Nvidia Rapids are used to train their models on GPUs. The resulting model artifact is stored in object storage (S3) and a reference to the model is added to GitLab.
- Versioning and the sign-off workflow are running in GitLab. Deeploy integrates with Gitlab for versioning, metadata, and sign-off.
- The model artifacts (ONNX) are pulled from S3 by Deeploy using the model reference in GitLab. Deeploy Dockerizes the artifact with pre-built Docker images. The same process is followed for explainers. The deployed image runs as a microservice using EKS.
- The backend of bunq’s banking platform provides the necessary data to train models.
- Output is shown in bunq’s Dashboard, providing ComOps with an admin environment to work on transaction monitoring. The feedback loop is monitored through Deeploy’s platform.
With Deeploy, bunq now has a future-proof MLOps setup to keep control of current and future models, explain the models to their ComOps agents and regulators, and continuously improve them based on human feedback, a true loop of Responsible AI.
