30 August 2022
4 min read
The financial services sector was one of the first industries to widely adopt predictive modeling, starting with Bayesian statistics in the 1960s and evolving after the advent of neural networks to deep learning and beyond. Machine learning applications in this industry are endless, whether in auditing, fraud detection, credit scoring, or others. These use cases have only increased as technology has improved and become more commercialized.
However, regulatory guidelines are still evolving as technology and applications rapidly change. Despite the widespread deployment of predictive and machine learning models, much of the research and frameworks around explainability and transparency in consumer credit lagged in adoption and the ability to be implemented at the same scale to ensure inclusive economic growth. As a result, modern models are often based on historically biased data and therefore impact individual lives.
In the United States, over 40 million people have no credit score, and more are classified as having bad credit. These two groups are often treated the same and denied financial support - decisions now usually made in tandem with data or machine learning models - and little recourse is provided as to why this decision was made or what they can do about it. Unfortunately, Black Americans are disproportionately impacted and comprise a large percentage of this group.
As we think about creating rights-preserving technology that is financially inclusive of diverse groups, we must ask ourselves:
Consumer credit panel discussion from the Stanford HAI event. Image from this video source.
These are the topics many of us across the private and public sectors wanted to explore and offer innovative research on, and why we collaborated with FinRegLab in what ultimately culminated in a white paper and symposium hosted by the U.S. Department of Commerce and National Institute of Standards and Technology, Stanford Institute for Human-Centered Artificial Intelligence (HAI), and the FinRegLab.
At RelationalAI, we focused on applying domain knowledge, or semantics, to counterfactual explanations to generate actionable and computationally efficient explanations. Before we get into this, let’s take a step back and look at the landscape of explainability in machine learning.
The panel focused on explainability and fairness in machine learning. Image from this video source.
Research in this space has been going on since the early to mid-2000s, with early explanation techniques focused on feature attribution scores coming out around 2016. This includes well-known techniques like LIME and SHAP. These techniques, released more broadly in 2016 and 2017, are still being improved today and focus on answering the question of which feature played the most significant role in influencing a prediction. Other explanation techniques include rules-based explanations, explanations for deep nets, etc.
While the state-of-the-art in explanations is constantly evolving as machine learning as a field continues to grow, what we at RelationalAI attempted to address in this research collaboration is the gap in ensuring consistent, actionable explanations that are also computationally efficient to produce. This is an important aspect to manage, especially in the context of adverse action, since any consumer-facing prediction (especially if declined) should be accompanied by a method of recourse that a customer could act on.
To do this, we incorporated the following research, GeCo, into the collaboration.
Illustration of a counterfactual explanation. Image from Max Schleich’s GeCo presentation.
At a high level, the way counterfactual explanations can be leveraged in the real world is that if a loan applicant submits their information and receives a denial as an output of a machine learning model, a counterfactual explanation is provided alongside that prediction that explicitly calls out what it would take to be approved.
In the simple example scenario above, increasing the submitted income by $200 and closing two accounts would be a form of actionable recourse by the applicant.
Description of how counterfactual explanations work. Image from Max Schleich’s GeCo presentation.
A common challenge with explanations, in general, is that many are not computationally efficient to produce. The realm of possibilities as to which variables can be toggled or adjusted to generate explanations is a massive search space. On top of that, many of the explanations generated are not even actionable by a consumer, resulting in significant amounts of wasted computation, money, and time.
For instance, we would not want to include any results that require a decrease in age, or starting and securing a Ph.D. in one year.
Example of how domain knowledge, or constraints, can be implemented to generate actionable recourse. Image from Max Schleich’s GeCo presentation.
The critical differentiator to our approach to counterfactual explanations is the incorporation of knowledge through constraints expressed via declarative programming.
This effectively does two things:
This is just one area where we see the opportunity to combine knowledge with data to unlock new capabilities in financial services, especially in the explanations space.
Like other areas of machine learning and AI, the field of explainable AI or responsible AI is constantly evolving. We also wanted to explore many different areas, but we have slated these for future research.
Other active research we would like to explore includes:
Given the complexity of bias in machine learning and AI, specifically bias in models used for consumer credit, it seems likely additional research and an ensemble of techniques are required to address different parts of the deployment process. There is not yet one model to rule them all.
All businesses regularly confront difficult decisions. Even when these decisions are constrained (perhaps by budgets) and the cost implications of the decisions are complex, optimization algorithms may nevertheless provide decisions that minimize cost and maximize benefit. Unfortunately, optimization can become very difficult as the number of decisions increases and this makes it an excellent use-case for quantum computing. Declarative languages like RelationalAI’s Rel make the integration of quantum optimizers simple.Read More
Rel is not a procedural language. It is a relational declarative language with roots in logic programming. Rel goes far beyond a simple type system by allowing you to define the rules your creations have to obey. Definitions are not methods that turn input into output, but rather rules that can be evaluated in both directions.Read More
Rel supports relations as arguments, recursion, variable arguments and inlined definitions that reference other inlined definitions. These features give you tremendous power so you can express your business logic clearly and concisely. Power that was typically reserved for procedural languages is now available in your models and queries.Read More