Getting to a nondiscriminatory outcome with AI

Proponents of the use of artificial intelligence trumpet the fact that a machine left to its own devices is able to make unbiased decisions and prevent fair lending violations.

Yet, a concept common to computer science from its earliest days is “garbage in, garbage out.” In other words, the underlying data used for a system to generate a decision could be responsible for perpetuating a fair lending bias because that underlying data was the result of past biased decisions.

“Since so many of the generative and machine learning AI applications are based on consuming historical data, the risk of encoding bad data, redlining for example, is very real,” warned a paper from consulting firm BlackFin Group.

AI has a tendency to repeat the past. “You’re building up on a set of histories,” said Andrew Weiss, a partner at BlackFin who was one of the authors of the report.
He is a former Fannie Mae executive who ran the group that developed Desktop Underwriter, which is rules-based technology.

In making a decision, the lender needs to be able to justify why it was made, Weiss said.

“Using AI to do the traditional underwriting job may not really be a good idea,” he said. “On the other hand, using AI [for] part of what underwriters do, which is examine documents and make sure that those documents contain valid data, that’s something that isn’t really subject to the same kind of judgment.”

Artificial intelligence has become a catch-all term for technologies that are capable of several different activities, and to some people, it’s like a “superpower,” said Tim Ray, CEO of VeriFast.

“There’s lots of nuance and edge cases and people are people; everyone has a story and everything’s not so black and white,” Ray pointed out. “So how do we use AI to help with those decisions?”

It only does what it is taught to do, he continued.

Agreeing with Ray about AI becoming a buzzword, “Almost everything is getting called AI these days,” Weiss commented. “People are relabeling old things that were invented before AI was ever really used.”

So far, only a few use cases for AI have been approved for real estate finance, said Mortgage Industry Standards Maintenance Organization President David Coleman. One is for code generation, and the other is for monitoring functions around documents.

“I haven’t talked to anyone yet that is looking at a production version of AI for decision making,” Coleman said. What has come out in MISMO workshops on the topic is that the industry wants more discussion about it, as well as the development of a taxonomy and a vocabulary.

He noted the government already has put out a dictionary with approximately 517 terms.

“We’re not looking to recreate anything but what we want to do is make sure this is where the industry is in making the best decisions,” Coleman said.

Its best use case in the mortgage process may not be in making the decision, but getting to the point where a decision can be made, several people interviewed for this article noted.

If a person is not making the decision, in theory using AI should lead to better or more unbiased outcomes, said Subodha Kumar, the Paul R. Anderson Distinguished Chair Professor of Statistics, Operations, and Data Science at Temple University.

“We have to understand [that] biases not only come from data biases but also come from algorithms,” Kumar said.

Some people have proposed that AI algorithms in mortgage lending be designed to exclude race as a characteristic.

“The problem is that even if you don’t use race data, there are many other features that will be related to race,” as thus used as a proxy, Kumar pointed out, adding it is not a problem related to mortgage; it has been raised when it comes to facial recognition.

But one of the beautiful things about AI is that it can sort through so many variables that allow for a more holistic view of that data, Kumar said. That allows for the reduction of bias.

“But the most important thing is that there will need to be oversight,” Kumar warned. Manual oversight provides checks and balances on the statistical side.

An algorithmic fairness technique mentioned in a paper by the National Fair Housing Alliance and Fairplay AI is called distribution matching.

It is a technique that uses additional objectives or targets so the outcome for any one group closes resembles that of the control group, said Michael Akinwumi, the NHFA’s chief responsible AI officer.

As a result, the outcomes are also distributed fairly, “so that the risk of discrimination is minimized,” Akinwumi said.

“When [AI] is used responsibly, it can actually expand the credit box for consumers,” he said, giving them more opportunities to obtain housing, both purchase and rental.

If the algorithms are trained to regard disparities in lending outcomes as another form of error, mortgage approval rates for Black and Hispanic homebuyers can be increased by between 5% to 13%, this study found.

As the Townstone Financial case proves — even though so far it has been a loss so far for the Consumer Financial Protection Bureau — fair lending enforcement actions aren’t limited to lending decisions, but also around marketing and customer contact.

Last September, the CFPB issued AI use guidance for credit denials, pointing out the technology is not a get out of jail free card when giving consumers the reasons why they did not get their desired product.

“Technology marketed as artificial intelligence is expanding the data used for lending decisions, and also growing the list of potential reasons for why credit is denied,” said CFPB Director Rohit Chopra, in a press release. “Creditors must be able to specifically explain their reasons for denial. There is no special exemption for artificial intelligence.”

In other words, using a complex algorithm does not abrogate the lender from its responsibility under the Equal Credit Opportunities Act.

“Where we really think it matters most is getting people off the phone, it’s getting people out of email, and letting them do their job,” said Hoyt Mann, president of Alanna.ai, which works with the title insurance industry.

Society wants instant 24/7 access to the information that they need, and AI is perfect for being inserted into the communication chain, Mann said.

But Mann has a strong opinion on one important aspect of AI usage: “We have to keep humans in the loop.

“Our work is going to change, but humans have to stay in the loop,” Mann continued. “There’s a risk for utilizing AI in different parts of your business and understanding the levels of risk is important.”

Using it for communication has a lower level of risk, as long as the lender is not giving out information that changes the course of the transaction, Mann said.

The Townstone case hinged on statements made over time on a radio program that the CFPB said discouraged a protected class from applying for a mortgage.

It comes back to the training data used for the AI system, Mann said, as well as how recent that information is, and importantly, how the users ask the question of the system.

When people ask questions of each other, it is in a context predefined by their roles.

“AI doesn’t necessarily have that unless you set it up,” Mann said. “People ask ambiguous things all the time, and without that context, the answers can be a little bit wonky.”

That is why users have to verify the answers AI gives. If anything, when giving an erroneous response, AI has been known to double down, so it is pertinent to ask where it got its answer from, Mann said.

Microsoft’s product is called Copilot; Mann termed that the perfect name, because “where [AI] needs to be is in the passenger seat. Not in the driver’s seat.”

Vishrut Malhotra, the CEO of Rexera, previously worked at large investment management firms on Wall Street, which would use quantitative algorithmic models to trade.

Those firms understand “that humans have conscious and subconscious biases, and those lead to poor decision making, which lead to poor performance in portfolios,” Malhotra said. So algorithms were created that could remove those biases for the most part were effective at that task.

“So I think there’s a lot we can learn from that industry in terms of how do we test? How do we look for bias? How do we look for poor decision making in these models?” Malhotra said “I think those ideas carry over even in the new world of AI.”

Step one is checking your data, Malhotra said, the next is to understand what the model does. Importantly, the model should not be a black box.

For any AI application the user has a complex number of steps or a complex workflow. “What we do is we break it up into smaller pieces and we ask each AI model to perform one small piece so that it doesn’t become this large black box model for us,” Malhotra said. “It’s easier to understand the decision making of a model if the steps are smaller.”

Rexera next pushes to the models to provide a detailed chain of thought, explaining how it came to the decision, so it can further mitigate the black box issue.

An approach to take for mitigation is to create an alternative model and use that as the check.

“Let’s say you have a model that makes some decisions and you are concerned that there may be bias in the data or within the model itself,” Malhotra said. “What you can do is create another model, whose job is to evaluate for bias and be sort of this bias detector.”

The second model can ask the first to convince it that no bias in the decision making existed.

Agreeing with Mann, Malhotra said the best way to think about these models right now is if they were interns, someone who needs the right oversight, training and support to perform their tasks.

An AI application that Rexera offers to clients is to evaluate if a loan could be eligible for purchase by Fannie Mae or Freddie Mac. But it does not decide if that loan will be approved. It is a low risk use case because Rexera is relying on quantitative data sets, Malhotra said.