> The Appraiser Coach > AMC Guide > OREP E&O ★★★★★ “They are knowledgeable, professional, and understand urgency.” – Joe Thweatt

Appraisal Bias? The Inconvenient Truth
of a Flawed Report

by Isaac Peck, Publisher

You can’t always judge a book by its cover or a report by its summary results. At least not without some scrutiny.

In the summer of 2022, in Baltimore, Maryland, the National Community Reinvestment Coalition (NCRC) conducted a “mystery shopper experiment” with the item being a residential appraisal. It concluded that appraisers are racially biased. Here’s what a lazy analysis failed to find.

The NCRC’s experiment had bi-racial couples engage local appraisers (who knew nothing of the experiment) to perform an appraisal of their home as a way of testing how appraisers would value the home based on race. In half the tests, the African American spouse answered the door and acted as if they needed an appraisal for a divorce or loan. The other half had the white spouse answer the door. In addition to a Black or white spouse, each house was “whitewashed” or “blackwashed” (as the NCRC puts it), meaning family photographs and cultural objects were removed to make the home racially neutral. Twelve fully-completed “matched pair” appraisal reports (from three homes) were ordered and delivered as part of the experiment.

The results from NCRC’s experiment were revealed in an October 2022 report titled Mystery-Shopper Testing In Home Appraisals Exposes Racial Bias Undermining Black Wealth, which have been widely cited and reported on by both local and national news outlets.

The conclusions the NCRC draws have been shared by the media and are obvious in the title: home appraisals are racially biased and undermine Black wealth.

Specifically, one of the key takeaways of the data, NCRC writes, is that appraisers assigned higher values to homes when they believe the owners are white ($7,000 higher per property). The report states that the two “most glaring” examples are 1) a home valued 12.9 percent higher when the white spouse presented ($40,000 variance), and 2) a home valued 9.1% higher with a white spouse ($46,000 variance).

What the Report Doesn’t Say
One key data point that the NCRC does not disclose is that the two most glaring examples of racial bias were performed by the same appraiser. We might call this appraiser an “outlier,” which in statistics is defined as “an observation that lies an abnormal distance from other values in a random sample from a population.”

In fact, if we remove this one appraiser as a data point, the data flips the other way! The results of NCRC’s experiment actually show that appraisers (on average) favor Black homeowners and assign their homes higher values.

(story continues below)

(story continues)

But perhaps more glaring is that the one “outlier” appraiser, who was so pivotal in skewing the data, is Black!

[Name Redacted], the appraiser who “undervalued” the first house in the NCRC’s “most glaring” example, and who “overvalued” the second house in the NCRC’s other key example, is a Black appraiser.

Blind to Data
Whether the NCRC failed to dig deeper or ignored what they found, this reveals a flawed conclusion. Nevertheless, the NCRC filed a discrimination complaint with the Department of Housing and Urban Development (HUD) against [Name Redacted] as a result of this experiment.

The HUD complaint against the Black appraiser reads:
“There is a significant disparity between how Respondent-valued a home that was presented as Black-owned, and how Respondent valued a home that was presented as White-owned on multiple occasions. This disparity constitutes a significant difference in treatment based on race.”

HUD has not responded to WRE as of this writing. The Black appraiser in question performed a total of two appraisals as part of the experiment.

Service Issues
Besides the NCRC’s data analysis issues explained above, the report also indicates that Black homeowners experienced “unprofessional treatment from appraisers,” while white homeowners did not. “One appraiser refused to complete an assignment for a Black homeowner. Another took 11 weeks to complete a report for a Black homeowner,” the NCRC writes.

Here too, the NCRC filed a discrimination complaint with HUD against the appraiser who took 11 weeks to complete a report for a Black homeowner (actor).

When WRE investigated, we learned that the slow appraiser called out in the report had a pretty good reason: he had experienced a medical emergency—which affected all of his reports during this time frame. The appraiser also reported that the homeowner/actor (and the NCRC) never asked him why the report was late before filing a HUD complaint against him.

It is also worth noting that this appraiser who turned in the report late actually valued the Black (actor) homeowner’s home at a higher value than his appraiser peers.  This is another inconvenient truth the NCRC fails to include in their report.

As to the appraiser who refused to complete the report. Maybe he or she saw something fishy and decided to steer clear. Not a bad idea, given that the NCRC filed discrimination complaints against two out of six appraisers who participated in the study (a 33 percent “hit rate,” according to WRE’s simple math).

Total Variance
Setting aside the fact that the two most egregious examples of racial bias were performed by the same (Black) appraiser and that with his data removed from the analysis the results actually show favoritism toward Black homeowners, the total variance of the basic data NCRC collected is worth looking at.

First, the NCRC reports that out of its twelve matched paired appraisals (six tests, two appraisals per test), the total of the valuations provided to the white homeowners is $2,418,000 and the total of the valuations provided to the Black homeowners is $2,377,000.

This is a 1.7 percent variance in valuations of the same homes, which is within the historic 5 percent variance range for appraisals. Regardless of the facts, NCRC writes that the results are indicative of “the severity of racial bias” within the profession and supports their argument that “sweeping and aggressive action” is needed from policymakers and the industry itself.

Conclusion
The NCRC’s report and experiment is a great example of how selectively presenting data and filtering statistics can misrepresent the reality of a situation.

For an organization that has “more than 20 years of experience…executing mystery shopper testing in housing and lending markets to probe for discriminatory business practices,” the fact that the NCRC has failed to disclose that both of the “most glaring” appraisal examples of racial bias in their experiment were performed by one appraiser—an outlier—and that this appraiser was in fact a Black appraiser, raises very serious questions about the credibility of the NCRC’s data.

The NCRC, however, considers the experiment a glowing success. In an online webinar, Director of Fair Housing at NCRC, Tracy McCracken, touted this as the first experiment of its kind and said its testing methodology was a promising tool to root out “racist” appraisers.

“We that work in Fair Housing and Fair Lending know that testing will be essential to root out appraisers that discriminate,” McCracken said.

In the same webinar, both McCracken and Jake Lilien, NCRC’s Counsel for Fair Housing Enforcement, indicated that they recommend additional testing specifically in Alabama, Georgia, Louisiana, Mississippi, North Carolina, and South Carolina. Lilien wrapped up the call by urging other Fair Housing organizations to reach out to him so he can help them run their own tests on appraisers.

(story continues below)

(story continues)

“There are certain things I don’t want to say publicly about how we did the testing, because who is to say we’re done testing appraisers. I have a feeling other groups may try to use our same practices and we don’t want appraisers to know what these practices are. So I can’t necessarily share with you specific details about how we contacted appraisers, what excuses we gave, etc. But if you work for a Fair Housing organization, reach out to me and I’d love to talk to you,” Lilien said.

Their methodology is so good, they’re eager to share it.

It appears that NCRC is planning more experiments in different locations in the coming months. Both McCracken and Lilien were broadcasting from Mobile, Alabama at the time.

Let’s hope they take the time to analyze the data as thoroughly as appraisers have to when before arriving at a conclusion.

Stay safe out there!

>> IMPORTANT: In an environment where somebody is always blaming the appraiser, you want a team with deep expertise in your corner. Specializing in appraiser insurance for over 20 years, OREP has built a network of attorneys and USPAP experts who have been defending appraisers for decades.

We’re proud to announce our partnership with trial attorney Craig Capilla, one of the most experienced appraiser attorneys in the country who has handled nearly 1,000 appraiser cases in his career. In addition to being on OREP’s litigation panel in several states, all OREP Members now enjoy a one-hour consultation with Mr. Capilla (at no charge) when facing a state board investigation, HUD discrimination complaint, or a CFPB inquiry.

OREP provides up to $250K of Discrimination Claim Coverage for individual appraisers and up to $500K of coverage for Appraisal Firms.

Click Here for a Quote (in 4 Minutes or less!)

About the Author

Isaac Peck is the Publisher of Working RE magazine and the President of OREP, a leading provider of E&O insurance for real estate professionals. OREP serves over 10,000 appraisers with comprehensive E&O coverage, competitive rates, and 14 hours of CE at no charge for OREP Members (CE not approved in IL, MN, GA). Visit OREP.org to learn more. Reach Isaac at isaac@orep.org or ( or (888) 347-5273. Calif. Lic. #4116465.

OREP Insurance Services, LLC. Calif. License #0K99465