Blessed by the Potato

The Globe has an article out this weekend on Emili, CMHC’s automated housing appraisal system.

To break it down, when someone wants to take out a mortgage, they go to the bank and say something like “I’d like to borrow $500,000 for this house that I just purchased for $550,000, and I’ll pay the other $50,000 with my own money.” The bank then has to make sure that the $500k they lend will be paid back, by looking at the income and creditworthiness of the borrower, and also at the value of the house, so that if there is a default that the value will cover the mortgage. Even with 10% down, a loan for $500k is not very secure if the property is only worth $400k. Emili is an automated system to determine that house value (and, as I understand it, some of the other aspects of the loan), which makes the whole process a lot faster and more efficient than sending an appraiser to check out the house.

The problem pointed out by the Globe is that Emili is too generous. There are lots of reasons given in the article as to why, including this gem:

A CMHC spokeswoman said that staff are aware of “a handful of cases” in which Emili approved a mortgage for a non-existent house.

But let’s think about it logically. Emili could be perfect, always assigning the correct value to the house. Personally, I find that unlikely for many reasons mentioned in the article, such as that Emili can’t see inside the house to assess the state of repair or the level of renovations, and that known errors exist.

Emili could be good enough, but with a few mistakes made here and there. That’s a fairly likely scenario, after all many times those small matters of internal shape are not that important to the valuation, especially if there’s a decent margin of safety built in or if the land value is a significant component of the valuation. If Emili is mostly accurate with a few inescapable random errors, then we should see mistakes made in both directions. We should hear reports of buyers caught in the emotion of a bidding war, only to find their mortgage rejected because Emili won’t support the valuation, or of Emili erroneously denying a mortgage because it reports a vacant lot after a house was destroyed by fire (not having the record of the replacement), or coming in too low on valuation for some other reason. I’ve been keeping my eyes open for these sorts of anecdotes for years, and haven’t seen them.

That leads me to what I believe is the actual situation: Emili systematically over-values real estate.

Now, some over-valuation is to be expected. Lenders don’t want to turn away business by cutting the appraisal too fine, and the insurers will build this into their models. They may also assume that prices generally go up (at a modest rate), and since it generally takes time for a default to occur some upward bias can be tolerated. But when extreme dislocations occur — such as bidding wars leading to “winning” offers hundreds of thousands of dollars too high, or certain neighbourhoods seeing annual appreciation way above the norm (like 20%/year) — then the system should be flagging those as problems and denying the loans. That would act as a natural brake on a bubble.

Though it is important on a national, system-wide basis to avoid bubbles, nobody on an individual level has much of an incentive to implement brakes. The banks want to lend (especially if they have CMHC covering their butts), the buyers want to buy, the sellers want to sell, and the ancillary agents want transaction volume. CMHC is one of the few entities that could play the role of a disinterested, rational appraiser, yet they too have no political will or desire to stop housing momentum and break deals by being strict with qualifying criteria. Indeed, contrary to Robert McLister’s opinion that “CMHC knows the risk of it botching property valuations en masse. It has the public, press and regulators breathing down its neck around the clock,” I’d say that the public, press, banks and mortgage brokers are breathing down its neck to allow transactions to proceed. So instead, Emili accounts for many things including the “…housing market conditions in which the property is located…” which to me reads as “becomes loose and permissive in hot housing markets.”

Rob McLister of CMT counters:

“Inevitably, people will read the Globe’s story and think that CMHC is using some back-of-the-napkin formula to judge property risk. That’s so far from the truth. Emili is not some 100-line computer program written by a college intern. It is multi-million dollar mission critical technology benefiting from the best available data and over two decades of R&D.”

For what it’s worth, I don’t doubt that. But it’s not open source, and we don’t know the assumptions that went into making that expensive, sophisticated valuation engine. For example, does it implicitly assume that buyers are rational? A single over-heated bidding war might raise a flag, but would 3 or 10 in an area upgrade the valuations of everything, as it’s then a pattern? Though fraud becomes less likely, the loans really are no more better supported in the long run. Similarly, in assessing risk the core assumption seems to be that valuation changes affect severity, while unemployment affects default rate — the two factors combining to make up the losses that CMHC may face, and the two factors being completely separate and orthogonal. Yet in the aftermath of a bubble, valuation changes also affect default rate as speculators walk away (even though they may remain gainfully employed), and unemployment as well (as construction grinds to a halt). But if that wasn’t observed in the dataset used to build the models, then it may not be accounted for.

As a parallel, consider the subprime mess in the US. I’m sure the ratings agencies had expensive teams of people and fancy computer systems to come up with the “mission critical” ratings for CDOs, yet every AAA handed out was in error, due to some flawed underlying assumptions and a lack of checks. For instance, an underlying assumption of building many of the CDOs and securitized portfolios is that not all the crappy subprime debt goes bad at once, so you can have a AAA slice from something made up of junk. Michael Lewis also highlighted one of the other flawed assumptions: that “average credit rating” meant something, when in fact a pool of 100 mortgages to people with a credit score of 650 is rather different than 50 mortgages to people with a 700 and 50 to those with 600.

I think that based on first principles and the housing market insanity we’ve seen in the last few years, it’s clear that whatever is inside the black box that is Emili is biased to the upside in its valuation methodology. While that doesn’t cause a housing bubble, it allows it — a tragedy given that CMHC is the ultimate holder of risk and should have its systems tuned to be more conservative.