Suneet Singal on questions to ask about automated valuation

Suneet Singal does not, on this site, claim to have built an automated valuation model. The models already exist, on lender desktops and on public listing pages, and the useful work is knowing which questions to ask before a number is treated as a value. An automated valuation model estimates a price or a range from data. It does not walk through the house. It is not, by itself, an appraisal, and it is not an inspection. This entry lists the questions: what data went in, what the error means, what the model was not trained on, and what a confidence score is actually counting. It quotes no current home price and no accuracy statistic for a named product. Those figures belong to a methodology note with a date. The entry is a reference for readers of the housing pages, not underwriting advice and not a credit decision.

An automated value is an estimate from data

An automated valuation model takes inputs it can read, such as prior sales, assessments, property characteristics, and nearby transactions, and produces an output it calls a value, a range, or a score. The output is an estimate under the model’s definition. It is not a contract price, which is what a buyer and seller agreed. It is not an appraiser’s opinion, which is a signed assignment with a scope of work and an effective date. The inspection-and-appraisal entry keeps those documents in their own folders. This entry is about the estimate that is sometimes asked to stand in for them.

The model can be used in more than one place. A listing site may show a public estimate to attract an audience. A lender may use a model to decide whether a full appraisal is required, or as one input among others. Those uses have different costs of being wrong. A public estimate that is 15 percent high is a marketing problem. A lending decision that is 15 percent high can become a collateral problem. Ask which use you are looking at before you ask how accurate the model is. Accuracy for a consumer page and accuracy for a credit decision are not one number, even when the brand name is the same.

Suneet Singal’s real estate and corporate finance background is a reason to read the output the way one would read any model in a credit memo: inputs, performance, limits, and what decision the number is allowed to support. The background is not a statement that a particular model was reviewed, approved, or built. No such review is published here. The questions are general. They apply to a number you did not build and do not control.

Ask what data went in

Ask which sales were eligible. Arm’s-length sales, sales between relatives, foreclosure sales, and new-construction sales behave differently, and a methodology either includes them or explains why it does not. Ask whether the model sees the subject property’s last sale, and how it treats a sale that is old. Ask whether listing prices and pending contracts are inputs. A listing price is not a sale, and using it as if it were one pulls the estimate toward the ask. That may be disclosed and reasonable. It has to be disclosed before a reader calls the output a market value.

Ask about the characteristics. Bedrooms, baths, square footage, lot size, and year built are common. Condition, view, renovation quality, and permitted work are often missing or weakly recorded. If the model does not have a condition input, it cannot have adjusted for condition, no matter how polished the page looks. Ask how fresh the characteristic data are, and which source supplied them. An assessment file, a listing feed, and an owner’s claim are three sources. In California, an assessed value has the further problem described in the property-tax entry: it may reflect a base year rather than the current market. A model that treats assessed value as a stand-in for market value has imported that system’s lag. Ask whether it does.

Ask about geography and time. Which transactions, within what distance or what neighborhood definition, over what months, were allowed to influence this address? A model can be national in branding and local in the comparables it uses, or national in a way that borrows strength from other cities. Those are different designs. The design is part of the answer to “what data?” A vendor who will not describe the design at least at this level has given you a number you cannot audit. You can still see the number. You cannot explain it.

Ask what the error means

Ask for the error metric in a full sentence. “Accurate” is not a metric. A median absolute percentage error of a stated size, on a stated sample, is a metric. A share of estimates that fell within 5 percent or 10 percent of a later sale is a metric. An average error that is near zero can hide large positive and negative misses that cancel. Ask which metric was chosen and why. Ask whether the sales used to compute it were held out from the data the model was trained on. A fit statistic and a predictive statistic answer different questions, and the proptech entry makes the same distinction for claims that are not about value.

Ask where the metric was computed. Error on a large metro’s ordinary resales can look tight while error on rural property, unique property, or the top of the price distribution is wide. If the methodology publishes one number for the country, ask whether a breakout exists for the kind of property in front of you. If it does not, do not borrow the national error and assign it to the house. The data-reading habit applies: the universe of the statistic has to match the universe of the sentence. A confidence interval printed on a page is only as relevant as the sample that calibrated it.

Ask about the direction of the miss, not only the size. In a lending context, overvaluation and undervaluation do not have symmetric consequences. In a listing context, they do not either. A model can be unbiased on average and still be high on a class of houses a particular decision cares about. Ask whether the vendor reports signed error, or only absolute error. If you are a consumer looking at a public estimate, the practical version of this question is simpler. Would the page still look confident if the next recorded sale were 20 percent away? If the page has no way to show that possibility, the confidence is a design choice, not a measurement you have seen.

Ask what the model was not trained on

Ask the exclusion list, and ask it as a list rather than as a reassurance. Houses with no prior sale, houses renovated after the last recorded sale, houses with unusual square footage, condominiums if the model is a house model, manufactured housing, new construction, properties with disaster damage, and markets with very few transactions are standard places to look. The model may still emit a number for those addresses. The question is whether the number is accompanied by a warning that the address sits outside the training experience. A warning is part of the product. A silent number is how an exclusion list fails in practice.

Photographs and text descriptions tempt people to think the model “saw” the house. Ask whether image features are actually inputs. If they are not, the photograph on the page is decoration beside the model, and condition problems visible to a person are invisible to the estimate. That is not a scandal. It is a scope. The inspection exists because scope of that kind is normal. Using the estimate to skip the inspection confuses a data product with a site visit. The inspection entry says what the visit can and cannot do. Neither document repairs the other’s blind spot automatically.

This page will not invent an exclusion list for a product it has not read. The questions are the list of topics. The vendor’s methodology is the list of answers. If you are reading a public page with no methodology link, write “exclusion list not disclosed” in the note. That sentence is allowed to be the entire finding. Suneet Singal’s site does not fill the blank with a guessed accuracy rate, and a reader should not either. A guessed rate would be a second model, a worse one, with no sample at all.

A confidence score is not a walkthrough

Many products show a confidence score, a star rating, or a range alongside the estimate. Ask what the score counts. It might measure how many comparable sales were found. It might measure agreement among internal models. It might measure the historical error for similar houses. Those are three different facts, and only the third is directly about being right. A high score that means “many comps were available” does not mean the comps were good matches on condition. Ask for the definition in one sentence, and keep that sentence next to the score so the score cannot drift into meaning “probability the number is correct” unless the vendor actually said that.

A range is more honest than a point, and it is still an output of the model. Ask what share of later sales fell inside the range, on the held-out sample. A range that looks wide can be the accurate one. A range that looks narrow is a claim of precision and should be the one you check hardest. Do not tighten the range by intuition because you have seen the house. Your knowledge of the house is the inspection and your own visit. It is not a recalibration of the model unless you are the model’s owner and you have a method. Writing a preferred number over the printed range is a new opinion. Label it as yours.

In a loan file, ask whether the figure is an automated valuation, an appraisal waiver or acceptance under a named program, or an appraisal signed by a licensed appraiser. Those phrases are not interchangeable. The Loan Estimate may show an appraisal fee or may not, and the fee line is a clue, not a definition. Ask the lender which document supports the value the loan is using. The CFPB’s owning-a-home and Loan Estimate pages describe the consumer-facing forms. They do not certify a model. They give you a place to see what service you are being charged for, which is one more question: if there is no appraisal fee, what value is the lender using, and under what program rules?

The appraisal and the inspection still have jobs

An appraisal is an opinion prepared under a scope of work, with a person accountable for it, using methods the report states. An automated model can be an input to that work or a substitute in a program that allows the substitute. It does not become an appraisal by being more convenient. If a report is signed, read the signature and the scope. If a webpage is unsigned, do not describe it as an appraisal in your notes. The distinction protects you when a later dispute asks which document you relied on.

An inspection looks at condition. A model that does not ingest a current inspection cannot have priced the repairs the inspector lists, except by accident through a prior sale that already reflected them. When an inspection finds work, the estimate you saw last week is stale in a specific way: its data did not include the finding. Update the note. Do not assume the model will agree with the repair list. Agreement is not its job unless its inputs say it is.

Public indexes remain a check on stories, not a value of the house. The FHFA house-price index tells you how a measured set of prices moved. The Census housing statistics tell you about stocks, rents, and values in a survey. NAR’s research tells you about sales in that association’s series. None of them is the automated estimate, and the automated estimate is none of them. If a model’s marketing says it is “based on the market,” ask which of these, or what other file, and then read that source’s header. The Sacramento and data-reading entries are the header instructions. A value model does not get to skip them because it also uses the word market.

Questions worth writing down

Write eight lines and stop. What decision will this number support? What data and what target did the model use? How fresh are those data? What error metric, on what held-out sample, for what property type? What was the model not trained on, and does it warn me? What does the confidence score or the range mean, literally? Is this an appraisal, a waiver, or a consumer estimate? What does the number not know about condition?

If the lines are unanswered, the number can stay on the screen and stay out of the sentence you tell someone else. Suneet Singal’s reference stops at the questions on purpose. Answering them for a named model would be a review, and a review would need the methodology, the sample, and a date this page does not pretend to have. Bring the questions to the lender, the listing site’s documentation, or the model provider. Keep their answers with the date they were given. An answer ages the way a housing release ages. The archive-date entry is the general form of that caution, and it applies to a methodology PDF as much as to an essay.

What this entry does not claim

Suneet Singal’s published professional background is in real estate and corporate finance. Company filings linked from this site record a chief executive officer departure from FC Global Realty Incorporated effective 2 January 2018 and an acting chief executive officer appointment at First Capital Investment Corporation effective 12 September 2017. The Suneet Singal Scholarship, Grant and Giving websites describe him as the founder of those programs. This entry explains a general subject. It is not a private track record, a current job title, an investment result, or a quote.

Continue with how to read a property-technology claim, why an article’s date is part of its meaning, and post-quantum planning in plain language.