Suneet Singal on base rates

Suneet Singal's page on base rates asks a plain question: how often does this kind of thing happen? The entry sits with the decisions essays beside the podcast The Opportunity Notebook with Suneet Singal. A base rate is the frequency of an outcome in a relevant class of cases, before you fall in love with the details of the case in front of you. The American Psychological Association's dictionary describes the base-rate fallacy as the error of ignoring that frequency or giving it too little weight. This page uses the dictionary's own illustration and does not invent a medical statistic. Daniel Kahneman's work on the outside view, including the decision-quality article he wrote with Dan Lovallo and Olivier Sibony, is the other main source. His published background is in real estate and corporate finance. That fact is biography, not a data set. A labeled hypothetical uses a neighborhood newsletter.

How often this kind of thing happens

A base rate is an answer to a frequency question. How often does this kind of outcome show up in a stated class of cases? The class might be new volunteer newsletters, or kitchens that change suppliers in a given year, or Saturdays in this month that rain hard enough to cancel an outdoor reading. The rate is not a moral judgment on the case in front of you. It is the record of the class, used so the case does not have to carry the whole weight of prediction by itself. If eight of the last ten similar attempts stalled, that fact belongs on the table even if this attempt has a charming founder. Charm is detail. The eight of ten is the base rate, and only if those ten really are the right class.

The question feels unkind, which is why it gets skipped. People hear it as a prediction that they personally will fail, or as a refusal to look at their evidence. It is neither. It is a request to see their evidence beside the frequency. Kahneman and Amos Tversky's 1973 paper On the Psychology of Prediction argued that intuitive predictions often follow representativeness: the outcome that resembles the story wins, and the prior probability of the outcome loses. You do not need the experiments to use the caution. When a story fits a picture in your head, ask what fraction of such stories end the way the picture ends. If you do not know the fraction, say you do not know. Unknown is a respectable line on a memo. Zero, written down because the story was vivid, is not.

Suneet Singal's podcast can raise questions about judgment. This page does not claim a base rate for his career, and it does not treat his real estate and corporate finance background as a sample. Biography is not a reference class. The class has to be defined by the kind of outcome you are predicting, not by the fame or the familiarity of the person in the room.

The case and the class

Every prediction mixes two ingredients. One is the case: the particular people, the particular plan, the particular reasons this time feels different. The other is the class: the set of cases you claim this one belongs to. The inside view, in the language Dan Lovallo and Daniel Kahneman use in their Harvard Business Review article on optimistic forecasts, stays inside the case and builds a story from the plan. The outside view starts from the class and asks how similar plans distributed their results. Their recommendation is to give the outside view a real place in the forecast, because the inside view is where optimism hides. The distribution is the base rate. The story is the adjustment. Adjustments should be earned by a specific difference, not by enthusiasm.

Choosing the class is part of the judgment, and it should be written down. Too narrow a class is a way to dodge bad news. Only our own past successes excludes the attempts that would lower the rate. Too broad a class is a way to make the rate useless. All human projects ever attempted does not tell a library whether a newsletter will last a season. A useful class shares the constraints that actually drive the outcome: volunteer labor, a weekly deadline, no paid editor, a town of this size. Write those constraints next to the class so a critic can say you have the wrong peers. A critic who improves the class is helping. A critic who only says we are unique is restating the inside view.

If the class has a published frequency, cite it and read the definition. If it does not, you may have a local count, and you should label the count as local. Three newsletters at this club, two of which stopped, are not a national statistic. They are three cases. Three cases can still beat zero cases, because they force a conversation about why those two stopped. They cannot support a precise probability. The expected-value page is where a frequency might become a weight, and only with the roughness left visible.

The dictionary's illustration, not a medical statistic

The APA Dictionary of Psychology defines the base-rate fallacy as a decision error in which information about the rate of a trait in a population is ignored or underweighted. The dictionary's own example is not medical. It says people might categorize a man as an engineer rather than a lawyer if they heard he enjoyed physics at school, even if they knew he was drawn from a population of 90 percent lawyers and 10 percent engineers. The resemblance between the detail and the stereotype of an engineer crowds out the frequency. This page repeats that example because it is the dictionary's, and because it keeps the lesson away from invented clinical numbers. Do not turn it into advice about hiring engineers. It is a picture of a weight going to the story instead of the class.

You can run the same picture on a friendlier topic. A plan that looks like a success story will be filed under success even when the class of similar plans mostly stalls. The detail that does the damage is the detail that resembles the desired outcome. Our editor loves writing, which resembles the stereotype of a lasting newsletter, can crowd out the frequency of newsletters that died when the editor got busy. Loving the work is real. It is the physics-class detail in the dictionary's example. It deserves a place as case information. It does not deserve to silence the class.

Kahneman, Lovallo, and Sibony, in the decision-quality checklist, ask whether the base case of a proposal is too optimistic, and whether a memorable analogy is doing too much work. Both questions are base-rate questions in working clothes. An analogy is a class of one, chosen because it is vivid. A base case is a forecast that may have been built entirely from the inside. The checklist is a way to make a group answer the frequency question before it accepts the story. Use it when the decision is heavy. Use the plain question, how often does this kind of thing happen, when the decision is smaller and a full review would be theater.

What to write when you do not have a rate

Often you do not have a clean frequency. The honest memo line is: class defined as follows, rate unknown, so the forecast is a judgment and not a count. That line blocks fake precision. It also blocks the pretense that uniqueness exempts you from the question. You looked. The class is empty of data. You are proceeding on judgment, with a review date. That is a respectable way to decide. Inventing a percentage to fill the blank is not. This page will not supply a typical failure rate for projects, marriages, businesses, or treatments. Any such number, dropped into an essay without a table, would be exactly the fake statistic the method forbids.

You can still gather a small, named set. List five attempts that meet the class definition, including attempts that failed and attempts by people you do not admire. Write the outcome of each in a word, not a paragraph. The fraction in that list is a sketch, and you will label it as a sketch of five. If you cannot find the failures because you only saved the successes, the sketch is biased and you should say that too. Lovallo and Kahneman's outside view is hard precisely because organizations archive the story of the win and misplace the cousins that stalled. Going to look for the cousins is the work. The arithmetic, once you have them, is the easy part.

A premortem can help people remember the cousins. Ask why similar plans failed, not only why this one might. The reasons will be particular, and some of them will reveal that your class was wrong. We fail when the editor is one person is a class definition trying to be born. Take it. Redefine the class as plans that depend on a single volunteer, and go look at those. The base rate you want is the one attached to the constraint that actually bites.

A labeled hypothetical about a newsletter

This is a labeled hypothetical. The club, the papers, and the outcomes are invented. A neighborhood club wants to launch a weekly printed newsletter. The editor is energetic, the first sample looks handsome, and the inside view says the town has been waiting for this. Someone asks for a base rate. Nobody has a published frequency for volunteer newsletters, and this page is not going to invent one. What the club does have, in the fiction, is a shelf of its own past: a poetry sheet that lasted four issues, a calendar that lasted a season, and a news sheet that is still copied monthly because one paid staffer at the community center owns it. Three cases. Two stopped while they depended on a single volunteer. One continues with a paid owner. The class the club writes down is volunteer publications at this club with no paid editor.

The sketch is too small for a percentage in a spreadsheet. It is large enough to change the plan. The handsome sample resembles success, in the sense of the dictionary's example, and the shelf says the volunteer versions stopped. The club does not conclude that the newsletter must fail. It concludes that the inside view left out the constraint that bit twice. The revised plan, still fictional, either finds a second editor before the first issue, or chooses a monthly sheet, or accepts the sketch openly and sets a review after four issues, which is when the poetry sheet died. The review trigger belongs on the decision memo. The base rate did not make the decision. It stopped the sample from being the only evidence in the room.

Nothing in the story is a study of journalism, a recommended format, or a claim about Suneet Singal. The numbers four issues and three past papers are props. If your club has different cousins, use your cousins. If you have no cousins and no published rate, write rate unknown and decide with your eyes open. What you should not write is that an energetic editor implies a lasting weekly, as if resemblance were a frequency.

How a rate should change a forecast

When you do have a rate you trust, let it speak before the story. Start the forecast at the class frequency, then list the specific differences that justify a move up or down. Our case has a paid printer, which none of the stalled cousins had. That is a difference. Our case has a passionate editor, which the stalled cousins also had. That is not a difference, even though it is the detail everyone wants to discuss. The discipline is tedious and fair. It is also the opposite of fake precision. You can say we start from two out of three local cousins stalling, a sketch, and we move only for differences they did not share. You cannot say the probability of success is 73 percent because the editor is good. The 73 came from nowhere.

Kahneman, Lovallo, and Sibony warn about optimistic base cases and about analogies to a memorable success. A single famous newsletter that lasted for decades is an analogy. It is allowed into the room as a story. It is not a base rate. The rate comes from the dull cousins, not from the most shareable survivor. If you cite a published table, read what the table counted. A survival rate for paid magazines is the wrong class for a volunteer sheet. Using it because it is a number is worse than having no number, because the memo will look sourced. Source and class are different achievements. Require both.

Expected value can take a weight from a base rate when the class is right and the roughness is confessed. It should not take a weight from a stereotype. The dictionary example is the picture of that mistake. Physics resembles engineer, so the frequency of lawyers is ignored. Energy resembles success, so the frequency of stalls is ignored. Put the frequency back in the first sentence of the forecast. Let the resemblance argue for a limited adjustment, in words.

When the case should outweigh the class

Base rates are not a veto by the average. A specific, checked difference can make the case unlike the class, and then the class should loosen its grip. The newsletter with a paid editor really is in a different class from the volunteer sheet, which is why the club in the hypothetical was right to notice the community-center paper as a different cousin rather than as proof that all paper survives. The error is to declare uniqueness before you look. The correction is to look, name the difference, and only then move. If the difference is hope, it does not count. If the difference is a paid role, a signed second volunteer, or a constraint the failures did not share, it can count. Write it where a skeptic can see it.

There is also a moral limit. A base rate about a group of people is a dangerous toy, and this page is not a permission slip to judge a person by a demographic frequency. The method here is for plans and outcomes of a defined practical kind: newsletters, schedules, repeated projects. Even there, the class has to be about the structure of the attempt, not about the identity of the attempter. If you cannot say the class without reaching for a stereotype, you do not have a base rate. You have the fallacy wearing a lab coat. Stop and return to the particulars you can actually check.

Used with that limit, the question how often does this kind of thing happen is one of the cheapest improvements you can make to a decision. It costs a little pride. It saves you from treating a vivid sample as a destiny. Ask it before the expected-value weights get typed, before the memo's forecast line hardens, and before the inside story becomes the only story the review date will remember. The class will sometimes agree with your hope. Let it. Agreement that survived the cousins is worth more than a story that never met them.

What this entry does not claim

Suneet Singal’s published professional background is in real estate and corporate finance. Company filings linked from this site record a chief executive officer departure from FC Global Realty Incorporated effective 2 January 2018 and an acting chief executive officer appointment at First Capital Investment Corporation effective 12 September 2017. The Suneet Singal Scholarship, Grant and Giving websites describe him as the founder of those programs. This entry explains a general subject. It is not a private track record, a current job title, an investment result, or a quote.

A base rate can become a rough weight on expected value, a cousin list inside a premortem, and a check on narrow framing, which isolates this case from the class it belongs to.