This post discusses the application of statistical methods when using the Market Approach. It emphasizes understanding the importance of sample size and how it affects the reliability of conclusions drawn from small samples. I hope that it helps you should you need to use or rebut statistically-based conclusions.
As an introduction, the Market Approach draws on different kinds of data:
- Prior transactions in subject company securities
- Shareholder agreement formulas
- Guideline public company data
- Guideline private company data
Prior transactions, even if there is only one, receive special attention under Revenue Ruling 59-60. If they meet the stipulated criteria (e.g., arms length, no duress, not part of other transactions, no major change from transaction date to valuation date, etc.), then they provide important value indications.
Price indications based on applicable agreements must meet the same criteria. For example, an agreement setting the price at book value, when fair market value can be shown by other methods to be much higher, may be deemed invalid for tax purposes.
Guideline public company data must be similar and relevant to the subject private company. For many well-documented reasons, we cannot value a single-location burger stand as if it were McDonald’s. Much research has been conducted to adjust public company valuations to those of private companies. We use public company rate of return data to build up private company discount rates. We have to do so with trepidation because of fundamental differences between public and (particularly, small) private companies.
In rare instances (in my experience), guideline public company data are sufficiently similar and relevant. As a segue to the meat of this article, you might ask “How many guideline public companies do we need to be statistically confident of the results?” The answer is: there is no minimum, as long as there is enough trading volume to make their market prices credible. The premise is that guideline public company prices are set by hundreds of (daily, weekly etc.) transactions. In theory, at least, you could rely on one very good public guideline company.
This brings us to the focus of this article: the use of guideline private company data and the Direct Market Data Method. Data on such transactions is available through IBA’s Market Data File, BizComps, and Pratt’s Stats. Regardless of which source(s) we use, we have to be aware of the effect of sample size on our value indication (in this case, the price to sales or price to earnings ratio that we develop and use to value the subject).
When I cook chili, I constantly taste it while it simmers for several hours to make sure it is (in my case) spicy enough. The idea is that a few spoonfuls tell what the whole pot tastes like.
The key to the taste test, which is a survey, is randomness. I stir the pot to get a thoroughly mixed and random taste. In statistical terms, my tastes are “samples” and the whole pot is the “population”. When using guideline private company data, we assume without verification that the transactions are random samples of the entire population (the industry) that is normally distributed (follows the bell curve). It stands to reason that the larger the sample size (the more transactions we find), the more confident we will be that the sample represents the population.
If we are valuing a dental practice, the IBA Market Data file has (as of this writing) 2,044 data points. (This is before screening out transactions which might be dated, too large or small, etc. Everything in this article applies to samples, whether screened down or not.) We are statistically very confident that this sample is sufficiently large to represent the industry. On the other hand, what if we find only 3, 5, 10 or 20 transactions, just to pick arbitrary numbers? This is where small sample size issues come up, because small samples are simply not as reliable as large ones. Small samples are far more prevalent than large ones, again, in my experience.
How well the sample represents the population (how confident we are that 2,044 dental practice data points represent the industry or 20 tastes represent my chili) is measured by two statistics: the margin of error and the confidence level.
We constantly read and hear news reports of various surveys. The results are quoted like this: “54% of Americans are in favor of X, with a margin of error of 3%.” (The media do not mention the confidence level. Their bad!) These terms mean that, were the SURVEY (sampling process) repeated 100 times, in 95 of the 100 repetitions, the sample mean would be 54% plus or minus 3% (i.e. from 51% to 57%). Stated another way, the 95% confidence interval is 51% to 57%.
Margin of error – the plus or minus 3% – decreases as sample size rises, but only to a certain point. A sample of 50 has a 14% margin of error while a sample of 1,000 has a 3% margin of error, and a sample of 3,000 has a 2% margin of error. Determining the margin of error for a sample of a given size is not hard, but there is a free, easy, reliable and instant online calculator at www.raosoft.com/samplesize.html. Let’s say we are valuing a business in an industry of 15,000 companies (I get that statistic from First Research’s report for the subject industry.) I want margin of error of plus or minus 5%. I want a 95% confidence level. I assume that the population of 15,000 companies is random and normal (see the assumption above) and enter 50% for the “response distribution” variable on the calculator. The answer: I need 377 observations in my sample. If I want a margin of error of plus or minus 1%, everything else the same, the sample size is (gulp!) 6,489. If I want a margin of error of 10% with 80% confidence (everything else the same), the sample size is 41. (As an aside, when I found this out, I resolved to taste my chili a lot more!) You can also use the model to work backwards; given a sample size, population total, and confidence level, you can calculate the margin of error. It is important to do this, particularly when you are dealing (as we often do) with really small samples of fewer than 10 guideline private transactions. In other words, by screening out transactions, you reduce the sample size, increase the margin of error and lower the confidence level.
Is there a minimum acceptable margin of error or confidence level associated with various standards of value and contexts for appraisals? I do not know. Moreover, neither did several attorneys that I consulted. All I can say is that the conventional confidence level mentioned most often in various studies is 95%.
To summarize, margins of error and confidence levels reveal how imprecise samples are. They help convert point estimates (e.g. “the price to sales ratio is 1.00”) into a range (plus or minus the margin of error) with a confidence level as to how much of the variation is reflected in the range.
Here’s an example (using the online calculator of how to use this information. We are rebutting an expert report that opined a price / sales ratio of 1.00 based on a sample size of 3 (which we agree are valid observations). First, we input a 5% margin of error, 95% confidence level, population of 20,000 and normal distribution (50% response). The calculated required sample size is 377! That is way larger than 3, so right off the bat we are onto something and not very confident about the expert’s opinion of 1.00.
Now, in the alternate scenarios box, enter sample size of 3, and compute a 56.58% margin of error for that sample size. We now know that based on a sample size of 3, the margin of error is plus or minus 57% and the confidence level is 95%. The small sample size means that the estimate of a price / sales ratio of 1.00 is hugely uncertain! Based on the small sample, 95% of the results will be between (1 + 0.57 =) 1.57 and (1 – 0.57 =) 0.43. The expert’s sample of 3 implies that the large majority of the values are between 0.43 and 1.57. The ratio of the boundaries is (1.57 / 0.43 =) over 3.6 times, a huge range that undermines confidence in the 1.00 multiple estimate, which is really going to be between 0.43 and 1.57 95% of the time.
We can go further using the assumption that the price / sales ratio is normally distributed. (That was the basis of the “50%” input assumption for the calculator.) Assume the sample size of 3 had a mean (average) price / sales ratio of 1.09 and a standard deviation of 0.03. (We calculated this from the expert’s market data.) Compare this to the expert’s opinion of 1.00. The difference between the sample mean of 1.09 and the expert’s opinion of 1.00 is 0.09. This is three standard deviations (= .09 / .03) below the sample mean. You can look up, in a table of the normal distribution (there are many free online calculators available), the probability that a sample result (in this case, the average price to sales ratio) will be three standard deviations below the population mean. You have to state a confidence level; let’s use 95%. At a 95% confidence level, the probability is 0.12%. Stated another way, the probability that the price / sales ratio is less than 3 standard deviations (“three sigma”) is 99.88%. We are highly confident that the price / sales ratio of 1.00 is far below what the data indicates the average should be. The expert will need to have good reasons to defend their opinion!