Empirical Probability Calculator

What empirical probability measures

Empirical probability is a count turned into a rate. You watch something happen, write down how many times it happened, and divide by how many tries you saw. Flip a coin 60 times and get 27 heads, and the empirical probability of heads in that run is 27 / 60 = 0.45.

The word empirical separates this from the probability that comes out of a model. A fair coin has a theoretical probability of 0.5 for heads before anyone flips it. The empirical number describes what your sample did, so it carries the noise of that sample. Getting 27 heads in 60 flips is close to half without being exactly half, and 45 percent is the honest answer for that run.

The same division goes by several names. Relative frequency, experimental probability, observed rate, hit rate, defect rate and batting average are all f / n with different words wrapped around them. OpenStax's introductory statistics text calls it relative frequency and notes that the values settle down as trials pile up.

One habit makes the number trustworthy: fix the denominator before you look at the data. Decide that a trial is one shot, one batch or one customer, then count every trial, including the failures. Choosing the denominator afterward is how a 3 for 4 streak turns into a 75 percent claim in a sales pitch.

The formula, worked three ways

The calculation is P(E) = f / n, where f is the number of times the event happened and n is the number of trials. The result always falls between 0 and 1, and multiplying by 100 gives the percent.

Free throws. A player makes 27 of 150 attempts. 27 / 150 = 0.18, so the empirical probability is 18 percent, the fraction is 9/50, and the odds in favor are 9 to 41. The odds against are 41 to 9.

Defects on a line. An inspector finds 3 bad parts in a batch of 8. 3 out of 8 comes to 0.375, which is 37.5 percent. Out of 400 parts at that rate you would expect 150 defects, which is the sort of number that justifies stopping the line.

No hits at all. A new email subject line gets 0 opens in 20 sends. The formula gives 0 / 20 = 0, and 0 is a real answer for this sample, but it is not a promise about the next 20 sends. A zero count needs a range rather than a point estimate, which is what the standard error rows on this page are for.

Watch the units of the denominator. 12 sales out of 40 leads is 30 percent, and 12 sales out of 40 phone calls is also 30 percent, but the two rates mean different things and should never be pooled into one number.

Empirical probability versus theoretical probability

Theory tells you what should happen over an unlimited number of trials. The empirical value tells you what did happen in the trials you ran. Comparing the two is the fastest way to see whether a coin, a die or a machine is behaving.

None of those gaps is alarming at those sample sizes. A 60 roll run of a fair die lands on six anywhere from about 5 to 15 times most of the time, so 8 sixes is unremarkable. The gap only becomes evidence when it is large compared with the margin of error, which is why the calculator reports a range next to the point estimate.

The two numbers converge as n grows. That is the law of large numbers: flip a coin 10 times and 70 percent heads is common, flip it 10,000 times and 52 percent is a big deviation. The coin did not change; the sample got big enough to show its true shape.

Reading the odds, the fraction and the one-in number

One count produces several equally valid descriptions, and each one suits a different audience.

Percent is what reports and dashboards use. 18 percent of attempts succeeded.

Fraction is the cleanest way to see the raw count. 27/150 reduces to 9/50 by dividing both numbers by 3, and the reduced form is easier to compare with other rates.

Odds compare successes with failures rather than with the total. With 27 successes and 123 failures the odds are 27 to 123, which reduces to 9 to 41. Odds and probability are not the same thing: a probability of 0.18 is odds of 9 to 41, not 9 to 50. Casinos and sportsbooks quote odds because payouts scale with the stake, while scientists and journalists quote probability because it compares directly with a model.

One in N is the form people repeat at the kitchen table. 18 percent is about 1 in 5.56, so a room of 28 people would hold roughly 5 who match. This form gets awkward below 1 percent: 0.2 percent is 1 in 500, which sounds rarer than "0.2 percent" even though the two are identical.

Conversions are quick. From probability to odds, use p / (1 - p). From odds of a to b, the probability is a / (a + b). From percent to one in N, divide 100 by the percent.

One count, five ways to write it, all from the same 27 of 150:

How far off can a short sample be?

A single rate hides its own uncertainty, so it helps to put a range around it. For a proportion, the standard error is the square root of p x (1 - p) / n. With p = 0.18 and n = 150, that is the square root of 0.18 x 0.82 / 150, or about 0.031. In percentage terms the standard error is 3.1 points.

A 95 percent range is roughly the estimate plus or minus 1.96 standard errors, so 18 percent plus or minus 6.1 points. That puts the plausible true rate between about 11.9 and 24.1 percent. If someone asks whether the process hits a 20 percent target, this sample cannot settle it, because 20 sits inside the range.

Sample size buys precision slowly. Doubling the trials divides the margin by the square root of 2, about 1.41, so 300 trials would narrow the margin from 6.1 to 4.3 points. Cutting the margin in half takes four times the work. Holding a margin of 3 points at a rate near 18 percent needs roughly 630 trials, which is why real polls cost money.

These are the same ideas behind the sampling and control charts in the NIST/SEMATECH e-Handbook of Statistical Methods, which is the reference many labs use when they put a bound on a measured rate.

The zero case needs its own rule. When nothing happens in n trials, the rule of three puts the upper 95 percent bound near 3 / n. Zero failures in 20 starts means the failure rate is probably under 15 percent, not zero. This page shows that bound in the reading line whenever the success count is 0.

The shrink is slow. Quadrupling the trial count only halves the margin:

What a rate lets you predict

A rate earns its keep when it points at a count you have not made yet. Expected count is trials times probability: at 0.18, a 400 part run should turn up about 72 defects, and a 250 call day should bring about 45 sales if the rate holds. Multiply the same way for a whole month and the number stops being abstract.

Expected value takes one more step and puts money on the outcome. If a 0.18 chance pays 20 dollars and a try costs 5, the expected return per try is 0.18 x 20 = 3.60 dollars against the 5 dollar cost, a loss of 1.40 dollars per try in the long run. That figure says nothing about the next try; it prices a thousand of them.

Sample planning runs the arithmetic backwards. Pick the margin you can live with, then solve for n. A 5 point margin at p = 0.18 needs 0.18 x 0.82 x (1.96 / 0.05) squared, or roughly 227 trials. The table in the previous section reads the same relationship from the other end.

Two cautions before anyone acts on the number. A rate measured on one population does not carry over to a different population without a reason, and a rate measured before a process changed describes the old process. Write the date and the population next to the figure, or the figure will outlive its meaning.

Five ways an observed rate goes wrong

Most bad rates come from the counting, not the arithmetic.

The denominator is wrong. Counting only the customers who replied turns a 4 percent response rate into 40 percent. Every trial that could have failed belongs in n.

The window moves. A store that reports its best 3 weeks as the annual rate is measuring a season, not a habit. Fix the period before you count.

The sample is too small to separate. 2 for 3 is 67 percent, but the 95 percent range runs from about 9 to 99 percent. Nothing about that sample supports a forecast.

The conditions changed. A machine that ran at a 1.2 percent defect rate on Monday and 0.4 percent after a repair has two rates, not one. Pooling them describes a machine that no longer exists.

The count is filtered. Recording only the calls that got answered leaves out the busy signals, and the resulting rate describes the answered calls only. That can still be a useful number, as long as the label says so.

None of these problems shows up in the decimal places. 0.1833 looks precise whether it came from 11 of 60 or 1,833 of 10,000, so the trial count belongs next to every rate you publish.