Outlier Formula - IQR Fences, Z-Scores, Examples

Key Points

• The standard outlier formula sets fences at $Q_1 - 1.5 \times \text{IQR}$ and $Q_3 + 1.5 \times \text{IQR}$, and flags any value beyond them.

• The NIST e-Handbook of Statistical Methods calls these inner fences, places outer fences at three times the IQR, and labels points beyond them mild and extreme outliers respectively.

• A second approach uses the Z-score $Z_i = \frac{x_i - \bar{x}}{s}$, measuring how many standard deviations a value sits from the mean.

• The NIST e-Handbook warns that the maximum possible Z-score is $\frac{n-1}{\sqrt{n}}$, so on small samples no value can reach a large Z-score however extreme it looks.

• Flagging a value as an outlier is not permission to delete it, because the rule identifies unusual values without explaining why they are unusual.

What Is the Outlier Formula?

The outlier formula identifies values sitting far enough from the bulk of a data set to deserve a second look, using quartile fences:

$$\text{lower fence} = Q_1 - 1.5 \times \text{IQR}, \qquad \text{upper fence} = Q_3 + 1.5 \times \text{IQR}$$

Any observation outside those bounds is flagged. The NIST/SEMATECH e-Handbook of Statistical Methods calls these inner fences, with outer fences at three times the interquartile range.

The Two Standard Outlier Formulas

The quartile method assumes nothing about the shape of the data. It builds the fences from $Q_1$, $Q_3$ and the interquartile range, which all ignore the tails, so the rule is not distorted by the values it is looking for.

The Z-score method measures distance from the mean in units of standard deviation:

$$Z_i = \frac{x_i - \bar{x}}{s}$$

Values with $\lvert Z_i \rvert$ above about 3 are commonly flagged. The weakness is circular: an extreme value inflates both $\bar{x}$ and $s$, shrinking its own Z-score and hiding it. NIST addresses that with a modified Z-score built from the median:

$$M_i = \frac{0.6745(x_i - \tilde{x})}{\text{MAD}}$$

where $\tilde{x}$ is the median and MAD is the median absolute deviation. NIST reports a recommendation that modified Z-scores above 3.5 in absolute value be labelled potential outliers.

Why Z-Scores Fail on Small Samples

This is the detail most outlier guides omit. The NIST e-Handbook notes that the maximum possible Z-score in a sample of size $n$ is $\frac{n-1}{\sqrt{n}}$.

For $n = 5$ that ceiling is $\frac{4}{\sqrt{5}} \approx 1.789$, so on five observations no value can ever reach a Z-score of 3, however obviously it stands apart.

So applying a "Z above 3" rule to a small sample guarantees finding nothing. For $n = 10$ the ceiling is about 2.846, still below 3, and the rule only becomes usable from about $n = 11$.

Variable Key

Examples of the Outlier Formula

Example 1

A data set has $Q_1 = 32$ and $Q_3 = 48$. Find the inner fences and test whether 78 is an outlier.

$$\text{IQR} = 48 - 32 = 16$$ $$\text{lower} = 32 - 1.5(16) = 8, \qquad \text{upper} = 48 + 1.5(16) = 72$$

Final answer: Since $78 > 72$, the value 78 is a mild outlier, beyond the upper inner fence but inside the outer fence at 96.

Example 2

Test whether 100 is an outlier in 2, 4, 5, 6, 100 using a Z-score above 3.

The Z-score rule looks natural here, because 100 is obviously unlike the rest. The mean is $\frac{117}{5} = 23.4$ and the sample standard deviation is about 42.56, so

$$Z = \frac{100 - 23.4}{42.56} \approx 1.80$$

That is nowhere near 3, so the rule says 100 is not an outlier. Yet it plainly is.

Two things went wrong together. The value inflated both the mean and the standard deviation it is measured against, and with $n = 5$ the largest possible Z-score is only $\frac{4}{\sqrt{5}} \approx 1.789$, so the test could never flag anything.

The quartile method has no such ceiling. Sorted, the data gives $Q_1 = 4$ and $Q_3 = 6$, so $\text{IQR} = 2$ and the upper fence is $6 + 1.5(2) = 9$:

$$100 > 9$$

Final answer: 100 is an outlier by the IQR rule, and beyond the outer fence at 12 it is an extreme one.

Example 3

Find the outlier fences for 12, 14, 15, 15, 16, 18, 19, 45.

With $N = 8$, the quartile positions are $0.25(9) = 2.25$ and $0.75(9) = 6.75$:

$$Q_1 = 14 + 0.25(15-14) = 14.25, \qquad Q_3 = 18 + 0.75(19-18) = 18.75$$ $$\text{IQR} = 4.5, \qquad \text{upper fence} = 18.75 + 1.5(4.5) = 25.5$$

Final answer: 45 is flagged, since $45 > 25.5$, and it also passes the outer fence at 32.25, making it extreme.

Example 4

A sample of 11 values has $\bar{x} = 50$ and $s = 8$. Is a value of 77 an outlier by the Z-score rule?

$$Z = \frac{77 - 50}{8} = 3.375$$

With $n = 11$, the ceiling is $\frac{10}{\sqrt{11}} \approx 3.015$, so a Z-score of 3.375 is not attainable. The figures given are therefore inconsistent with a sample of this size.

Final answer: The value exceeds the maximum Z-score for $n = 11$, so the stated mean, standard deviation and sample size cannot all be correct. Checking a Z-score against the ceiling catches transcription errors in reported summary statistics.

Example 5

Use the modified Z-score on 10, 11, 12, 13, 40.

The median is 12, and the absolute deviations from it are $2, 1, 0, 1, 28$, whose median is 1, so $\text{MAD} = 1$:

$$M = \frac{0.6745(40 - 12)}{1} = 18.89$$

Final answer: $M \approx 18.89$, far above the 3.5 threshold, so 40 is flagged.

The median and MAD both ignore the extreme value, so the modified score is not dragged down by it as the ordinary Z-score was in Example 2.

An Outlier Is Not Automatically an Error

The NIST e-Handbook treats outlier identification as labelling, and names Grubbs' test as the recommended route when testing for a single outlier. None of that decides what to do with the point.

A flagged value can be a keying error, a genuine rare event, or the most informative observation in the set. Deciding needs to know where the data came from, which no formula supplies.

Tripping Points to Avoid

Mistake 1: Deleting flagged values as a matter of routine

Where it slips in: The rule returns a clean verdict, so the verdict gets treated as an instruction.

Don't do this: Remove every value beyond a fence before analysing the data.

The correct way: Investigate first, and if a point is removed, say which rule flagged it and why removal was justified.

Students who ask what the flagged value would mean if it were real, before deciding anything, keep the genuinely interesting observations. The reflex to tidy data is where real findings get discarded.

Mistake 2: Using a Z-score threshold on a small sample

Where it slips in: The "Z above 3" rule is widely quoted without its sample-size condition.

Don't do this: Apply a Z-score cutoff of 3 to a sample of five or ten values.

The correct way: Check the ceiling $\frac{n-1}{\sqrt{n}}$ first. Below about $n = 11$ the threshold is unreachable, so use the quartile fences or the modified Z-score instead.

Mistake 3: Recomputing the fences after removing a point

Where it slips in: Removing one outlier changes the quartiles, which moves the fences, which flags another point, and so on.

Don't do this: Iterate the rule until nothing is flagged.

The correct way: Apply the fences once, to the original data. Repeated application will eventually flag almost anything, because each removal tightens the spread the next pass measures against.

What to Remember About the Outlier Formula

• The outlier formula flags values beyond $Q_1 - 1.5 \times \text{IQR}$ or $Q_3 + 1.5 \times \text{IQR}$.

• NIST calls these inner fences and adds outer fences at three times the IQR, separating mild from extreme outliers.

• The Z-score alternative is self-defeating on small samples, because the maximum attainable score is $\frac{n-1}{\sqrt{n}}$.

• A flagged point is a candidate for investigation, never an automatic deletion.

• To work through data handling with a teacher, explore a high school math tutor or math tutoring.

Quick Self-Check, Try These

Find the inner fences for 5, 7, 8, 9, 10, 12, 30 and say whether 30 is flagged. Then compute the maximum possible Z-score for this sample size and decide whether a Z-score rule could have caught it.

Want a live Bhanzu trainer to walk through more statistics problems? Book a free demo class.

Read More

• Mean Median Mode Formula

• Quartile Formula

• Sample Standard Deviation Formula

• Mean Deviation Formula

• Percentile Formula