Variance is a key concept in statistics and econometrics that influences a great deal of formulae, statistical tests, model setups, estimation strategies, and more. It’s one of the major characteristics of random variables and probability distributions, which are ubiquitous in statistics and economic models.

This is also a case of a very accurate name: variance is a quantity that measures how much a random variable can be expected to differ from its expected value (or its mean). In other words, variance measures how much something varies from the value we’d expect it to take on.

Along with bias, variance is one of the main factors by which economists, statisticians and data scientists judge estimators, helping them decide which ones to use. Thus, understanding both concepts will help new students of economics — as well as statistics and data science — to dive into the weeds of models and make the best judgments possible.

Mathematical definition of variance

As with bias, the definition of variance can — well — vary, at least when we rely on words. The mathematical definition, below, is more precise:

\begin{equation*}
$Var(x) = \mathbb{E}(X^2) - \mathbb{E}(X)^2$ 
\end{equation*}

Let’s break this formula down. “Var” simply stands for variance, and so “Var(x)” means “the variance of x”. Meanwhile, the expectation symbol (the fancy “E”) gives the expected value, or mean, of the quantity inside it — check out our article on mathematical symbols if you’re unfamiliar.

This equation states that variance of x is the expected value of “x squared” minus the squared “expected value of x”*. Notice that both terms in this equation are squared somehow. This is very important, for reasons we’ll soon get to.

Recall that variance measures how much a random variable deviates from its mean. There are two ways to vary from a mean: being too small or too large. When we want to measure this variation, we want a nice, straightforward number that gives us a good sense of how far the variable’s specific realizations could be from its mean*.

*Note that much of this language is, while good for building intuition, very imprecise. Words are not very good at capturing truths about statistics or probability — the invested reader should lean on the mathematical statements first and foremost for these concepts.

The most natural way to generate such a number would be to measure multiple realizations of a random variable, then add up all of the deviations from the mean. We could add all of these deviations together to get the number. But, this method has a fatal flaw. “Negative”, or too-small, deviations can cancel out with “positive”, or too-large deviations, when we simply add them together. This is why we square the values.

As a quick example, suppose we have a random variable y with mean 0, and we record the following realizations of y from an experiment: 1, -3, 2, 2, -1, and -1. The sum of the deviations is 0, so if we looked at this sum alone, we might think that y doesn’t vary at all. But none of the outcomes were 0 — so obviously, y must have at least some variance!

We can easily solve this problem by squaring the values of the deviations, then adding them together and finding the average. So, for example, the variance of this example y is the average of 1 + 9 + 4 + 4 + 1 + 1, which is 20/6 = 10/3. Now, we have a single number that tells us how much realizations of the random variable y will likely vary from its mean**.

**Again, language can be imprecise here where math is very clear. This statement is NOT implying that the realizations of y are bounded, only that the realizations of y vary from its mean with some frequency. There could still be extremely large or extremely small deviations from the mean far outside of this implied “range”, just with very low probability. Variance doesn’t tell us any information about what values y can or cannot take on; it only tells us how “spread out” the realizations of y are relative to its mean.

This is why it’s so important that we use squared values in the variance formula. Squares are always positive, and so we will always get a non-negative measure of a variance. The positive and negative deviations won’t cancel each other out.

Finally, note that it is actually possible for a quantity to have zero variance. All constants — like the number 38 — have zero variance. (It’s a little weird to talk about constants in this way, because constants aren’t random variables, but it’s a very clear illustration all the same.)

High variance is (usually) a bad thing

One of the main things to know about variance is that, typically, we want as little of it as possible. The more variance we have, the harder it is to tell what that random variable is supposed to be varying around — in other words, what its expected value is. This makes drawing conclusions from datasets difficult.

Let’s illustrate this point with the following table. It contains the realizations of two random variables, X1 and X2, both of which are Normally distributed. In other words, we draw a random number from a Normal distribution 6 times for each.

Try to use the table to answer the question: what are the expected values of X1 and X2? In other words, what values are these random variables centered around? Give the table a look and see if you can guess.

The answer? Both X1 and X2 are centered around 0; in other words, their mean is 0. This might have been fairly easy to guess with X1, which has many values close to 0. But X2 has a much higher variance than X1, which means its realizations are much more spread out, and it’s harder to tell that it’s centered around 0.

For the curious, these values were generated from the following distributions: X1 ~ N(0,1) and X2 ~ N(0,15).

Variance’s role in the analysis of real data

When we gather data to run a regression, hoping to test hypotheses and learn something about the real world, inevitably the variables we collect will have some variance. This isn’t a bad thing — in fact, we need at least some variance in order to conduct any analysis. If variables never changed, we wouldn’t be able to tell how other factors affect them.

Still, too much variance is a bad thing. The table above clearly showed one reason why: when values fluctuate wildly, it’s hard to tell what the expected value of a random variable is. Of course, we have computers that can parse data for us, but even a computer cannot magically find the expected value without enough data.

Let’s suppose we do want to find the expected values of these random variables, using the data we have. This is an extremely common analysis. If we fed the X2 column into a program and asked it to find the expected value for us, it’d give us -4.64. That’s pretty far off from 0. Meanwhile, if we used a computer to find the expected value of the X1 column, we’d get -0.67. Much closer! The lower variance certainly helped.

There is a simple way to improve the accuracy of our analysis, even for the X2 column: collect more data! This is where the Law of Large Numbers and other helpful statistics theorems come in. If we were to generate, say, 10000 random draws from these distributions, we would get estimates for both expected values very close to zero. If you wish to see this for yourself, there are many helpful tutorials online about how to code a Normal distribution and conduct these draws in your statistical software program of choice. Give it a shot!

Population vs. sample variance

It’s important to mention the difference between the sample variance and the population variance. The sample variance is what we’ve collected in the data. For example, if we were to use the numbers in the table above and find the estimated variance given the numbers for X1, we’d have generated a sample variance.

The population variance is the true variance of the process underlying the random variable. In the case of X1, that variance is 1, since we generated the draws using a normal distribution with variance 1. This is an important distinction to know about, since it’s usually obvious from the context which is being discussed, but seldom actually mentioned.

Heteroscedasticity vs. homoscedasticity

Finally, it’s worth mentioning the concepts of homoscedasticity and heteroscedasticity as they are intimately linked to variance and commonly discussed. When the variance of a random variable is unchanging, it’s called homoscedastic. This is the “default” case, which enables much of our statistical inference to be valid.

However, sometimes random variables have a variance that changes. This can be over time or over different values of the underlying random variable. For example, spending by low-income households feature lower variance than spending of high-income households. Larger companies typically have bigger fluctuations in profit than smaller companies do.

Heteroscedasticity invalidates our usual methods of drawing conclusions from data unless we notice and correct for it. This is a topic that is often discussed in statistics and econometrics courses. Consider taking some coursework in these subjects to learn more!

Good to Know

Standard deviation is another key concept in statistics that measures the variance of a random variable. It’s simply the square root of the variance. Since the variance is essentially the deviations from the mean squared, the standard deviation is thus the deviations from the mean put back into the same units that the mean is measured in.

For this reason, statisticians, data scientists and economists might use the standard deviation rather than variance for the purposes of extended discussion, since it’s a more intuitive measure. It’s easier to see when a particular value is very unlikely when thinking in terms of standard deviation.

Suppose we know that a random variable has a mean of 0 and a variance of 4, and a realization of that variable is — say — 6, it seems that this realization isn’t terribly unlikely. Still, this means we know that the standard deviation is 2, and the value 6 is three times that. Turns out, this is quite unlikely!