Nobody wants to be biased! But rather than suggest someone isn’t a fair judge, “bias” in economics has a very specific — and perhaps equally distasteful — meaning.

This econometric concept is borrowed straight from the field of statistics, just like its cousin, variance. Both bias and variance have very important roles to play in our analysis of real-world data via regression analysis, and they’re linked together through the bias-variance tradeoff.

Regression analysis is an important tool in the modern economists’ toolkit (as well as any other scientist or data analyst!). With it, we have methods to test economic theories and understand how certain processes or variables affect others. In our article on regression analysis, we touched on several factors that can complicate an analysis and lead to incorrect conclusions. Bias is one such factor.

Understanding bias

Bias means that an estimator we’re using to learn about a real-world phenomenon is not accurate. It’s not hard to see that this is a bad thing! On the flip side, if an estimator is unbiased, it’s (usually) accurate and thus we can trust it without doing any further work.

The linguistic definition of bias varies and can be misleading, but the mathematical definition is precise. We’ll use it to explore the concept of bias further. It states:

\begin{equation*}
\mathbb{E}(\hat{\beta}) - \beta = 0
\end{equation*}

Or, equivalently,

\begin{equation*}
\mathbb{E}(\hat{\beta}) = \beta
\end{equation*}

This equation says “the expectation of our estimator of beta minus the true value of beta is zero”, or equivalently, “the expectation of our estimator of beta equals the true value of beta”.

Beta is simply a Greek letter that’s most often used to denote a regression coefficient; here, it symbolizes the effect we’re interested in studying. The expectation symbol (the fancy “E”) is a very important part of this definition: it gives the expected value, or mean, of the quantity inside it. (If you need touching up on mathematical symbols, we’ve got you covered.) Let’s dig deeper into how this works.

The expectation of a constant is trivial, since constants don’t vary and the mean of a constant is itself. So for example, the expectation of 4 is just 4. Not very exciting. But when it comes to random variables that can take on many different values — like “the number of heads in 5 coin flips” — the expectation is suddenly very useful. It tells us what our “best bet” is: the average (or most likely) outcome. Of course, in 5 coin flips, the expected number of heads is 2.5, and it’s impossible to get 2.5 heads out of 5 flips. But, this is still an intuitive way to think about it. Therefore, the expectation of our estimator \hat{\beta} tells us its expected value, or in other words, what we’d guess it would be if we ran an experiment to measure it.

When we’re using this estimator \(\hat{\beta}\) to study, say, the effect of car sales on carbon dioxide emissions, we want its expected value to be the true effect that selling a car has on emissions. If our estimator is wrong on average — say, it oversells the true effect and makes selling cars look worse than it actually is (or vice versa) — then it is biased. In other words, if the estimator’s expectation is not equal to the true, real-world effect (represented by \(\beta\) ), it is biased!

Meanwhile, estimators that satisfy this criteria are unbiased, which is great news! Unbiased estimators are usually preferred over biased ones, all else equal.

Examples of bias in economics

Sadly, in economics and many sciences, naturally-occurring unbiased estimators are few and far between. This is often the case because of how complicated and interconnected economic factors are. When one factor affects another in economics, we usually need to work hard to disentangle the effects of everything else in the economy on those two things. If we can do so, we will be able to study them in “isolation”. This will lead us to getting an unbiased estimator, but it can be devilishly tricky.

For example, we can’t simply throw investment and GDP into a regression and expect to get an unbiased estimate for “how much investment increases GDP”. This is a classic endogeneity problem, because investment increases GDP, but a higher GDP means an economy will have more funds to invest! And even besides this issue, there are many factors that increase both investment and GDP at the same time: a fall in interest rates, for example, can spur investment and increase GDP. Therefore we must account for interest rates when considering this relationship.

In fact, once you learn how to think about bias in economics, it becomes difficult to find an estimator that isn’t biased. The effects of education on income are biased because of people’s innate ability — those with more ability tend to get more schooling, and those with more schooling tend to earn higher wages. An estimate of how city parks affect people’s life satisfaction would be biased, too — wealthier neighborhoods can afford more parks, but also likely have higher life satisfaction than poorer neighborhoods because they’re wealthier (not because of the parks). The effects of policing on crime can be biased because high-crime areas attract more police attention. Analyzing these biased scenarios improperly can lead to erroneous results — with this last example, a simple regression might suggest that more police cause more crime!

There are nearly infinite examples of how an estimator could be biased in economics. It seems important, then, that we understand what the effects of bias on our analysis are.

Bias causes inference to be inaccurate

As we discussed above, the mathematical definition of bias tells us that if an estimator is biased, it’s inaccurate. Let’s expand on this idea further.

The regression coefficient \(\hat{\beta}\) is supposed to give an idea of the effect size of x on y. This number is usually calculated with a computer program and reported with supplementary statistics that tell the economist how “significant” the result is. But if the estimator is biased, a significant result might lead us to conclude that the true real-world effect size is close to the biased estimator’s value. By the definition of bias, this is wrong! (Think of our policing example above.)

Even worse, our estimate of the sample mean is often used to construct so-called confidence intervals. If we have a biased estimator, the sample mean will also be biased. Confidence intervals are ranges that tell the researcher “with 95% likelihood, the true effect size lies within this range” (or a similar percentage). For example, if we find that 1 additional year of schooling increases future income by $5000, a confidence interval might range from $4300 to $5700. Then, the researcher is led to believe that the true population value lies within that interval with a high probability — it’s at least $4300 or not more than $5700.

However, if the estimator was biased this can be completely wrong. We can’t observe innate ability (we can use imperfect measures like IQ scores, but let’s ignore that for now), which increases both the likelihood of getting more education and the likelihood of making a higher income. This means that our simple estimator is biased upwards, since the regression falsely attributes all of “innate ability”’s power for increasing income to “education” instead. Therefore, it’s possible the true effect would be only $3900, which is completely outside of the biased confidence interval that the researcher is now very confident in!

Correcting bias

If we end up with a biased estimator, not all is lost. There are methods to correct bias. Most of these involve accurately diagnosing the form of the bias and ensuring that appropriate steps are taken to resolve the issue.

Let’s return to some of our earlier examples. Suppose we wanted to measure the effect of parks on people’s life satisfaction, and we run a regression to capture this effect. If we don’t have data on wealth included, clearly we have an omitted variable problem and the estimator will be biased. But there’s an easy solution in this case: collect data on wealth of the areas used in the study, and this will ensure the effect of wealth is not captured by our “parks” variable. This can remove the bias.

Sometimes, it’s not possible to collect data that we might need to remove the bias. Consider “attenuation bias”, which is caused by measurement error that systematically under- or over-states the true values — if our measurements themselves are biased, clearly we can’t simply fix that by collecting more data. One great example of this is self-reported income: people tend to overstate their income in surveys, which leads to a bias we cannot remove. In cases like these, we can turn to advanced regression and statistical inference methods to help us analyze the data and attempt to recover the truth. Still, it’s better to simply avoid these scenarios if at all possible.

Identifying and correcting bias in an economic or data science model is complex and can require years of studying to master. Still, this section should have given a good intuition for how it can be done.

We’ve spent quite a while now discussing bias and its effects, and clearly, ceteris paribus, it’s better to have an estimator that is unbiased. And yet…

A little bias is often acceptable

Bias is sometimes not only okay, but even welcome in an analysis.

Of course, this begs a question. Why on earth would we ever want an estimator to be biased?! Has this whole article been a lie?

The unfortunate truth is that bias and its cousin, variance, have an interconnected relationship. This forces economists, statisticians, and data scientists to make a choice between how much of either one they’re willing to tolerate.

In short, scientists can increase bias to reduce variance, so in cases where variance is too high (creating an “unstable” estimator) we might introduce bias to settle down the variance. In this case, we can reduce the total error of the regression equation, which improves our analysis. Say that increasing bias by 10% reduces variance by 50%; this will lower the overall error of our analysis, and is a worthwhile trade.

Still, in general, bias isn’t something to go looking for. Now you should be well equipped to recognize it!

Good to Know

Small sample sizes are often an issue in regression analysis. Because of this, every economist or data scientist prefers to have more data rather than less in their dataset. To see this, consider a coin flip. When we flip a coin, we expect it to be heads half the time and tails the other half of the time. But, it’s entirely possible that we flip a coin 10 times and get heads 8 times instead of 5. If we were running a regression and used this experiment as our estimate, we’d get an 80% chance of heads.

This seems like a problem — if we didn’t already know beforehand that a fair coin has a 50% chance of landing on heads, how could we tell that this estimate is wrong? And, is this a case of bias?

This example — and indeed, all of the examples in this article — highlight that theory is very important. When studying bias and analyzing data, it’s imperative that the data scientist understands what the data actually means. This will help them make the right decisions in their analysis, and help them to make correct takeaways.

So, if this coin experiment was run and the data scientist had a solid theory about it — like that coins should land on heads 50% of the time — they could make an informed guess that either they’ve observed a slightly unexpected outcome, or the coin could be weighted. Then, they can figure out how to test these hypotheses and learn more.

Further Reading

Advanced readers might be interested to know that, shockingly, there exist estimators that are biased but provably better than unbiased options — even if variances are comparable! The James-Stein estimator is one such intentionally biased estimator that improves upon the usual “default” results.

The classic method for “solving” a regression equation, Ordinary Least Squares (or OLS), minimizes the squared error terms in the regression. This results in coefficient values that can be unbiased (if the researcher used OLS correctly) and that are as close to the true values as we can get, given the variations in the random data we collected.

The James-Stein estimator is built on a concept that shocked statisticians when it was initially introduced in 1961. Put simply: we can bias our own estimates downward in order to reduce the sum of squared error even more than OLS can. Then, since we have less error on average, we are closer to discovering the true population value overall, even though we purposefully biased the estimators downwards to understate the effect size!

Estimators that are strictly worse in a total-error sense than other estimators in this way are called “inadmissible”. Thus, statisticians proved OLS is inadmissible. And later, they also proved that the James-Stein estimator itself is also inadmissible! This begs many questions, not least of which is why we’d ever use OLS (or other unbiased estimators) if we can purposefully bias our estimators to get “better” results.

Alas, this topic goes much beyond the scope of this article, but the answer lies in the bias-variance tradeoff, the specific research question at hand, and the (size of the) dataset used among other factors. Consider taking courses in statistics, data science, or econometrics to learn more!