What does uncertainty really mean?

In science, we try to report the results of our research in careful terms so as not to over- or under-state the implications. In particular, we must acknowledge that all measurements have some associated "error." What the layperson usually thinks of as error is that a mistake was made. This is not the meaning of error when scientists speak with statistics.

I think a relatively simple example is best. If you wanted to record the temperatures in the shade under your porch compared with the temperatures in the sun every Saturday over the course of a year, you might end up with 104 observations. 52 of them might be of the temperature under the porch and the other 52 would be of the temperature in the sun. Let's say you took those observations at about 5:00 every evening. For this thought experiment, you're using two simple dial-read thermometers for the measurements (like in this image).


Let's examine some of the sources of uncertainty (error).

1. The dial is graduated to 2 degrees F of precision. That is, each small mark indicates 2 degrees F. You can guess that about half-way between two of the small marks is one degree. You cannot be much more precise than that.
   
2. Each mark has some width. Should the temperature be read as 52 degrees F on the right edge, the left edge, or the center of the mark? Were you being consistent in the reading every Saturday, and for both thermometers?
3. The needle of the thermometer does not stay a constant size; in warmer weather, it expands slightly and it contracts slightly in cold weather. Does this matter? It depends on how precise you want to be.
4. Are you reading the thermometers at the exact same time every day?
5. This particular thermometer is on a swivel mount; is it always in the same spot?
6. How well calibrated are your thermometers?
7. Do both thermometers record the same temperature when placed in the same location for the same amount of time?
   

Each of the seven points above add some amount of uncertainty, noise, error, whatever you want to call it, to every measurement, and we need to understand that uncertainty and report it. All of these errors must be accounted for, but we don't exactly remove those errors/noise when we report the temperature trends for the year, we report the errors as well as the measurements. There are various ways to make sure we get good estimates for all of the various errors. I don't have space for a class on statistical analysis or error propagation in scientific analysis. If you're truly interested, I can recommend a bunch of books to read.

Now, the measurements of a single yard do not necessarily say a whole lot about the temperatures in your neighborhood. Let's assume everyone in the neighborhood took the same measurements for their backyard and we wanted to report the neighborhood's average temperatures for each week of the year. Every reading is going to be different and when compiled together, there will be an uncertainty associated with each average. For example, we might report the neighborhood temperature as:
week 9; 45 +- 7 degrees F
week 10; 46 +- 9 degrees F
week 11; 42 +- 12 degrees F
week 12; 49 +- 3 degrees F

That +-7 degrees F for week 9 has some very important implications, and it's terribly imprecise at the moment. Is that 1-, 2-, or 3-sigma standard deviation? Did you include the systematic corrections? Did you account for all of the errors? How did you calculate this 7 degrees?

I just threw out a bunch of terms that may not be familiar. Let me define one of them. What is standard deviation? It's a measure of the variability of the observations. You have a ~68.3% confidence that any of your measurements are one standard deviation away from the mean value. Let's say that everyone was so careful in their measurements and calibration that the +-7 degrees F for week 9 is the 3-sigma standard deviation. That means that 99.7% of the neighborhood measurements were within 7 degrees of the mean value, 45 degrees F. If you only reported 1-sigma, only 68.3% of your measurements were within 7 degrees of the mean.

If you were to continue these measurements over a decade, you'd be able to say something about the long-term change and the short-term variability of the temperatures in your neighborhood. If you do this right, you might notice a pattern. Here's a good discussion of long-term change vs. short-term variability; I think it's written by someone who knows how to educate people.

What's the point of this terribly long and boring discussion? Well, scientists report the errors/uncertainty in their data as a method of limiting the strength of their conclusions. When they report that 99.7% of their measurements fall within a range and that range indicates such-and-such, that means there's only a 3/1000 chance of something outside that range being important to their conclusions.

[rant]
Lots of people willingly or ignorantly seem to see scientific "uncertainty" or "error" or any other scientific language as a way to wiggle out of doing anything they don't want to do.
[/rant]

Obviously, scientists are only human; some may be sloppy and some may be dishonest. That's what peer-review and reproduction of results is about. Sometimes fraud slips through (and it's always reported as if this happens regularly), sometimes mistakes are made, but those are usually corrected rather quickly and are an embarrassment to the journal publishing the incorrect results. Often models are only approximations to reality. That's fine; they give us a way of understanding complex situations, not as a way of saying exactly what will happen.

Let's go back to measuring temperatures. Let's say that you were told that over the past 100 years, the mean temperature of the earth's lower atmosphere has increased by 0.75 degrees C, and that the data plotted here are plotted with a 95% confidence interval. (Note that these are just measurements, not models or approximations. The error bars are from the kinds of errors I discussed above.)



You are also told that this increase is very likely (>90% to >99%) to have been caused by human activity.

What does this mean?

First, what does that 95% confidence interval mean? It means that those little grey bars on the plot here are 2-sigma error bars. It essentially means that 19 of 20 observations were within the range of the bar-ends. Notice that while the errors were relatively large early in the measurements, the old measurements rarely overlap with the new measurements. That is, even if you were to take the warmest measurements from 1850 and the coldest measurements from 2000, we're still significantly warmer.

What is the meaning that there's a 90% probability that human activity has caused this warming? It means that after all the known errors in the measurements and in the models have been accounted for, we're virtually certain that this warming is due to human activity. It means that all of the other possible sources of error account for less than a 10% chance that global climate change is not mostly human-induced. It means that those who advocate doing nothing to slow down CO2 emissions are counting on a--at best--1 in 10 chance of being right.

When scientists speak in terms of uncertainty, they're being careful to note that there is noise associated with all measurements. When laypeople glom on to this language as a way to avoid doing anything because "there's uncertainty" or "there are errors in the data", they're being the opposite of careful. They're claiming that a 1 in 10 chance of not causing 1000 or more years of difficulties for our posterity is the right bet to make.

By the way, there is not a single climate model out there that can reproduce the observed mean temperature increases without including anthropogenic sources. None. There is no controversy among scientists. There may be controversy among politicians, media, and special interests, but no honest climate scientist thinks that humans are not a major cause of global climate change.

Who would take their children or grandchildren on a flight with an airline if they knew that it had a record of crashing nine times for every ten flights it operated? Who would take a flight with an airline if it even had a 1 in 10 chance of crashing?

So, why are we doing this with the future?