All Book Reviews.

Book Reviews: How We Know What Isn't So: The Fallibility of Human Reason in Everyday Life, by Thomas Gilovich

If you watch sports at any level, you’re bound to hear variants of these phrases:

“She’s on a roll!”

“He’s on quite a run!”

“She’s unstoppable!”

“What an awful streak!”

“He’s in the zone!”

The implication being that streaks are a natural part of athletics. But what, exactly, constitutes a streak? We each have an intuitive feel for hot streaks -- it’s that time when you’re hitting every pitch, timing your overhand smashes perfectly, and draining every putt. And similarly for cold streaks -- the pitcher has your number, the net is a 6-foot chain-link fence, and you’re hitting every tee shot onto the fairway... for the next hole.

Are streaks real? They must be -- we experience them all the time!

But let’s look a little deeper. Take basketball’s Kobe Bryant for instance. What does it mean for Kobe to experience streaks? If we look at his pattern of hits and misses, how can we quantify the presence or absence of streaks? One reasonable hypothesis is that once Kobe hits a couple shots in row, then he’s in the zone, thereby increasing his chance of hitting his next shot. Or if he misses a couple shots in a row, he loses confidence, thereby decreasing his chance of hitting his next shot. This hypothesis implies that his shots are not independent rolls of the dice, e.g. the outcome of one shot influences the outcome of other shots.

Having taken our share of stats classes, we can test this hypothesis. What we do, in fact, is to assume the opposite:

Null hypothesis: Kobe’s shots are independent (no streaks).

And then we see if we can reject this null hypothesis, based on the data.

One way to test this hypothesis is to perform a runs test. A runs test makes no distributional assumptions about the underlying data -- it merely looks at the number of runs in the data, and compares this to what would be expected when drawing truly independent random numbers.

In the spirit of scientific investigation, we recorded the outcome of each of Kobe’s field-goal shots for the 1st game of the western conference finals between Los Angeles and San Antonio (May 19, 2001). Here is the data (O Missed shot, X = Hit shot):

O O O X X X X X X X O O X O O O (halftime) X X O X O O X X O X X O O O X O X O X X O

[Sports Illustrated lists Kobe as hitting 19/35 shots from the field for this game -- this could be due to a different treatment of shot attempts where a foul occurs. Our analysis is based on the sequence shown above.]

Looking at the first half, Kobe came out cold, then got red hot, then cooled off again before halftime. And in fact, here is a sampling of the announcer’s first-half comments:

“What an awful start” (after he missed three in a row)

“The basket is looking mighty big right now” (after he hit 5 in a row)

“This is an unstoppable run!” (after he hit the 7th in a row, and coincidentally right before the run did, in fact, stop).

Faced with this overwhelming evidence, we must conclude that streaks exist, right? Well... we’ll perform our runs test, just to confirm our intuition.

Using Minitab:

Total Observations: 37

Field-goal percentage: 51.35% (19 shots made out of 37 total)

Observed number of runs: 19

Expected number of runs (under null hypothesis of independence): 19.4865

p-value: 0.8710 Cannot reject at alpha = 0.05!

Not only do we not reject the null hypothesis, we’re not even close to rejecting it. The number of runs is frightfully close to what we would expect, assuming that Kobe’s shots are completely independent. So we are at a loss -- our intuition told us that Kobe’s game was filled with streaks, but the statistics said no: there were exactly as many streaks as one would expect from totally independent shots.

Now one example does not a life-changing experience make, but it’s enough to make one stop and think. Perhaps if we look at other games for Kobe we will get a different result. Or maybe our error was in the technical definition of a streak. And that’s where we turn to Dr. Gilovich. Gilovich argues that no, if we look at more games, we’ll spend a lot of time watching basketball (which might be enjoyable), but we won’t prove the presence of streaks. And we can twist the definition of streak every which way, but we won’t find a reasonable definition that results in statistical evidence of streaks.

If we admit the possibility that streaks don’t exist, this raises a host of interesting questions:

Why do we believe in streaks in the first place?

Why do we believe so strongly in streaks?

Is our belief in streaks irrational?

If we are presented with data opposing the existence of streaks, will we change our minds?

Gilovich explores each of these questions in depth in this book. He covers sports, but also a host of other areas in everyday life where the things that we know turn out to not be so. Taken together, it’s an eye-opening experience to see how our intuition can lead us astray, particularly in the presence of large quantities of data. As wafer fabs generate millions of statistical data points every day, our intuition has plenty of opportunities for mischief. In our effort to coax information from this vast sea of data, the distinction between belief (an untested hypothesis) and fact is an important one. Gilovich’s book provides useful insight into how easily we are fooled. Pick up a copy -- it’s a worthwhile read.