What are the odds?

After centuries of dispute, a probability theory, rooted in common sense, wins out

June 05, 2011|By Michael Washburn, Globe Correspondent

THE THEORY THAT WOULD NOT DIE: How Bayes’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines and Emerged Triumphant from Two Centuries of Controversy
By Sharon Bertsch McGrayne
Yale, 336 pp., $27.50

What’s the plural of “statistician’’? A quarrel.

It’s a minor but telling point that having read Sharon Bertsch McGrayne’s “The Theory That Would Not Die’’ I find jokes like this kind of funny. McGrayne’s book transforms what many would consider to be a history of boredom — the philosophical and historical foundations of a debate between warring factions of statisticians — into an intellectual romp touching on, among other topics, military ingenuity, the origins of modern epidemiology, and the theological foundation of modern mathematics.

The book’s central debate involves competing epistemologies — in this case, a philosophy that supports scientific decision-making. The seemingly obvious idea is the brainchild of Thomas Bayes, an 18th-century renegade priest and amateur mathematician. “Bayes’s rule’’ seems banal in its simplicity. “[B]y updating our initial belief about something,’’ McGrayne writes, “with objective new information, we get a new and improved belief.’’ Isn’t this how everyone negotiates the world? But the implications for scientific inquiry were staggering, and have remained contentious until the beginning of our 21st century. Why? Notice that the rule commences with belief. Starting from informed belief — hunches, guesses, intuition — Bayes, or actually his more sophisticated successors, devised formal ways of measuring whether additional information added or detracted from the probability that one’s initial notion was true.

Bayes’s theoretical, epistemological opponent is “frequentism.’’ For frequentists, belief means nothing; objectivity and validity occur only through repeated observation of a replicable phenomenon until enough data is amassed for a meaningful sample. The theoretical structure of frequentist statistics presumes that something that has never happened in the past can never happen in the future. This distorts reality. As McGrayne shows, for frequentists, statistically speaking, airplanes couldn’t collide in mid-air. Until, of course, they do, at which point it’s a bit late to calculate the probability of error or determine an actuarial table.

Advertisement
Advertisement
|
|
|
|