Geologist’s Prediction Cited by Nate Silver
David Bowman’s Earthquake Prediction Discussed in Best-Selling Book
Nov. 26, 2012
Nate Silver's book is currently No. 4 on the N.Y. Times list of bestsellers.
Partly as a result of Nate Silver's successful forecasting of the 2012 presidential election on the New York Times political blog FiveThirtyEight.com, the statistician and writer's first book, “The Signal and the Noise: Why So Many Predictions Fail — but Some Don't,” quickly vaulted to the top of the bestseller lists, where it remains today. In the course of his study of the world of prediction, Silver cites the work of Cal State Fullerton's own David Bowman, professor and chair of the Department of Geological Sciences, as an example of the difficulty we have distinguishing a true signal from a universe of noisy data. An excerpt follows:
There is another type of error in which an earthquake of a given magnitude is deemed unlikely or impossible in a region―and then it happens. David Bowman, a former student of Keilis-Borok who is now the chair of the Department of Geological Sciences at Cal State Fullerton, had redoubled his efforts at earthquake prediction after the Great Sumatra Earthquake of 2004, the devastating magnitude 9.2 disaster that produced a tsunami and killed 230,000 people. Bowman’s technique, like Keilis-Borok’s, was highly mathematically driven and used medium-size earthquakes to predict major ones. However, it was more elegant and ambitious, proposing a theory called accelerated moment release that attempted to quantify the amount of stress at different points in a fault system. In contrast to Keilis-Borok’s approach, Bowman’s system allowed him to forecast the likelihood of an earthquake along any portion of a fault; thus, he was not just predicting where earthquakes would hit, but also where they were unlikely to occur.
Bowman and his team did achieve some initial success; the massive after-shock in Sumatra in March 2005, measuring magnitude 8.6, had its epicenter in an area his method identified as high-risk. However, a paper that he published in 2006 also suggested that there was a particularly low risk of an earthquake on another portion of the fault, in the Indian Ocean adjacent to the Indonesian province of Bengkulu. Just a year later, in September 2007, a series of earthquakes hit exactly that area, culminating in a magnitude 8.5. Fortunately, the earthquakes occurred far enough offshore that fatalities were light, but it was devastating to Bowman’s theory.
Between a Rock and a Hard Place
After the model’s failure in 2007, Bowman did something that forecasters very rarely do. Rather than blame the failure on bad luck (his model had allowed for some possibility of an earthquake near Bengkulu, just not a high one), he reexamined his model and decided his model and his approach to predicting earthquakes was fundamentally flawed ― and gave up on it. “I’m a failed predictor,” Bowman told me in 2010. “I did a bold and stupid thing ― I made a testable prediction. That’s what we’re supposed to do, but it can bite you when you’re wrong.”
Bowman’s idea had been to identify the root causes of earthquakes ― stress accumulating along a fault line ― and formulate predictions from there. In fact, he wanted to understand how stress was changing and evolving throughout the entire system; his approach was motivated by chaos theory.
Chaos theory is a demon that can be tamed ― weather forecasters did so, at least in part. But weather forecasters have a much better theoretical understanding of the earth’s atmosphere than seismologists do of the earth’s crust. They know, more or less, how weather works, right down to the molecular level. Seismologists don’t have that advantage.
“It’s easy for climate systems,” Bowman reflected. “If they want to see what’s happening in the atmosphere, they just have to look up. We’re looking at rock. Most events occur at a depth of fifteen kilometers underground. We don’t have a hope of drilling down there, realistically ― sci-fi movies aside. That’s the fundamental problem. There’s no way to directly measure the stress.”
Without that theoretical understanding, seismologists have to resort to purely statistical methods to predict earthquakes. You can create a statistical variable called “stress” in your model, as Bowman tried to do. But since there’s no way to measure it directly, that variable is still just expressed as a mathematical function of past earthquakes. Bowman thinks that purely statistical approaches like these are unlikely to work. “The data set is incredibly noisy,” he says. “There’s not enough to do anything statistically significant in testing hypotheses.”
What happens in systems with noisy data and underdeveloped theory ― like earthquake prediction and parts of economics and political science ― is a two-step process. First, people start to mistake the noise for a signal. Second, this noise pollutes journals, blogs, and news accounts with false alarms, undermining good science and setting back our ability to understand how the system really works.
Reprinted from “The Signal and the Noise: Why So Many Predictions Fail — but Some Don't” by Nate Silver, published in 2012 by Penguin Press, copyright © Nate Silver, 2012, all rights reserved. Copies of “The Signal and the Noise” may be ordered from the Penguin Group.