Useless Arithmetic and Inconvenient Truths
A review of Useless Arithmetic: Why Environmental Scientists Can’t Predict the Future by Orrin H. Pilkey and Linda Pilkey-Jarvis, Columbia University Press, 2007.
My story begins with the intriguing title of a new book — Useless Arithmetic: Why Environmental Scientists Can’t Predict the Future (Pilkey 2007). The authors are a father and daughter team. The father is Orrin H. Pilkey, an emeritus professor of geology at Duke University’s Nicolas School of the Environment. He lives in Hillsborough, North Carolina. The father is a prolific author and expert in shoreline developments. The daughter, Linda Pilkey-Jarvis, is also a geologist. She hails from McCleary, Washington, working in Washington State’s Department of Ecology, managing the state’s oil spill programs.
The book is a delight to read. The Pilkeys recount dozens of scientific vignettes, unfolding like detective stories, of scientists gone astray, lost following their predictive models to unexpected consequences and tragic failures. As the Pilkeys make clear, science has not been very successful in predicting or managing environmental changes. The problems, they argue, are inherent in any attempt to model complex natural and human systems. Predictions from any computer simulations of any complex reiterative dynamic processes are not worth the binary code they were written in, nor the supercomputers they were run on. The book reads like a series of parables, each illustrates what Whitehead meant by “the Fallacy of Misplaced Concreteness.” The problem is endemic to all modeling of any complex environmental or human process.
Chapter Four alone should be required reading for anyone concerned with the debate over climate change. To address the larger question, the authors begin by pulling on the string of sea level change. Readers get a brief tutorial on eustatic and isostatic changes in sea level. Eustatic variations are changes in the volume of liquid water in the Earth’s oceans, more or less depending on the amount of glaciated ice, atmospheric water, and geologically bounded water captured in aquifers, lakes, soil, and rock. Isostatic changes in sea levels are dramatic geological changes in the contours of Earth’s ocean basin, increasing or decreasing the volume of the ocean containers. When the ocean basin is smaller, global sea levels rise everywhere. The ocean cup runneth over unto all of the continents. Or as the case may be in the reverse, sea levels can also drop dramatically.
These dynamics and others have been at work on the Earth since its beginning. Major climate changes in the past have been caused by wobbles in the Earth’s axis of rotation. Indeed, the magnetic poles have even flipped – south becomes north and north becomes south. Our orbit around the sun is also ever so slightly out of kilter. Our sun too is dynamic, sometimes overly exuberant in bathing the Earth with excess solar energy, and sometimes too little. In addition, there are disruptions caused by volcanic activity and terrestrial impacts. And life itself is also an important part of the story, like the invention of photosynthesis or the formation of large hydrocarbon deposits hundreds of millions of years ago. All of these can dramatically impact global climate and maybe even your vacation plans this summer.
Climate change is hardly front-page news for geologists; climate change is the whole story from beginning to end. Geologists read this story from the text of rock, mud, water, ice, and air, in the half-lives of radioactive isotopes, in the orientation of magnetic sediments, in geological deposits, in the traces of ancient glaciers, mountain ranges, canyons, fossils, bygone oceans, and tectonic plates. The 4.5 billion year old Earth story is one of continuous and dramatic metamorphoses on a time scale difficult to imagine; unless, of course, you happen be a geologist – or in this case, two geologists.
This is the backdrop to the Pilkeys’ exploration of useless arithmetic in the current debate on anthropogenic global climate change. Their message undermines everyone and every position in the current global debate about global climate change. The book came out before the release of the 2007 Intergovernmental Panel on Climate Change (IPCC) Report, but we do get a careful analysis of the 2001 IPCC Report. Perhaps this section can be updated in future releases of the book, even though there is no problem in extrapolating from 2001 to 2007, unlike some of the other extrapolations discussed in their book.
There are about fifteen major climate models used by scientists around the world. Favored are bottom-up models, involving a long chain of events and very complicated computer simulations running on supercomputers. This approach uses a great aggregation of models, and models of models, all the way up. In other words, it is models all the way down too. The assumption here is that the more variables included in the meta-model, the better the meta-model. Another approach, the minority view, favors top-down models, focusing only on larger systems – simplify, averaging, estimating, testing, but not presuming to include every potentially relevant variable. Predicting future sea-levels, of course, is only one piece of the climate puzzle. Up or down, the Pilkeys profess:
What a daunting task faces those who choose to predict the futures of the sea-level rise! We have seen that the factors affecting the rate are numerous and not well understood. Even if our understanding improves, the global system simply defies accurate and quantitative prediction because of its complexity. (76)
Their argument is not whether our climate cup is half full or half empty. Geologists have a different perspective on time. Their earthy timescale is some 4.5 billion years. All rock is ultimately metamorphic rock. And this includes the concrete, steel, and glass monuments of human engineering and architecture built in cities around the world. Imagine my beloved New York City, and every other at some point in the future, crushed under mile-thick glacier ice, or perhaps absorbed back into the molten core of the Earth through normal plate tectonics, or perhaps someday under the ocean. A geologist knows, it is only a matter of time — hot and cold, sea levels up and down, round and round the sun — before there are dramatic changes on our restless and creative planet. Maybe this will happen soon, maybe suddenly, and maybe not for a long time, at least relative to the scale of human life, but it will happen, if the past is any guide.
The American Petroleum Institute and Dick Cheney should take no pleasure in the Pilkeys’ thorough challenge to the global climate-change prediction industry. Anthropogenic climate change may be a real concern. And furthermore, the same types of modeling errors and unknowns presumably also call into question industry models of global petroleum reserves. The Pilkeys’ real argument is that no scientist can offer cogent predictions of the Earth’s climate – too hot, too cold, or just right. No matter how much data is collected, no matter how sophisticated the computer program, no matter how powerful the supercomputer employed to run the simulation. Complex natural systems cannot be modeled in a way that generates useful predictions. There are too many variables, too many feedback loops between variables, and the system is dynamic in ways that we do not understand and cannot represent mathematically.
In the case of climate change, a short list of variables and feedback loops might begin:
- the absorption of CO2 by the ocean,
- the heat exchange between the oceans and the atmosphere,
- the effect of cloud cover,
- variations in the Earth’s albedo,
- ocean current circulation,
- local climate perturbations,
- long-term climate cycles
- arctic ice melt,
- release of methane from melting artic tundra,
- health of phyloplankton,
- variations in amounts and types of precipitation, and
- many more confounders large and small.
Any of these variables could accentuate or ameliorate climate change and could do so with runaway dynamics. The authors are leaning agnostic to pessimistic on the prospects for near-term climate change (resulting from anthropogenic causes). It may not be all that bad. It may even be worse. We have no way of knowing, in spite of the $2 billion-per-year industry funded by the United States government to studying climate change. The Pilkeys use strong words to criticize these expenditures:
Assumption upon assumption, uncertainty upon uncertainty, and simplification upon simplification are combined to give an ultimate and inevitably shaky answer, which is then scaled up beyond the persistence time to make long-term predictions of the future of sea-level rise. Aside from the frailty of the assumptions, there remains ordering complexity: the lack of understanding of the timing and intensity of each variable. (82)
The authors advocate instead a qualitative methodology that settles with tendencies, directions, and magnitudes of change. A supercomputer is not required to document actual glacial declines around the world over the last few decades. Before-and-after photographs from a tourist camera of Muir Lake, Alaska from 1941 and 2004 provide compelling evidence for major changes (83). Over twenty years of space telemetry and ground observations in Antarctic give us disturbing short-term trends. Over a three-year period, the West Antarctic Ice Sheet lost 36 cubic miles of ice per year. The complete melting of the West Antarctic Ice Sheet alone would produce a 13-foot global sea level rise (78). Maybe you should rebook that summer vacation after all.
The Pilkeys certainly seem to think that global climate change is a serious problem. It is just that “A serious societal debate about ‘solutions’ can never occur so long as modelers hold out the probability, just around the corner, of accurate projections of future climates and sea-levels” (86). There will be no accurate projections.
Along with their scathing critique, the authors do manage a backhanded compliment to climate change modelers, at least by way of a negative comparison to their own guild in applied geology. They write:
The publications of this diverse international group (IPCC) are filled with painfully long discussions about error, uncertainties, and missing data. The objectivity of these global change modelers stands in stark contrast to the arrogance of the coastal engineers or the overconfidence of ground water modelers (79).
It is not that mathematical predictions are always impossible. Far from it. At one point, the authors quote reassuringly the New York Times for June 7, 2004:
In New York City sunrise will be at 5:25 am. Eastern time on Tuesday, and Venus is to begin leaving the solar disc at 7:06 am, when the sun is 17 degrees above the horizon. The planet’s final contact with the sun’s edge should occur about 7:26 am when the sun is 20 degrees high. There will be another transit on June 6, 2012…” (34)
It is comforting that some things can be known with certainty. I can plan on another transit of Venus in 2012. Predictive success is thought to be the sine qua non in most science, technology, and engineering fields. Regularity and reproducibility have traditionally been seen as one of the hallmarks of science. I count on it every time I log onto this computer, get on an airplane, or take an elevator to the 40th floor. In some domains, however, science is going to need to let go of prediction. Two things have changed:
1) the rise of complexity and
2) the rise of computation.
Environmental and human processes have always been complex. This is not new. It is just that now we have a lot more insights and background information. We know a lot more of the details, so we are compelled by the known facts at every turn to ask more and more complex questions. This is true in many disciplines, but for the Pilkeys, it is the key to understanding our human power in affecting major environment changes by our actions. For instance, they launch the first chapter showing how industrial fishing wiped out the North Atlantic cod fisheries, in spite of mathematical models predicting levels for maximum sustainable yields.
The complexity challenge also arises because of the availability of the computer. Every scientific discipline has been dramatically changed over the last twenty years by the availability of computers. Scientists can now collect enormous datasets, query the datasets, and run computer simulations. Without computers, there would be an epistemic bias towards asking simpler questions and ignoring questions that were thought to be beyond the capabilities of science.
Climate change is only one of two-dozen different kinds of quantitative modeling projects that the Pilkeys discuss in their book. Each example demonstrates failures of quantitative modeling, including:
- maximum sustainable yield and the Atlantic cod fishery,
- plans for storing highly radioactive nuclear waste in Yuka Mountain,
- invasive weed species,
- 1972 Club of Rome Report, Limits to Growth,
- McNamara’s management of the Vietnam War,
- abandoned pit mines water toxicity,
- forecasting on Wall Street,
- Enron collapse,
- EPA second-hand smoke studies
- Lord Kelvin and the age of the Earth
- soil erosion on sandy coasts
- engineered beaches
- salt-marsh grass
- Brown Tree Snakes on the Island of Guam
We also get a thorough introduction to Orrin Pilkey’s specialty — developed shorelines, treated in two chapters and the appendix. These should be required reading for anyone living in the coastal communities on any of the seven seas.
Already in the second chapter, the Pilkeys begin to develop a typology for modeling. This comes with a long list of common modeling errors. This genealogy of models – mathematical, applied, quantitative, qualitative, statistical, epidemiology, simulations, analytic, numerical, static, dynamic, conceptual – are all discussed with an eye to how the model employed can distort our understanding of reality. Other sources of reality distortion result from computer coding, uncertain debugging and quality assurance in computer programming, algorithmic biases based on important assumptions, situational bias, model-tweaking, pessimist and optimist biases, advocacy and politically correct biases. All of this, compounded and confounded by increasing complexity, causes us to often ask the wrong questions. We don’t look back. I will refer to these as tragic errors, distortions that arise because we are always imperfect humans being. We are finite and mortal. We make mistakes.
There are other sets of modeling distortion. Let’s call these complexity errors, becausethese errors result from the nature of complexity itself. Our models necessarily make assumptions about partially known and unknown relationships, expressed in ordering complexity with different valences, intensities, and vectors. There are negative and positive feedback, linear and nonlinear systems, deterministic or probabilistic strategies. So too I might add exponentially more data sets, but also exponentially more models. It’s models all the way down.
Complex models often exhibit sensitivity. This means that when some small variable is changed, the system changes dramatically. Complex models can exhibit sensitivity to initial conditions, variations in guiding assumptions, and minor modifications in ordering the parameters. The Pilkeys remind us that “the sensitivity of the parameters in the equation is what is being determined, not the sensitivity of the parameters of nature.”(25) The italics is theirs, so let me restate and interpret.
There are two problems that need to be solved in every model of complexity. First, what is the ordering of complexity in the system, the timing and intensity of different parameters? And second, how does one best “re-represent” this ordering of these parameters and complexities mathematically on a computer? Algorithms need to be imagined. Relationships defined. Data collected. Data analyzed. Values assumed. Code written. Models tested. Simulations run. And all of this — the algorithms, the lines of code, sets of data, computer storage and processing — have all been growing exponentially over the last three decades. But, and this is what the Pilkeys are emphasizing, as a simulation leading to predictions, the computer model is only simulating and testing itself. The computer re-representation is not “run” on the actual complex natural phenomena.
The Pilkeys show that substituting mathematics for nature is itself a source of errors in modeling nature. What is most illuminating are the varied ways that models are corrupted and misguided. What is the impact of substituting laboratory measurements for nature? What happens when we scale up short-term predictions into long-term predictions? What happens when one chooses and omits different parameters in a model of nature? What we do not know about initial conditions in a model of nature? What happens with the intrusion of forces from outside of a particular model of nature?
The Pilkeys are advocates of qualitative modeling, which at best can be used only to predict general directions of change and possible magnitudes. Qualitative modeling will not presume to offer a numerical answer with a range of error. The approach asks why, how, and what if. Qualitative modeling can also use large datasets, computer simulations, and lots of arithmetic, but they used to explore different scenarios, contingencies, and normative relationships. At the end, there is also humility and uncertainty, multiple scenarios, and no hard and fast predictions. The authors offer the following chart, in other words, a model of modeling (200).
Scenario Planning <<—>> Strategic Planning or Mathematical Modeling
Qualitative input <<—>> Quantitative input
Exploits uncertainties <<—>> Minimizes uncertainty
Long-range planning <<—>> Short-term planning
Multiple answers <<—>> Single answer
Planning for the future <<—>> Predicting the future
Hypothetical events <<—>> Predetermined goals
The bad news about complex predictions is that we don’t know anything and we can’t know anything — not about future climate change, not about storing radioactive waste over eons, not about managing declining fisheries or invasive weed species. Science is butting its head against more and more complexity horizons, my term, not theirs. Science discovers complexity horizon that it cannot cross, but cannot yet accept. This is not a problem that can be solved with bigger datasets, more code, more powerful supercomputers, and less flawed and politicized science.
We cannot look over this horizon of complexity, in part, because we are mortal humans with normal human problems. We do not have a God’s eye view of the world and ourselves. This means that science will always be distorted by political and economic interests, the culture and personalities of the scientists at that time. Even if we could minimize all of these “externalities”, science is still confronted with the problem of complexity itself. When the phenomenon is networked, reiterative, nonlinear, creative; then prediction will not work.
The Pilkeys focus on environmental changes, but I suspect that many scientists are on similar wild-goose chases when it comes to hope for understanding and controlling complex genetic systems, developmental biology, cognitive neurosciences, and a whole slough of other phenomena. Complexity is not just more; it is something new. There are known limits to computational complexity (Harel 2000). There are known limits to science (Barrow 1999). And the really creative processes in nature and by humans in nature tend to be complex distributed systems, not amenable to deterministic modeling (Kelly 1994) This is the greatest challenge for science today. It is also a challenge to any applied bioethics or environmental ethics, because the consequences of actions cannot be known in advance.
Again science produces lots of useful and reliable predictions. Mathematical modeling works well enough with simpler systems, like plotting the motion of the stars and planets in the evening sky or designing a modern bridge with stress-engineering of concrete and steel under variable loads and conditions. Multiply the variables, however, add a lot feedback loops, grow the complexity of a system, and suddenly predictive modeling becomes an exercise in futility. Predictive modeling cannot yield valid predictions for any complex natural and human related processes. This is truly the Earth shattering story, which really should be on the front page of the New York Times, not to mention Fox News. This story is about the approaching limits of science, at least a certain kind of science.
After goring so many sacred cows, it is perhaps understandable that the Pilkeys resist the temptation to move into metaphysics, philosophy, and applied ethics. These iconoclasts have already gotten themselves into a lot of hot water with their colleagues. One conclusion to be drawn is that humanity is now thrust willy-nilly into the role of managing the Earth, not that we really know what we are doing. The Pilkeys advocate a qualitative modeling approach, which aspires to predict mere tendencies, directions, and magnitudes of changing systems. After all of the qualifications and caveats though, I am not sure qualitative modeling has much more to offer in the way of certainty, comfort, or a clear plan of action. The future will always be shrouded in a cloud of uncertainty.
And that is the bad news enumerated in Useless Arithmetic. Humans will never have the complete know-how, even though we certainly have increasing can-do. Humans have themselves become an important variable in the future evolution of the planet. This book offers no comfort or consolation. The Pilkeys offer no hard and fast predictions.
The good news is that we live and think in a networked universe. Our environment is networked, as are our networked bodies with our networked brains in our networked culture. Let’s call it a metanexus. You and I are surrounded by, constituted by, and are also ourselves dynamic components within all kinds of complex distributed systems. These systems transcend us and form us, even as we also participate in their transformation. The universe is metanexus all the way down. These complex distributed systems exhibit creative intelligence, even elegance, though not unfailingly to our benefit. Still some amazement and gratitude are evoked. This seems like a promising point of departure for a new theology of nature based on a rather different understanding of nature (and science). I also find it hopeful that science has known theoretical and practical limits. Do not get me wrong. Push the mechanistic, reductionist, and predictive envelope as far as possible. Without the skeptics like the Pilkeys, however, there would be no way of escaping from “misplaced concreteness”.
Science must now recognize that there are non-reducible emergent, transcendent systems, which seem to constitute many of the most interesting and creative phenomena in our contextual universe – ecosystems, genomics, brains, and culture. No amount of mathematical modeling, computer simulations, reiterative databases, and paradigm filtering will get us beyond this horizon of complexity. We may hope that an “Invisible Hand”, reputably at play in free economic markets to the maximum benefit of all, is also at play in the free evolution of technology, culture, and the planet. We won’t know for certain, but the very hope itself now becomes a variable in our future modeling and doings.
None of this relieves us of the risks and responsibilities of taking action. We have to make choices. We have to project desirable outcomes. Let us try to model, design, and build for sustainable and better futures. Expect adaptation. Think geology.
How should governments, business, and citizens respond to the real and/or perceived threat of global climate change? The Pilkeys don’t really say. Perhaps the question is as perplexing as asking how one would plan for and respond to a dramatic non-anthropogenic climate change? Still I wish they had been more explicit in their recommendations for the stray business leaders, elected leaders, and eclectic citizens who might pick up this book.
For my part, we need to deemphasize climate change and look at other variables. There are many compelling arguments for radically reducing fossil fuel consumptions. These reasons do not depend on prognostications of climate models. Reducing fossil fuel consumption will improve local environmental air and water quality. It will increase health, safety and quality of life. It will slow resource depletion. Reducing fossil fuel consumption can improve the bottom-line for individuals, corporations, and entire economies. There are also important national security interests at risk, if we do not dramatically reduce fossil fuel consumption. We don’t need a global climate change scare, in order to justify, rationalize, or motivate, what should already be obvious and sound public and private policy. It is in the best interest of the United States and the world to dramatically reduce fossil fuel consumption, especially through increased efficiency, while also developing alternative energy sources. I wonder whether the Pilkeys would agree. After reading their chapter on nuclear waste storage, I doubt they would be enthusiastic about increasing nuclear power production as one of those alternative strategies. Again, the authors leave us hanging, perhaps intentionally.
Useless Arthmetic is a book that should be adopted widely in college courses because professors and students both need to read it. It is directly relevant in departments of engineering, environmental science, economics, public policy, medicine, sociology, psychology, history of science, law schools, computer science, and applied mathematics. I would also add departments of philosophy, religion, and theology, who have a vested interest in understanding the content, practices, limits, and interpretations of science.
In the end, the qualitative modeling advocated by the Pilkeys will also fail to make useful predictions. Perhaps their approach offers more understanding with less explanation. When they do fail, they will do so humbly and with multiple scenarios in their back pocket. This may not be very satisfying. Remember that humans are being asked to make major political and economic decisions in response to an unknown threat of anthropogenic climate change. And that is just the tip of the iceberg, so to speak, of the many and varied complex ways that humans and nature interact.
The Pilkeys call for an adaptive management. To this we might add adaptive epistemology. This strategy is the most potentially transformative take-home from the book, but very few examples are offered. It would be nice if they developed adaptive management and adaptive epistemology with lots of specific examples. How do corporations, governments, and people actually implement an adaptive management strategy? How would scientists practice adaptive epistemology? Perhaps their next book will offer stories of successful case studies, the lessons learned, the successes counted, and the adaptations made. We need a lot more examples of successful case studies in the world today.
Barrow, John D. (1999).Impossibility: The Limits of Science and the Science of Limits. New York, Oxford University Press.
Harel, David (2000).Computers Ltd.: What They Really Can’t Do. New York, Oxford University Press.
Kelly, Kevin (1994).Out of Control: The New Biology of Machines, Social Systems, and the Economic World. New York, Addison-Wesley.
Pilkey, Orrin H. and Linda Plkey-Jarvis (2007).Useless Arithmetic: Why Environmental Scientists Can’t Predict the Future. New York, Columb a University Press.</