Wednesday, April 27, 2011

Constructing a Scientific Theory

As someone interested in the science of climate, I often see claims that the current paths used by researchers to investigate our earth are not 'science'. That is, part of the program of climate science uses physical models based on both classical and quantum physics to re-produce and predict past and future climatic conditions, respectively. Some believe that since some of these model outputs cannot be 'falsified', the entire endeavor is un-scientific.

I do agree that there is some truth to parts of these claims. Prof. Roger Pielke Sr. has long been critical of aspects of climate model use and makes important points. Today in fact, in response to a paper published in the Geophysical Research Letters, Peilke the Elder pointed out,

'Moreover, they (the authors of said paper) write “[t]he credibility of model-simulated cold extremes is evaluated through both bias scores relative to reanalysis data in the past and multi-model agreement in the future.” The testing against reanalysis data for the period 1991-2000 is robust science. However, bias scores using“multi-model agreement in the future” is a fundamentally incorrect approach.

Now, in this post, Pielke the Elder pulls no punches as to how he feels about this paper and the research program that he believes it more broadly represents. He calls the approach 'scientifically flawed', 'fails as robust science' and represents a 'failure of the scientific method'.

But he never says that it is not 'science'. To him, it's just bad 'science'.

I think this is an important distinction to make in the context of this notion of 'science'. Some commenters are stuck on the idea that there are very strict rules, mostly coming from esoteric and mostly useless (in my opinion) philosophical texts, that parse out the exact way in which 'science' is to be done. A course of investigation that follows this path can bear the name 'science', while anything less is disparaged and not worth second glance. These proponents of a squeaky clean notion of 'science' often point to the simplest of physical laws to makes their case. Most times confusing the process in which that physical knowledge was discovered and verified.

What one finds in the process of doing actual scientific research that the process is quite different from these 'rules of science', however. One finds often there is not a clear-cut 'answer' to a specific situation. The data are consistent with multiple models or the models cannot account for the full variation that the data spell out or several other, more complex situations can emerge from a given research track. In such a situation, which is very, very common in scientific research, the practitioner finds that the 'rules of science' are not very helpful.  Specific hypotheses cannot be rigorously tested, yet the structure of the scientific community necessitates 'a paper'.  It becomes a matter of the researcher's judgment as to what explanation seems most reasonable.

Sometimes that judgment is good. Other times it's not so good.

From that point, other researchers can pursue the problem further, with different methods and approaches that may shed light on an aspect unseen originally. Replication can better confirm at times, but also create more confusion based on what is already known in that given situation. And so the process continues.

Which brings us to the larger point. 'Science' is what needs to be done to 'solve' a particular research situation. That might mean using a computer simulation of a physical or biological model with known errors. It might mean using an experimental technique not analytically designed for the experiment you want to run. Ideally, one would find the absolutely best technique for a given research 'job' and use that technique to its utmost ability. Unfortunately, one often finds oneself in a situation where any data helps better inform one's judgment about what is happening. Therefore, we are willing to use techniques that suffer poorer resolution of the pertinent dynamics or numerical calculations with known problems. It may simply be a resource problem, but it can't stop us from trying to do 'science'.

All of that said, if one is using a known problematic technique, that fact has be very transparent when the work is reported to the community. In the case of the paper highlighted by Peilke the Elder, the group in question is overconfident in the approach they're using. They are not forthcoming with the fact that the models they use are not good at predicting the parameters.

I think that's what makes their paper 'bad science'.  But it's 'science' nonetheless.

Saturday, April 23, 2011

Climate Change Debate: Is Science Really at Its Heart?

Prof. Judith Curry of the Georgia Institute of Technology has recently presented a post in which she seems to point out that in all of the analysis concerning the failure of cap and trade and other climate-related legislation, commentors and experts are neglecting the role of the state of climate science in forming people's opinions on policy possibilities. She points to her testimony to Congress in which she discusses the poor communication of uncertainty and over-simplication of the science itself. From there she states,

'These are the issues that concern skeptics and scientists and professionals from other technical fields. And these are the people that influence politicians and other thought leaders on this subject.'

While I think there is some truth is pointing out that scientists and researchers from fields outside of climate science find some conclusions hard to believe, I challenge that this has any meaningful effect on 'politicians and other thought leaders'. In the same way that I would like to see some well-conditioned evidence to support many of claims I read in the media concerning climate impacts, it would be nice to see a citation for a serious study that looks into the political effect of belief in climate science among a very small contingent of research educated members of the public.

It seems to me that there is strong cross-correlation between the fields from which these individuals come (economics and engineering, mostly) and their political identity (conservative). A belief that the science is questionable in this group of individuals would simply be a variation that could be explained by politics and has really nothing to do with the science itself.

And I think that's the point. Dr. Curry points out,

'Steve McIntyre starkly disagrees with Olson’s diagnosis of the problem, in a post entitled “The Smug Loop“, and calls for an “engineering quality exposition” of the climate change science...'

Unfortunately, McIntyre's audit is not going to disprove the increase in surface temperatures in the last century. Maybe the number will decrease slightly, but it won't 'go away'. His ideas of what 'good science' are will simply become more common practice among the small group of researchers whose work he would like to more critically investigate. In my opinion, I agree with the notion that more transparency is better, but I am not of the attitude that we are going to discover some incredible cover-up in which these researchers have lied to the scientific community and themselves via some kind of audit. So, such an undertaking is not really about the science in of itself, but the perceptions of how the science is done, which is a distinction worth noting in the context of this type of discussion.

Other scientific factors will also not affect the take home point.

Understanding natural variability will be very important as civilization continues to adapt to our planet. But expanding natural variability is not going to invalidate the notion we are contributing to an increase of the greenhouse effect by burning stuff. It also will not invalidate that there will be both good and bad outcomes for particular places on the planet due to a continued increase of the greenhouse effect.

As we learn more and more about the coupling of different processes in the climate, we are not going to simultaneously find out we have no effect on it.

I think there are aspects of the actual science that need to be better brought to the forefront of public knowledge. That we are so dependent on climate models producing future possible climates to inform our decision making is pretty far from ideal. The lack of agreement between sets of climate models parameterized to optimize specific climate processes makes confirmation of specific outcomes difficult to interpret. And there is an uncertain amount of uncertainty involved in establishing what outcomes are reasonable and which are not. Most of the general public does not know these things, but I'm not sure that this fact matters in the context of a policy debate. If scientists are transparent about these issues, that's basically all we can hope for at this point. Maybe that's a bigger 'if' than some of us would like, but here we are.

But again, that doesn't have to do with the quality of the science itself. That has to do with inappropriate communication of science. Either downplaying the role of what we do not understand or over-generalizing the import of the aspects of climate we do understand.

The take home point even accounting for poorly communicated levels of uncertainty and possibly over-exaggerated results is still that we are affecting the world around us. Knowing that, the questions become: what type of impact do we want to have and what do we need to know to have that impact?

These minor aspects of the scientific facts and science communication, however, are not changing the average person's or politician's mind on how to respond to 'the threat of climate change'. To me, the entire climate debate in the political realm has NOTHING to do with science. A quick peruse of WUWT will show that they are just as uninterested in science as proponents of policies pushed by projections of major climate 'tipping points' for which we have no evidence to validate.

The main issues that many research educated people have with climate science are informed by politics. They don't want the government telling them what to do. In order to motivate that narrative, they use whatever credentials they've got and see what sticks. So far, I agree with Nisbet that this strategy has had an impact ranging from marginal to negligible.

On the flip side, people who believe in government intervention feel that we should do 'something' to combat climate change, despite the fact that there are really no potential policies that have been demonstrated to actually achieve that goal. Meanwhile, as Dr. Curry has pointed out many times, there is a stalemate with both sides of the debate trying to determine what is happening. 'Skeptics' are more than willing to accept blame because, to them, it bolsters their position that there are real questions concerning the most fundamental aspects of the science, despite the complete lack of any evidence to support such a conclusion.

Prof. Curry does come to a place that is interested in solving the hardest problems facing climate science at the moment. Some of those techniques are being used already via the collaboration accessible on the Berkeley Open Intrastructure for Network Computing (BOINC). This collaboration has already had some results, though not with the impact the authors likely intended, in my mind at least.

All in all, I think we need to move past this notion that there is something scientific involved in the collective inability of individual nations and the global community to agree on a path to forge with respect to climate change. Up to this point, there is no reason to believe such a conclusion.