ToxSci Advance Access originally published online on August 31, 2006
Toxicological Sciences 2006 94(2):256-260; doi:10.1093/toxsci/kfl095
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
From Science to Policy—When are Scientific Results Certain Enough?
Division of Environmental Medicine, Norwegian Institute of Public Health, NO-0403 Oslo, Norway
1 To whom correspondence should be addressed at Department of Air Pollution and Noise, Division of Environmental Medicine, Norwegian Institute of Public Health, P.O. Box 4404 Nydalen, NO-0403 Oslo, Norway. Fax: +47 22042686. E-mail: annike.irene.totlandsdal{at}fhi.no.
Received May 23, 2006; accepted August 25, 2006
 |
ABSTRACT |
|---|
 TOP ABSTRACT BACKGROUNDDEALING WITH UNCERTAINTY AND...CONCLUDING REMARKSREFERENCES |
|---|
The amount of scientific findings linking air pollution with adverse health effects is continuously growing and indicates a need for action to improve air quality. On 21 September 2005, the European Commission published a new draft directive on air quality, as part of the Thematic Strategy on air pollution. This is a long-term plan on how to reduce air pollution in the European Union in the next 15 years. Immediately after its release, the Commission received criticism for not going far enough from various instances, such as research institutions and nongovernmental organizations concerned with health and environmental effects of air pollution. One policy argument for not taking full measures was the argument of scientific uncertainty. In light of this air quality strategy and the corresponding criticism which ensued, the present article discusses how the ambiguity of scientific uncertainty may contribute to impeding the process of translating scientific findings into concrete policy options. As complete certainty is likely to never be achieved, the question arises whether it is possible to determine and agree on clear and applicable definitions of certain levels of scientific certainty. The case referred to in this paper clearly demonstrates a situation with discordant views on the uncertainty of scientific findings. More discussion on how to define scientific uncertainty and how to deal with it would be beneficial for both the scientific and the political communities. Finally, it is important to recognize that scientific evidence is not the only driver influencing policy decisions.
Key Words: policy; science; uncertainty; air pollution; health.
|
BACKGROUND |
|---|
|
TOPABSTRACTBACKGROUNDDEALING WITH UNCERTAINTY AND...CONCLUDING REMARKSREFERENCES |
|---|
On 21 September 2005, the European Commission published a new draft directive (European Commission, 2005
) on air quality, as part of the Thematic Strategy on air pollution. This is a long-term plan on how to reduce air pollution in the European Union (EU) in the next 15 years. According to the Commission's analysis, 370,000 premature deaths in Europe in 2000 were related to exposure to particulate matter (PM) and ozone. By implementing the Commission's plans this number is expected to be reduced to 230,000 by the year 2020. Immediately after its release, the Commission received criticism for not going far enough from various instances, such as research institutions and nongovernmental organizations concerned with health and environmental effects of air pollution. More specifically, in a letter signed by 37 air pollution and health scientists across Europe, the suggested cap value of PM2.5 (fine PM) is accused of being far too high to adequately protect human health (IRAS [Institute for Risk Assessment Sciences], 2005). Further criticism was expressed as, "‘uncertainty’ is being used as an argument to support this high value whereas research, in no small measure paid for by the Commission, has reduced uncertainty to the extent that an evidence-based pollution reduction policy can now be strongly supported" (IRAS, 2005). Another point of criticism has been the current lack of legally binding of the strategy's exposure reduction plans.
From the content of the directive, it is obvious that the EU recognizes the current levels of air pollution as a significant threat to human health and the necessity of taking actions. As a result, the EU has chosen a strategy with an annual budget of 7.1 billion Euros. This strategy is predicted to reduce, but not to remove, the adverse health effects of air pollution. A great number of citizens will still be suffering from polluted air in 2020 and the corresponding costs of damage to human health alone are estimated to between 189 and 609 billion Euros per annum (European Commission, 2005). This means that the health damage caused by air pollution is estimated to cost between 27 and 86 times more than the suggested strategy for improvement. It can therefore be questioned if not the use of more resources and stricter guidelines may be justified. In this strategy, the Commission suggests a cap value of 25 µg/m3 for PM2.5, in order to protect against high exposure levels. As an argument for choosing this particular value, it is stated in the directive that it "takes account of the inherent uncertainties in our current knowledge about the risks of PM2.5" (European Commission, 2005). It is this part of the directive in particular which provokes the scientists, who claim that research, with sufficient certainty, has demonstrated that such a level has been associated with significant adverse effects (IRAS, 2005). The scientists also mention that complete certainty never will be achieved and that there are reasons to think that the estimated risks are on the low side, rather than on the high (IRAS, 2005).
In short, the European Commission and the scientists agree about the health impact of particulate air pollution and the need for action to protect against such effects. However, there is an obvious discrepancy in what they consider as sufficiently certain scientific evidence. To demonstrate the benefits of the strategy, the Commission refers to their cost-benefit analysis. This analysis is based on input from research which is considered as too uncertain for a lower PM2.5 cap value, but certain enough for the selection of a strategy. The cost-benefit analysis shows that significant effects of PM will remain in 2020 after having implemented the strategy. Despite this, the Commission suggests a PM2.5 cap value regarded as "highly unprotective" by the scientists and to delay the decision of making other targets of the strategy legally binding. The selection of a strategy, which is known to give limited health benefits on something which is recognized as a significant health threat, can be seen as a demonstration of science not being the only driver of environmental policies.
In light of this air quality strategy and the corresponding criticism which ensued, the present article discusses how the ambiguity of scientific uncertainty may contribute to impeding the process of translating scientific findings into concrete policy options. There are of course factors additional to scientific uncertainty which may influence a risk assessment or a cost-benefit analysis which serves as key information in discussions leading to policy decisions. It is for instance important to ensure that the risk assessments are scientifically objective by giving weight to both positive and negative studies and by examining all available evidence. Closely linked to the process of summarizing and interpreting scientific evidence is the concept "weight of the evidence." This concept has recently been comprehensively reviewed (Weed, 2005) and indicates a need to reach consensus on its meaning and the use of its associated methods. The importance of reaching consensus on the concept's meaning is also applicable to scientific uncertainty.
|
DEALING WITH UNCERTAINTY AND ITS AMBIGUITY |
|---|
|
TOPABSTRACTBACKGROUNDDEALING WITH UNCERTAINTY AND...CONCLUDING REMARKSREFERENCES |
|---|
Airborne PM's ability to induce both respiratory and cardiovascular morbidity and mortality is well documented. Also, toxicological evidence tends to support the epidemiological results. An important activity in research of PM effects on health has been to meet the criticism on single city/area studies often with a single air quality monitoring station by launching large multicenter studies using the same methodology. Despite this, it has proven difficult to define an air quality standard which adequately protects human health. One factor contributing to this is that a threshold level, below which effects are unlikely to occur, has not been identified. No threshold level means that all exposure levels can lead to some degree of adverse health outcome. Although lower exposure levels are likely to cause less harm, the challenge still remains to then determine an acceptable risk level. Additional factors making the development of guidelines for PM difficult are uncertainties induced by for instance the wide range of techniques used for exposure assessment, variation in individuals' susceptibility and variation in composition of PM in time and by location. The latter factor implies that the health benefit of compliance with a certain PM standard might vary from negligible to considerable within and between various populations. Risks can be estimated when both an outcome and its probability of occurrence are well known. This requires knowledge about the mechanism underlying the association between the cause and the outcome. With regard to PM, no consensus on the precise mechanism behind its associations with health effects has yet been reached, despite many investigated hypotheses. Most likely several mechanisms are of importance. Based on this information on PM's potential to adversely affect human health, it can be concluded that compliance with the suggested strategy may on one hand be beneficial, have no effect, or on the other hand be harmful on an individual level.
With regard to different types of air pollutants, there are wide gaps in our current knowledge and thus uncertainty with respect to their potential to affect human health. To provide an adequate level of protection and an acceptable margin of safety, uncertainty factors have been used in setting guidelines for inhalation of different air pollutants. Such uncertainty factors are designed to take into account factors influencing the data on adverse effects of pollutants on human health. The quality of the data on effects, possible interactions with other components, the type of adverse effect investigated, extrapolation of effects observed in other species to effects in humans, extrapolation from high exposure effects to low exposure effects, interindividual variations in sensitivity and from small groups selected for studies to the general population are all examples of factors influencing the size of the uncertainty factor.
For carcinogenic air pollutants only unit risk evaluation is generally applied and no safe level can be recommended, whereas for noncarcinogenic compounds uncertainty factors have varied between almost zero (for carbon monoxide) and 1000 (dichloroethane) (WHO Regional Office for Europe, 2000). These wide differences in uncertainty factors may be due to variations in the quality of the data and the potential of these compounds to damage human health. This demonstrates that uncertainty is dealt with in different ways. Uncertainties regarding exposure levels and adverse health effects for many air pollutants have apparently been far greater than for particles, e.g., dichloroethane, benzene, nitrogen dioxide, and acid particles (Grahame and Schlesinger, 2005; WHO Regional Office for Europe, 2000). However, the impact of low level guidelines and even more so of rigorous standards would be far greater for particles than for example dichloroethane because exposure to the latter compound is not ubiquitous and thus an estimated health impact would be much less than that for particles.
The fact of not knowing what we do not know makes it impossible to foresee every outcome of a cause or to identify all causes for an outcome. This might partly explain the commission's hesitations for a more stringent PM cap value. Obtaining compliance with stringent air quality standards can become very costly, depending on emission levels. Expensive investments in emission reduction implicate expectations of noticeable improvement. This does not only mean improvement in terms of achieving compliance with air quality standards. A reduction in incidence and prevalence of adverse health outcomes should also be achieved. As such expectations are likely to increase with increasing use of resources, a stringent strategy may result in deep disappointment and even mistrust should it turn out to be a failure.
The possibility for an improvement strategy to result in unforeseen adverse effects is a consequence of the persisting gaps in our knowledge. This can be adverse effects on the specific situation a strategy is aimed at, as well to other areas which initially were not expected to be influenced at all. With regard to PM emissions, it has for instance been hypothesized that filtering out larger-sized particles may cause an increase in the concentration of smaller-sized particles. The idea behind this hypothesis is that small particles under normal ambient conditions are partly removed by aggregation onto larger particles. Together with observations indicating that smaller-sized particles might be more harmful to health than larger particles, this illustrates how a measure aimed at improvement might result in the opposite, just because we do not know everything.
How should policy makers cope with this ever-existing uncertainty? In their directive, the Commission seems convinced about "inaction" not being an option, but uncertain about the effectiveness of making the exposure reduction targets legally binding and lowering the PM2.5 cap value. Based on the current scientific knowledge with its "inherent uncertainties" the Commission has decided to take a certain amount of measures. Such an intermediate approach may reduce the impacts of unforeseen outcomes and protect against potential accusations for nonaction or for having wasted resources. The decision of taking measures against a potential threat, based on scientific knowledge with "inherent uncertainties," can be linked with the general ideas behind the so-called "precautionary principle"; "better to be late than sorry." According to EU's Communication on the Precautionary Principle, this principle will be applied "where preliminary objective scientific evaluation, indicates that there are reasonable grounds for concern that the potentially dangerous effects on the environment, human, animal or plant health may be inconsistent with the high level of protection chosen for the Community" (European Commission, 2000).
There are various definitions of the precautionary principle. The EU builds upon Principle 15 of the 1992 Rio Declaration on Environment and Development which states that, "In order to protect the environment, the precautionary approach shall be widely applied by States according to their capabilities. Where there are threats of serious or irreversible damage, lack of full scientific certainty shall not be used as a reason for postponing cost-effective measures to prevent environmental degradation" (UNEP [United Nations Environment Programme], 1992). This definition, as other definitions of the precautionary principle, contains ambiguous terms which makes the approach interpretable in many ways, and thus applicable for most situations. What is for instance seen as a "threat"? How severe must the threat be to meet the criteria for taking precautionary measures? Furthermore, as "full scientific certainty" is likely to never be achieved, and as gaps in knowledge might influence what we perceive as a threat, are the criteria for reaction according to the principle exclusive? The arbitrariness and ambiguity of the precautionary principle has been criticized (Marchant and Mossman, 2004).
To what extent EU's strategy on air quality can be seen as acting according to the precautionary principle depends on how EU's guidelines for application of the precautionary principle are interpreted. These guidelines state for instance that "measures based on the precautionary principle must not be disproportionate to the desired level of protection and must not aim at zero risk, something which rarely exists" (European Commission, 2000). Another guideline is, "Risk reduction measures should include less restrictive alternatives which make it possible to achieve an equivalent level of protection, such as appropriate treatment, reduction of exposure, tightening of controls, adoption of provisional limits, recommendations for populations at risk, etc." (European Commission, 2000). The strategy's correspondence with these guidelines might partly explain why the Commission does not go as "far" as some would have liked it to go regarding air quality. However, due to several ambiguous terms, the same guidelines may also be used to defend the criticism on the strategy. What is for instance meant by "desired level of protection" in the guideline on proportionality? Desired by whom? What is meant by "less restrictive alternatives"? Less restrictive than what? In their letter the scientists mention that the proposed PM2.5 cap value of 25 µg/m3 is significantly higher than the range of values of 12–20 µg/m3 proposed in earlier position papers of the Clean Air for Europe activity (IRAS, 2005). Compared to those values the strategy's cap value is "less restrictive," but would not also a cap value of 21 µg/m3 be "less restrictive"? Which level of "less restrictive" can be accepted?
It is likely that any value, even zero, which most would agree on as being an unrealistic target, would generate discussion. This is not the least due to the wide range of stakeholders affected by policy decisions, representing and defending different interests. Stringent air quality standards may for instance evoke reactions from the industry and transport sectors, as emission reductions implicate extra costs. Contrary to this, less stringent standards may evoke reactions from stakeholders concerned with health and environment. It is interesting to read in the scientists' letter that EU's suggested "cap" of an annual average PM2.5 concentration of 25 µg/m3, represents a level which most member states already comply with (IRAS, 2005). This implies that most member states will have the possibility to postpone reduction measures, or even increase their emissions, since this cap value is the strategy's only target which is legally binding. For some countries the strategy may thus lead to nonaction, which is contradictive to what is written in the directive about the necessity for taking actions. With this is mind it can be questioned to what extent scientific knowledge plays a significant role in policy decisions, as compared to other factors such as stimulation of economic growth, perception of other risks, public awareness, and maintenance of goodwill. Science contributes with information that can be used for prediction, prevention as well as treatment of certain threats. A purely science-based policy, however, might fail, as successes of policy measures also depend on acceptance among those affected.
In the United States, the federal Clean Air Act requires the Environmental Protection Agency (EPA) to carry out a periodic review and revision, where appropriate, of the scientific criteria and the National Ambient Air Quality Standards for criteria air pollutants such as PM. In a review of EPA's methods to determine the carcinogenic risk associated with exposure to hazardous air pollutants by The National Academy of Sciences it was recommend to conduct formal uncertainty analysis, which may be specifically useful in identifying where additional research is likely to resolve major uncertainties (NRC [National Research Council], 1994). Other recommendations with regard to uncertainty were to develop guidelines for quantifying and communicating uncertainty as it occurs into each step in the risk assessment process, and when ranking risks to consider the uncertainties in each estimate rather than ranking solely on the basis of point estimate value. Recently, the White House Office of Management and Budget (OMB) released a proposed Risk Assessment Bulletin which is a guide intended to enhance the technical quality and objectivity of risk assessments that are used by federal agencies in regulatory decision making (OMB, 2006). According to this bulletin influential risk assessments (in the bulletin defined as: a risk assessment the agency reasonable can determine will have or does have a clear and substantial impact on important public policies or private sector decisions) are to meet certain additional standards. With regard to uncertainty influential risk assessments should characterize uncertainty with a sensitivity analysis and, where feasible, through use of a numeric distribution (OMB, 2006). A recent workshop, "Bridging Components Along the Exposure-Dose-Response Continuum," focused on tools that can help scientists address uncertainty. The workshop highlighted the need for environmental and toxicological scientists to more fully utilize the techniques of probabilistic risk assessment for uncertainty analysis. Thus, the uncertainty issue has been developed further to facilitate the assessment of scientific results and maybe their use in policies.
|
CONCLUDING REMARKS |
|---|
|
TOPABSTRACTBACKGROUNDDEALING WITH UNCERTAINTY AND...CONCLUDING REMARKSREFERENCES |
|---|
This essay, written in light of an air quality strategy with corresponding criticism, demonstrates how discordant perceptions of uncertainty may offer great challenges when science is to be translated into policy. Despite more research, uncertainty will always be present. Uncertainty is thus something we will have to learn to deal with. With regard to the air quality strategy, it might be beneficial for the scientists and the decision makers to come together to clarify the discrepancy in their interpretations of scientific uncertainty and to set common goals for further research. In 1965, Sir Austin Bradford Hill made the following statement about the incompleteness of scientific knowledge: "All scientific work is incomplete—whether it be observational or experimental. All scientific work is liable to be upset or modified by advancing knowledge. That does not confer upon us as a freedom to ignore the knowledge we already have, or to postpone the action that it appears to demand at a given time" (Hill, 1965
). The precautionary principle can be seen as a response to this statement and offers a way for decision makers to deal with uncertainty. However, as with uncertainty, we are unfortunately also in this principle faced with ambiguous terms which causes confusion and discussion. Finally, it is important to realize that uncertainty is not the only challenge in the translation of science into policy. In fact, science will have to compete with other factors as for instance the importance of economic growth, public awareness, and perceptions of other risks. Undertaking policy decisions are very complex processes, eventually depending on a group's perception and judgment based on both information and pressure from various instances. A decrease in scientific uncertainty will therefore not necessarily increase science's influence in policy decisions.
|
ACKNOWLEDGMENTS |
|---|
Conflicts of interest: None declared.
|
REFERENCES |
|---|
|
TOPABSTRACTBACKGROUNDDEALING WITH UNCERTAINTY AND...CONCLUDING REMARKSREFERENCES |
|---|
European Commission. (2000) Communication from the commission on the precautionary principle. Available at: http://europa.eu.int/comm/environment/docum/20001_en.htm [accessed 28 April 2006].
European Commission. (2005) Communication from the commission to the council and the European parliament, Thematic Strategy on Air Pollution. Available at: http://europa.eu.int/comm/environment/air/cafe/ [accessed 28 April 2006].
Grahame T and Schlesinger R. (2005) Evaluating the health risk from secondary sulfates in eastern North American regional ambient air particulate matter. Inhal. Toxicol. 17:15–27.[CrossRef][Web of Science][Medline]
Hill AB. (1965) The environment and disease: association or causation? Proc. R. Soc. Med. 58:295–300.[Web of Science][Medline]
IRAS (Institute for Risk Assessment Sciences). (2005) Researchers' letter to the European Parliament on proposal of new PM 2.5 directive. Available at: http://www.iras.uu.nl/EPletter/EUletterFINAL.pdf [accessed 28 April 2006].
Marchant GE and Mossman KL. (2004) Arbitrary and Capricious: The Precautionary Principle in the European Union Courts 1st ed (American Enterprise Institute, Washington, DC).
NRC (National Research Council). (1994) Science and Judgment in Risk Assessment(National Academy Press, Washington, DC).
OMB (Office of Management and Budget). (2006) Proposed risk assessment bulletin. Available at: http://www.whitehouse.gov/omb/inforeg/proposed_risk_assessment_bulletin_010906.pdf [accessed 25 August 2006].
UNEP (United Nations Environment Programme). (1992) Rio Declaration on Environment and Development. Available at: http://www.unep.org/Documents.multilingual/Default.asp?DocumentID=78&ArticleID=1163 [accessed 28 April 2006].
Weed DL. (2005) Weight of the evidence: A review of concept and methods. Risk Anal. 25:1545–1557.[CrossRef][Web of Science][Medline]
WHO Regional Office for Europe. (2000) Air Quality Guidelines for Europe 2nd ed European Series, No 91. WHO Regional Publications, Copenhagen.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||