Research Themes: (C) Addressing uncertainty in risk assessments

No risk analysis can be conducted with full scientific certainty. Significant uncertainty arises from poor understanding of causal mechanisms in ecological systems and from limited data to describe components of a risk assessment model. For example, population growth rates are essential to characterizing population dynamics and spread rates (components of exposure assessment), but population growth has proven exceptionally difficult to predict for species introduced into new environments. Biologists often respond to this uncertainty by calling for more data. Student research within this theme will address three interrelated questions: (1) when does increased quantification enhance the value of risk assessments; (2) do different approaches for characterizing uncertainty lead to different risk management decisions; and (3) when does increased quantification reduce conflicts over risk management decisions?

Treatments of uncertainty in risk assessment vary. Expert judgments have figured extensively in qualitative risk assessments, but usually treat uncertainty in such general terms that it has little influence on risk characterization. In quantitative risk assessments, uncertainty has often been modeled with unrealistic probability distributions. Farm-to-table risk assessments for food-borne hazards, such as Salmonella enteridis in eggs and E. coli 0157:H7 in beef, both involving Kuzma, have pioneered methods to account for limits of knowledge about model inputs in large food and agricultural systems. More recently, Bartell and Nair (2004) studied how narrowing the range of uncertainty for parameters would improve understanding of establishment risk of the Asian longhorned beetle. Economic modeling can enhance such analyses by assessing the value of reducing parameter uncertainty, for both biological and economic parameters. To be sure, uncertainty analysis is not always warranted, nor does it always lead to better management decisions, for example, when screening indicates the risk is below levels of concern, the cost of reducing exposure is low, or characterization of the nature and extent of the hazard is inadequate to permit even a bounding estimate.

An innovative aspect of our ERA research is the application of worst-case analysis tools (bounding assessments), which have been successfully used in engineering systems to elucidate their worst-case behavior, given modeling error, uncertainty and exogenous disturbances. These techniques can directly assess sensitivity of the results to individual model uncertainty and are less data intensive than probabilistic models, yet can better inform decision makers. For example, worst-case and probabilistic analysis applied to the NASA X-38 Crew Return Vehicle prior to its first test flight revealed the effects of aerodynamic and mechanical model error on the performance of the vehicle. The probabilistic analysis methods failed to identify values of aerodynamic coefficients that would cause instability, whereas the worst-case analysis techniques successfully validated the flight control system and identified worst-case aerodynamic coefficients. Application to ERA will require refinement of quantitative ecological models and overall performance objectives for potentially affected ecosystems. Worst-case analysis would be used in concert with probabilistic analysis to clarify the role model parameters play in the analysis of such models as resistance evolution, non-target effects, and net-fitness for assessing gene flow.

A substantial economic literature on the value of information applies to the value of resolving uncertainty in parameters. Once quantitative models are developed and performance objectives are established, we can ask questions such as: is it preferable to devote research funds to learning about the effectiveness of control techniques or about the speed of an organism’s spread? The key parameters of the model can be estimated from existing scientific knowledge, and uncertainty can be incorporated via probability distributions for those parameters. We will then assess possible scenarios, each with different parameter sets, to find the optimal course of action under each scenario. Under complete uncertainty, managers are assumed to follow a course of action where the control variables take on the expected value of the various optimal strategies. If the uncertainty is completely resolved, the control can be tailored to the true state of the world. The value of information can be calculated as the difference in expected value of overall benefits when parameter values are perfectly known at the outset versus when coefficients become known. Reduced variability in the parameters also has value and can be estimated.

The PFOA methodology recognizes that uncertainty can result not only from lack of scientific information, but also from lack of knowledge of individual and social values. By timely presentation of the best available scientific information to all stakeholders, PFOA reduces the misinformation and misinterpretation associated with conflict-ridden issues. It provides opportunity for discussion, leading to understanding of which values stakeholders share in common and those on which they differ. It also allows scientists to learn of concerns about the limits of scientific knowledge. IGERT students will learn about the full range of approaches to address uncertainty, both quantitative and qualitative, including worst-case analysis tools, optimization models, and multi-stakeholder deliberation.

A key strength of our IGERT faculty is its breadth of experience with regulatory agencies (e.g., EPA, USDA, FWS) and risk-assessment frameworks. Research groups will link risk assessors (Adgate, Andow, Hueston, Kapuscinski, Kuzma, Ragsdale, Venette), external partners (Table 1), economists (Haight, Homans, Hurley) and biologists with specialized expertise (Galatowitsch, Heimpel, Newman, Sadowsky, Shaw, Tilman). In their retrospective analyses of risk assessments, students will determine whether probabilistic methods or uncertainty analyses were employed and in what form(s). Uncertainty in the model and data will be quantified, when possible, for inclusion in models. Sociologists and governance specialists (Nelson, Schurman) will help structure social science questions about uncertainty in societal discourse, governance, and decision-making. Student teams will characterize the results of the risk assessments and evaluate how the public perceived the results. Finally, students will assess the role of uncertainty analysis in affecting the choice of risk management options.

These retrospective analyses will help IGERT students define appropriate ERA approaches (quantitative or qualitative) for their own studies. Where lack of information has prevented the past use of quantitative methods, students will work with faculty to design experiments to fill information gaps. For example, Venette prepared a qualitative assessment of the risks posed by a moth species, known only to occur in Mexico and South America. Even in the face of extremely limited information about this species, this analysis revealed that this pest threatens US agriculture and ecosystems and warrants quarantine. Quantitative models were needed to evaluate the efficacy of potential quarantine treatments.

In this Section

A) Evaluating effectiveness of regulation using retrospective analysis

B) Improving theory and models for ecological risk assessment

C) Addressing uncertainty in risk assessments

D) Managing introduced species and genotypes and post-removal strategies