CC BY
13
EU ADVANCES IN IDENTIFYING SOURCES OF UNCERTAINTY IN RISK
ANALYSES
K. Lauridsen and I. Kozine •
Systems Analysis Department, Risoe National Laboratory, Roskilde, Denmark A. Amendola and M. Fiori EC-Joint Research Centre, Ispra, Italy
INTRODUCTION
Quantitative Risk Assessment (QRA) aims at the modelling of stochastic uncertainties associated with the occurrence and circumstances of a major accident. But the process itself of carrying out a QRA implies several uncertainties. For the implementation of the risk assessment procedure a variety of techniques and models must be used, and uncertainties are introduced due to imperfect knowledge and expert judgement. Because QRA is used as input in many decisions related to the control of major accident hazards and the need for accuracy in the results increases, the adequate management of these uncertainties gains increased importance.
This paper presents the scope and some main results of a European project on the ASSessment of Uncertainties in Risk ANalysis of Chemical Establishments (ASSURANCE). The project aims at identifying the uncertainties associated with risk analysis of major industrial hazards and assessing the way these uncertainties can affect the final outcome of risk studies and of the relevant decisions based on that outcome. In order to achieve this goal, a number of benchmark exercises/case studies have been performed by the partners and the results were analysed in a modular and structured way. A reference plant served as the basis for a realistic description of these case studies. For this particular project an ammonia storage plant was selected, consisting of cryogenic and pressurised storage tanks, together with import loading/unloading facilities and the relevant piping. This installation was analysed independently by each partner, using common input data and boundary conditions, but different methods, tools and assumptions. The results were then compared and discrepancies identified, discussed and explained.
In order to permit the step-wise comparison of the results and to assess the contribution of each factor and each phase of Risk Assessment to the overall discrepancy, the analysis was divided in the various phases (Hazard identification, frequency estimation, consequence assessment, and overall risk assessment), and the results of each phase were compared. Moreover, detailed exercises addressing particular issues within each phase (e.g. source-term definition, dispersion modelling, vulnerability modelling, etc.) were performed, in order to give more insights on the factors affecting the overall discrepancies in the results.
Concerning the quantification of risk, a structured procedure was followed in reporting and comparison of results. This procedure required not only the assessment of the risk profile, i.e. estimate of the level of risk at each point in the area around the plant, expressed in the form of isorisk curves for individual risk and F-N curves for societal risk, but also the assessment of intermediate results. These included:
• Assessment of the frequencies associated to accident scenarios,
• calculation of release/evaporation rates and conditions,
• modelling of dispersion (including detailed results of selected scenarios), and
• dose-response calculations.
FINDINGS CONCERNING HAZARD IDENTIFICATION
Comparison of the approaches to hazard identification showed that the partners had used many different methods. As a matter of fact, no two partners had used exactly the same method, although some of the methods, of course, are of similar nature. The methods used were:
• HAZard and OPerability analysis (HAZOP)
• Master Logic Diagram (MLD)
• Structured What-IF Technique (SWIFT)
• Hazard Identification by Area Audit (HIAA)
• Function analysis and Hazard and Consequences Analysis
• HAZardous SCenario ANalysis (HAZSCAN)
• Use of (national) standard checklists based on accumulated experience from past accidents and past (detailed) studies.
These methods can be grouped into three general types of approach:
• Methods based on a top-down analysis, mainly represented by the Master Logic Diagram, which has a form similar to Fault Trees, starting from a top event and going down to combinations of basic events that can initiate an accident
• Methods based on a bottom-up analysis, like HAZOP, SWIFT and HAZSCAN, which investigate whether deviations of the process variables and failures of individual devices can initiate an accident
• Methods based on the systematic use of standard checklists, after division of the plant into areas. Here, the accumulated experience from past accidents and studies is combined with systematic rules to identify the areas that deserve a more detailed analysis.
Even though the partners had used different methods for the hazard identification they had all identified the accident scenarios that must be considered the most severe. But, due partly to the use of different methods, each partner had some scenarios that other partners did not have in their list of selected scenarios. Therefore, in order to have a common basis for comparison of the methods used in the following quantified risk analysis phase, 11 reference scenarios were agreed for further analysis by everybody together with possible additional scenarios identified by the individual participants. Examples of the reference scenarios are:
• Full section rupture of an 8" import pipeline
• Full section rupture of a specific 4" pipeline
• Full section rupture or disconnection of the loading/unloading arm to a ship
• Catastrophic rupture of the cryogenic tank
• Catastrophic rupture of a pressurised tank
FINDINGS CONCERNING THE QUANTIFICATION OF RISK
One of the ways the participants were requested to present the results of their quantitative risk analyses was by means of iso-risk curves for the individual risk, i.e. curves on a map where the risk is the same for all points on the curve. The individual risk is the probability that a person staying unprotected in the same location around the clock during one year will die as a consequence of an accident in the facility considered. The two curves in the example shown in Figure 1 are the maximum and minimum distances found by the participants for an annual fatality risk of 10-5. As can be seen, there are rather large differences; the diameter of the outer curve is roughly 2 km.
Although care was taken to specify reference scenarios well, major differences were seen in the partners' risk results for these scenarios. Some causes that could easily be identified were:
• some remaining misunderstandings concerning plant data and specification of the reference scenarios
• differences in data used for failure probabilities
• different assumptions used concerning release duration.
The uncertainty assessment for the quantitative analysis phase was carried out separately for the frequency assessments and consequence modelling of hazardous scenarios. This separation allowed the identification of root causes of the deviation in risk assessments and their range among the research teams.
Frequency assessment
Uncertainty analysis in frequency assessment was based on the understanding that there were three different types of uncertainty: modelling, completeness and parameter uncertainty. Modelling uncertainty results from the use of different analysis models (fault trees, event trees and simplified generic-based models). Completeness uncertainty is due to differences in the number and nomenclature of basic/intermediate and initiating events included in the modelling. And parameter uncertainty is due to different numerical inputs (basic/initiating event frequencies, in particular) used to assess the net frequency of a hazard.
Not all types of uncertainty can be analysed quantitatively. Thus, modelling uncertainty in principle allows quantification only if the input data are the same. Having the same inputs for modelling would reveal the variability in outputs. If inputs cannot be made the same, it is impossible to distinguish whether the variability in final assessments is caused by different inputs or the models themselves. After having analysed all the approaches for frequency assessments it became obvious that the inputs could not be made the same because the sets of basic and initiating events are very different and overlap only to some extent. It was concluded that for risk analysis studies it is hardly possible to split up completeness and modelling uncertainty analysis and to perform their quantitative categorisation. Yet, these two kinds of uncertainty can be analysed in a descriptive way through the review of different approaches and basic and intermediate events included in the analyses.
Figure 2 illustrates the deviation in frequencies found by six research teams for one scenario. As can be seen, there are deviations of more than one order of magnitude between some teams in this case. An investigation into the root causes of these deviations revealed differences in data sources, in the use of data from the same source and in the interpretation of plant data.
1,E- 04
TO 0
>s 1,E- 05 0
^ 1,E- 06 o c 0 3
1,E- 07
1,E- 08
m
1 2 3 4 5 6
Research teams
Figure 2 Frequencies of the rupture of a pressurised ammonia tank
Consequence assessment
The consequence assessment included four phases: (i) outflow calculations; (ii) pool formation and evaporation (whenever applicable); (iii) dispersion; and (iv) dose/response modelling. Since the assessment of the implications of differences in dose/response (vulnerability) models is straightforward, the project focused on the first three phases, and the final results reported were in most cases concentration- and dose endpoints. In particular, three concentration endpoints were used, corresponding to concentration levels of 6200, 1000 and 500 ppm, and three dose endpoints, corresponding to dose levels equivalent to 30 minutes exposure to a constant concentration of ammonia equal to the above values.
In addition to these endpoints, a number of intermediate results were calculated and reported (e.g. outflow rates and conditions, pool dimension and characteristics, percentage of droplets, evaporation rates and behaviour of the cloud).
The comparison of the calculated endpoints for the reference scenarios revealed again noteworthy discrepancies. In general, the sources of uncertainty in the consequence assessment can be divided in the following categories:
• Scenario completeness and correctness
• Uncertainty in definition of scenarios/ambiguity
• Modelling uncertainty, including the description of physical phenomena and the detailed model characteristics, constants and parameters
• Input assumptions/boundary conditions/interface between models
• Simplifications made throughout the analysis
• Overall level of "conservatism" of the analyst.
CONCLUSIONS AND RECOMMENDATIONS
The findings mentioned above and other observations during the project led to the identification of the following types of uncertainty and variability that influence the results of risk analyses and for which an effort is needed in order to minimise their influence:
Misunderstandings or lack of knowledge about plant layout and operation Completeness of hazard identification
Modelling uncertainty (failure modelling and consequence modelling) Data uncertainty
Variability in, for instance, weather conditions or plant operational state
• Variability due to the fact that probabilities are not fixed numbers but distributions.
The first one, lack of knowledge/misunderstandings about plant layout and operation, may have had a particularly great influence in this project because the interaction between the individual risk analyst and the plant staff was not typical for a normal risk analysis. Due to the geographical diversity of the consortium it was not possible to visit the plant more than once, and in order to give all partners equal conditions all communication with the plant staff went through the co-ordinators of the exercise.
The differences experienced between the partners' results suggest the need for:
• Recommendations concerning the use of standardised approaches to risk analysis in Europe
• Recommendations for common data sources for failure rates of components.
In any case it will be necessary to ascertain that risk analyses performed in different EU countries by different analysts are comparable and lead to similar results.
After the project has finished results from it will be made available at the JRC's MAHB server (http://mahbsrv.jrc.it/) in order to secure their availability to the international community of risk analysts.
ACKNOWLEDGEMENTS
ASSURANCE was a benchmark exercise and the outcomes of the project are the joint effort of many project participants in addition to the authors of the present paper. O. Aneziris, D.A. Carter, S. Coulter, P. Crossthwaite, M. Fiori, S. Gadd, I.L. Hirst, D. Hourtolou, C.M.A. Jansen, Y. Malmen, M. Molag, A. Montesi, Z. Nivolianitou, I.A. Papazoglou, G. Spadoni and J. Tiihonen are all gratefully acknowledged for their work and for providing the background data. Special thanks are expressed to the anonymous company and its safety engineers. Without the availability of the plant and their co-operation and help to resolve the numerous questions of the partners, this project would never have been realised. Last but not least, the funding from the European Commission under the ENVIRONMENT programme is kindly acknowledged.
