CC BY
22
CO <1>
CD
>
> RISK ASSESSMENT FRAMEWORK:
APPLICATION OF BAYESIAN BELIEF NETWORKS IN AN AMMUNITION
| DELABORATION PROJECT
o
" Slobodan B. Malbasic3, Stefan V. Duricb
q a Ministry of Defence of the Republic of Serbia, Sector for Material
Resources, Belgrade, Republic of Serbia, q e-mail: Slobodan.malbasic@mod.gov.rs,
UJ ORCID iD: http://orcid.org/0000-0001-5053-4033
b University of Kragujevac, Faculty of Engineering, Kragujevac, Republic of Serbia, e-mail: sdjuric@kg.ac.rs,
ORCID iD: http://orcid.org/0000-0003-0660-7551
DOI: 10.5937/vojtehg67-20900; https://doi.org/10.5937/vojtehg67-20900
^ FILED: Project management
< ARTICLE TYPE: Review paper
0 ARTICAL LANGUAGE: English
>o Summary:
1 Models that represent real problems have been relying so far on historical data to draw upon conclusions. One negative aspect of these models was that they could not predict future states based on real data instantly
o collected or new sources of risk that suddenly appeared. To overcome this
problem, this work presents the process of building a realistic predictive model using Bayesian Belief Networks (BBNs) and the AgenaRisk software. BBNs are a direct representation of real problems where their graphical structure represents real causal connections and not just a flow of information. Software tools providing algorithms for dealing with conditional probabilities have been developed. The Bayesian Theorem, a theoretical background for conditional probability, was also explained in the paper. Another benefit of using BBNs is that the reasoning process can operate by propagating information in any direction (top-down or bottom-up) which makes it a powerful tool in risk assessment and a decision-making process. The paper also provides the core principles and the power of BBNs and their application in the project planning phase for ammunition delaboration (resolving problems of surplus and obsolete ammunition in stockpiles), where risk assessment is one of the required processes which helps in making a final decision for project approval or
not. The sensitivity and SWOT analyses are also performed as valuable and helpful tools for validation and making conclusions. 55
Key words: conditional probability, Bayesian Belief Networks, risk £ assessment, sensitivity analysis, SWOT analysis.
Introduction
o
(D t? £i
The project risk management process is seen as a process that accompanies the project through its life cycle. The Project Management Book of Knowledge recognizes risk management as one of knowledge | areas (together with its inside processes) that need to be addressed during project planning and can have a significant impact on the project success.
A number of variations for the risk management process have been proposed, (Marcelino-Sádaba et al, 2014), (Petrovic et al, 2010), (Andrejic et al, 2011), (Malbasic et al, 2016). According to (Fang & Marle, 2012), there is a general agreement on what is included in the process but differences exist in "the level of details and assignment of activities to steps and phases". Based on the previous resources, the main processes for risk management are: risk planning, risk assessment, risk mitigation, risk monitoring, and documentation.
Stakeholders are the ones among many who constantly insist on risk management/assessment processes because they want to be protected against different consequences (financial or legal) if some unwanted risk occurs (internal and external source of risk), or at least to be warned against potential problems. Their ultimate goal is to have a project fj successfully finished. Besides the aforementioned, project managers have to consider a number of other parameters such as safety, security, social and environmental issues, which are interrelated and hence increase the complexity of problems. This complexity leads to the existence of a network of interdependent risks (Fang & Marle, 2012).
The risk management/assessment process requires tools for its implementation, and many tolls have been developed so far. Adoption of certain tools depends on several reasons (investment for the implementation is a significant one) but one of the most important is what benefits a tool can provide to a system, (Raz & Michael, 2001). The same authors argue that many of the developed tools are based on the concept of probability and impact, assessed through qualitative and quantitative approaches.
In most cases, these approaches focus their calculations on several independent risks, emphasizing those of a high value, and then take ^
(D
m
c <5
W (D
Ht
m
mitigation measures. Often, they are able to take into account complex interrelations between them, influences, causes and consequences, but in the end, they still concentrate on a single risk and cannot calculate £ influences that exist between them. We can also argue which ° approaches are better and why.
In order to overcome and resolve the mentioned issues, this paper presents a process of risk assessment using Bayesian belief networks oc and their application in the project planning phase for ammunition delaboration as part of the Trust Fund project, where it is necessary to make a decision for the project approval or against it. In the ammunition
LLJ CC
o delaboration project, insufficient attention has been paid to risk
< assessment in the planning phase. This process has been mainly done ° by forming a list of the most frequent risks that have appeared in similar 5 past or current projects. It is obvious that this approach needs to be m changed and adapted to new circumstances using BBNs.
Bayesian belief networks also use qualitative and quantitative
< approaches. In this case, the qualitative approach is the process of graphical representation of the relations among variables (structural learning) while the quantitative approach relies on conditional probability among variables (parameter learning), (Lee at al, 2009).
C/D <
-J
CD >o
X LU H O
Introduction to Bayesian Belief Networks (BBNs) -theoretical and graphical background
It can be said that any event (A) is a statement about conditional probability, because we have made this statement with background knowledge or context (K), so it would be accurate to write conditional o probability as P(A|K), (Fenton & Neil, 2011).
> From the scientific point of view, we explain the previous by
introducing a hypothesis - H, beliefs, evidence (E) and conditional probabilities P(H|E) and this process is called probabilistic reasoning. So, for calculating P(H|E), we use Bayes Theorem,
P(H/E)__ pM!i.m. (1)
where:
P(H) - the prior probability of the hypothesis H, P(E) - the prior probability of the evidence E,
P(H/E) is the probability of H, conditional on a new piece of evidence E or a posterior belief about H, P(E/H) is the probability of E given the H.
Bayes Theorem tells us how to calculate conditional probabilities. In our case, it tells us how to calculate the conditional probabilities of the H given the new evidence (E). It also tells us that this probability depends on three things: the prior probabilities of H and E, and the conditional probabilities of E given the H.
Now, from this very simple but basic explanation, we are transferring from conditional probability into the visualization of the above mentioned situation and a BBN. As we said earlier, it makes no sense to assign a direct probability (the node E or the child node) without considering the events it is conditional on (the node H or the parent nodes).
For the purpose of further explanation, let us assume that both variables are discrete and have just two possible states: true and false, with prior probabilities as shown in the Figure.
CD
CO £± С
О Ф
с?
о
.Q
.Œ
Ф T3
Table 1 - Basic two-node Bayesian Network Таблица 1 - Базовая концепция байесовской сети с двумя элементами Табела 1 - Основни концепт Ба^есове мреже са два елемента
ф m с Œ
<Л
Ф &
m
го о
"œ о. го
о
With initial probabilities for the hypothesis, we do not know anything about the evidence, meaning that the probability of E is 50%.
It is important to note that the Bayesian calculation should never be done manually. Different tools are created to help the modeling process and to run a simulation with the Bayesian algorithm in the background -AgenRisk software is one of them.
In light of new evidence, we enter the Conditional Probability table (CPT) for the evidence E. This means that, for each state of H (2 states), we define probability for the states of E (2 states) and get a matrix for the CPT as explained in Table 2.
(Л (Л Ф (Л (Л
ro
(Л
en "rô ф
w
•o
>1Л
ro
.Q ГО
со ф
CD
"с5 >
О
см
ОС LLJ
ОС ZD О
О _|
< с;
X О ш
I—
>-
ОС <
(Л <
-J
О >о
X ш I—
о
о >
Table 2 - Propagation through the Bayesian network Таблица 2 - Процесс расчета в байесовской сети Табела 2 - Процес израчунаваъа у Баjесовоj мрежи
HVPOTESIS False True
False 0,9 0,2
True 0,1 0,8
Initial probability for evidence has slightly changed.
Now we ran a simulation entering that the evidence has a true value and observe how our hypothesis changes. We have 94.9 % that the initial hypothesis is true.
Vice versa, in the situation when the evidence has a false value, the initial hypothesis will be 65.8% false.
The conclusion form Table 2 is as follows: at first glance, do not rely on the initial probability (or make a decision) until you see new evidence. New evidence might cause some changes in the initial states and help to make a better decision, based on real data.
With more variables, states and dependencies between variables, the risk asssessment problem becomes more complex, hence a Bayesian Belief Nework (BBN). Figure 1 shows a complex BBN with the explanation of the nodes utilities. The BBN structure consists of the qualitative structure (graph structure) and the quantitative components (probability tables). It is a Directed Acyclic Graph (DAG) with an associated set of Conditional Probability Tables (CPT), as shown in Figure 1 (Fan & Yu, 2004).
A node represents event occurrence (a variable of interest in the problem), the arrows (directed edges) between the nodes mean the relationships of events i.e. a dependency structure within the problem, while other nodes serve as:
- Utility node - representing the quantity of interest, generating a numerical value, and helping to rank the alternatives in order to obtain the best option,
- Decision node - representing the alternatives for the decision maker,
- Chance node - probabilistic quantities.
TRAINING
DECISION NODE
CHANCE NODE
UTILITY NODE
CD
CO £± cp
О Ф
с? £i
О .Q
m ф
T3
Manager capability High Low
Developer capability High Low High Low
Pr ("product quality="High") 0.9 0.85 0.35 0.15
Pr ("product quality="Low") 0.1 0.15 0.65 0.85
Figure 1 - Example of a Bayesian Decision Network - BDN Рис. 1 - Пример динамической байесовской сети Слика 1 - Пример динамичке Ба^есове мреже
ф m с Œ
<Л Ф
na
m
го о
"ü cp го
i¿
о
According to (Marcot & Penman, 2019), BBN models with inclusion of decision and utility nodes create Bayesian decision networks (BDNs). They can be also highly useful in the risk assessment process. The nodes that have no parents are called "root nodes"and the nodes without children are called "leaf nodes".
Application of BBNs
The use of BNs is spreading to almost all areas: safety and reliability modeling, operational risk in finance, information retrieval, environment, medicine or, according to (Fenton & Neil, 2013) and (Weber et al, 2012), to modeling operational risk, system reliability modeling,
(Л (Л Ф (Л (Л
ro
(Л
en "rö ф
w
'9
>1Л
ro
CO <D
CD
"o >
CD
O CM
of
UJ
a.
Z) O O
_l <
o
X
o
LU
H ^
a. <
H
<
CD >o
X LU H O
O >
dependability, risk analysis and maintenance as well as to architecture design developing models to capture change impact analysis (Tang et al, 2007), data mining, determining and explicitly displaying the relationship among variables, representing expert knowledge and combining expert knowledge and empirical data, and identifying key uncertainties (Marcot & Penman, 2019).
In addition to the previously mentioned, Bayesian Belief networks also have a variety of applications in the following fields:
- In a risk assessment approach, to improve the resilience of a seaport system (giving a flexible tool to the safety analysts to increase resilience strategy), (John et al, 2016),
- In a project management assessment modeling framework that calculates costs, benefits and returns on investments (use hybrid and dynamic BBNs, case study for agricultural development projects), (Yet et al, 2016),
- In modeling large and complex infrastructure systems (addressing one of the major obstacles i.e. the exponentially increasing amount of information that needs to be stored as the number of components in the system increases), (Tien & Der Kiureghian, 2016),
- In medical decision support systems (overcoming problems of complex, unstructured and incomplete patient questionnaires and interviews that inevitably contain examples of repetitive, redundant and contradictory responses and to ensure the BN model can be used for the interventional analysis), (Constantinou et al, 2016),
- Modeling research on ecosystem service (ESS), (Landuyt et al, 2013) or,
- In land forces, to aid reasoning and decision making under uncertainty (Starr & Shi, 2004).
Having in mind the previous explanations, the benefits of using BBNs are:
- Explicitly modelling causal factors.
- Reasoning from the effect to the cause and vice versa.
- Overturning previous beliefs in the light of new evidence.
- Making predictions with incomplete data.
- Combining diverse types of evidence including both subjective
beliefs and objective data.
- Arriving at decisions based on visible auditable reasoning.
As it is stated in (Marcot & Penman, 2019), BBNs are probabilistic models (filled with real data) which help us to "investigate the consequences of conditions or deducing conditions resulting in an outcome".
o
AgenaRisk Software
The AgenaRisk software tool (Fenton & Neil, 2013) has been used by some of the world's leading organizations to model risk and improve ^ decision making across a range of industry sectors and to implement solutions to a range of critical business and safety problems.
AgenaRisk is a powerful tool which overcomes problems that existed with the previous versions of BBN tools, making BBN building much easier (each node type is associated with an extensive set of probability distributions which can be chosen from a predefined list), making |j calculations or a decision process more accurate and giving a variety of solutions for a wide range of end users.
AgenaRisk Lite version 7.0 that has been used for modeling in this paper consists of: risk map, risk table, risk explorer views and risk graphs and has some powerful and advanced features for creating the Node Probability Table (NPT): rank node, simulation node, partitioned expression, and continuous graphs. For the created model, the software provides a various set of tools for analysis and optimization such as: sensitivity analysis, multivariate analysis, compound sum analysis, and it creates a node probability table based on spreadsheet data.
As a free download version, it has some constraints regarding the saving mode for ranked, simulation nodes and multiple Bayesian network objects. Also, there is no maintenance and upgrade support. Recently, this AgenaRisk Lite version has been withdrawn from the site and has been replaced with the 14 day free trial of a new AgenaRisk 10 version. Anyway, all developed models can run under this new version. The reason for this is a custodian effort to further promote the ppl commercial/academic subscription license version only. ®
Trust Funds project policy
The policy of the Trust Fund projects is to assist countries (financially and managerially) with the safe destruction of stockpiles of surplus and obsolete landmines, weapons and ammunition. There are various reasons for this approach. The destruction of surplus stockpiles of arms and ammunition reduces the threat to individual partner countries, the wider region and ensures that such materials are not ^ subject to any proliferations. When it is possible, the project can use country facilities and resources for project completion and can hire local population.
(D
w
(D
oa
m
co
CO <D
CD
"o >
CD
O CM
of
UJ
a.
Z) O
o <
o
X
o
UJ
H ^
a. <
H
<
CD >o
X UJ
H O
O >
Modeling of Trust Funds Project
After a request is initiated from individual partner countries, based on the voluntary basis and an extensive negotiation process, the "Lead nation" is chosen and it is responsible for gathering political and financial support for the project as well as for selecting the executing agent for the project.
Different agencies have been often appointed to act as the executing agent for demilitarization projects by the lead nation, contributing to the project through: development of a feasibility study, technical advice, management activities, overseeing the project development, and ensuring a competitive bidding process. It is very important for legal agreements to be in place between the parties involved in the process.
For the final approval of the project proposal from a higher authority, several elements or preconditions need to be in place: a feasibility study developed by the executing agency, the donation countries, the threshold level of donations, and a clear financial picture (donated money -enough to start the project, costs for running the project - donated money decreased for management and administrative costs).
The whole process of negotiations, gathering political and financial support, and the development of the feasibility study takes some time and is full of uncertainty since it depends a lot on the preconditions and is subjected to everyday risk, known and unknown.
The existing process and the structure for Trust Funds projects involves a number of participants from different jurisdictions, for example, for the feasibility study or for collecting donations and normative legal regulations. The project manager has the main responsibility for the feasibility study. The existing structure within the Trust Fund policy for the project approval does not provide the visibility of the whole process, the progress of one component is not reflected in the other dependent component, coordination is necessary and sometimes difficult, and the project manager sublimates all the information even though he has no jurisdiction over the whole process. All the mentioned things and their interconnections pose a great risk to project execution.
In order to overcome these problems and risks, it is necessary to model the process and causal connections and to reflect the daily changes of individual components on other components (positive or negative) as well as on the final decision.
o
Risk assessment framework for the Trust Fund project
The modeling process with the help of BBNs overcomes most of the problems and risks listed before. For better understanding the whole ^ process, it is necessary to turn them into a causal model with a Bayesian network, Diagram 1. Most of the previously mentioned is presented in the risk framework model, except legal agreement.
The whole process of building the risk framework and running the simulation is explained in the continuation of this text. The Risk identification process has been performed using the interactions and jjj the elements that characterize the Trust Fund policy. Determining the risk interactions is actually the process of building a risk map and the map was modeled using the network structure.
The question that needs to be asked is: "How to build a risk map"? According to (Fenton & Neil, 2013), the following steps are useful in building a risk map:
- Consider the set of risk events from a given perspective.
- Identify the risk triggers for the identified risk events.
- Identify the consequences and mitigations for the identified risk
events.
- Define probabilities for the risk events.
- Generate risk predictions for the issues such as simulation,
backward reasoning, and a what-if scenario.
(D
w
(D
oa
m
By chaining together different risks, we can model multiple risks, risks from different perspectives, and common causes, consequences | and mitigate all within the same model.
Another question related to building risk maps is risk perspective. It is obvious that there are different views or perspectives of risk: stakeholder perspective (owners, shareholders, employees, suppliers), customer/user or manager perspective, and local community perspectives. Generally, different experts consider risk at very different levels of granularity and perspective (Wright, 2011). While risk for someone (stakeholder, risk responsibility authority, etc.) can be an opportunity, for others it can be a cause, a consequence or a mitigation and this is something that can be a limitation in constructing a risk map since it has an impact on how a risk map will be constructed. What is good in this approach is that once risk events are identified from a particular perspective, there will be very little or no ambiguity at all about the causal structure.
co
со ф
CD
"с5 >
0
01
Q¿ LLJ
СП ZD О
о <
с; 2: х о ш
н >-
СП <
со <
-J
о ■о
X ш н о
о >
MODEL CONSTRUCTION IS ALSO PROCESS OF BUILDING RISK MAP
RISK ASSESSMENT AND RISK ANALYSIS PROCESS
* 1 .................... r
ENTERING EVIDENCY
i
RUN SIMULATION
SENSITIVITY ANALYSIS
RISK PROFILE AND RESIDUAL RISK
Diagram 1 - Framework for the risk assessment process using a BBN and the AgenaRisk
software
Диаграмма 1 - Рамки для анализа рисков при использовании программного обеспечения AgenaRisk, с внедренным модулем для условной вероятности Диаграм 1 - Оквир за процену ризика коришПешем софтвера AgenaRisk са угра^еним модулом за условну вероватноЬу
Risk assessment and risk analysis: A tool for modeling and entering evidence is the AgenaRisk software that uses the BBN tool for modeling and conditional probability. Performing the evaluation process is as follows: running a simulation is a process of measuring the
interactions between risks and comparing the results with the predefined boundaries for an identified risk. The sensitivity analysis is also S performed to enhance the reliability of the network analysis phase and to define which and how the identified risks influence the main variable or pp the decision variable - project approval. When new evidence appears from the environment, from expert knowledge or as an input from the sensitivity analysis, the optimization or re-evaluation process of the model starts.
This optimization/or response phase is performed until all balance ° between elements is found (i.e. effectiveness of the mitigation measures are in place) and all identified risks are within the defined boundaries. The simulation is helpful for estimating the effects of the mitigation measures.
The end of the process is a risk profile. This profile consists of the list of identified and measured risks, meaning that some risks still exist in the system (i.e. residual risk) but they are under control and constantly monitored.
Construction of the framwork for risk assessment using the AgenaRisk software &
Further steps in this paper comprise the following: model developing
using the AgenaRisk software, specifying the variables, entering the &
w
(D
m
probabilities, a case study (back and forth propagation), the validation of the model (sensitivity analysis) and the conclusions.
Based on the policy of Trust Funds (activities and conditional | dependence) and the basic principles for constructing a model within the AgenaRisk software together with user perspectives, a model was created as shown in Figure 2. The established model provides a visual image of the process where each node represents the potential risks identified in the process.
The model represents a chain of events with uncertainties that will be assigned later and gives clear visibility of the risk map. The ability to decompose a risk problem into chains of interrelated events and variables should make the risk analysis more meaningful, practical and coherent.
The model also represents the integration of different levels of decision makers involved in this process: international organization, different countries, the government level, and the factory/customer level.
со ф
CD
15 >
О
см
ОС LLJ
ОС
о
о _|
< с;
X О ш
н
>-
ОС <
(Л <
-J
О >о
X ш н
о
о >
Рис. 2 -
Figure 2 - Basic Trust Fund model in the AgenaRisk environment Базовая модель проекта, представленная в программном обеспечении Слика 2 - Основни модел прожекта приказан у софтверу
Based on the explanation presented earlier for the creation of a Bayesian Belief Decision Network, the utility nodes in the proposed model are: donation level, left for project, fixed costs and buy equipment (Figure 2). The chance nodes are: lead nation donation, country donation, donations and flexibility study, while the node "project approval" is the decision node.
Data acquisition problem and the process of entering the probabilities
Assigning the probability tables in a risk map is not always an easy task. The process requires expert knowledge or relevant statistical data, well suited for decision making. Expert knowledge is especially needed in a case where the existing data cannot be extended except for the incorporation of expert knowledge.
Depending on a problem in question, one example of acquiring data is given in (John et al, 2016) where they explain that audit reports from maintenance departments framed in the probabilistic way can be a valuable source.
When there are not enough data, purely subjective values can be supplied and it is essential to make the most of what is given.
(Constantinou et al, 2016) focus their work on complex data problems that come from poorly structured questionnaires and interviews (with inevitable examples of repetitive, redundant and contradictory
responses, different classes of data) and on how to transform them to be useful for inclusion in BNs.
AgenaRisk provides a wealth of tools to make the task of entering probability as easy as possible. It can be done manually through filling the Node/Conditional Probability Table (NPTs/CPTs), using expressions or through a process called "learn tables from spreadsheet".
Since this is not the first time to run such a project in this particular facility, but the first time to model it, the expert knowledge from people who once were involved in the first project was now available (through the interview process and data collection). This helped us to distinguish between important and less important elements in the modeling process.
Also, to overcome the problem of information shortage, the model was developed as a post-appraisal process of the project planning phase, when we had enough information or feedback from the reality to build it up.
After entering the NTP for the nodes, the initial probabilities are as in Figure 3.
CD
CO £± CP
О Ф
с? £i
О .Q
Ф T3
Ф
m
с ®
(Л Ф
oa
CO
ro о
CP
m J^ о
Figure 3 - Risk Map model with the initial probabilities for the NTF Policy Рис. 3 - Карта рисков с инициальной вероятностью Слика 3 - Мапа ризика са инициалним вероватноЪама
For the use in the AgenaRisk software, the following variables are created (with their abbreviations):
- Ranked nodes are: Lead nation donation (LND), Country donation (CD), Donations (D), Feasibility Study (FS), and Project approval (PA). This is important because, to be of any use to define the Node Probability Table (NPT), the node type has to be Ranked.
(Л (Л Ф (Л (Л
m
<л СП
"TO Ф
со
'9
>1Л
m .a
CO <1>
CD
Simulation node: Donation level, Left for Project, Fixed costs, and Buy equipment, were the last two are constant (pre-defined costs).
° The NPT for the parent nodes can be generated manually (when
filling in the table, three states can be chosen: low, medium, high; the software provides up to 5 states). For quantifying the strength of the oc relationships among variables and for forming the CPT (for the child node), a software option is used for entering the weight and partitioned expressions for the parent nodes..
UJ
a.
Z) O
0
CA The use of the weight expression in the model is as follows:
z
5 - For the variable Donations where the variable LN has weighted m influence by 2:1 against the CD, and
£ - For the variable Project Approval where the variable FS has < weighted influence by 2:1 against the DL.
The use of the Partitioned expression for creating the NPT for the variable Donation level is useful to create the NPT using different w expressions for each combination of the parent states. In our model, low, medium and high statuses for the variable D were created using the 2 TNormal expression type with the mean, the variable and the bounds that actually represent low, medium and high levels of donations.
1 LLI I—
o
Utilization of the utility nodes: Using the Arithmetic Expressions, the variable LFP was created. The arithmetic value for this is: donation ó level minus two constants. The constant is also a simulation node. The constant can be used in a combination with the Arithmetic Expressions for calculating another simulation node values.
The constant variable "Bye Equipment" refers to Capability improvements costs (in logistic support, new process machines, etc.). The constant variable "Fixed Costs" refers to Management and contingency costs.
What is left, or the variable "Left for Project", refers to Operational costs for running projects, in this case for the delaboration activity.
Running the model (case study)
One of the most powerful features of AgenaRisk is the ability to compare different scenarios side by side. Models are used to generate predictions about the variables LFP and PA in the case of different donations from the variables CD and LN (these scenarios simulate financial risk). The Feasibility study (FS) variable has a high value in both scenarios:
- Scenario 1: CD and LN have donated a small amount of money. This means that donators are not interested enough to support the project.
- Scenario 2: CD and LN have donated a big amount of money. This means that the project should have enough support for start and further running.
CD
CO £± CP
О Ф
с?
о
.Q
.Œ
Ф T3
The question is how these states influence the variables Project approval, Donation level and Left for project (Table 3):
- Scenario 1: It is obvious that a smaller donation has a smaller
influence on decision makers whether to accept a project, especially in the situation when project fixed costs reduce the donation sum necessary for project approval (Project approval (only 30% of high probability thanks to the variable FS with weighted influence by 2.5:1.5 against the DL), Donation level (55.9), and Left for project (5.9).
- Scenario 2: The situation is different in the case of a bigger
donation: Project approval (86% of high probability thanks to the variable FS with weighted influence by 2.5:1.5 against the DL), Donation level (135.5), and Left for project (84.5).
Table 3 - Data comparison after running both scenarios Таблица 3 - Сопоставление данных после изучения случая Табела 3 - Поре^еше података након студне случаjа
ф m с Œ
<Л
Ф &
m
го о
"œ с го
о
Scenario 1 Small Scenario 2 Big
Donation Level 55.961 135.54
Left for project 5.931 84.528
Donations 66.8% 89%
Project approval 30% 86%
A visualization of these scenarios is presented in Figures 4 and 5. In Figure 6, the comparison of two scenarios is shown.
(Л (Л Ф (Л (Л
ro
(Л
en "rö ф
w
'9
>1Л
ro
-Q
ro
629
со <u
CD
15 >
О CM
DC LLJ
DC
О
О <
с; 2: x о ш
>-
DC <
Figure 4 - Risk Map model with the initial probabilities for scenario 1 Рис. 4 - Карта рисков с инициальной вероятностью по сценарию 1 Слика 4 - Мапа ризика са иници]алним вероватноЬама за сценарио 1
<л <
-j
О ■О
X ш I-
О
О >
Figure 5 - Risk Map model with the initial probabilities for scenario 2 Рис. 5 - Карта рисков с инициальной вероятностью по сценарию 2 Слика 5 - Мапа ризика са иници]алним вероватноЬама за сценарио 2
Figure 6 - Comparing two data scenarios Рис. 6 - Сравнение двух сценариев Слика 6 - Поре^еше два сценариа
СО
со £± cp
О ф
с? £i
О .Q
т ф тз
Back propagation capabilities
Another very useful tool in using the AgenaRisk software is the "back propagation" option, meaning that a value for the last variable in the chain (or the child node) can be defined and after running the model, the values for other variables in the chain (or for the parent nodes) are obtained.
Now, let the probability of project approval be high (scenario 3), so let us see which probabilities other variables need to have. The scenario gives the threshold level, i.e. if we want to have 100% of project approval, what would be the minimum level of donations and the donations level?
ф m
с ®
(Л Ф
oa
CO
ro о
cp m
J^
о
Scenario 3 gives the important information of the threshold level for donations in order to have the probability of project approval of 100%. In real situations, this percentage and the donation level can be lower (other variables have also their influences) and in that case the project can be run in phases, meaning that the next phase can start when there is enough money on the account. The threshold level is one of the boundaries in the risk assessment process (the obtained values are compared with this one), meaning that, in case there are lower values, the new threshold value optimization process should be run again.
(Л (Л Ф (Л (Л
m
<л СП
"TO Ф
со
'9
>1Л
m .a
со ф
CD
"с5 >
0
01
ОС LLJ
ОС ZD О
О _|
< с;
X О ш
I—
>-
ОС <
Figure 7 - Risk Map model for"back-propagation" scenario 3 Рис. 7 - Карта рисков по сценарию 3 (снизу вверх) Слика 7 - Мапа ризика за сценарио 3 (одздо на горе)
<л <
-J
О ■О
X ш I—
О
О >
Table 4 - Data after running scenario 3 Таблица 4 - Полученные данные по сценарию 3 Табела 4 - Добц'ени подаци за сценарио 3
Scenario 3 PA Big
Donation Level 126
Left for project 76
Donations 74%
Project approval 100%
As it is presented in Figure 7, the node FS has a significant influence (with 100%) on our targeted node (Project approval). One reason is that in this model, this node has twice as big impact as the other nodes, due to a pure practical reason. In order to run a project of interest (in our case it is a delaboration project), we need to have a facility/factory and there is no better way to represent that idea in the model than through the variable FS. In our case, the feasibility study shows that the project can be run in the designated facility with all preconditions fulfilled: line for demilitarization, quality system, environmental protection engaged, operational health at high level, skilled workforce, adequate machines, etc.
o
Sensitivity analysis
A very useful tool to check the built model is to perform the sensitivity analysis. Although there are different interpretations of the ^ meaning of this analysis ("What is important for model or system development?", "What is important for calculated measures of uncertainty?"), in this case, we tried to find out which nodes have the greatest impact on the "targeted" node PA (Project approval). This is important for a risk assessment process and also to see which node/nodes to pay attention to. |j
The sensitivity analysis is presented through a tornado graph were the bar length corresponds to the sensitivity which a paticular variable has regarding the targeted variable. The largest bar appears at the top of the graph (depicting the highest sensitivity).
The sensitivity analysis was done for the case scenario where all variables already have their prior probabilities. A further interpretation means that with "high probability" for the variable PA, the influence from the variable FS ranges from 0.019 (when the FS has low probability) up to 0.776 (when the PA has high probability). The same explantion is valid for other variables. By comparing the influences of other variables, we can conclude that almost every variable (in the state of its high value), ^ except constant variables, has a big infulence (ranging form 0.229 up to 0.799) on the FS to reach its high value (0.7) as well, meaning that, in the risk assessment proces, each of them requires special attention, Figure 8.
For the purpose of this work and a further explanation of the power of the sensitivity analysis, we have run the sensitivity analysis for case scenario 2 (variables have "high" values) in order to define which variable would have a big influence on the variable PA.
Figure 9 depicts quite well a real situation where the variable PA depends on the amount of money collected - Donation level and Left for project (amount of money needed to run it) - knowing that other variables (CD and LN) already have "high"values. A high probability of influence for the constant variable means that, as a representer of fixed cost, this variable should be lower meaning that the variable LFP would ® be higher enough for running the project (comparing to the treshold level).
Knowing the sensitivity of some nodes to the targeted node, we should try to estimate the states of these nodes with as much accuracy as possible.
(D
w
(D
oa
m
o
co
со <u
CD
"о
>
О CM
DC LLJ
DC ZD О
о <
с;
Z X О ш
i>-
DC <
(Л <
-J
О >о
X ш н
о
О >
Figure 8 - Sensistivity data when the variable FS has the highest value (other variables
have prior probabilities) Рис. 8 - Диаграмма анализа чувствительности для тех случаев, когда значение переменной FS является наибольшим (остальные переменные соотносятся с предыдущими вероятностями) Слика 8 - Диаграм анализе осетъивости у случа]у када вариабла FS има на^веЬу вредност (остале вариабле има]у претходне вероватноПе)
Figure 9 - Sensitivity anlaysis for case scenario 2 Рис. 9 - Анализ чувствительности по сценарию 2 Слика 9 - Анализа осетъивости за сценарио 2
Conclusions
One of the main ideas of this paper was not to rely only on mathematical and statistical elements during risk assessment, but also to incude modeling and reasoning procedures. This work gives its contribution in several ways:
- Provides a risk assessment framework applicable during the
project planning phase in the ammunition delaboration process that has not been used so far in similar projects.
- Presents how a simplified framework can provide valuable results
related to potential risk contributors.
- Introduces probabilities into the risk assessment process as an
advanced approach comparing to statistical data, through the use of BBNs.
The proposed model has brought some innovative elements. The Trust Fund Policy was studied from the risk management perspective. The model itself presents a clear and visual risk map explaining how risk emerges or how it is connected, thus providing a good base for a risk identification process, including an optimization process through which new data are incorporated in light of new evidence. Finally, the approved project with its residual risk is a real picture of the risk profile. This risk profile should be documented and carefully monitored.
Several scenarios were evaluated, some of them were not covered, but those which were covered had a significant influence on the decision making process and provided enough elements to make a right decision. The evaluated scenarios are also risk indicators a lot of attention should be paid to.
For performing all the aforementioned, we used most of the advantages that the AgenaRisk software provides (Gadeberg & Luedeling, nd):
- Models built using BBNs provide a real tool to update belief in
some uncertainty event when we observe new evidence about the event (in our case, about new donations, feasibility study beliefs, etc).
- One helpful feature of BBNs is the option to integrate expert
knowledge with data, which could prove to be a cost-effective way to assess development projects.
- BBNs are well suited to address uncertainties about benefits and
costs due to their ability to work without precise numbers and to incorporate expert knowledge.
CO <D
CD
"o >
CD
O CM
of
UJ
a.
Z) O
o <
o
X
o
UJ
H ^
a. <
H
<
CD >o
X UJ
H O
O >
- Decision makers can use the BN framework by entering values
related to the project budget, impact and risks into the model.
The sensitivity analysis is another powerful tool of the applied software which helps highlight the significancy of some variables. In this paper, the sensitivity analysis was run to define which variables have the greatest impact on the targeted variables, i.e which variable is of the biggest interest for/in the presented problem (to make a decision about the project approval). Decision makers can devise necessary schemes to optimize the process or some operations within the process based on the impact factors.
What can be a limitation in constructing a risk map, or, generally, in risk definition, is that different experts consider risk at very different levels of granularity and perspective. While risk can be an opportunity for someone (stakeholders, risk responsibility authorities, etc), for others, risk can be a cause, a consequence or a mitigation. This can have an impact on how a risk map is to be constructed.
Another problem might be a case when someone is uncomfortable with the reliance on expert assessments and the inclusion of cause-effect relationships that have not been confirmed in controlled experiments. For that reason, the presented model was made as a post-project appraisal when we have enough data to incorporate in it.
As stated in (Fenton & Neil, 2013), special attention needs to be paid in the process of decomposing a problem into classes of events and relationships (with enough granularity to be meaningful and accurate enough for the purpose required), states of variables and probabilities that reflect our best knowledge (we have supposed that probabilities for some variables are T-normal although real-life situations are different in most cases).
For the purpose of defining positive and negative aspects of the proposed model, we have also done a SWOT analysis where we highlighted the following aspects of using BBNs and the AgenaRisk software:
- Strength: visibility of the process through a graphical
interpretation, risk measurement using probability, defining causal relationships, use of expert knowledge upon empirical data in case of lack of data, defining uncertainty through the probabilistic set of rules, possibility of using validation tools, reasoning process in light of new evidence, possibility to develop a model for a type of problems related to project planning.
- Weaknesses: oversimplification of the model due to the limitation
of using the AgenaRisk Lite version, lack of historical data, using S knowledge of only one or two experts which can lead to subjectivity, modeling this type of the problem for the first time, and lack of experience with BBNs which can lead to misunderstandings.
- Opportunities: growing interest in using BBNs can lead to
improved models, use of the full version of AgenRisk will provide numerous options for modeling, use of recent advances in BBNs (object oriented BNs, dynamic BNs, hybrid BNs, integrated BNs, hybrid BNs, neural networks, (Marcot & Penman, 2019)), and possibilities for expanding the model including other variables.
- Threats: low acceptance of this model in a wider community
(academic, public, political, etc.), and availability of only a full AgenRisk version for purchasing.
For further work, a new model needs to be complex, to cover different fields, and to be developed, if possible, in a new version AgenaRisk 10.0. It is also necessary to depict the interactions between organizational, human and technical factors/risks.
References
CO £± Cp
O (D C?
(D
m
c ®
w
(D &
CO
Andrejic, M., Borovic, B., & Pamucar, D. 2011. Managing project using a project management approach. Vojnotehnicki glasnik/Military Technical Courier, 59(2), pp.142-175. Available at:
https://doi.org/10.5937/vojtehg1102142A (in Serbian).
Constantinou, A.C., Fenton, N., Marsh, W., & Radlinski, L. 2016. From complex questionnaire and interviewing data to intelligent Bayesian network models for medical decision support. Artificial Intelligence in Medicine, 67, pp.75-93. Available at: https://doi.org/10.1016/j.artmed.2016.01.002.
Fan, C-F., & Yu, Y-C. 2004. BBN-based software project risk management. Journal of System and Software, 73(2), pp.193-203. Available at: https://doi.org/10.1016/jjss.2003.12.032.
Fang, C., & Marle, F. 2012. A simulation-based risk network model for <§ decision support in project risk management. Decision Support Systems, 52(3), pp.635-644. Available at: https://doi.org/10.1016Zj.dss.2011.10.021.
Fenton, N., & Neil, M. 2011. The use of Bayes and causal modeling in decision makin, uncertainty and risk. [online] Available at: https://pdfs.semanticscholar.org/92dc/7cf5f483f5ebe9a0fffc5afe6e87bc5627e5.p -o df. Accessed: 20.04.2018.
.Q (Ü
CO
CO <D
CD
"o >
CD
O CM
of
UJ
a.
Z) O O
_l <
o
X
o
UJ
H ^
a. <
H
<
CD >o
X LU H O
O >
Fenton, N., & Neil, M. 2013. Risk Assessment and Decision Analysis with Bayesian Network. Boca Raton: CRC Press Taylor & Francis Group.
Gadeberg, M., & Luedeling, E. Can we build a better project: assessing complexities in development projects. [online] Available at: https://wle.cgiar.org/thrive/2016/06/01/can-we-build-better-project-assessing-complexities-development-projects. Accessed: 10.09.2016.
John, A., Yang, Z., Riahi, R., & Wang, J. 2016. A risk assessment approach to improve the resilience of a seaport system using Bayesian networks. Ocean Engineering, 111, pp.136-147. Available at: https://doi.org/10.1016/j.oceaneng.2015.10.048.
Landuyt, D., Broekx, S., D'hondt, R., Engelen, G., Aertsens, J., & Goethals, P.L.M. 2013. A review of Bayesian belief networks in ecosystem service modeling. Environmental Modelling & Software, 46, pp.1-11. Available at: https://doi.org/10.1016/j.envsoft.2013.03.011.
Lee, E., Park, Y., & Shin, J.G. 2009. Large engineering project risk management using a Bayesian belief network. Expert Systems with Applications, 36(3-Part2), pp.5880-5887. Available at: https://doi.org/10.1016Zj.eswa.2008.07.057.
Malbasic, S., Tancic, L., & Petrovic, V. 2016. Technology risk assessment as part of risk management process. Serbian Project Management Journal, 6(1), pp.51-62.
Marcelino-Sádaba, S., Pérez-Ezcurdia, A., Echeverría Lazcano, A.M., & Villanueva, P. 2014. Project risk management methodology for small firms. International Journal of Project Management, 32(2), pp.327-340. Available at: https://doi.org/10.1016/j.ijproman.2013.05.009.
Marcot, B.G., & Penman, T.D. 2019. Advances in Bayesian network modeling: Integration of modeling technologies. Environemntal Modeling & Software, 111, pp.386-393. Available at:
https://doi.org/10.1016/j.envsoft.2018.09.016.
Raz, T., & Michael, E. 2001. Use and benefits of tools for project risk management. International Journal of Project Management, 19(1), pp.9-17. Available at: https://doi.org/10.1016/S0263-7863(99)00036-8.
Starr, C., & Shi, P. 2004. An Introduction to Bayesian Belief Networks and their Applications to Land Operations. [online] Available at: https://www.researchgate.net/publication/267240702_An_Introduction_to_Bayes ian. Accessed: 15.09.2016.
Tang, A., Nicholson, A., & Jin, Y., & Han, J. 2007. Using Bayesian belief networks for change impact analysis in architecture design. Journal of Systems and Software, 80(1), pp.127-148. Available at: https://doi.org/10.1016/jjss.2006.04.004.
Tien, I., & Der Kiureghian, A. 2016. Algorithms for Bayesian network modeling and reliability assessment of infrastructure systems. Reliability Engineering & System Safety, 156, pp. 134-147. Available at: https://doi.org/10.1016/j.ress.2016.07.022.
Weber, P., Medina-Oliva, G., Simon, C., & lung, B. 2012. Overview on § Bayesian networks application for dependability, risk analysis and maintenance. Engineering Applications of Artificial Intelligence, 25(4), pp.671-682. Available at: https://doi.org/10.1016/j.engappai.2010.06.002.
Wright, E. 2011. Risk Management in Public Contracting. USA National Institute of Governmental Purchasing (under LEAP program).
Yet, B., Constantinou, A., Fenton, N., Neil, M., Luedeling, E., & Shepherd, K. 2016. A Bayesian network framework for project cost, benefit and risk analysis with an agricultural development case study. Expert Systems with Applications, 60, pp.141-155. Available at:
https://doi.org/10.1016/j.eswa.2016.05.005.
МЕТОДОЛОГИЯ АНАЛИЗА РИСКОВ: ПРИМЕНЕНИЕ БАЙЕСОВСКИХ СЕТЕЙ ВЕРОЯТНОСТИ В ПРОЕКТЕ УТИЛИЗАЦИИ БОЕПРИПАСОВ
Слободан Б. Малбашича, Стефан В. Джурич6
a Министерство обороны Республики Сербия, Департамент материальных
ресурсов, г. Белград, Республика Сербия,
б Университет в г. Крагуевац, Факультет инженерных наук,
г. Крагуевац, Республика Сербия
РУБРИКА ГРНТИ: 78.00.00 ВОЕННОЕ ДЕЛО;
78.01.81 Измерения, контроль и управление качеством.
ВИД СТАТЬИ: обзорная статья ЯЗЫК СТАТЬИ: английский
со £± cp
О ф
с? £Í
Испытание образцов вооружения и военной техники ^
ф ш
с ®
<л ф
ш
Резюме: ®
Модели, представляющие реальные проблемы при принятии решений, зачастую руководствуются историческими данными. Негативный аспект данных моделей заключается в том, что они не могут предусмотреть обстоятельства в будущем, которые основаны на реальных событиях и новых источниках риска. Для преодоления этой проблемы в данной статье представлен процесс разработки реальной предиктивной модели с применением байесовской сети вероятности и программного обеспечения AgenaRisk. Байесовские сети <§ вероятности напрямую отражают реальные проблемы т посредством графических структур, которые представляют не только поток информации, но и условные связи. В качестве теоретического обоснования в данной работе приведена и объяснена теорема Байеса. Преимущество использования ю байесовых сетей вероятности в процессе принятия решений, ® заключается в том, что этот процесс производится в „двух «
СП
со
о
направлениях" (снизу вверх и наоборот), что делает его мощнейшим инструментом для оценки рисков в процессе принятия решений. В статье также представлены базовые [¡5 принципы и значение байесовых сетей и их применение в
процессе разработки проекта по утилизации боеприпасов (утилизация запасов устаревших и непригодных боеприпасов), в котором оценка рисков является одной из сложнейших задач, c\i непосредственно влияющей на принятие окончательного
ос решения о запуске проекта. Наряду с вышеперечисленными
^ методами для обоснования принятия окончательных решений
g проведены: анализ чувствительности и SWOT анализ.
^ Ключевые слова: условная вероятность, байесовская сеть
< вероятности, оценка рисков, анализ чувствительности, SWOT анализ.
х
Ш МЕТОДОЛОГША ЗА ПРОЦЕНУ РИЗИКА: ПРИМЕНА БАJЕСОВИХ
>- МРЕЖА ВЕРОВАТНОЪЕ У nPOJEKTy ДЕЛАБОРАЦШЕ
МУНИЦШЕ
Слободан Б. Малбаши^8, Стефан В. Ъури^ a Министарство одбране Републике Срби]е, Сектор за матери]алне
ресурсе, Београд, Република Срби]а, б Универзитет у Крагу]евцу, Факултет инженерских наука,
< Крагу]евац, Република Срби]а d 2
ОБЛАСТ: менадмент про]ектима
о ВРСТА ЧЛАНКА: прегледни рад
х иЕЗИК ЧЛАНКА: енглески
ш н О
Сажетак:
Модели ко\и репрезентуу реалне проблеме приликом доношена ^ закъучака веЯином се осла^у на историске податке. Негативан
аспект ових модела jесте да они не могу да предвиде будуЯа стаъа заснована на тренутно прикупъеним подацима као и новим изворима ризика. Да би се превазишао оваj проблем, у раду jе приказан процес изградъе реалног предиктивног модела коришЯеъем Баjесових мрежа вероватноЯе и софтвера AgenaRisk. Ба]есове мреже вероватноЯе наjдиректниjе репрезентуу реалне проблеме преко графичке структуре ща представка условне везе, а не само токове информациа. Развиени су и софтвери щи имаjу алгоритме за рачунаъе условних вероватноЯа. Као теоретска основа користи се Баjесова теорема ща jе тако^е обjашfoена у овом раду. Друга предност коришЯеъа Баjесових мрежа вероватноЯе jесте процес закъучиваъа щи се може вршити у „оба правца" (одозго надоле и обратно), што га чини веома моЯним алатом у процени ризика и процесу закъучиваъа.
Тако^е, у раду су приказани основни принципи и предности примене Баjесових мрежа у процесу припреме проjекта делаборацуе § муницие (решаваъе вишкова и неперспективне муницие у складиштима). У ъему jе процена ризика jедан од захтеваних активности ко\и помаже у процесу доношена коначне одлуке за покретаъе или непокретаъе проjекта. Анализа осетливости и SWOT анализа примеъени су као корисни алати за валидациу и доношеъе коначних заклучака.
Клучне речи: условна вероватноЬа, Ба]'есове мреже вероватноПе, процена ризика, анализа осетливости, SWOT анализа.
СО £± CP
О ф
с?
Paper received on / Дата получения работы / Датум приема чланка: 11.03.2019. Manuscript corrections submitted on / Дата получения исправленной версии работы / Датум достав^а^а исправки рукописа: 12.05.2019.
Paper accepted for publishing on / Дата окончательного согласования работы / Датум коначног прихвата^а чланка за об]ав^ива^е: 14.05.2019.
© 2019 The Authors. Published by Vojnotehnicki glasnik / Military Technical Courier (www.vtg.mod.gov.rs, втг.мо.упр.срб). This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/rs/).
© 2019 Авторы. Опубликовано в «Военно-технический вестник / Vojnotehnicki glasnik / Military _
Technical Courier» (www.vtg.mod.gov.rs, втг.мо.упр.срб). Данная статья в открытом доступе и m
распространяется в соответствии с лицензией «Creative Commons» §
(http://creativecommons.org/licenses/by/3.0/rs/). <g
© 2019 Аутори. Обjавио Воjнотехнички гласник / Vojnotehnicki glasnik / Military Technical Courier (www.vtg.mod.gov.rs, втг.мо.упр.срб). Ово jе чланак отвореног приступа и дистрибуира се у складу са Creative Commons licencom (http://creativecommons.org/licenses/by/3.0/rs/).
"5
о p
m
ф
£ m
(Л (Л
ф
(Л (Л
m <л
со
'9
>1Л
m .a
