How to assign weights to different factors in vulnerability analysis: towards a justification of a heuristic technique Текст научной статьи по специальности «Математика»

Mathematical

Structures and Modeling 2014. N. 2(30). PP. 87-98

UDC 004.056

HOW TO ASSIGN WEIGHTS TO DIFFERENT FACTORS IN VULNERABILITY ANALYSIS: TOWARDS A JUSTIFICATION OF A HEURISTIC TECHNIQUE

Beverly Rivera1'2

Research Assistant, PhD student, e-mail: [email protected]

Irbis Gallegos1 Research Scientist, e-mail: [email protected] Vladik Kreinovich2 Ph.D. (Math.), Professor' e-mail: [email protected]

1 Regional Cyber and Energy Security Center RCES Computational Science Program, University of Texas at El Paso, El Paso, TX 79968,

USA

Abstract. The main objective of vulnerability analysis is to select the alternative which is the least vulnerable. To make this selection, we must describe the vulnerability of each alternative by a single number — then we will select the alternative with the smallest value of this vulnerability index. Usually, there are many aspects of vulnerability: vulnerability of a certain asset to a storm, to a terrorist attack, to hackers' attack, etc. For each aspect, we can usually gauge the corresponding vulnerability, the difficulty is how to combine these partial vulnerabilities into a single weighted value. In our previous research, we proposed an empirical idea of selecting the weights proportionally to the number of times the corresponding aspect is mentioned in the corresponding standards and requirements. This idea was shown to lead to reasonable results. In this paper, we provide a possible theoretical explanation for this empirically successful idea.

Keywords: vulnerability analysis, weighted average, heuristic method, probabilistic justification.

1. Assigning Weights to Different Factors in Vulnerability Analysis: Formulation of the Problem

Need for vulnerability analysis. When it turns out that an important system is vulnerable — to a storm, to a terrorist attack, to hackers' attack, etc. — we need to protect it. Usually, there are many different ways to protect the same system. It is therefore desirable to select the protection scheme which guarantees the largest degree of protection within the given budget. The corresponding analysis of different vulnerability aspects is known as vulnerability analysis; see, e.g., [2,8,11-14].

Vulnerability analysis: reminder. Among several possible alternative schemes for protecting a system, we must select a one under which the system will be the least vulnerable. As we have mentioned, there are many different aspects of vulnerability. Usually, it is known how to gauge the vulnerability v, of each aspect i. Thus, each alternative can be characterized by the corresponding vulnerability values (vi,... , vn). Some alternatives result in smaller vulnerability of one of the assets, other alternatives leave this asset more vulnerable but provide more protection to other assets.

To be able to compare different alternatives, we need to characterize each alternative by a single vulnerability index v — an index that would combine the values v1,..., vn corresponding to different aspects: v = f (v1,... ,vn).

If one of the vulnerabilities v, increases, then the overall vulnerability index v must also increase (or at least remain the same, but not decrease). Thus, the combination function f (v1,... ,vn) must be increasing in each of its variables v,.

Vulnerability analysis: important challenge. While there are well-developed methods for gauging each aspect of vulnerability, there is no well-established way of combining the resulting values v1,..., vn into a single criterion v = f (v1,..., vn).

Usually, vulnerabilities v, are reasonably small; so terms which are quadratic (or of higher order) in v, can be usually safely ignored. As a result, we can expand the (unknown) function f(v1,...,vn) in Taylor series in v, and keep only linear terms in this expansion. As a result, we get a linear dependence

v = co + E c, ■ v, (1)

,=1

for some coefficients c,.

Comparison between different alternatives does not change if we subtract the same constant c0 from all the combined values: v < v' if and only if v — c0 < v' — c0.

n

Thus, we can safely assume that c0 = 0 and v = E c, ■ v,.

,=1

Similarly, comparison does not change if we re-scale all the values, e.g., divide

n

them by the same constant E c,. This is equivalent to considering a new (re-scaled)

,=1

combined function

Ec ■ v,

f(v1,...,vn)

,=1

where

w,

Ec*

,=1

def

E

,=1

w, • v,

E cj j=1

(2)

(3)

For these new weights, we have

E w =1

(4)

The fact the function must be increasing implies that wi > 0.

The important challenge is how to compute the corresponding weights wi.

Heuristic solution. In [4,15,17], we proposed an empirical idea of selecting the weights proportionally to the the frequency with which the corresponding aspect is mentioned in the corresponding standards and requirements. This idea was shown to lead to reasonable results.

Remaining problem and what we do in this paper. A big problem is that the above approach is purely heuristic, it does not have a solid theoretical explanation.

In this paper, we provide a possible theoretical explanation for this empirically successful idea.

2. Possible Theoretical Explanation

Main idea. We consider the situation in which the only information about the importance of different aspects is how frequently these aspects are mentioned in the corresponding documents. In this case, the only information that we can use to compute the weight wi assigned to the i-th aspect is the frequency fi with which this aspect is mentioned in the documents. In other words, we take wi = F(fi), where F(x) is an algorithm which is used to compute the weight based on the frequency.

Our goal is to formulate reasonable requirements on the function F(x) and find all the functions F(x) which satisfy this requirement.

First requirement: monotonicity. The more frequently the aspect is mentioned, the more important it is; thus, if fi > fj, we must have wi = F(fi) > F(f) = Wj. In mathematical terms, this means that the function F(f) must be increasing.

Second requirement: the weights must add up to one. Another natural requirement is that for every combination of frequencies fi,..., fn for which

n

£ fi = 1, (5)

i=i

the resulting weights must add up to 1:

nn

£ wi = £ f(fi) = i. (6)

i=i i=i

We are now ready to formulate our main result.

n

Proposition 1. Let F : [0,1] ^ [0,1] be an increasing function for which Y^ fi = 1

i=i

n

implies F(fi) = 1. Then, F(x) = x.

i=i

Comment. So, it is reasonable to use the frequencies as weights. This justifies the above empirically successful heuristic idea.

Proof.

1°. Let us first prove that F(1) = 1.

This follows from our main requirement when n =1 and f1 = 1. In this case, the requirement (6) leads to F(f1) = F(1) = 1.

2°. Let us prove that F(0) = 0.

n

Let us consider n = 2, f1 = 0, and f2 = 1. Then, E f = 1 and therefore,

,=1

n

EF(f,) = F(0) + F(1) = 1. Since we already know that F(1) = 1, we thus

,=1

conclude that F(0) = 1 — F(1) = 1 — 1 = 0.

3°. Let us prove that for every m > 2, we have F ( — ) = —.

\my m

1n

Let us consider n = m and f1 = ... = fn = —. Then, E f = 1 and therefore,

m ¿=1

E F(f,) = m ■ F ( —)=1. We thus conclude that F[ — J = —. j=1 \m J \m J m

/ k \ k

4°. Let us prove that for every k < m, we have F — ) = —.

mm

k1 Let us consider n = m — k + 1, f1 = —, and f2 = ... = fm_k+1 = —. Then,

m m

n

Ef = 1 and therefore,

,=1

£ F (fi) = F + (m - k) ■ F = 1.

\m/ \m/

\ / \ /

(7)

We already know that F ( — ) = —. Thus, we have

mm

F =1 - (m - k) ■ F f—1=1 - (m - k) ■ - = —. (8)

\my \my mm

The statement is proven.

5°. We have already proven that for every rational number r, we have F(r) = r. To complete the proof, we need to show that F(x) = x for every real number from the interval [0,1], not only for rational numbers.

Let x be any real number from the interval (0,1). Let

x = 0.xix2... xn ..., xi G {0,1}, (9)

be its binary expansion. Then, for every n, we have

4 d=f 0.xi... xra < x < d=f 4 + 2-n. (10)

As n tends to infinity, we have ^ x and Un ^ X.

Due to monotonicity, we have F(£n) < F(x) < F(un). Both bounds and un are rational numbers, so we have F(£n) = and F(un) < un. Thus, the above inequality takes the form < F(x) < un. In the limit n ^ œ, when ^ x and un ^ x, we get x < F (x) < x and thus, F (x) = x. The proposition is proven.

Possible fuzzy extension. Our current analysis is aimed at situations when we are absolutely sure which aspects are mentioned in each statement. In practice, however, standards and documents are written in natural language, and a natural language is often imprecise ("fuzzy"). As a result, in many cases, we can only decide with some degree of certainty whether a given phrase refers to this particular aspect.

A natural way to describe such degrees of certainty is by using fuzzy logic,

technique specifically designed to capture imprecision of natural language; see,

e.g., [6,10,19]. In this case, instead of the exact frequency f — which is defined n '

as a ratio between the number n of mentions of the i-th aspect and the total

number N of all mentions - we can use the ratio ^, where ^ is a fuzzy cardinality

of the (fuzzy) set of all mentions of the i-th aspects — which is usually defined as the sum of membership degrees (= degrees of certainty) for all the words from the documents.

3. Towards a More General Approach

What we did: reminder. In the previous section, we proved that if we select the i-th weight wi depending only on the i-th frequency, then the only reasonable selection is F(x) = x.

A more general approach. Alternatively, we can compute a "pre-weight" F(fi) based on the frequency, and then we can normalize the pre-weights to make sure that they add up to one, i.e., take

F (fi) (11) wi = -n-. (11)

E F(fk) k=i

Remaining problem. In this more general approach, how to select the function F(f)?

What we do in this section. In this section, we describe reasonable requirements on this function F(f), and we describe all possible functions F(f) which satisfy these requirements.

First requirement: monotonicity. Our first requirement is that aspects which are mentioned more frequently should be given larger weights. In other words, if

fi > fj, then we should have

Wi = Wi = > = Wj. (12)

E F(fk) E F(fk) k=i k=1

n

Multiplying both sides of this inequality by the sum E F(fk), we conclude that

k=i

F(fi) > F(fj), i.e., that the function F(f) should be monotonic.

Second requirement: independence from irrelevant factors. Let us assume that we have four aspects, and that the i-th aspect is mentioned ni times in the corresponding document. In this case, the frequency fi of the i-th aspect is equal to

fi = + + + . (13)

ni + n,2 + n + n,4

Based on these frequencies, we compute the weights wi, and then select the alternative for which the overall vulnerability

Wi ■ Vi + W2 ■ V2 + W3 ■ V3 + W4 ■ V4 (14)

is the smallest possible.

In particular, we may consider the case when for this particular problem, the fourth aspect is irrelevant, i.e., for which v4 = 0. In this case, the overall vulnerability is equal to

Wi ■ Vi + W2 ■ V2 + W3 ■ V3. (15)

On the other hand, since the fourth aspect is irrelevant for our problem, it makes sense to ignore mentions of this aspect, i.e., to consider only the values ni, n2, and n3. In this approach, we get new values of the frequencies:

fi = +ni + . (16)

ni + n2 + n3

Based on these new frequencies f-, we can now compute the new weights w-, and then select the alternative for which the overall vulnerability

W- ■ vi + w2 ■ V2 + W3 ■ V3 (17)

is the smallest possible.

The resulting selection should be the same for both criteria. As we have mentioned, the optimizing problem does not change if we simply multiply the objective function by a constant. So, if w- = A ■ Wi for some A, these two objective

W •

functions lead to the exactly same selection. In this case, the trade-off — between

Wj

w' W'

each two aspects is the same: —i = —. However, if we have a different trade-off

Wj Wj

between individual criteria, then we may end up with different selections. Thus, to make sure that the selections are the same, we must guarantee that Wf = —.

Wj Wj

Substituting the formulas for the weights into the expression for the weight

ratio, we can conclude that — = F(/i ). Thus, the above requirement takes the

— F (/j)

F (/•) F (/)

form * = , s. One can check that the new frequencies / can be obtained

F (/j) F (/j)

from the previous ones by multiplying by the same constant:

n _ m + + m.3 + m.4 _m*_

/ = , , = , , ' , , , = k' (18) mi + m2 + ma mi + m2 + ma mi + m2 + ma + m,4

where we denoted

k def mi + m + m.3 + m4 (^

mi + m,2 + m3

Thus, the above requirement takes the form F(k ) = F(/i). This should be true

F(k ■ /j) F(/j)

for all possible values of /*, /, and k. Once we postulate that, we arrive at the following result.

Proposition 2. An increasing function F : [0,1] ^ [0,1] satisfies the property

F(k ■ /) F(/)

F(k ■ fj ) F(fj)

(20)

for a// possible real values k, / and / if and on/y if F(/) = C ■ /a for some a > 0.

Comments.

• The previous case corresponds to a = 1, so this is indeed a generalization of the formula described in the previous section.

• If we multiply all the values F(/) by a constant C, then the normalizing sum is also multiplied by the same constant, so the resulting weights do not change:

f/) C ■ /» /«

— = -n- = -n- = -n-. (21)

e f(/fc) e c ■ /a e /a k=i k=i k=i

Thus, from the viewpoint of application to vulnerability, it is sufficient to consider only functions

F(/) = /a. (22)

Proof.

1°. First, it is easy to check that for all possible values C and a > 0, the function F(/) = C ■ /a is increasing and satisfies the desired property. So, to complete our proof, we need to check that each increasing function which satisfies this property has this form.

2°. The desired property can be equivalently reformulated as

F(k ■ fi) _ F(k ■ fj)

F(fi) F (fj) *

This equality holds for all possible values of fi and fj. This means that the ratio

F(k ■ f)

—^ does not depend on f, it only depends on k. Let us denote this ratio by F(f)

c(k). Then, we get F.f) = c(k), i.e., equivalently, F(k ■ f) = c(k) ■ F(f).

F(f)

3°. Since k ■ f = f ■ k, we have F(k ■ f) = F(f ■ k), i.e., c(k) ■ F(f) = c(f) ■ F(k).

F(f) F(k)

Dividing both sides by c(k) ■ c(f), we conclude that = . This equality

c(f) c(k)

F(f)

holds for all possible values of f and k. This means that the ratio . does not

c(f)

depend on f at all, it is a constant. We will denote this constant by C. From the

F(f)

condition = C, we conclude that F(f) = C ■ c(f). So, to prove our results, it

c(f)

is sufficient to find the function c(f).

4°. Substituting the expression F(f) = C ■ c(f) into the formula F(k ■ f) = c(k) ■ F(f), we get C ■ c(k ■ f) = c(k) ■ C ■ c(f). Dividing both sides of this equality by C, we conclude that c(k ■ f) = c(k) ■ c(f). Let us use this equality to find the function c(f).

5°. For k = f = 1, we get c(1) = c(1)2. Since c(k) = 0, we conclude that c(1) = 1.

6°. Let us denote c(2) by q. Let us prove that for every integer n, we have c(2i/n) = qi/n.

Indeed, for f = 2i/n, we have f ■ f ■ ... ■ f (n times) = 2, thus, q = c(2) = c(f) ■... ■ c(f) (n times) = (c(f ))n. Therefore, we conclude that indeed, c(f) = 2i/n.

7°. Let us prove that for every two integers m and n, we have c(2m/n) = qm/n. Indeed, we have 2m/n = 2i/n ■... ■ 2i/n (m times). Therefore, we have

c(2m/n) = c(2i/n) ■ ... ■ c(2i/n) (m times) = (c(2i/n)m. (23)

We already know that c(2i/n) = qi/n; thus, we conclude that c(2m/n) = (qi/n)m = qm/n. The statement is proven.

8°. So, for rational values r, we have c(2r) = qr. Let us denote a d=f log2(q). By definition of a logarithm, this means that q = 2a. Thus, for x = 2r, we have

qr = (2a)r = = (2r )a = xa. (24)

So, for values x for which log2(x) is a rational number, we get c(x) = xa.

Similarly to the proof of Proposition 1, we can use monotonicity to conclude that this equality c(x) = xa holds for all real values x. We have already proven that F(x) = C ■ c(x), thus we have F(x) = C ■ xa. The proposition is proven.

4. Possible Probabilistic Interpretation of the Above Formulas

Formulation of the problem. In the above text, we justified the empirical formula F(x) = x without using any probabilities — since we do not know any probabilities that we could use here.

However, in the ideal situation, when we know the exact probability of every possible outcome and we know the exact consequences of each outcome, a rational decision maker should use probabilities — namely, a rational decision maker should select an alternative for which the expected value of the utility is the largest; see, e.g., [3,7,9,16].

From this viewpoint, it would be nice to show that the above heuristic solution is not only reasonable in the above abstract sense, but that it actually makes perfect sense under certain reasonable assumptions about probability distributions.

What we do in this section. In this section, on the example of two aspects vi and v2, we show that there are probability distributions for which the weights —* should be exactly equal to frequencies.

Towards a formal description of the problem. Let us assume that the actual weights of two aspects are —i and —2 = 1 — —i. Let us also assume that vulnerabilities v are independent random variables. For simplicity, we can assume that these two variables are identically distributed.

In each situation, if the first vulnerability aspect is more important, i.e., if —i ■ vi > —2■v2, then the document mentions the first aspect. If the second vulnerability aspect is more important, i.e., if —i ■ vi < —2 ■ v2, then the document mentions the second aspect. In this case, the frequency / with which the first aspect is mentioned is equal to the probability that the first aspect is most important, i.e., the probability that —i ■ vi > —2 ■ v2:

/i = P (—i ■ Vi >—2 ■ V2). (25)

We would like to justify the situation in which / = — so we have

—i = P(—i ■ vi > —2 ■ v2). (26)

This equality must hold for all possible values of —i.

Analysis of the problem and the resulting solution. The desired equality

can be equivalently reformulated as P( — > —2 ) = —i. Since —2 = 1 — —i,

\V2 — ij

we get P | — >-—- ) = —i. To simplify computations, it is convenient to use

\V2 —i J

logarithms: then ratio becomes a difference, and we get P(ln(vi)—ln(v2) > z) = —i, where we denoted z = ln —i

—i

Let us describe —i in terms of z. From the definition of z, we conclude that

ez = i-—i = -1 — 1. (27)

—i —i

Thus, — = 1 + ez, and w1 = —1—. So, we conclude that

w1 1 + ez

P(ln(vi) - ln(v2) > z) = .

1 + ez

The probability of the opposite event ln(v1) — ln(v2) < z is equal to one minus this probability:

1 ez

P(ln(vi) — ln(v2) < z) = 1 — -— = -—. (28)

1 + ez 1 + ez

This means that for the auxiliary random variable £ = ln(v1) — ln(v2), the cumu-

def

lative distribution function (z) ==: P(£ < z) is equal to (z) ^ z. This

distribution is known as a logistic distribution; see, e.g., [1,5,18].

It is known that one way to obtain a logistic distribution is to consider the distribution of ln(v1) — ln(v2), where v1 and v2 are are independent and exponentially distributed. Thus, the desired formula w, = f (i.e., F(x) = x) corresponds to a reasonable situation when both vulnerabilities are exponentially distributed.

5. Conclusion

In vulnerability analysis, it is important to adequately describe the overall vulnerability of a system. For most systems, there are many different aspects of vulnerability; to estimate the overall vulnerability of a system, it is necessary to combine vulnerability values corresponding to different aspects of vulnerability — e.g., by producing a weighted average of different vulnerability values. For such a combination to adequately describe an overall vulnerability, we need to use appropriate weights.

In the previous papers, we proposed to take, as a weight of each aspects, the relative frequency with which this particular aspect of vulnerability is mentioned in the corresponding standards and requirements. This heuristic proposal was shown to lead to reasonable results. In this paper, we provide a possible theoretical explanation for this heuristic idea.

Acknowledgments

This work was supported by the University of Texas at El Paso Regional Cyber and Energy Security Center (RCES) supported by the City of El Paso's Planning and Economic Development division.

This work was also supported in part by the National Science Foundation grants HRD-0734825 and HRD-1242122 (Cyber-ShARE Center of Excellence) and DUE-0926721.

References

1. Balakrishnan N. Handbook of the Logistic Distribution. New York: Marcel Dekker, 1992.

2. Department of Energy, Electricity Subsector Cybersecurity Capability Maturity Model (ES-C2M2), Version 1.0, 2012.

URL: http://energy.gov/oe/services/cybersecurity/electricity-subsector-cybersecurity-capability-maturity-model-es-c2m2.

3. Fishburn P.C. Nonlinear Preference and Utility Theory. Baltimore, Maryland: John Hopkins Press, 1988.

4. Gallegos I. et al. System, Method and Apparatus for Assessing a Risk of one or More Assets within an Operational Technology Infrastructure. US Patent N. 61/725,474. 2012.

5. Johnson N.L., Kotz S. and Balakrishnan N. Continuous Univariate Distributions, V. 2. New York: Wiley, 1995.

6. Klir G. and Yuan B. Fuzzy Sets and Fuzzy Logic. Prentice Hall, Upper Saddle River, New Jersey, 1995.

7. Luce R.D. and Raiffa R. Games and Decisions: Introduction and Critical Survey. New York: Dover, 1989.

8. National Electric Sector CyberSecurity Organization Resource (NESCOR), Electric Sector Failure Scenarios and Impact Analyses, Version 1.0, 2012. URL: http:// www.smartgrid.epri.com/nescor.aspx.

9. Nguyen H.T., Kreinovich V., Wu B. and Xiang, G. Computing Statistics under Interval and Fuzzy Uncertainty. Berlin, Heidelberg: Springer Verlag, 2012.

10. Nguyen H.T. and Walker E.A. A First Course in Fuzzy Logic. Boca Raton, Florida: Chapman and Hall/CRC, 2006.

11. National Institute of Standard and Technology (NIST), Guide for Mapping Types of Information and Information Systems for Security Categories // NIST Special Publication 800-60, Volume 1, Revision 1, 2008.

12. National Institute of Standard and Technology (NIST), Guide for Conducting Risk Assessment // NIST Special Publication 800-30, Revision 1, 2011.

13. National Institute of Standard and Technology (NIST), Guide to Industrial Control Systems (ICS) Security // NIST Special Publication 800-82, 2011.

14. National Institute of Standard and Technology (NIST), Security and Privacy Controls for Federal Information Systems and Organizations // NIST Special Publication 800-53, Revision 4, 2012.

15. Perez L. Regional Cyber and Energy Security (RCES) Center 2012 Annual Progress Report — Year 1, El Paso, Texas, June 2013. URL: https://www.elpasotexas. gov/muni_clerk/agenda/07-30-13/07301315C.pdf.

16. Raiffa H. Decision Analysis. Columbus, Ohio: McGraw-Hill, 1997.

17. Regional Cyber and Energy Security (RCES) Center at the University of Texas at El Paso, Developing a Framework to Improve Critical Infrastructure Cyber Security, National Institute for Standards and Technology (NIST) Report, April 2013. URL: http://csrc.nist.gov/cyberframework/rfi_comments/ rces_center_040113.pdf.

18. Sheskin D.J. Handbook of Parametric and Nonparametric Statistical Procedures. Boca Raton, Florida: Chapman & Hall/CRC, 2011.

19. Zadeh L.A. Fuzzy sets // Information and Control. 1965. V. 8. P. 338-353.

ВЫБОР ВЕСОВ ДЛЯ РАЗЛИЧНЫХ ФАКТОРОВ В АНАЛИЗЕ УЯЗВИМОСТЕЙ: НА ПУТИ К ОБОСНОВАНИЮ ЭВРИСТИЧЕСКОГО

МЕТОДА

Б. Ривера1

научный сотрудник, аспирант, e-mail: [email protected]

И. Гальегос1 учёный-исследователь, e-mail: [email protected]

В. Крейнович2 к.ф.-м.н., профессор, e-mail: [email protected]

1 Региональный центр кибернетической и энергетической безопасности (RCES) 2Техасский университет в Эль Пасо, США

Аннотация. Основная цель анализа уязвимости — выбор такой альтернативы, которая обеспечивает наименьшую степень уязвимости. Чтобы сделать этот выбор, мы должны описать степень уязвимости каждой альтернативы одним числом. Далее мы выберем вариант с наименьшим значением этого показателя уязвимости. Как правило, есть много аспектов уязвимости: можно рассматривать уязвимость определённого актива к стихийным бедствиям, к терактам, к атакам хакеров и т.д. Для каждого аспекта, мы обычно можем оценить соответствующую уязвимость. Трудность заключается в том, как перевести эти частичные уязвимости в единый взвешенный показатель. В нашем предыдущем исследовании мы предложили эмпирический метод выбора весовых коэффициентов пропорционально количеству упоминаний соответствующего аспекта уязвимости в стандартах и требованиях. Как было показано, данная идея является состоятельной на практике. В настоящей статье мы предоставляем её возможное теоретическое объяснение.

Ключевые слова: анализ уязвимостей, взвешенное среднее, эвристический метод, вероятностное обоснование.