CC BY
8
RESEARCHES IN IDENTIFICATION OF LOGICAL AND PROBABILISTIC RISK MODELS WITH GROUPS OF INCOMPATIBLE
EVENTS
Solojentsev E.D., Rybakov A.V. •
Institute of Mechanical Engineering Problems of RAS, sol@sapr.ipme.ru
Abstract: In this paper the results of the researches in identification of the logical and probabilistic (LP) risk models with groups of incompatible events are presented. The dependence of the criterion function on several parameters has been investigated. The parameters include: the total number of optimisations, the amplitude of parameters increments, the initial value of the criterion function (CF), the choice of identical or different amplitudes of increments for different parameters, objects risks distribution. An effective technology of defining the global extreme in the identification of LP-risk model for the calculation time, appreciable to practice has been suggested.
Key words: risk, logic, probability, model, identification, incompatible events
The logical and probabilistic risk models are almost twice as accurate and have seven times better robustness than other known classification methods [1,2]. However the task of multi-parameter and multi-criteria optimisation for training LP-models is characterised by exclusive difficulty [1-3]. In the process of identification of LP-risk models in business according to statistical data there arise a number of additional features and difficulties [1,2]:
• The criterion function Fmax (CF) is a number of correctly recognised good and bad objects, i.e. it accepts the integer values and it is stepped;
• CF has some local extrema, and depends on the high number of real positive arguments;
• The derivatives of the criterion function with respect to probabilities P1jr cannot be computed.
Fig.? The ev$nfif&wkaVgfnlEoj th £ ontereof dO? FmlfomWparMteeS&Ff qpincy of the
grade in the objects of the "object-signs" table, P1jr is the probability of the event-grade in GIE, PJr is the probability of the event-grade to be substituted into the probability formula. The sums of the probabilities both Wjr and P1jr in GIE equal 1. Connection of these probabilities are considered in [1].
The criterion function Fmax , presented in Fig.1, depends only on two arguments and changes with steps equal to 2. The platforms have different sizes. The arguments P11 and P12 belong to the interval [0,1], but their sizes can differ substantially. While approaching the extreme the platforms decrease in size.
The optimisation can get «stick» at any «platform», not reaching the maximum or crossing the maximum. The character of changing the criterion function in the multivariate space remains the same. Let us remind that the optimisation arguments space dimension for the credit risk LP-model equals 94 [1].
1. IDENTIFICATION OF LP-RISK MODELS
The risk object is described by a large number of signs, every sign has several grades. These signs and grades correspond to random events, which lead to a failure [1,2]. The events-signs (j=1,n) have logical connections and events-grades for each event-sign ( r=1,Nj) form groups of incompatible events (GIE).
The identification of the P-risk model consists in the determination of optimal probabilities Pjr , r = 1, Nj; j = 1, n, corresponding to events-grades. Let us formulate the identification (training) problem for a B- risk model [1,2 ].
Available data: the 'object-signs' table with Ng good and Nb bad objects and the risk B-model; Expected results: to determine the probabilities of Pjr ,r = 1, Nj; j = 1, n for events-grades and the acceptable risk Pad, dividing the objects into good and bad according the amount of risk. We need: to maximise the criterion function, which is the number of correctly classified objects:
(1) F = Nbs + Ngs ^ MAX,
where Ngs and Nbs are the numbers of objects classified as good and bad using both by statistics and the P- risk model (both estimates should coincide ). From (1) it follows, that the errors or accuracy indicators of the P-risk model in the classification of good Eg and bad Eb objects and in the classification of the whole set Em are equal:
(2) Eg = (Ng - Ngs) / Ng;Eb = (Nb - Nbs) / Nb;Em = (N - F)/ N. Assumed restrictions:
1) probabilities Pjr and P1jr must satisfy the stipulation:
(3) 0 < Pjr < 1, j = 1~ft; r = 1Nj.
2) the average risks of objects Pm based on the P- risk model and on the table Pav must be equal; while training the P- risk model we must correct the Pjr probabilities on every step of iterative training under the formula
(4) Pjr = PF * (Pav / Pm); j = 1n; r = 1Nj.
3) the acceptable risk Pad must be determined with the given ratio of incorrectly classified good and bad objects, because of non-equivalence losses at their wrong classification:
(5) Egb = (Ng - NgS) / (Nb - Nbs).
2. OPTIMISATION IN THE IDENTIFICATION TASK
Identification of the LP- risk model by the random search method is based on the ideas used in the training of neural networks [4]. With reference to the identification task of the LP- risk model, the following formula for the calculation of the changes of events-grades probabilities may be put down:
(6) dP1 jr = K 1*(1/ Nt) * tg(K3); j = 1nr = 1N
where: K1 is a coefficient; Nt is the current number of optimisation; K3 is a random number from [n/2, + n/2], n is a number of events-signs, Nj is a number of events-grades in each GIE, i.a. for every event-sign.
In the formula (6) the CF is a current error in training. The number of optimisations Nt, before the end of the training process, can be very big. The «tangent» operation is the consequence of the training error distribution recording to Cauchy. Theoretically, this error is distributed according to the normal law, but not spend a lot of time on tabulated values calculation, we use the distribution of the training error under the Cauchy's law. It allows to reduce in 100 times the calculation time, which otherwise, for real problems, would continue for days and weeks.
For failure risk LP-model training the following modification of the formula (6) is suggested [1]:
(7) dP1 jr = K1 * (Nopt - Nt) * tg(K3), j = \nj = 1N,
where: Nopt is the given number of optimisations. The new values of P1jr and Pjr, obtained at F > Fmax on every step Nt of optimisation are considered optimal and saved.
In the LP-risk model identification task, the criterion function cannot exceed the total number of objects in the statistical data. The formula (7) is quite applicable, but the time of calculation is too big (about 10 hours for a session of optimisation).
To reduce the time of calculation, in the formula (7) the "tangent" operation is eliminated. As a result the following expression is obtained [3]:
(8) dP 1 jr = K 1 * (Nopt - Nt) * K 3, j = 1,nj = 1,Nj.
Using (7,8) the optimization happens so: if F>Fmax, then we remember the new P1jr and Pjr. If the criterion function does not strictly increase after the chosen number of trials Nmc in Monte-Karlo, then Fmax is reduced by 2-4 units and optimisation continues.
In spite of the investigation in optimisation, carried out before, where the formulas (7) and (8) were used [1,2], the problem of optimisation in the identification task of LP-risk models is far from the final solution. The following fact proves it. In one of the research with the huge number of optimisations Nopt=245 000 and with the constant, almost optimal, value of the increment dP1jr, we obtained Fmax = 824 instead of Fmax = 810 at the usual number of optimisations Nopt « 245. We had to carry out special investigations, the results of which are adduced below.
3. INVESTIGATIONS IN IDENTIFICATION / OPTIMISATION
If we generate a random number K3 in the interval [-1, +1], then the absolute values of increments of probabilities dP1jr, multiplied by 100, are transformed in percents (%). It is convenient, for practically it solves the problem of the evaluation of probabilities P1jr accuracy. For example, if the increment is dP1jr=0.0005, it equals 0.05 % . We can say that the probability P1jr with the accuracy 0.05 % is evaluated .
Using the formula (8), in the beginning of optimisation we have the following maximum amplitude of probabilities increments :
(9) AP1max = K*Nopt .
In the end of optimisation the maximum amplitude of probabilities increments equals 0. Let us designate the current amplitude of probabilities increments as AP1. There is an optimal interval OPT of the amplitudes increments AP1, which position and width are unknown (Fig. 2). For the big values of AP1 there is a small probability of increasing Fmax, and for small values of AP1 there is a high probability to stop at the local extreme of the reached value Fmax (see Fig.1).
Fig.2. Graphs of relation between the number of optimisations Nopt and increments amplitudes API
The optimisation process ( of training the LP-risk model) should be long enough in the optimal OPT interval . The value of dNopt duration in the optimal OPT interval is equal
(10) dNopt = (OPT * Nopt ) / AP1max.
It also depends on the number of optimisations Nopt and the maximum amplitude of the increment dP1max. The more Nopt is and the less AP1max is , the longer is the duration of dNopt. The purpose of this work is the investigation of the dependence of the criterion function (accuracy of LP- risk model) on the following parameters in the training formula (8):
1. The number of optimisations Nopt;
2. The increment minimum amplitude AP1min, at which the optimisation is still possible;
3. The initial value of the criterion function Fbeg ;
4. The choice of identical or different amplitudes API for different grades;
5. The increment maximum amplitude AP1max;
6. Objects risk distribution in the statistical data.
Let us illustrate it. A question arises, whether to choose the identical or different values of increments amplitudes API for all events-grades ? In other words, whether the amplitudes AP1Jr should depend on the values of probabilities P1jr ? In the training formulas of the LP-risk model (7) and (8) the increments amplitudes AP1Jr are identical for all events-grades and do not depend on the values of their probabilities P1jr. The increments dP1jr differ only because of the random simulation of the K3 coefficient.
The model investigations for the LP-model of the credit risk were made on the PC. The credit risk structural LP-model has 20 events-signs (correspondingly GIE) and 94 events-grades. The credit risk L-function is [1,2] :
(11) Y = X1U X 2 U...U X 20
Verbally it can be formulated as follows: a failure occurs, if any one, or any two, ... or all initiating events happen. After the orthogonalization of the L-function (11) the following P-risk model for the evaluation of the credit risk has been obtained:
(12) P = Pi + P 201 + P 3Q1Q 2 +....
The investigations were carried out in a set of 1000 credits, 700 of which were good and 300 - bad [5]. For calculation investigations we used the Software , designed in the object-oriented languages Visual C+++ and Java.
3.1 The choice of parameters Nopt , AP1min , Fbeg
In comparison with the optimal variant Fmax = 824, the initial variant had the probabilities P1jr without the last four signs. So the optimisation starts at Fbeg = 690-760. Such solution allowed to reduce calculation time.
The calculations were made for two values of increments maximum amplitudes: 1) AP1max =0.05 (5 %) , 2) AP1max = 0.1 (10 %) . We used the following numbers of optimisations Nopt: 150, 300, 500, 750, 1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000.
The results of investigations presented in Table 1 (Var.2-21) and Fig.3 , allow to make the following conclusions:
1) The criterion function Fmax (column 6 in Table 1 and Fig.3 ) asymptotically increases with the growth of the number of Nopt optimisation ;
2) The minimum amplitude AP1 min (column 9) equals approximately 0.0025 ( 0.25 %); at the smaller values of AP1min the optimisation does not happen and the number of the last optimisation Nend (column 10) is less, than the given number of Nopt optimisations. It is necessary to modify the law of the change of AP1 during the training process , adding the constant line AP1min (Fig.4). It increases the chance to get the greater value of Fmax;
3) The big value of Nopt can lead to the disappearance of the B-C line (Fig. 4), which undoubtedly will deteriorate the process of optimisation.
4) The initial value of Fbeg (column 5) should not be lowered, as it often leads to low final values of Fmax (Fig. 5) because of the unsuccessful trajectory of optimisation process; in the considered case it is possible to accept Fbeg =750-760.
Taking into consideration the just made conclusions , instead of the formula (8) the following
formula for training the LP-risk model is suggested:
(13) If AP1 < AP1 min , then dP1jr = AP1 mm * K3 ,
If AP1 > AP1min, then dP1jr = K1* (Nopt - Nt) * K3.
The optimisation results using the formula (13) under AP1min = 0.0025 (0.25 %), different AP1max = 0.098, 0.09, 0.03 (9.8 %, 9 %, 3 %), a rather large number of optimisations Nopt=5000-12000 and the high Fbeg =745 in Table 1 ( var. 22-24) are shown. In all variants high values of Fmax =812-822 have been obtained.
Table 1. The investigations results in the choice of optimisation parameters
N Nopt K, AP1ma Fbeg Fmax dPc AP1min Nend Notes
1 2 3 4 5 6 7 8 9 10
1 2000 0.0001 0.2 776 786 0.204 0.1987 20
2 300 0.000165 0.05 756 794 0.1969 0.00198 289 (3)
3 300 0.00033 0.1 712 790 0.221 0.00429 288 (3)
4 750 0.0000665 0.05 756 802 0.1641 0.00545 669 (3)
5 750 0.000133 0.1 692 790 0.2052 0.01316 652 (3)
6 1000 0.00005 0.05 750 802 0.1867 0.00350 931 (3)
7 1000 0.0001 0.1 708 792 0.2174 0.01580 843 (3)
8 2000 0.000025 0.05 776 808 0.1595 0.00747 1702 (3)
9 2000 0.00005 0.1 724 798 0.1802 0.01405 1720 (3)
10 3000 0.0000166 0.05 748 806 0.1867 0.00699 2581 (3)
11 3000 0.000033 0.1 708 806 0.1867 0.00501 2849 (3)
12 4000 0.0000125 0.05 744 812 0.1945 0.00791 3368 (3)
13 4000 0.000025 0.1 740 802 0.2121 0.00862 3656 (3)
14 5000 0.00001 0.05 754 806 0.1663 0.00556 4445 (3)
15 5000 0.00002 0.1 738 803 0.1586 0.00400 4801 (3)
16 6000 0.000016 0.1 710 810 0.1598 0.00625 5610 (3)
17 6000 0.0000183 0.109 736 810 0.1618 0.00495 5730 (3)
18 7000 0.0000071 0.05 764 810 0.2096 0.00407 6430 (3)
19 7000 0.0000142 0.1 734 810 0.1692 0.00745 6479 (3)
20 8000 0.0000062 0.05 764 810 0.1755 0.00985 6425 (3)
N Nopt K AP1ma Fbeg Fmax dPc AP1min Nend Notes
21 8000 0.0000125 0.1 718 814 0.1802 0.00286 7772 (3)
22 12000 0.0000075 0.09 772 812 0.1737 0.0025 11754 (10)
23 8000 0.00000375 0.03 780 820 0.1526 0.0025 7662 (10)
24 8000 0.00000875 0.07 744 814 0.1733 0.0025 7801 (10)
25 5000 0.0000043 0.0215 812 820 0.1462 0.0025 23 (13)
26 5000 0.00000043 0.0025 810 824 0.1511 0.0025 34 (13)
27 8000 0.00000002 0.0025 810 826 0.1538 0.0025 678 (13)
28 8000 0.0000025 0.00458 806 822 0.1604 0.00609 507 (13)
29 8000 0.00000312 0.00572 806 822 0.1677 0.00452 1757 (13)
F
A max
815 810 805 800 795 790 785
AP1„
B
C
AP1„
D
No,
Nopt
Fig.4. The graph of the current amplitude of increment API modification
F
max 820
810
800
790
780
-1-1-1-1-1-
680 700 720 740 760 780 Fh
Fig. 5. Dependence of the criterion function Fmax on its initial value
-70A
0
3.2 Different amplitudes AP1jr of increments for different grades
It should be noted, that the probabilities P1jr depend on: a number of grades in GIE, the frequencies of Wjr grades in objects and the grades contributions in the classification errors of objects. In the formula of training the LP-risk model (8) the increments amplitudes AP1jr are identical for all events-grades and do not depend on the magnitude of their probabilities P1jr.
Let us change the formula of the training LP-risk model so that it takes into account the value of probability for each grade
(14) dP1jr = K1 * (Nopt - Nt) * K3 * P1 r..
Here the amplitudes for every event grade are equal
(15) APljr = K*(Nopt -N)*P1jr
and the formula (14) can be the following:
(16) dP1 jr = AP1 jr * K3.
Let us also put down the formula (14) with the following modification: (17) dP1 jr = K1 * (Nopt - Nt) *((1 - a) + a * P1 jr )* K3,
where a is a coefficient from the interval [0 < a < 1]. It determines the formula (8) at a=0, the formula (14) at a=1 and all the modifications at other values of a.
In the formula (13) let us take into account the limitations, introduced earlier in the formula (8), and we shall get the following expression for training the LP-risk model:
(18) If AP1jr < AP1 min , then dP1jr = AP1min ,
If AP1jr > AP1 min, then dP1jr = K1* (Nopt - Nt) *((1 - a) + a * P1 jr )* K3,
The investigations results in optimisation using the formula (18) at a=1 (AP1max = 2.15 % , 0.25 % , 0.45 %, 0.57%) are represented in Table 1 (Var.25-29). They show that the high values of the Fmax =822-826 can be obtained at the limited number of optimisation attempts Nend (column 10). Actually the first optimisation already gives the high value of CF (Fbeg=806-810). The optimisation process ends at Nend = 23-1750 instead of the given numbers of optimisations Nopt=5000-8000 (column 6). It seems, that the number of optimisations Nopt can be essentially reduced. To verify this hypothesis some extra investigations have been carried out.
The investigations were carried out at small numbers of optimisations Nopt = 600, 450, 300, 150, 100, 50 and K1=0.00033, 0.00025, 0.00015, 0.0001. The increments maximum amplitude AP1max varied in an interval 0.5% - 20% from P1jr. In Table 2 the CF values and the difference between maximum and minimum risks of objects in the statistics Fmax / APc are shown. The results of the investigations should be considered as good (Fmax =810-822) and completely confirming the effectiveness of the formulas (14), (17) and (18).
Also the investigations of the influence of a parameter on the optimisation results have been carried out. It was done at the small numbers of optimisations Nopt=150 and Kj=0.00015. The maximum amplitude of an increment AP1max equals 0.0225* P1jr.
Table 2. Values of Fmax / APc at the small number of optimisations Nopt and a=1
Number of
optimizations, NoDt K1=0.00033 K1=0.00025 K1=0.00015 K1=0.0001
600 798 / 0.248 796 / 0.225 810 / 0.180 810 / 0.149
450 802 / 0.217 804 / 0.187 814 / 0.162 819 / 0.161
300 810/ 0.146 810 / 0.174 816/ 0.147 820 / 0.162
225 810/ 0.154 811 / 0.152 818 / 0.148 821 / 0.146
150 816/ 0.145 820 / 0.156 822 / 0.148 822 / 0.147
100 818/ 0.146 820 / 0.149 820 / 0.151 820 / 0.153
50 822 / 0.151 820 / 0.146 820 / 0.152 820 / 0.148
The investigations results, represented in Table 3, also confirm the effectiveness of the formulas (14),(17) and (18) at a=1. Really, at a=1 Fmax equals 820, and at a=0 Fmax equals 802.
Table 3. Values Fmax at different values of a
Value a 0.0 0.1 0.2 0.3 0.4 0.5 0. 6 0.7 0 . 8 1.0
Value Fmax 802 800 798 804 808 810 808 810 818 820
3.3 Determination of the amplitude AP1max and the global extreme Fm
Let us consider again the choice of the increment maximum amplitude of probabilities AP1max. The results of the change of Fmax at the change of AP1max=K1*Nopt in the interval 0.5-20 % of P1jr are represented in Table 2. They demonstrate that the higher is AP1max the less is Fmax. In Fig.6 the dynamics and the results of optimisation for five variants, having Nopt =2000, are shown:
Variant 1: AP1max Variant 2: AP1max Variant 3: AP1max Variant 4: AP1max Variant 5: AP1max
F
1 m
820
opt
-0.05(5%), Fmax =808 (Var.8 in Tablel), training under the formula (3); =0.1(10%), Fmax =798(Var.9 in Tablel), training under the formula (3); =0.05 (5%), Fmax = 820, training under the formula (14) with a=1; =0.1 (10%), Fmax = 804, training under the formula (14) with a=1; -0.2 (20%),Fmax =786(Var.1 in Table1),training under the formula (14).
820 816 812 808 804 800 796 792 788 784 780 776
810
Var. 5 I
0.02 0.04 0.06 0.08 0.1 0.18 0.2 AP1„
Fig.6. Dynamics and results of optimization depending on the AP1max parameter
0.14
0.15 0.16 0.17 0.18 0.19
Fig.7.The connection ofparameters Fmax and APc
0.20
APc
0
Variants 4 and 5 with high AP1max, despite using the effective formula (18) and a=1, have bad training dynamics and results. In these variants CF are correspondingly 786 and 804. The optimisation process finishes early, (Nend=1608 and Nend=20). Additional optimisation attempts Nopt - Nend have not increased CF. This example confirms that the increment amplitude AP1max should not be more than 0.02 -0.05 (2-5 %).
We check the calculation of the global extreme of the CF by the graph (Fig.7). The function Fmax has an extreme at some value of the difference APc between the maximum risk and the minimum risk of objects in statistics [2 ]. This difference, constructed for variants of computational investigations , presented in Table 1 and 2, demonstrates the robustness (stability) of solutions at a small dispersion of APc in the area of the global extreme of CF.
4. CONCLUSION
In the investigations the following main results have been obtained:
1. The effective technology of the criterion function global extreme search in the tasks of identification of LP-risk models under statistical data has been offered .It permits to solve the task of multi-parameter multi-criteria optimisation with integer CF for the time, applicable to practice (less than before).
2. We suggest to generate in the training formula a random number K3 in the interval [-1, +1]. It permits to consider the absolute values of increments dP1jr, multiplied by 100, in percents (%) ) and to estimate the accuracy of probabilities P1jr.
3. In the technology of the CF global extreme search, the following regularities of changing the CF should be used:
• The CF asymptotically increases with the growth of Nopt optimisation number ;
• The minimum amplitude AP1min of probabilities P1jr increments is established by 2-3 test calculations; at smaller values of AP1min the optimisation does not happen (less than 0.25 %);
• The initial CF Fbeg should not be lowered , as low values more often result in low final values of Fmax because of the unsuccessful trajectory of the optimisation process;
• Maximum amplitude of increments of AP1max must not exceed 0.02 - 0.05 (2-5%), as the training speed lows down and the value of the CF Fmax becomes less.
4. For the criterion function global extreme search new , more effective formulas of training (14), (17), (18) have been suggested ; they use different amplitudes of increments for probabilities of different events-grades.
5. It has been confirmed that we can test the determination of the global extreme of CF Fmax by the graph of change of Fmax in the function of difference APc between maximum and minimum risks of objects in statistics. The function Fmax has an extreme at a certain value of APc .
REFERENCES
1. Solojentsev E.D., Karasev V.V.,Solojentsev V.E. Logic and probabilistic models of risk in banks, business and quality / edited by E.D.Solojentsev. SPb.: Nauka, 1999.-120 p.
2. Solojentsev E.D., Karassev V.V. (2001) Risk logic and probabilistic models in business and identification of risk models. - Informatica 25 (2001) 49-55.
3. Taha X. Introduce in research of operations . v.1,2. Moscow, Mir , 1985.
4. Wasserman Philips D. Neural Computing Theory and Practice. ANSA Research , Inc. VAN NOSTRARD REINOLD, New York, 1990.
5. Seitz J., Stickel E. Consumer Loan Analysis Using Neural Network. Adaptive Intelligent Systems. Proceed. of the Bankai Workshop, Brussels, 14-19 October, 1992. P.177-189
