CC BY
8
MSC 90B06
DOI: 10.14529/ mmp220411
SMOOTH APPROXIMATION OF THE QUANTILE FUNCTION DERIVATIVES
V.R. Sobol1, R.O. Torishnyy1
1Moscow Aviation Institute, Moscow, Russian Federation E-mail: vitsobol@mail.ru, arenas-26@yandex.ru
In this paper, a smooth approximation of the second-order derivatives of quantile function is provided. The convergence of approximations of the first and second order derivatives of quantile function is studied in cases when there exists a deterministic equivalent for the corresponding stochastic programming problem. The quantile function is one of common criteria in stochastic programming problems. The first-order derivative of quantile function can be represented as a ratio of partial derivatives of probability function. Using smooth approximation of probability function and its derivatives we obtain approximations of these derivatives in the form of volume integrals. Approximation of the second-order derivative is obtained directly as derivative of the first-order derivative. A numerical example is provided to evaluate the accuracy of the presented approximations.
Keywords: stochastic programming; probability function; quantile function and its derivatives.
Introduction
Wide range of design problems are represented as mathematical programming problems, where the optimization criterion represents the system performance, which depends on chosen optimization vector (strategy) and some undetermined parameters. Consideration of the undetermined effects as random parameters appears very effective in many engineering, financial, and social problems. Addition of random parameters in the model turns the objective and constraints to random functions. This fact leads us to consideration of optimization problems in which the objective or constraints are in the form of a probability function or a quantile function [1]. The probability function equals to probability that a specified level is not exceeded by the loss value. The quantile function is the minimum loss level, which is not exceeded with a given probability. The choice of the loss level or reliability level depends on the specifics of the system considered.
There are vast numerical methods and algorithms developed to solve non-stochastic optimization problems. These methods often assume that we can calculate or estimate first and/or second order derivatives of the criterion - estimate its gradient and Hessian. But this technique can not be applied directly to stochastic optimization problems. Calculation of a probability function gradient implies the integration over surface [2]. Only in specific cases direct calculation can be implemented through volume integration [3]. Other approaches to derive the gradient approximations are described for example in [4,5], but they are limited by the type of distribution or other stochastic mechanisms.
One effective approach to estimate the derivatives of probability function is to use the smooth approximation of probability function. The key idea is to replace the Heaviside function inside the probability function with its smooth approximation that is a sigmoid function [6]. Then the approximation of probability function along with its gradient takes a form of volume integrals. Smooth approximation of probability function and its derivatives along with the proof of convergence of the approximations were first presented in [6] for one-dimensional random vector with absolutely continuous distribution. In [7] the same results were obtained in the case of continuous random vector.
Smooth approximation of the second-order derivatives of probability function was first described in [8]. The application of second-order optimization algorithms for stochastic optimization problems was described. The approximations of the second-order derivatives were presented as expectations or volume integrals, which can be calculated using Monte-Carlo method. The proof of convergence of approximations of the second-order derivatives was not considered in [8], since exact formulas are not available. But numerical examples show that smooth approximations of the second-order derivatives converge to ones estimated using finite differences. The present paper continues the research considered in [8] and presents a smooth approximation of the second-order derivatives of quantile function. The convergence analysis in some basic setups is provided. Numerical example is given. The example shows that smooth approximations have lower variance than finite-difference estimates and provide more accurate estimates.
The paper is organized as follows. In Section 1 all necessary formulas and statements on the smooth approximations are provided. In Section 2 the convergence analysis is made for some basic cases of the loss function. In Section 3 numerical example is given, comparing finite-difference estimates with the smooth approximations of quantile function derivatives. In conclusion an overview of the paper is provided.
1. Theoretical Part
Consider a complete probability space (Q, F, P), an absolutely continuous random vector X with the support G C Rn, a probability density function f (x), and a loss function g(u,x) depending on a strategy u G U, U C Rm. The function g(u,x) is considered to be smooth and strictly piecewise monotonic with respect to x. The probability function is equal to probability that a random value g(u,X) do not exceed a specified level p, while the quantile function is equal to minimal loss level, which is not exceeded with a given probability a:
Pip(u) = P {g(u, X) < p} = i Q(p - g(u, x))f (x)dx, (1)
G
Pa(u) = min{p : (u) > a}, (2)
where ©(•) is the Heaviside function.
Since g(u,X) is the loss function, we can consider two optimization problems [1]:
Pa (u) ^ min, (3)
Pv (u) ^ max. (4)
According to [6] we replace Heaviside function with a sigmoid function to get a differentiable approximation of the probability function. The sigmoid function and its derivatives are defined as follows
= (5)
S'e(t) = 9So(t)(1 - So(t)), St(t) = e2Sd(t)(1 - So(t))(1 - 2So(t)) (6)
with the parameter 9 > 0 corresponding to the steepness of the sigmoid function. The approximation of the probability function is defined as:
PO(u) = J So(P - g(u,x))f (x)dx = E(So(p - g(u,X))), (7)
G
The approximation of the probability function derivatives are defined as:
/ Se- g(u, x))f (x)dx = E {S'e- g(u, X))), (8)
(u)
G
-к— = - Se(<p- g(u, x)) f(x)dx = -E I Se(<p - g(u, X))——— I . (9)
i *J i \ i /
G
The following statements were proven [6,7]:
lim Pi(u) = P {g(u,X) < p} , (10)
d d
= ("J
d d
Approximation of the second-order derivatives of the probability function [8] is
d2 Pe (u) / \
= E^-g^X))^^^^^^)-^^-^«^))^.^^)). (13)
Approximation of quantile function derivatives are given in [6]:
сЩи) д _1__д E (S>(Va(u) - д(и,Х))д>щ(и,Х))
da E (S'e{tpa{u) - д{и,Х))У дщ E (S'e(tpa(u) — g(u, X)))
(14)
In the last ratio, we denote the numerator and denominator by V(u) and W(u), respectively:
V(u) 4 E [S'(ya(u) - g(u, X))g'Ut(u,X)] , (15)
W(u) 4 E[S'(ya(u) - g(u, X))]. (16)
Approximation of the second partial derivative of the quantile function is defined as:
sva(u) VUj(u)W(u) - V(u)WUj(u)
du duj W 2(u)
where the derivatives of the functions V(u) and W(u) are defined as:
(17)
(u) = E Isв(v»(u) - g(u,X))g'UiUi(u,X)
+ E
W' (u) = E
Í д
S'éi'Mu) - g{u,X))g'Ui(u,X) tpa(u) -g'Uj{u,X) S'e(<pa(u) - g(u,X)) ^j-tpa(u) - д'и.(и,Х)^
, (18) (19)
2. Convergence Analysis
We assess the convergence of approximations for 3 cases, where the quantile function can be defined explicitly (the deterministic equivalent exists).
2.1. Separable Loss Function
Let us consider the separable loss function, where u G U C and X G :
g(u,X) = gi(u) + g2(X ).
In this case the quantile function and its derivatives are equal to
dpa(u) dgi(u) d2pa(u) d2gi(u)
Pa(u) = gi(u) + [g2(X )]a,
dui du ' duiduj duduj
where [g2(X)]a denotes the quantile of the random variable g2(X). In this case
V(u) = E [so(pa(u) - g(u,XMu.(u)] = glu.(u)E [so(pa(u) - g(u,X))], W(u) = E [S'o(pa(u) - g(u, X))].
The ratio of V(u) and W(u) gives g'lu. (u) = -^-gi(u):
dpjju) V(u) dpa{u)^ d2pj(u) d2pa(u)_ dui W (u) dui ' dui duj dui duj
2.2. Product of Two Functions
Next, consider the loss function to be a product of two functions:
g(u,X) = gi(u)g2(X),
where gl(u) > 0 for every u G U. The quantile function and its derivatives are equal to { \ ( \\ (vw dV»(u) dg^u) d2paju) d2glju)
= -Q^r = =
The auxiliary functions V(u) and W(u) are defined as:
V(u) = E [so(pa(u) - g(u,X))g'ui(u)g2(X^ = glUi(u)E[so(pa(u) - g(u,X))g2(X)],
W(u) = E[Sfd(pa(u) - g(u, X))].
The smooth approximation of the quantile function derivative is equal to:
d<fj{u) = E [S'('Mu) - g(u,X))g2(X)] d dut E [S'e(<pa(u)-g(u,X))] dut9l[U)*
The smooth approximation of the quantile function gradient takes the following form:
= VmM ■ E|
e[so(pa(u) - g(u,X))]
The vectors Vpda(u) and Vgl(u) are codirectional. Smooth approximation of the second-order derivative is a linear combination of the second-order derivative of the loss function and the product of the first-order derivatives with coefficient converging to zero.
2.3. One-Dimensional Case
Let X G Rl, i.e. n = 1, and suppose that g(u,x) is a strictly increasing and left-continuous function with respect to x for every u. This case was studied in [1, Chapter 4]:
pa (u) = g(u, [X]a). Using the properties of the sigmoid function described in [6] we state that:
g' (u x)
V(u) = E[S>(pa(u) - д(и,Х))д>щ(и,Х)} ff '
Wx (u,x)
as 9 —У то,
1
W(u) = E [S>(pa(u) - g(u, X))] —--
l9x(u,x)l
9щиa (u,x)
X=[X ]a
as 9 — то,
Vf' / 4 uui LL j \ '
и Xй) TT7-77
' |gX (u,x)l
x=[X ]0
X=[X ]c
WlL . (u) — 0 as 9 — то.
Therefore:
dpi (u) V (u)
dui
W (u)
— W (u, [X]a) as 9 — то,
d2pea(u) Vj(u)W(u) - V(u)Wj(u)
dui duj
W2 (u)
— giui (u, [X}a) as 9 — то-
3. Numerical Example
Let us compare smooth approximation of quantile function derivatives with the finite difference estimates based on sample quantiles. We use a sample of size 100000 to calculate sample quantiles and to calculate expectations in (14) - (19). The parameter a is equal to 0,8, 9 is equal to 30. To find a sample quantile for a given vector u we use the sample {Xi}f=l of size N, then obtain a sample of loss function values {gi(u)}f=l, where gi(u) = g(u,Xi). Let g(i) (u) be the i-th element of the variational series:
g(l)(u) < g(2)(u) < ... < g(N)(u)-The sample quantile is a statistical estimate of the quantile function [9]:
pa(u) = g([aN])(u), (20)
where [■] denotes the integer part.
Example 1. Let us consider a two-dimensional case with a bilinear loss function and multivariate normal distribution:
g(u,X) = uiXi + u2X2, X - N (m, K) ,
m =(0) , K =(o15 5) •
The quantile function can be found explicitly:
pa(u) = xa^u\ + UiU2 + 2 where xa is the a-quantile for standard Gaussian distribution.
The comparison of the smooth approximation of quantile function derivatives with the finite-difference estimates and with exact derivatives is presented in Figure. The smooth approximation of the first-order derivatives is close to the exact derivative and have lower variance. The smooth approximation of the second-order derivative has correct sign and slope, but is more affected by noise when compared to the first-order approximations.
Conclusion
We provide a smooth approximation of the second-order derivatives of quantile function. Approximations of the first and second order derivatives of quantile function are expressed via partial derivatives of the probability function, which have a form of volume integrals and can be calculated using Monte-Carlo method. We provide convergence analysis for special cases of loss function and give a numerical example to analyse the accuracy of the smooth approximations of quantile function derivatives. Numerical example shows that the smooth approximation of quantile function derivatives is close to exact values.
Acknowledgements. The reported research was funded by RSCF, project number 22-2100213, https://rscf.ru/project/22-21-00213/.
References
1. Kibzun A.I., Kan Yu.S. Stochastic Programming Problems with Probability and Quantile Functions. London, John Wiley & Sons, 1996.
2. Raik E. The Differentiability in the Parameter of the Probability Function and Optimization of the Probability Function via the Stochastic Pseudogradient Method. Proceedings of Academy of Sciences of the Estonian SSR. Physics. Mathematics, 1975, vol. 24, no. 1, pp. 3-9.
3. Uryas'ev S. Derivatives of Probability Functions and Some Applications. Annals of Operations Research, 1995, vol. 56, pp. 287-311. D01:10.1007/BF02031712
4. Henrion R. Gradient Estimates for Gaussian Distribution Functions: Application to Probabilistically Constrained Optimization Problems. Numerical Algebra, Control and Optimization, 2012, vol. 2, no. 4, pp. 655-668. D0I:10.3934/naco.2012.2.655
5. Pflug G., Weisshaupt H. Probability Gradient Estimation by Set-Valued Calculus and Applications in Network Design. SIAM Journal on Optimization, 2005, vol. 15, no. 3, pp. 898-914. DOI:10.1137/S1052623403431639
6. Sobol V.R., Torishnyi R.O. On Smooth Approximation of Probabilistic Criteria in Stochastic Programming Problems. SPIIRAS Proceedings, 2020, vol. 19, no. 1, pp. 181-217. DOI:10.15622/sp.2020.19.1.7
7. Sobol V., Torishnyi R. Smooth Approximation of Probability and Quantile Functions: Vector Generalization and its Applications. Journal of Physics: Conference Series, 2021, vol. 1925, article ID 012034.
8. Torishyi, R. Application of the Second-Order Optimization Methods to the Stochastic Programming Problems with Probability Function. Trudy MAI, 2021, no. 121, 27 p.
9. Cox D.R., Hinkley D.V. Theoretical Statistics. London, Chapman and Hall, 1979.
Received May 13, 2022
УДК 519.856 DOI: 10.14529/mmp220411
О ГЛАДКОЙ АППРОКСИМАЦИИ ПРОИЗВОДНЫХ ФУНКЦИИ КВАНТИЛИ
В.Р. Соболь1, Р.О. Торишный1
1Московский авиационный институт (НИУ), г. Москва, Российская Федерация
В статье предложена гладкая аппроксимация вторых производных функции квантили. Сходимость аппроксимаций первых и вторых производных функции квантили исследуется в случаях, когда для соответствующей задачи стохастического программирования существует детерминированный эквивалент. Функция квантили является одним из основных критериев в задачах стохастического программирования. Производная первого порядка может быть представлена как отношение частных производных функции вероятности. Используя гладкую аппроксимацию функции вероятности и ее производных, эти производные аппроксимируются в форме объемных интегралов. Аппроксимация второй производной определяется непосредственно дифференцированием аппроксимации первой производной. Для оценки точности представленных аппроксимаций приведен численный пример.
Ключевые слова: стохастическое программирование; функция вероятности; функция квантили и ее производные.
Исследование выполнено за счет гранта Российского научного фонда № 22-2100213, https://rscf.ru/project/22-21-00213/
Литература
1. Кибзун, А.И. Задачи стохастического программирования с вероятностными критериями / А.И. Кибзун, Ю.С. Кан. - М.: Физматлит, 2009.
2. Raik, E. The Differentiability in the Parameter of the Probability Function and Optimization of the Probability Function Via the Stochastic Pseudogradient Method / E. Raik // Proceedings of Academy of Sciences of the Estonian SSR. Physics. Mathematics. - 1975. -V. 24, № 1. - P. 3-9.
3. Uryas'ev, S. Derivatives of Probability Functions and Some Applications / S. Uryas'ev // Annals of Operations Research. - 1995. - V. 56. - P. 287-311.
4. Henrion, R. Gradient Estimates for Gaussian Distribution Functions: Application to Probabilistically Constrained Optimization Problems / R. Henrion // Numerical Algebra, Control and Optimization. - 2012. - V. 2, № 4. - P. 655-668.
5. Pflug, G. Probability Gradient Estimation by Set-Valued Calculus and Applications in Network Design / G. Pflug, H. Weisshaupt // SIAM Journal on Optimization. - 2005. -V. 15, № 3. - P. 898-914.
6. Соболь, В.Р. О гладкой аппроксимации вероятностных критериев в задачах стохастического программирования / В.Р. Соболь, Р.О. Торишный // Труды СПИИРАН. - 2020. -Т. 19, № 1. - С. 180-217.
7. Sobol, V. Smooth Approximation of Probability and Quantile Functions: Vector Generalization and its Applications / V. Sobol, R. Torishnyi // Journal of Physics: Conference Series. - 2021. - V. 1925. - № 012034.
8. Торишный, Р.О. О применении численных методов второго порядка к задачам стохастического программирования с функцией вероятности /Р.О. Торишный // Труды МАИ. -2021. - № 121. - 27 c.
9. Кокс, Д. Теоретическая статистика / Д. Кокс, Д. Хинкли. - М.: Мир, 1978.
Виталий Романович Соболь, кандидат физико-математических наук, доцент, кафедра теории вероятностей и компьютерного моделирования, Московский авиационный институт (национальный исследовательский университет) (г. Москва, Российская Федерация), vitsobol@mail.ru.
Роман Олегович Торишный, ассистент, кафедра теории вероятностей и компьютерного моделирования, Московский авиационный институт (национальный исследовательский университет) (г. Москва, Российская Федерация), arenas-26@yandex.ru.
Поступила в редакцию 13 мая 2022 г.
