WorksMathematics
Statistical robustness and time consistency in risk averse optimization
Abstract
The aim of this manuscript is to investigate risk measures which are not necessarily convex. We show that law invariant coherent risk measures are not robust with respect to contamination of the probability measure, in the sense that an arbitrarily small contamination can change their values by an arbitrary amount. This contrasts with the Value-at-Risk, which is robust against such contamination. In addition, we show that for a given law invariant risk measure and a given policy, one can construct, under mild conditions, conditional risk measures with respect to which this policy is time consistent, in the frameworks of both Markov Decision Processes and Stochastic Optimal Control.
Introduction
In this manuscript, we study risk measures which do not necessarily satisfy the convexity axiom, and their use in sequential decision making under uncertainty. Modern theory of risk measures was started in [2], where the axioms of the so-called coherent risk measures were formulated. An important example of a coherent risk measure is the Average Value-at-Risk, which can be written as the infimum of a family of expectations [7]. More generally, spectral risk measures with monotonically nondecreasing spectral functions are coherent [1], and on a nonatomic probability space, law invariant coherent risk measures can be represented as the supremum of a set of coherent spectral risk measures (e.g., [12], Section 6.3.4). Coherent risk measures also possess the dual representation, which relates the risk averse and distributionally robust approaches to stochastic optimization (e.g., [12], Section 6.3). On the other hand, the Value-at-Risk, which is widely used in applications, does not satisfy the convexity axiom and does not have such dual representation.
The convexity of risk measures, however, comes at a price from the statistical point of view. It is well known that the Value-at-Risk is much less sensitive to outliers than the expected value (cf., [4, 8]). That is, when the distribution is contaminated by a small mass placed far away, the Value-at-Risk is not changed much, while the expected value can be moved by an arbitrary amount. It appears that the same property of statistical nonrobustness is shared by a large class of coherent risk measures. Indeed, the Average Value-at-Risk and spectral coherent risk measures are concave with respect to the probability measure, and consequently an arbitrarily small contamination of the distribution can increase their values by an arbitrary amount. Actually some authors argue that this is an advantage of the Average Value-at-Risk compared with the Value-at-Risk, since it accounts for possible large losses (cf., [7]). In any case, law invariant coherent risk measures are not robust from the statistical point of view, which motivates us to consider risk measures that do not satisfy the convexity axiom.
For sequential decision making, an approach to risk averse multistage stochastic programming based on nested compositions of conditional risk measures was suggested in [10], and was extended to Markov Decision Processes (MDPs) in [9]. A construction of nested risk functionals which does not rely on the convexity of risk measures was recently discussed in [13], for both Stochastic Optimal Control (SOC) and MDPs.
Closely related to the nested formulation is the time consistency of policies, that is, whether a policy optimal at the initial stage remains optimal at every later stage conditional on the history of the process. It is well known that the nested formulation is time consistent, while this is not the case for static formulations based on the Average Value-at-Risk. It should be noted that optimality at every later stage is measured by conditional risk measures, and hence time consistency depends on the choice of these conditional risk measures (e.g., [6]). For a given static risk measure, a natural choice is the one for which the static risk measure is equal to the composition of the conditional risk measures, since in that case the static problem can be written in the nested form. However, such a choice with law invariant coherent conditional risk measures exists only for the expectation and the max-risk measure (cf., [5], [11]), and in particular does not exist for the Average Value-at-Risk. In addition, even when such a choice exists, an optimal policy is not necessarily time consistent if the conditional risk measures are not strictly monotone [14, 15]. In this manuscript, for a given policy such that no policy is uniformly better, we construct conditional risk measures with respect to which this policy is time consistent, where the static risk measure is not necessarily convex.
The main contribution of this manuscript can be summarized as follows.
We show that risk measures concave in the probability measure, as well as their suprema, are not robust with respect to contamination of the probability measure. In particular, this
applies to spectral coherent risk measures, such as the Average Value-at-Risk, and to law invariant coherent risk measures on nonatomic probability spaces. This contrasts with the Value-at-Risk, which is robust against such contamination.
For an arbitrary law invariant risk measure satisfying the axioms of monotonicity, translation equivariance and positive homogeneity, and a policy such that no policy is uniformly better, we construct conditional risk measures with respect to which this policy is time consistent. The construction is based on the tail events on which the given policy is not worse than any other policy, and is carried out for both MDPs and SOC.
The remainder of this manuscript is organized as follows. In § 2, we discuss risk functionals which are not necessarily convex, and their concavity and statistical robustness with respect to the probability measure. In § 3.1 and § 3.2, we construct conditional risk measures with respect to which a given policy is time consistent, in the MDP and SOC settings, respectively. We conclude the manuscript in § 4.
We use the following notation and terminology. For , we denote . A set equipped with its sigma algebra is called a measurable space. By we denote the set of all probability measures on . For , is called the probability space. It is said that is nonatomic, if for any such that , there exists such that and . By we denote the Dirac measure of mass one at the point .
For and measurable variable we denote by its expected value, and by
its cumulative distribution function (cdf). By
we denote the left side quantile. In particular if we write and for the respective cdf and quantile. By we denote the set of right side continuous monotonically nondecreasing functions such that and , i.e. is the space of cdfs. By we denote the space of random variables such that
Risk functionals
Let be a measurable space. Assume that with every probability measure is associated a linear space of measurable functions (random variables) and a functional . Consider the following axioms (conditions) that may satisfy.
(A1) (monotonicity) if and , -a.s., then .
(A2) (translation equivariance) for and , it follows that .
(A3) (positive homogeneity) if and , then .
(A4) (convexity) if and , then .
In order to emphasize that the functional depends on , we write it as . The corresponding linear space may also depend on , but we suppress this in the notation. In any case, we assume that if and , then . A typical example of the space is , . Note that it follows from axiom (A1) that if , -a.s., then . It follows from axiom (A3) that .
Definition 2.1. It is said that functional is a risk measure if it satisfies axioms (A1) and (A2).
Functionals satisfying axioms (A1)-(A4) are called coherent risk measures [2]. An important example of a non-convex risk measure is the Value-at-Risk, , where
That is, is the left side -quantile of the distribution of . In that example the corresponding space consists of all measurable . The functional satisfies axioms (A1)-(A3), but not the convexity axiom (A4). An important example of a coherent risk measure is the Average Value-at-Risk (also called Conditional Value-at-Risk, expected shortfall, expected tail loss), for ,
The first of the above representations of the Average Value-at-Risk is due to [7]. In that example, the respective space is .
It is said that are distributionally equivalent (with respect to ), if their cdfs do coincide, i.e., for all .
Definition 2.2. It is said that is law invariant if for any distributionally equivalent it follows that .
That is, a law invariant risk measure is a function of the cdf , i.e., it can be represented as
where is a mapping from the space of cdfs into . For example, for the corresponding
defined for such that for all . For the corresponding mapping is
Risk measure is called spectral if it can be represented in the form
where is a density function on . Spectral risk measures were introduced in Acerbi [1]. Spectral risk measures are law invariant. It was shown in [1] that the spectral risk measure is coherent in the sense of [2], if the corresponding density is monotonically nondecreasing on . When the space , it is assumed that the spectral function belongs to the dual space .
Remark 2.1. Assuming that the probability space is nonatomic, it is possible to show that any law invariant coherent risk measure can be represented as the supremum of a set of coherent spectral risk measures (e.g., [12 Section 6.3.4]). This motivates us to consider risk measures of the form
where is a set of probability measures on the interval .
Definition 2.3. We refer to risk measures of the form (5) as the generalized risk measures. We say that a generalized risk measure is generalized spectral if the set is a singleton.
The generalized risk measure also depends on a choice of the set . In order to emphasize this, we sometimes write it as . The generalized risk measures are law invariant and satisfy axioms (A1)-(A3), but may not satisfy the convexity axiom (A4). By the above we have that a generalized risk measure is coherent if the probability measures have monotonically nondecreasing densities. An important example of a non-convex generalized spectral risk measure is , which can be written in the form (5) with being the singleton .
Concavity of risk measures
Definition 2.4. A law invariant functional is said to be concave in , if for and it follows for all that
Such concavity property of risk measures is useful, for example, in construction of upper bounds in risk averse SDDP type algorithms [3], and in establishing existence of non-randomized policies in risk averse MDPs [13].
There is a large class of risk functionals which are concave with respect to the probability distribution. For example consider risk functionals of the form , where is a subset of a finite dimensional vector space and is a real valued function. The risk measure is of that form. In that case is given by the infimum of functionals linear in , and hence is concave in . Spectral coherent risk measures are representable as the corresponding integral of the Average Value-at-Risk measures and hence are concave in .
We refer to the interval equipped with its Borel sigma algebra as the standard sample space. We can view generalized risk measures as defined on the standard sample space. Consider the Bernoulli probability measure , . With a law invariant risk measure and we can associate the function
Since for ,
it follows that if is concave in , then the function is concave on for all . For we write for the corresponding function.
Let us consider some examples. Suppose that . Then we have that for , for and for . Thus for , and for . Consider for some . Then for ,
Here cannot be represented as the supremum of functions concave on the interval , and hence cannot be represented as the supremum of law invariant risk measures concave in .
Consider now . In that case by (2) we have that for ,
Of course this function is concave on .
Statistical robustness of risk functionals
It is well known that the Value-at-Risk (quantile) functionals are much less sensitive to outliers than the expected value . That is, for a probability measure supported on the real line and consider the contamination of the original distribution by the scaled Dirac measure. Then
By choosing large enough, the contaminated expectation can be moved from the expectation by an arbitrary amount for any positive value of . On the other hand the measure will not be changed much by contamination for . The respective theory of the so-called Robust Statistics was well discussed already more than 40 years ago (cf., [4], [8]).
Suppose now that risk measure satisfies axioms (A1)-(A3) and is concave in . Then for ,
where the last equality holds since is the distribution of the constant , and hence by axioms (A2)-(A3). As in the expected value example, the risk can be moved from by an arbitrary amount for any positive value of . As was already mentioned, spectral coherent risk measures are concave in , and hence are not robust from the statistical point of view. Moreover, consider functionals , where is a family of risk measures such that for every the risk measure is law invariant, satisfies axioms (A1)-(A3) and is concave in . Then by (10)
and hence is not robust from the statistical point of view. In particular, it follows from Remark 2.1 that law invariant coherent risk measures on a nonatomic probability space are not robust from the statistical point of view.
Time consistency
In this section, we discuss time consistency of policies with respect to nested formulations of risk averse problems. Specifically, in § 3.1, for a given policy of an MDP such that no policy is uniformly better, we construct conditional risk measures with respect to which this policy is time consistent, and in § 3.2, we carry out a similar construction for SOC.
Markov Decision Processes
Consider a finite horizon MDP with state spaces , action spaces , cost functions , and transition probabilities , . A (non-randomized) policy is defined by mappings . A policy is Markovian if , , are functions of rather than the whole history . To a policy corresponds the probability distribution defined on histories with , (cf., [16]).
We say that a policy is time consistent if at every stage , conditional on the history , the policy is optimal. In order to make this precise we need to define what we mean by optimality at every stage . Suppose that for every the state space is a measurable space. Consider the set of Markovian policies. Let be a law invariant risk measure with being a probability measure defined on . Let , , be reference transition kernels. Then to a policy corresponds the nested functional given by the composition of conditional risk measures (cf., [13 Section 4]). Let us make the following observation.
Remark 3.1. Let be the set of probability measures on a measurable space and be a law invariant risk measure associated with . For a given and a set such that , consider the induced measure supported on , i.e., for any measurable set . Consider the corresponding risk measure . Then for any measurable such that for all , we have that and are distributionally equivalent with respect to and hence .
This motivates the following construction. Let be a given policy (for example, can be an optimal policy with respect to for the total cost). Assume that there is no policy uniformly better than , that is:
Assumption 3.1. For every policy and all there is a set of positive measure with respect to the reference probability measure on , such that for a.e. , where
Consider the probability measure on induced by . That is, for a measurable set ,
Note that the measure is supported on the set and depends on and . Consider the corresponding conditional risk measures . In the following, we show that under Assumption 3.1, policy is time consistent with respect to the nested formulation determined by risk measures constructed from .
Specifically, we construct risk measures such that is time consistent with respect to the corresponding nested formulation. Throughout the rest of our discussion, we assume that also satisfies axiom (A3), and hence for any probability measure . With the measures and the function defined above, we define the risk measure of as
For any , we can write the value function of under the nested formulation determined by recursively as and
Proposition 3.1. Suppose Assumption 3.1 holds. Then for any , and , we have
That is, is time consistent with respect to the nested formulation determined by .
Proof. Since and , we obtain from (12) with that for . Clearly the desired claim holds at . Suppose the claim holds at . Then we have
where (a) follows from Assumption 3.1 and (A1), (b) follows from (11), (c) follows from , and (d) follows from the induction hypothesis. Hence the claim holds at . The proof is then completed.
It could be noted that the construction of does not rely on the policies being Markovian. Consider history dependent policies with , and suppose that for every policy and every history there is a set of positive measure with respect to such that for a.e. . Then the conclusion of Proposition 3.1 holds for history dependent policies, by following the same construction with replaced by . The same applies to the SOC setting discussed in § 3.2 below.
Stochastic Optimal Control
Let us now consider the stochastic optimal control (SOC) problem
where the state evolves according to , , from a deterministic initial state , and the minimization is over policies that determine the controls . Here , and are nonempty closed sets, is the random data at stage , is the Borel sigma algebra of , and both the cost function and the mapping are continuous.
Throughout the rest of our discussion, we assume that the distribution of does not depend on states and controls, and is given by the product of the marginal distributions of on . For each , let us consider a law invariant risk measure $\mathcal{R}^{P_t} satisfying axioms (A1)-(A3). The value functions of the risk averse counterpart of the SOC problem then depend only on the state, and satisfy the dynamic programming equations and
where . In view of this, we restrict our attention to the set of Markovian policies , .
Similar to § 3.1, for a given policy , we proceed to construct risk measures such that is time consistent with respect to the corresponding nested formulation.
Assumption 3.2. For , any policy and any , there is a set such that and for a.e. , where
and .
For and , consider the probability measure on induced by , that is, for measurable . We then define the risk measure of as
For any , we can write the value function of under the nested formulation determined by recursively as and
Proposition 3.2. Suppose Assumption 3.2 holds. Then for any , and , we have
That is, is time consistent with respect to the nested formulation determined by .
Proof. Similar to the proof of Proposition 3.1, we obtain from (14) with and (13) that for . Clearly the desired claim holds at . Suppose the claim holds at . With , we have
where (a) follows from Assumption 3.2 and (A1), (b) follows from (13), (c) follows from , and (d) follows from the induction hypothesis with for any . Hence the claim holds at . The proof is then completed.
Concluding remarks
In this manuscript, we discuss risk measures which are not necessarily convex. We show that concavity in the probability measure, which holds for a large class of coherent risk measures, makes the risk sensitive to an arbitrarily small contamination of the distribution, and hence such risk measures are not robust from the statistical point of view. Our findings suggest that when statistical robustness is of concern, it is natural to consider risk measures without the convexity axiom, such as the Value-at-Risk.
For any law invariant risk measure satisfying axioms (A1)-(A3) and any policy such that no policy is uniformly better, we construct conditional risk measures with respect to which this policy is time consistent, in both the MDP and SOC frameworks. It should be noted that the constructed risk measures depend on the given policy and are not law invariant. The important question of how to solve the resulting non-convex problems numerically is beyond the scope of this manuscript, and can be a topic of future research.
Acknowledgments We used Claude (Anthropic) to produce initial drafts of selected paragraphs, based on the main results provided by the authors, and to polish the writing and correct typos. All content was subsequently revised and verified by the authors.
References
- [1]C. Acerbi. Spectral measures of risk: A coherent representation of subjective risk aversion. Journal of Banking & Finance, 26:1505–1518, 2002.DOI
- [2]P. Artzner, F. Delbaen, J.-M. Eber, and D. Heath. Coherent measures of risk. Mathematical Finance, 9:203–228, 1999.
- [3]V. Guigues, A. Shapiro, and Y. Cheng. Risk-averse stochastic optimal control: An efficiently computable statistical upper bound. Operations Research Letters, 51:393 – 400, 2023.arxiv.org/abs/2112.09757
- [4]P. Huber. Robust statistics. John Wiley & Sons, 1981.
- [5]M. Kupper and W. Schachermayer. Representation results for law invariant time consistent functions. Mathematics and Financial Economics, pages 189–210, 2009.DOI
- [6]A. Pichler, R. P. Liu, and A. Shapiro. Risk-averse stochastic programming: time consistency and optimal stopping. Operations Research, 70:2439–2455, 2022.arxiv.org/abs/1808.10807
- [7]R.T Rockafellar and S. Uryasev. Conditional value-at-risk for general loss distributions. J. of Banking and Finance, 26(7):1443–1471, 2002.DOI
- [8]P.J. Rousseeuw and A.M. Leroy. Robust Regression and Outlier Detection. John Wiley & Sons, 1987.
- [9]A. Ruszczyński. Risk-averse dynamic programming for Markov decision processes. Mathematical Programming, 125:235–261, 2010.DOI
- [10]A. Ruszczyński and A. Shapiro. Conditional risk mappings. Mathematics of Operations Research, 31:544–561, 2006.
- [11]A. Shapiro. Time consistency of dynamic risk measures. Operations Research Letters, 40:436–439, 2012.DOI
- [12]A. Shapiro, D. Dentcheva, and A. Ruszczyński. Lectures on Stochastic Programming: Modeling and Theory. SIAM, Philadelphia, third edition, 2021.
- [13]A. Shapiro and Y. Li. Risk-averse formulations of Stochastic Optimal Control and Markov Decision Processes. arXiv preprint arXiv:2505.16651, 2025.arxiv.org/abs/2505.16651
- [14]A. Shapiro and K. Ugurlu. Decomposability and time consistency of risk averse multistage programs. Operations Research Letters, 44:663–665, 2016.arxiv.org/abs/1806.01497
- [15]A. Shapiro and L. Xin. Time inconsistency of optimal policies of distributionally robust inventory models. Operations Research, 68:1576–1584, 2020.DOI
- [16]Ionescu Tulcea. Mesures dans les espaces produits. Atti Accad. Naz. Lincei Rend, 7:208 – 211, 1949.