WorksComputer Science
Risk-Averse Decision-Focused Learning: Minimizing CVaR of Regret
Abstract
Many combinatorial optimization (CO) problems involve parameters that are uncertain when the problem must be solved. In such settings, the parameters have to be predicted from correlated contextual information before a solution can be computed. Decision-focused learning (DFL) addresses this by training the predictive model to produce predictions that optimize downstream decision quality. Most existing DFL methods are risk-neutral and minimize the expected regret. In high-stakes applications this can lead to undesirable outcomes, because decision-makers must also account for the tail risk of decision quality. Prior risk-sensitive approaches typically model uncertainty explicitly and solve a stochastic or robust optimization problem at inference time, which increases computational complexity. We instead propose a DFL framework that achieves risk-sensitive behavior while retaining point predictions and deterministic optimization at inference time. Starting from the dual representation of the Conditional Value-at-Risk (CVaR) and adding entropy regularization, we derive a smooth surrogate for the CVaR of regret. Its gradient induces an instance-dependent reweighting within each minibatch that emphasizes high-risk outcomes, so the approach can turn existing surrogate-loss DFL methods, including the SPO+ loss, into risk-sensitive variants. When instantiated with SPO+, the resulting objective admits a generalization bound on the CVaR of the true regret. Experiments on knapsack, portfolio, shortest-path, and energy-aware scheduling problems show substantial reductions in tail regret when the uncertain parameters exhibit heterogeneous tail behavior.
Introduction
A wide range of real-world decision-making problems can be modeled as combinatorial optimization (CO) problems. Examples include network optimization [21], production planning [35], and vehicle routing [49]. In practice, these problems often depend on parameters that are unknown at decision time, such as traffic congestion in routing or future costs in production planning. To handle this uncertainty, there has been growing interest in treating such problems as contextual stochastic optimization problems [39]. This perspective gives rise to predict-then-optimize (PtO) frameworks [28], in which a machine learning (ML) model first predicts the unknown parameters from contextual features, and the resulting instantiated optimization problem is then solved.
A straightforward approach to PtO problems is prediction-focused learning (PFL), in which the ML model is trained to predict the unknown parameters as accurately as possible with a standard supervised loss such as the mean squared error (MSE). A key limitation of PFL is that it ignores how prediction errors affect the downstream decision, even though not all prediction errors are equally harmful. Consider a routing problem with three possible paths that take 10, 12, and 30 minutes to traverse. Predicting the travel time of the third path accurately is largely irrelevant, as long as it is predicted to be clearly worse than the other two. In contrast, a small error on the first two paths may change the chosen route. Because PFL penalizes errors of equal magnitude equally, it does not align learning with downstream decision quality.
Decision-focused learning (DFL) [28] addresses this limitation by embedding the optimization problem into the training loop of the predictive model. Instead of minimizing the error of the predicted parameters, DFL minimizes a decision loss such as the regret, which measures the suboptimality of the decisions induced by the predicted parameters. The model is thus trained specifically to make predictions that lead to good decisions. Gradient-based DFL requires differentiating through the CO problem, which is generally not possible because the optimal solution of a CO problem does not change smoothly with its parameters. A widely used remedy is to train with a surrogate loss [15, 31] whose subgradients provide useful descent directions for the regret.
Most existing DFL methods, however, adopt a risk-neutral perspective and minimize the expected regret. In many applications the average regret is not the primary concern. Decision-makers often operate in environments where rare but severe outcomes dominate operational risk. Consider dispatching emergency vehicles over a road network with uncertain travel times: a route that is fastest on average may occasionally incur extreme delays, which can be unacceptable in such a high-stakes setting. In these cases the objective should be to minimize the tail risk rather than the average regret.
Risk-sensitive decision making is commonly modeled with measures such as the Conditional Value-at-Risk (CVaR), which evaluates the average of the worst fraction of outcomes. Recent research has started to incorporate such risk measures into PtO. Existing approaches either learn a conditional distribution over the parameters and solve a risk-averse stochastic optimization problem by sample average approximation [50], or learn uncertainty or ambiguity sets and solve a robust or distributionally robust optimization problem at inference time [9, 24, 45]. Although these approaches target tail risk, they increase computational complexity, because a stochastic or (distributionally) robust optimization problem must be solved every time a decision is made [10].
Many applications, however, require fast inference, and solving a complex risk-averse optimization problem for every decision may be computationally prohibitive. This motivates the question: can risk-sensitive decision making be achieved while retaining simple point predictions and deterministic optimization at inference time?
To answer this question, we start from the dual formulation of CVaR and derive, through entropy regularization, a smooth surrogate for the tail regret. The derivation shows that risk-sensitive DFL can be achieved by a principled, instance-dependent reweighting of subgradients, where the weights follow from the first-order optimality conditions of the regularized objective. Instances with larger risk receive higher weight, while lower-risk instances remain informative rather than being discarded. We call this approach REweighted Subgradients for Tail Regret Optimization (RESTRO). RESTRO operates on top of existing DFL losses, such as the SPO+ surrogate loss [15], and thereby turns standard risk-neutral DFL methods into risk-sensitive variants without increasing the complexity of solving at inference time.
To evaluate the framework, we introduce benchmark datasets that expose the trade-off between expected regret and tail regret. Building on the PyEPO framework [47], we construct knapsack, portfolio, and shortest-path instances in which candidate solutions exhibit heterogeneous tail behavior: some solutions minimize the expected regret, while others minimize the CVaR of regret.
The contributions of this paper are as follows.
We derive a smooth, entropy-regularized surrogate for the CVaR of regret and show, via Danskin’s theorem, that its gradient is the gradient of the underlying DFL loss scaled by a closed-form instance weight. This yields RESTRO, a plug-in reweighting scheme for any surrogate-loss DFL method.
For the SPO+ instantiation, we prove a generalization bound: with high probability, the population CVaR of the true regret is upper-bounded by the empirical RESTRO objective plus a complexity term, and we quantify the bias introduced by the entropy smoothing.
We introduce benchmark datasets with heterogeneous long-tail cost behavior, in which minimizing expected regret and minimizing tail regret lead to different decisions.
In an extensive empirical study on knapsack, portfolio, shortest-path, and energy-aware scheduling problems, we show that RESTRO substantially reduces tail regret on heterogeneous data without sacrificing performance in standard risk-neutral settings.
The remainder of the paper is organized as follows. § 2 reviews related literature. § 3 presents the methodology, including the smooth CVaR surrogate, its gradient, the training procedure, and the generalization guarantee. § 4 reports the experimental evaluation, and § 5 concludes.
Related Literature
Decision-focused learning. Contextual stochastic optimization has received increasing attention in the operations research and ML communities, leading to the predict-then-optimize (PtO) framework. Bertsimas and Kallus [7] formalized PtO as a two-stage procedure in which a model first predicts uncertain parameters from contextual features and then solves an optimization problem with these estimates. Decision-focused learning (DFL), in contrast, trains the predictive model to optimize downstream decision quality directly. One family of DFL techniques achieves this through surrogate losses [31, 25, 15, 48, 4, 26]. Another family differentiates through a smoothed optimization problem, obtained either by regularization [27, 2] or by perturbation [36, 5, 33, 43]. To compute solutions and their gradients, smoothing-based methods rely on differentiable solvers such as cvxpylayers [1] and OptNet [2]. Approximate approaches such as DYS-Net (McKenzie et al. [30]) avoid exact optimization by using projected gradient methods. Mandi et al. [29] recently proposed using differentiable solvers to minimize surrogate losses, and Schutte et al. [42] proposed alternative loss functions that make the empirical estimation of regret more robust. Only few methods tackle directly the minimization of the regret through mathematical programming formulations (Bucarey López et al. [8]). All of these methods, however, target risk-neutral objectives and minimize the average regret.
Risk-sensitive decision-focused learning. Several recent works pursue risk-averse DFL by modeling uncertainty explicitly. Gen-DFL (Wang et al. [50]) uses generative models to learn a contextual distribution over high-risk regions and improves robustness by sampling at test time. Predict-then-Calibrate (Sun et al. [45]) constructs calibrated uncertainty sets that are used in a distributionally robust optimization (DRO) problem at inference. Differentiable DRO layers (Ma et al. [24]) and end-to-end conditional robust optimization (Chenreddy and Delage [9]) learn ambiguity or uncertainty sets and solve a (distributionally) robust optimization problem at test time. In contrast to these approaches, we directly train a point estimator to minimize the CVaR of regret. This enables risk-sensitive decision making with the computational efficiency of deterministic solving at inference time. § 3.2 further explains why minimizing the tail of the regret differs from minimizing the tail of the realized objective value, which is the target of Gen-DFL.
Distributionally robust and long-tailed learning. Our approach is also connected to distributionally robust stochastic optimization (Duchi and Namkoong [14]) for empirical risk minimization in ML. This line of work (Sagawa* et al. 2020, Qi et al., Li et al. [23]) minimizes the worst-case rather than the average training loss in order to improve generalization to unseen test instances (Fidon et al. [16], Namkoong and Duchi [32], Duchi and Namkoong [13]). Several works in this area, such as Equalization Loss (Tan et al. [46]) and DropLoss (Hsieh et al. [18]), study reweighting strategies for long-tailed learning (Samuel and Chechik [41]). Among these, re-weighted gradient descent (Kumar et al. [22]) is most closely related to our approach, as it also reweights the instances of each minibatch during gradient updates.
Methodology
This section first introduces the PtO setting and the SPO+ surrogate loss (§ 3.1). § 3.2 explains why we target the tail of the regret rather than the tail of the realized objective value. § 3.4–§ 3.6 derive the smooth CVaR surrogate, its gradient, and the resulting training procedure. § 3.7 establishes a generalization guarantee for the SPO+ instantiation.
Preliminaries: Decision-Focused Learning
We consider combinatorial optimization (CO) problems with a linear objective function, of the form
where is the cost vector and is the feasible set. We denote the optimal objective value by and write for when the argument is clear from context. In the PtO setting, the cost vector is unknown at decision time. Instead, we observe contextual features and predict the cost vector as , where is a predictive model, typically a neural network or a linear regressor, with parameters . The quality of a prediction is not measured by how well it approximates , but by how good the resulting decision is under the true cost vector . This is captured by the regret, the excess cost incurred by deciding with the predicted rather than the true cost vector. Given a dataset of feature–cost pairs , the empirical average regret is
Training therefore seeks parameters that minimize the empirical average regret (2). Direct gradient-based minimization of (2) is, however, not possible. This is because for CO problems, is a piecewise constant function of . Small changes in the predicted costs typically do not change the optimal solution, so the gradient with respect to is zero almost everywhere.
The SPO+ surrogate loss. The non-differentiability of the regret is commonly addressed with surrogate losses that approximate the regret while providing informative (sub)gradients. A widely used surrogate is the SPO+ loss (Elmachtoub and Grigas [15]), which we use as the main instantiation in this paper. It is defined through the perturbed cost vector and the corresponding decision :
A subgradient with respect to the predicted cost is
An important property of SPO+ is that it upper-bounds the regret pointwise, that is, for all and [15]. We use this property in § 3.7. Training with SPO+ minimizes the empirical average surrogate loss
whose gradient with respect to the model parameters follows from the chain rule:
This formulation minimizes the average regret over the dataset. In the remainder of this section, we adapt the framework to minimize the CVaR of the regret instead.
Conditional Value-at-Risk. For a random loss function and a risk level , measures the expected loss in the worst fraction of outcomes, beyond the -quantile (the Value-at-Risk). Its Rockafellar–Uryasev representation [38] is
where . For observed losses , replacing the expectation by the sample average gives the empirical CVaR
A minimizer of (8) is the empirical -quantile of the losses, which we denote by . When is an integer, is the average of the largest losses.
In this paper, instead of the average in (2), we aim to minimize the empirical CVaR of regret:
For example, with , this is the average regret over the 10% of instances with the highest regret.
Motivation: Tail Regret Versus Tail Objective Value
Our objective differs from robustness objectives that minimize the tail risk of the realized objective value, such as the one targeted by Gen-DFL [50]. The distinction is subtle but important: such objectives focus on adverse outcome scenarios, whereas we focus on adverse regret scenarios. The two do not in general lead to the same optimizer.
Let denote a set of cost scenarios. For a candidate decision , the objective value in scenario is . A tail-risk objective over outcomes evaluates the tail of . For instance, if we use CVaR as the risk measure, we will consider the CVaR of the realized objectives across all scenarios in the following way:
This objective asks whether the decision has a low cost in adverse scenarios. It does not ask whether the decision is close to the best possible decision in each scenario. Regret, in contrast, compares the decision against the scenario-specific optimum . The regret of in scenario is , and our risk-sensitive objective focuses on the tail of these values:
This objective asks in which scenarios the decision is most suboptimal relative to the best decision available for that scenario. Because the scenarios that form the tail in (11) need not be those that form the tail in (10), minimizing the tail objective value and minimizing the tail regret are generally different problems:
The distinction matters in practice. In portfolio optimization, a tail-risk objective over realized returns focuses on scenarios in which all assets perform poorly. This can lead to highly conservative decisions, such as investing in only a small subset of assets or avoiding risky assets altogether. Such a decision may perform well in extreme loss scenarios, but it gives up large gains in favorable scenarios, where the scenario-wise optimal portfolio achieves a much higher return. The conservative decision can therefore incur a large regret: it protects against adverse outcomes in an absolute sense, but it does not stay close to the scenario-wise optimal decision.
Illustrative example. Consider a shortest-path problem with two routes, A and B, and two scenarios: a normal scenario with probability 0.9 and a rare scenario with probability 0.1. Route A always takes 30 minutes. But Route B takes 20 minutes in the normal scenario and 60 minutes in the rare scenario, for instance due to a railway crossing. Route A therefore has a regret of 10 minutes in the normal scenario and zero in the rare scenario. Route B has zero regret in the normal scenario and 30 minutes in the rare scenario. The CVaR of the regret is 10 for Route A, whose regret never exceeds 10, and 30 for Route B. The CVaR of the travel time is 30 for Route A and 60 for Route B. Therefore, both CVaR objectives prefer Route A.
Now suppose we replace Route B with Route B′, which takes 40 minutes in the normal scenario but just 5 minutes in the rare scenario, for instance because a tunnel that is rarely open is very fast when available. In this example, Route A incurs zero regret in the normal scenario and 25 minutes of regret in the rare scenario. Route B′, on the other hand, has 10 minutes of regret in the normal scenario and zero regret in the rare scenario. The CVaR of the travel time prefers Route A to Route B′, 30 against 40. The CVaR of the regret, however, prefers Route B′ to Route A, 10 against 25. Table 1 summarizes both cases. Minimizing the CVaR of the objective value and minimizing the CVaR of the regret can therefore lead to different decisions.
| Case 1 | Case 2 | |||
| Travel times and regrets (in minutes) in the illustrative example. Bold entries mark the route preferred by each CVaR objective. | A | B | A | B′ |
| Travel time, normal scenario () | 30 | 20 | 30 | 40 |
| Travel time, rare scenario () | 30 | 60 | 30 | 5 |
| Regret, normal scenario | 10 | 0 | 0 | 10 |
| Regret, rare scenario | 0 | 30 | 25 | 0 |
| CVaR of travel time | 30 | 60 | 30 | 40 |
| CVaR of regret | 10 | 30 | 25 | 10 |
Table 1. Travel times and regrets (in minutes) in the illustrative example. Bold entries mark the route preferred by each CVaR objective.
From scenarios to instances
In the illustrative example, the CVaR of both regret and travel time is evaluated across two scenarios. In PtO problems, these scenarios correspond to realisations of the cost parameters drawn from their conditional distribution given the contextual features. In practice, however, we observe only one realised scenario for each context. Consider an ambulance dispatcher who chooses a route each morning. The dispatcher observes only the travel times that realized that morning. Alternative travel times under the same context remain unobserved. Thus, each context in the training set is paired with a single realised cost vector. Hence we cannot directly compute the empirical CVaR for each context. We can, however, compute the empirical CVaR of regret across the instances. We will show that this measure yields an upper bound on the average per-context CVaR of regret.
To make this precise, let and . The decision is a deterministic function of the context, so the model, , defines a decision policy, and is the regret this policy incurs. We first formalize these two tail measures and then illustrate their differences.
The per-instance tail computes the CVaR of regret under the conditional distribution of given , then takes the expectation of this conditional CVaR over the marginal distribution of :
The joint-distribution tail computes the CVaR of regret under the joint distribution of and :
Both expressions follow from the Rockafellar–Uryasev representation (7). They differ in the threshold . In (14), each context has its own threshold, so the tail is formed within each context. In (15), one threshold is formed across contexts. RESTRO computes a single threshold per minibatch and therefore targets the joint distribution tail. The per-instance tail generalizes the notion of CVaR in the example. The per-instance tail cannot be estimated from one observation per context without modeling the conditional distribution of given . Generative and robust approaches take this route Wang et al. [50], Chenreddy and Delage [9], at the cost of solving a stochastic or robust problem at inference time. The empirical CVaR in (9), on the other hand, estimates the CVaR from i.i.d. samples drawn from the joint distribution Soma and Yoshida [44].
We highlight that for risk-neutral regret, per-instance and joint-distribution measures coincide, because . That is why the distinction between per-instance and joint distribution need not be considered in risk-neutral DFL. However, we have to consider their difference, as we deal with CVaR. We next prove that CVaR computed using the joint-distribution yields an upper bound on the average per-context CVaR.
Lemma 1. For every and every ,
Proof. For a threshold and a context , define
Now, for any , we can write the following using linearity of expectation
From the definition of CVaR:
Taking the expectation over on both sides preserves the inequality:
Note that this is true for all values of . So,
Equality. Suppose the conditional distribution of is the same for every . Then does not depend on , and both sides equal .
Lemma 1 has two implications. First, for every , the joint distribution tail upper-bounds the average per-instance tail. The objective (9) estimates the joint distribution tail, so a small value of (9) also implies a small average per-instance tail, without modeling the conditional distribution of given . The two tails coincide when the conditional distribution of the regret does not depend on the context. Second, when this distribution varies across contexts, the joint distribution tail is more conservative. Under a shared threshold, instances from contexts in which the policy incurs high regret exceed the threshold more often. The tail therefore contains both unfavorable realizations within a context and contexts in which the policy performs poorly. For a decision-maker who faces a stream of instances, joint distribution tail is the right metric to minimize. This choice follows standard practice in risk-averse machine learning, where CVaR is computed across training examples rather than over the conditional distribution of the output for each input ([11, 44]).
A Smooth Surrogate of the CVaR of Regret
The objective (9) cannot be minimized directly with gradients, for two reasons. First, each regret term is piecewise constant in , as discussed in § 3.1. Second, the empirical CVaR is non-smooth in the losses: the set of instances that form the tail changes abruptly as the losses change. A surrogate such as SPO+ addresses the first issue by replacing the regret in each term. In this paper, we address the second issue by deriving a smooth surrogate of the CVaR itself.
We first reformulate the empirical CVaR through its dual representation, which introduces instance weights, and then add entropy regularization that smooths the otherwise binary weight assignment. The dual ([3]) of the empirical CVaR can be expressed in the following form:
Let . Substituting gives and , so that
From this perspective, CVaR is a reweighting of the training instances that has higher weight on high-loss instances and lower weight on low-loss ones. The variable is the weight of instance in the CVaR objective. The constraints distribute a total weight of over the dataset, with at most unit weight per instance, and maximizing concentrates this budget on the instances with the highest losses, effectively selecting the worst fraction of samples. Because a linear objective over a polytope attains its maximum at a vertex, the optimal weights are binary: each instance is either fully selected () or discarded ().1 The resulting objective therefore depends non-smoothly on the losses, since small changes in can abruptly change which instances are selected. To obtain a smooth surrogate, we replace this hard selection with soft weights by adding a negative binary-entropy term, which encourages interior solutions :
The parameter controls the sharpness of the relaxation. A large yields near-binary instance weights and a solution close to the original CVaR objective, that is, a hard selection of the worst fraction of samples. A small induces stronger smoothing and spreads the weight more evenly across samples.
To derive the optimal instance weights, we form the Lagrangian associated with the equality constraint,
and set its derivative with respect to to zero, which characterizes the interior optimum:
Solving for the optimal weight gives
where is the sigmoid function.
The threshold : exact computation and quantile approximation. The Lagrange multiplier is determined by the budget constraint, that is, it is the solution of
The left-hand side is continuous and strictly decreasing in , and it ranges from to 0, so (22) has a unique solution that can be computed to any precision by bisection. As , the sigmoid approaches a step function, and approaches the empirical -quantile of the losses, which is the minimizer in (8). In practice, we therefore approximate the threshold by , which avoids solving (22) in every minibatch. § 4.4 shows empirically that both choices lead to nearly identical results and that the approximation is generally faster. With either threshold, the smooth CVaR surrogate reads
Normalization. To make the sharpness parameter independent of the absolute scale of the losses, we can normalize the losses by the threshold, which gives
Equation (24) is equivalent to (23) with the sharpness rescaled to . It assigns high weight to instances whose loss exceeds the threshold and low weight to those below it, and controls the sharpness of this weighting. Normalization is a practical scaling choice rather than an essential ingredient; § 4.5 shows that it does not materially change the results. Unless stated otherwise, the experiments use the normalized form (24) with the quantile threshold .
Gradient of the Smooth Surrogate
The smooth CVaR objective depends on the model parameters through the instance losses , which are functions of the predicted costs . We now derive its gradient with respect to these predictions, which enables gradient-based training of the predictive model.
Define the smooth CVaR surrogate as the optimal value of the entropy-regularized problem (18),
where and is the instance-level DFL loss (e.g., ). We use Danskin’s theorem [12, 6]: if is concave in and admits a unique maximizer , then the gradient of the max-function equals the partial derivative of with respect to , evaluated at the maximizer. The entropy term is strictly concave and the constraint set is convex, so both conditions hold.
To see why the dependence of on can be ignored, apply the chain rule:
At the optimum, the first-order conditions of the inner maximization (Equation (20)) make the derivative of the Lagrangian with respect to each vanish. Moreover, since for all , the multiplier contribution is zero, so the indirect effect (the second term) vanishes. The entropy term does not depend on , so its partial derivative with respect to is also zero. Only the direct effect remains, and
Hence the optimal weights can be treated as constants during backpropagation. Substituting the closed-form weights of (24) gives the explicit gradient
This reveals a key practical advantage: the risk-sensitive gradient is simply the gradient of the loss scaled by the instance weight . The approach can therefore be applied as a plug-in to any existing DFL loss by reweighting the gradients before the parameter update. For example, with the SPO+ surrogate, (4) gives
which is the standard SPO+ subgradient scaled by the CVaR-induced instance weight.
Risk-Sensitive Decision-Focused Training
Algorithm 1 summarizes the resulting risk-sensitive DFL procedure. The standard DFL pipeline of predicting, optimizing, and computing the DFL loss remains unchanged. Our modification introduces instance-dependent weights that rescale the gradients according to each instance’s contribution to the tail of the decision regret.
In practice, we optimize a stochastic approximation of the empirical CVaR objective using minibatches. At each iteration, we compute the CVaR statistics, including the threshold , over the current minibatch rather than over the full dataset. This yields a minibatch-based stochastic estimate of the CVaR gradient and allows scalable training. We refer to the resulting framework as REweighted Subgradients for Tail Regret Optimization (RESTRO). It can be combined with any decision-focused loss. Our main instantiation uses the SPO+ surrogate and is denoted by .
Generalization Guarantee for RESTRO with SPO+
Although the reweighting mechanism applies to any DFL loss, we will prove generalization guarantee considering SPO+ as the DFL loss. Our proof rests on a chain of three upper bounds. First, because SPO+ upper-bounds the regret pointwise, the CVaR of the SPO+ loss upper-bounds the CVaR of the regret. Second, the population CVaR of the SPO+ loss is close to its empirical CVaR, uniformly over the hypothesis class. Third, the entropy-regularized objective that RESTRO minimizes upper-bounds the empirical CVaR of the SPO+ loss. therefore minimizes a controlled upper bound on the tail regret. This section makes the argument precise. Surrogates such as IMLE and PFY do not upper-bound the regret, so the generalization guarantee does not extend to them directly.
Algorithm 1 Decision-focused learning with risk-sensitive reweighting (RESTRO)
Require: Training data , model , risk level , sharpness , learning rate
while not converged do
Sample minibatch
for do
DFL loss of choice (e.g., SPO+)
end for
or solve (22) by bisection
for all
treated as constants
end while
Notation. We write for a feature–cost pair drawn from a distribution , and , , for the training data. The predictive model ranges over the hypothesis class , and we write for a generic element of . For , we define the regret and the SPO+ loss of a pair as
so that the instance losses of § 3.4 are . Since this section only concerns SPO+, we omit the superscript on . For a random variable , is defined by (7), and we write for the empirical CVaR (8) of the losses . As in § 3.4, .
The RESTRO objective. Let denote the binary entropy, with . We define the RESTRO objective as
The entropy term is the same as the one in (18), written with . Closing the box constraints does not change the maximum, because the maximizer (21) is interior. Hence , where is the smooth surrogate (25) with . By (27), its gradient is , which is the RESTRO update direction up to the constant factor . The guarantee below therefore applies to the objective whose gradient RESTRO follows, computed on the full training set with the exact threshold and the unnormalized weights (23). The minibatch quantile threshold and the normalization (24) are practical approximations of this objective that the bound does not cover. § 4 shows that they lead to nearly identical results.
Assumption 1 (Standing assumptions)
(A1) are i.i.d. draws from , and .
(A2) The feasible set is compact, the cost vector is bounded almost surely, and the hypothesis class is uniformly bounded. Consequently, there exists such that
The middle inequality is the pointwise upper bound of SPO+ on the regret (§ 3.1 and Elmachtoub and Grigas [15]).
(A3) The sharpness parameter satisfies .
The bound is stated in terms of the complexity of the class of hinge-transformed SPO+ losses that appears in the Rockafellar–Uryasev representation (7):
where plays the role of the threshold variable in (7). Let , and let
denote the empirical and the expected Rademacher complexity of , where are i.i.d. Rademacher variables.
Proposition 1 (Generalization bound for RESTRO with SPO+). Let Assumption 1 hold, and let . Then, with probability at least , simultaneously for every ,
Proof. Step 1: the entropy term is slack. Since on and , every feasible in (30) satisfies . Taking the maximum over the feasible set, dividing by , and using the dual representation (17) of the empirical CVaR gives
Step 2: Rockafellar–Uryasev representation on a bounded range. For a random variable , let , so that by (7). If almost surely, the minimum is attained on . For , we have , hence , which is nonincreasing in because . For , we have , hence . Applying this to under and to the empirical distribution of gives
Step 3: one-sided comparison of minima. For any functions and , if attains , then . Taking and as the objectives of the two displays in Step 2, the terms cancel, and
where .
Step 4: uniform convergence over . The supremum in (32) is bounded by the supremum over all of , which ranges over and jointly. Every takes values in , so replacing a single sample changes by at most . By McDiarmid’s inequality and symmetrization, with probability at least ,
Combining (32) and (33) gives, with probability at least and simultaneously for all ,
Step 5: conclusion. Chaining (31) into (34) gives . By (A2), almost surely, and is monotone, so , which yields the claim. □
:::
The bound in Proposition 1 involves the expected Rademacher complexity , which cannot be computed from the data. The following remark gives a fully data-dependent version.
Remark 1 (Data-dependent version). Replacing one sample point changes by at most , since every takes values in . Applying McDiarmid’s inequality to the lower tail, which excludes the event that the empirical complexity is unusually small, gives
Setting the right-hand side to gives, with probability at least ,
Applying (33) with in place of and taking a union bound, with probability at least ,
Proposition 1 therefore also holds with
Inequality (31) is one-sided, since the entropy term can only increase the objective. The next remark quantifies how much the smoothing can overestimate the empirical CVaR, and how the sharpness parameter should scale with to keep this bias below the statistical error.
Remark 2 (Smoothing bias of the entropy term). The gap in (31) is controlled by rather than by . By concavity of and , Jensen’s inequality gives , hence
The smoothing bias is therefore at most of order , and this bound does not improve with . Keeping it at or below the statistical rate requires . With this scaling, the RESTRO objective is a consistent surrogate for the empirical CVaR.
Experimental Evaluation
This section evaluates RESTRO empirically. Section 4.1 describes the benchmark problems and our heterogeneous long-tail data generation, and Section 4.2 the experimental setup. Section 4.3 reports the main results. The remaining subsections compare the exact and approximate thresholds (Section 4.4), and an energy-aware scheduling problem (Section 4.6), and study the effect of normalization (Section 4.5).
Synthetic Cost Generation with Heterogeneous Long-Tail Perturbations
We build on PyEPO ([47]), a widely used benchmark for PtO problems. PyEPO generates cost vectors for CO problems synthetically through a nonlinear mapping of contextual features. In the shortest-path setting, for example, the cost of edge in instance is generated as
where is the contextual feature vector of instance , is a random binary matrix shared across all instances, and is a multiplicative noise term sampled independently for each cost coefficient. In this construction, all costs are generated with the same transformation and noise model, so the induced distributions of the cost coefficients share the same parametric form and differ mainly in scale. As a result, the expectation and the CVaR tend to be positively correlated, and decisions optimized for average performance often remain optimal with respect to tail risk. To create a genuine mismatch between average performance and tail risk, we introduce datasets in which the decisions that are optimal on average differ from those that are optimal under tail-risk criteria.
Tail-aware synthetic data generation. We generate datasets for three CO problems: shortest path, knapsack, and portfolio optimization. For all three, we modify the PyEPO generators to produce cost vectors with heterogeneous long-tail behavior. The modification is heterogeneous because it applies only to a subset of the cost coefficients: the coefficients in follow long-tailed distributions, while the others follow the usual PyEPO distributions. The aim is to create coefficients that look attractive under average performance but become undesirable under tail-risk criteria. This setting mimics real-world environments in which only some assets, routes, or items are exposed to rare but severe disruptions, while the others show regular variability.
Shortest path. We first generate with the standard PyEPO nonlinear transformation (36). For the special edges , we rescale the cost as
where with probability and with probability , and . Since , these edges have a lower average cost but can incur large costs in the tail. We then apply the PyEPO multiplicative noise to all edges.
Knapsack. For knapsack, we use the PyEPO construction of the base item values and rescale the values of the special items as in the shortest-path problem, with with probability , with probability , and . The special items are taken from the high-weight end of the item ordering, so they are resource intensive. Since , these items are profitable on average, but in 20% of the cases they contribute little value while still consuming substantial capacity. We then apply the PyEPO nonlinear rescaling and multiplicative noise.
Portfolio. For portfolio optimization, the base mean returns are computed with the PyEPO mapping
For the special assets , we rescale the base mean return as , where with probability 0.8, with probability 0.2, and . Since , these assets have higher expected returns but can generate losses in the adverse regime. The final returns add factor-driven and idiosyncratic noise, , where , , and is the noise level.
Experimental Setup
We evaluate the RESTRO framework instantiated with the SPO+ loss, denoted , on portfolio optimization with 50 assets, shortest path on a grid, and knapsack with 50 items. We compare against a prediction-focused baseline trained with MSE, standard SPO+ (Elmachtoub and Grigas [15]). We report results on the standard PyEPO datasets with homogeneous cost vectors and on our heterogeneous long-tail datasets.
For each problem, we use 800 training instances and 200 validation instances to tune the method-specific hyperparameters: the learning rate, the minibatch size, and, for , the sharpness . Table 2 lists the selected hyperparameters. After hyperparameter selection, we evaluate on 1000 test instances. We compute the regret of each test instance and report the CVaR with over the 1000 regret values. We repeat this procedure for 5 random seeds, with independently generated training and test instances for each seed, and report the mean and standard deviation of the CVaR across the seeds.
| Problem | Data | Method | Batch size | Learning rate | |
| Portfolio (50 assets) | Heterogeneous | MSE | 32 | – | 0.005 |
| Portfolio (50 assets) | Heterogeneous | SPO+ | 128 | – | 0.005 |
| Portfolio (50 assets) | Heterogeneous | (exact ) | 128 | 50 | 0.005 |
| Portfolio (50 assets) | Heterogeneous | (quantile ) | 128 | 10 | 0.005 |
| Portfolio (50 assets) | Homogeneous | MSE | 128 | – | 0.005 |
| Portfolio (50 assets) | Homogeneous | SPO+ | 64 | – | 0.005 |
| Portfolio (50 assets) | Homogeneous | (exact ) | 64 | 1 | 0.005 |
| Portfolio (50 assets) | Homogeneous | (quantile ) | 64 | 0.1 | 0.005 |
| Shortest path ( grid) | Heterogeneous | MSE | 128 | – | 0.005 |
| Shortest path ( grid) | Heterogeneous | SPO+ | 128 | – | 0.05 |
| Shortest path ( grid) | Heterogeneous | (exact ) | 64 | 10 | 0.005 |
| Shortest path ( grid) | Heterogeneous | (quantile ) | 64 | 50 | 0.005 |
| Shortest path ( grid) | Homogeneous | MSE | 64 | – | 0.005 |
| Shortest path ( grid) | Homogeneous | SPO+ | 128 | – | 0.005 |
| Shortest path ( grid) | Homogeneous | (exact ) | 128 | 0.1 | 0.005 |
| Shortest path ( grid) | Homogeneous | (quantile ) | 128 | 0.1 | 0.005 |
| Knapsack (50 items) | Heterogeneous | MSE | 128 | – | 0.05 |
| Knapsack (50 items) | Heterogeneous | SPO+ | 64 | – | 0.05 |
| Knapsack (50 items) | Heterogeneous | (exact ) | 32 | 50 | 0.005 |
| Knapsack (50 items) | Heterogeneous | (quantile ) | 32 | 10 | 0.005 |
| Knapsack (50 items) | Homogeneous | MSE | 64 | – | 0.005 |
| Knapsack (50 items) | Homogeneous | SPO+ | 64 | – | 0.05 |
| Knapsack (50 items) | Homogeneous | (exact ) | 32 | 0.1 | 0.05 |
| Knapsack (50 items) | Homogeneous | (quantile ) | 128 | 1 | 0.05 |
Table 2. Selected Hyperparameters for Each Problem, Dataset, and Method. Hyperparameters were selected on 200 validation instances after training on 800 instances.
The ML models are implemented in PyTorch (Paszke et al. [34]), and the CO problems are solved with Gurobi (Gurobi Optimization [17]). The experiments were run on an Intel i7-13800H CPU with 14 cores and 20 threads, and 32 GB of RAM.
Experimental Results with homogeneous and heterogeneous datasets
Table 3 reports the results on the standard PyEPO datasets with homogeneous cost vectors, and Table 4 the results on our heterogeneous long-tail datasets.
| Problem | MSE | SPO+ | (exact ) | (quantile ) |
| Portfolio | 0.00 ± 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 |
| Shortest path | 0.53 ± 0.11 | 0.38 ± 0.07 | 0.38 ± 0.06 | 0.39 ± 0.07 |
| Knapsack | 2.30 ± 0.30 | 1.61 ± 0.07 | 1.61 ± 0.11 | 1.62 ± 0.09 |
Table 3. CVaR of Regret on the PyEPO Datasets with Homogeneous Cost Vectors. Mean ± standard deviation over 5 random seeds; each value is the CVaR over 1000 test instances.
| Problem | MSE | SPO+ | (exact ) | (quantile ) |
| Portfolio | 0.43 ± 0.01 | 0.42 ± 0.01 | 0.39 ± 0.01 | 0.39 ± 0.01 |
| Shortest path | 5.36 ± 0.83 | 11.04 ± 0.96 | 4.99 ± 0.63 | 5.07 ± 0.55 |
| Knapsack | 10.73 ± 1.50 | 20.54 ± 5.11 | 8.18 ± 0.84 | 8.07 ± 0.90 |
Table 4. CVaR of Regret on the Datasets with Heterogeneous Long-Tail Cost Vectors. Mean ± standard deviation over 5 random seeds; each value is the CVaR over 1000 test instances.
Performance on homogeneous cost vectors. On the homogeneous datasets, SPO+ already achieves a very low CVaR of regret, which leaves little room for improvement. This is expected: on these datasets, average performance and tail risk are closely aligned, so decisions that are near-optimal in expectation also tend to perform well under the CVaR criterion. Consistently, hyperparameter tuning typically selected small values of , which produce nearly uniform instance weights and make RESTRO behave similarly to SPO+.
Performance on heterogeneous cost vectors. On the heterogeneous benchmarks (Table 4), achieves the lowest CVaR of regret on shortest path and knapsack, and it is competitive with the best method on portfolio. Because SPO+ weights all instances equally, it achieves a low mean regret but performs poorly on the long-tail instances with high regret. We highlight that RESTRO achieves very low CVaR of regret by assigning more weight to instances that induce high regret. These results support subgradient reweighting as an effective way to minimize the CVaR of regret.
Exact Versus Approximate Threshold
In the main experiments, the threshold is approximated by the empirical -quantile of the minibatch losses. Here we compare this approximation with the exact threshold , obtained by solving (22) by bisection in every minibatch. Table 5 reports the CVaR of regret and the training time for both variants, together with MSE and SPO+.
| Problem | Method | Heterogeneous CVaR | Heterogeneous Time (s) | Homogeneous CVaR | Homogeneous Time (s) |
| Shortest path | MSE | ||||
| Shortest path | SPO+ | ||||
| Shortest path | (exact ) | ||||
| Shortest path | (quantile ) | ||||
| Knapsack | MSE | ||||
| Knapsack | SPO+ | ||||
| Knapsack | (exact ) | ||||
| Knapsack | (quantile ) | ||||
| Portfolio | MSE | ||||
| Portfolio | SPO+ | ||||
| Portfolio | (exact ) | ||||
| Portfolio | (quantile ) |
Table 5. Exact Threshold Versus Quantile Approximation : CVaR of Regret and Training Time (s).
The exact threshold and the quantile approximation produce nearly identical CVaR values on all three problems and in both data regimes. The small differences are within the reported variability and do not change any conclusion. The quantile approximation is also generally faster, because it avoids solving the threshold equation by bisection in every minibatch. The approximation used in the main experiments thus preserves the performance of the exact formulation while reducing its computational cost.
Effect of Normalization
The main experiments normalize the losses by the threshold, as in (24), to control the scale of the sigmoid transition and reduce the dependence of on the absolute magnitude of the losses. We test whether this normalization materially affects the results by comparing the normalized form (24) with the unnormalized form (23), each with the exact threshold and with the quantile approximation. Table 6 shows that normalization does not materially change the CVaR values in any of the tested settings, and the small runtime differences are not systematic. The empirical gains of RESTRO therefore come from the tail-aware reweighting rather than from the normalization, which is a practical scaling choice.
| Problem | Normalized | Heterogeneous CVaR | Heterogeneous Time (s) | Homogeneous CVaR | Homogeneous Time (s) | |
| Shortest path | Exact | No | ||||
| Shortest path | Exact | Yes | ||||
| Shortest path | Quantile | No | ||||
| Shortest path | Quantile | Yes | ||||
| Knapsack | Exact | No | ||||
| Knapsack | Exact | Yes | ||||
| Knapsack | Quantile | No | ||||
| Knapsack | Quantile | Yes | ||||
| Portfolio | Exact | No | ||||
| Portfolio | Exact | Yes | ||||
| Portfolio | Quantile | No | ||||
| Portfolio | Quantile | Yes |
Table 6. Effect of Normalizing the Losses by the Threshold: CVaR of Regret and Training Time (s) of .
Energy-Aware Scheduling
Finally, to test RESTRO beyond the PyEPO generators, we evaluate it on the energy-cost-aware scheduling benchmark used in the DFL literature ((Mandl et al. 2024)), with an instance of 10 tasks. The cost parameters are electricity prices that must be predicted from historical features. We consider the original data and a heterogeneous variant with long-tail perturbations. Table 7 reports the results.
| Data | Method | CVaR | Time (s) |
| Original | MSE | 166,331.07 | 35.90 |
| Original | SPO+ (cached) | 159,213.45 | 98.10 |
| Original | SPO+ (non-cached) | 159,032.17 | 1,102.86 |
| Original | (exact ) | 157,237.15 | 106.83 |
| Original | (quantile ) | 157,176.50 | 97.88 |
| Heterogeneous | MSE | 242,105.05 | 35.41 |
| Heterogeneous | SPO+ (cached) | 238,957.60 | 103.35 |
| Heterogeneous | (exact ) | 231,704.70 | 108.97 |
| Heterogeneous | (quantile ) | 227,848.71 | 107.40 |
Table 7. Energy-Aware Scheduling with 10 Tasks: CVaR of Regret and Training Time (s).
On the original data, all DFL methods improve over MSE, and RESTRO slightly improves over SPO+. On the heterogeneous variant whereas RESTRO reduces the CVaR of regret below both MSE and SPO+. We have observed the same behavior on the synthetic heterogeneous datasets.
Mean standard deviation over 5 random seeds. Shortest path on a grid, knapsack with 50 items, portfolio with 50 assets. The exact variant uses the unnormalized weights (23) and the quantile variant the normalized weights (24); Table 6 reports all four combinations.
Mean standard deviation over 5 random seeds.
Mean over 5 random seeds. Non-cached SPO+ was run only on the original data.
Conclusion
We proposed a risk-sensitive DFL framework, RESTRO, that turns existing DFL methods such as SPO+ into variants that minimize the CVaR of regret. The framework integrates with standard DFL pipelines by reweighting gradients at the minibatch level, without changes to the underlying solver or model architecture. A key advantage is that it relies on point predictions of the uncertain parameters followed by a deterministic optimization problem. This distinguishes it from prior risk-sensitive approaches that solve more complex stochastic or robust optimization problems to account for tail risk. For the SPO+ instantiation, we showed that the RESTRO objective controls an upper bound on the population CVaR of the true regret. Empirically, RESTRO substantially reduces tail regret on knapsack, portfolio, shortest-path, and energy-aware scheduling problems, especially when the cost parameters exhibit heterogeneous tail behavior, and it remains competitive when they do not.
Limitations and future work. Although the reweighting mechanism can be combined with any DFL surrogate, its benefit depends on the underlying loss: the improvements are strongest and most consistent for SPO+, which is also the instantiation covered by our theoretical guarantee. Extending the guarantee to surrogates without a pointwise regret upper bound, such as IMLE and PFY, and to methods that differentiate through smoothed optimization problems is an interesting direction. More broadly, this paper considers uncertainty in the objective parameters of CO problems. Uncertainty in the constraint parameters is more challenging, because it affects both feasibility and optimality, and recent DFL works have begun to study this setting (Hu et al. 2023a,b, Mandi et al. 2026). In future work, we aim to extend RESTRO to uncertain constraint parameters.
References
- [1]Agrawal A, Amos B, Barratt S, Boyd S, Diamond S, Kolter JZ (2019) Differentiable convex optimization layers. Advances in neural information processing systems 32.arxiv.org/abs/1910.12430
- [2]Amos B, Kolter JZ (2017) Optnet: Differentiable optimization as a layer in neural networks. International Conference on Machine Learning, 136–145 (PMLR).arxiv.org/abs/1703.00443
- [3]Ang M, Sun J, Yao Q (2018) On the dual representation of coherent risk measures. Annals of Operations Research 262(1):29–46.DOI
- [4]Berden S, Mahmutoğulları A, Tsouros D, Guns T (2025) Solver-free decision-focused learning for linear optimization problems. Belgrave D, Zhang C, Lin H, Pascan u R, Koniusz P, Ghassemi M, Chen N, eds., Advances in Neural Information Processing Systems, volume 38, 158127–158145 (Curran Associates, Inc.).arxiv.org/abs/2505.22224
- [5]Berthet Q, Blondel M, Teboul O, Cuturi M, Vert JP, Bach F (2020) Learning with differentiable perturbed optimizers. Advances in neural information processing systems 33:9508–9519.
- [6]Bertsekas DP (1997) Nonlinear programming. Journal of the Operational Research Society 48(3):334–334.
- [7]Bertsimas D, Kallus N (2020) From predictive to prescriptive analytics. Management Science 66(3):1025–1044.arxiv.org/abs/1402.5481
- [8]Bucarey López V, Calderón S, Muñoz G, Semet F (2026) Decision-focused predictions via pessimistic bilevel optimization: complexity and algorithms: V. bucarey lópez et al. Computational Optimization and Applications 1–31.arxiv.org/abs/2312.17640
- [9]Chenreddy AR, Delage E (2024) End-to-end conditional robust optimization. Uncertainty in Artificial Intelligence, 736–748 (PMLR).arxiv.org/abs/2403.04670
- [10]Chicoisne R, Ordóñez F, Espinoza D (2018) Risk averse shortest paths: A computational study. INFORMS Journal on Computing 30(3):539–553.DOI
- [11]Curi S, Levy KY, Jegelka S, Krause A (2020) Adaptive sampling for stochastic risk-averse learning. Advances in Neural Information Processing Systems, volume 33.arxiv.org/abs/1910.12511
- [12]Danskin JM (1966) The theory of max-min, with applications. SIAM Journal on Applied Mathematics 14(4):641–664.DOI
- [13]Duchi J, Namkoong H (2019) Variance-based regularization with convex objectives. Journal of Machine Learning Research 20(68):1–55.
- [14]Duchi JC, Namkoong H (2021) Learning models with uniform performance via distributionally robust optimization. The Annals of Statistics 49(3):1378–1406.arxiv.org/abs/1810.08750
- [15]Elmachtoub AN, Grigas P (2022) Smart “predict, then optimize”. Management Science 68(1):9–26.arxiv.org/abs/1710.08005
- [16]Fidon L, Aertsen M, Deprest T, Emam D, Guffens F, Mufti N, Van Elslander E, Schwartz E, Ebner M, Prayer D, Kasprian G, David AL, Melbourne A, Ourselin S, Deprest J, Langs G, Vercauteren T (2022) Distributionally robust deep learning using hardness weighted sampling. Machine Learning for Biomedical Imaging 1(PIPPI 2021):1–61, ISSN 2766-905X, URL http://dx.doi.org/10.59275/j.melba.2022-8b6a.
- [17]Gurobi Optimization L (2021) Gurobi optimizer reference manual. http://www.gurobi.com.
- [18]Hsieh T, Robb E, Chen H, Huang J (2021) Droploss for long-tail instance segmentation. Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, 1549–1557 (AAAI Press), URL https://doi.org/10.1609/aaai.v35i2.16246.
- [21]Huang T, Dilkina B (2020) Enhancing seismic resilience of water pipe networks. Proceedings of the 3rd ACM SIGCAS Conference on Computing and Sustainable Societies, 44–52, COMPASS ’20 (New York, NY, USA: Association for Computing Machinery), ISBN 9781450371292, URL http://dx.doi.org/10.1145/3378393.3402246.
- [22]Kumar R, Majmundar KA, Nagaraj DM, Suggala A (2024) Stochastic re-weighted gradient descent via distributionally robust optimization. Transactions on Machine Learning Research ISSN 2835-8856, URL https://openreview.net/forum?id=KCf5CLAXZq.arxiv.org/abs/2306.09222
- [23]Li T, Beirami A, Sanjabi M, Smith V (2021) Tilted empirical risk minimization. International Conference on Learning Representations, URL https://openreview.net/forum?id=K5YasWXZT30.arxiv.org/abs/2007.01162
- [24]Ma X, Ning C, Du W (2024) Differentiable distributionally robust optimization layers. Forty-first International Conference on Machine Learning, URL https://openreview.net/forum?id=HUJK9dFOW6.arxiv.org/abs/2406.16571
- [25]Mandi J, Bucarey V, Tchomba MMK, Guns T (2022) Decision-focused learning: Through the lens of learning to rank. Chaudhuri K, Jegelka S, Song L, Szepesvari C, Niu G, Sabato S, eds., Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, 14935–14947 (PMLR).arxiv.org/abs/2112.03609
- [26]Mandi J, Defresne M, Berden S, Guns T (2026) Feasibility-aware decision-focused learning for predicting parameters in the constraints. The Thirty-ninth Annual Conference on Neural Information Processing Systems, URL https://openreview.net/forum?id=eTmvwxohRx.arxiv.org/abs/2510.04951
- [27]Mandi J, Guns T (2020) Interior point solving for lp-based prediction+ optimisation. Advances in Neural Information Processing Systems 33:7272–7282.arxiv.org/abs/2010.13943
- [28]Mandi J, Kotary J, Berden S, Mulamba M, Bucarey V, Guns T, Fioretto F (2024) Decision-focused learning: Foundations, state of the art, benchmark and future opportunities. Journal of Artificial Intelligence Research 80:1623–1701.DOI
- [29]Mandi J, Mahmutogullari AI, Berden S, Guns T (2025) Minimizing surrogate losses for decision-focused learning using differentiable optimization. ECAI 2025 - 28th European Conference on Artificial Intelligence, 25-30 October 2025, Bologna, Italy - Including 14th Conference on Prestigious Applications of Intelligent Systems (PAIS 2025), volume 413 of Frontiers in Artificial Intelligence and Applications, 3888–3895 (IOS Press).
- [30]McKenzie D, Heaton H, Fung SW (2024) Differentiating through integer linear programs with quadratic regularization and davis-yin splitting. Transactions on Machine Learning Research ISSN 2835-8856, URL https://openreview.net/forum?id=H8IaxrANW1.arxiv.org/abs/2301.13395
- [31]Mulamba M, Mandi J, Diligenti M, Lombardi M, Bucarey V, Guns T (2021) Contrastive losses and solution caching for predict-and-optimize. Zhou ZH, ed., Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, 2833–2840 (International Joint Conferences on Artificial Intelligence Organization), URL http://dx.doi.org/10.24963/ijcai.2021/390, main Track.
- [32]Namkoong H, Duchi JC (2016) Stochastic gradient methods for distributionally robust optimization with f-divergences. Advances in neural information processing systems 29.
- [33]Niepert M, Minervini P, Franceschi L (2021) Implicit mle: Backpropagating through discrete exponential family distributions. Ranzato M, Beygelzimer A, Dauphin Y, Liang P, Vaughan JW, eds., Advances in Neural Information Processing Systems, volume 34, 14567–14579 (Curran Associates, Inc.).arxiv.org/abs/2106.01798
- [34]Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, et al. (2019) Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32.arxiv.org/abs/1912.01703
- [35]Pochet Y, Wolsey LA (2006) Production planning by mixed integer programming (Springer).DOI
- [36]Pogančić MV, Paulus A, Musil V, Martius G, Rolinek M (2020) Differentiation of blackbox combinatorial solvers. International Conference on Learning Representations.
- [38]Rockafellar RT, Uryasev S (2000) Optimization of conditional value-at-risk. Journal of risk 2:21–42.DOI
- [39]Sadana U, Chenreddy A, Delage E, Forel A, Frejinger E, Vidal T (2025) A survey of contextual optimization methods for decision-making under uncertainty. European Journal of Operational Research 320(2):271–289, ISSN 0377-2217, URL https://doi.org/10.1016/j.ejor.2024.03.020.
- [41]Samuel D, Chechik G (2021) Distributional robustness loss for long-tail learning. 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, 9475–9484 (IEEE).arxiv.org/abs/2104.03066
- [42]Schutte N, Postek K, Yorke-Smith N (2024) Robust losses for decision-focused learning. Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence (IJCAI-24), 4868–4875, URL http://dx.doi.org/10.24963/ijcai.2024/538.
- [43]Silvestri M, Berden S, Signorelli G, Mahmutoğulları AI, Mandi J, Amos B, Guns T, Lombardi M (2026) Score function gradient estimation to widen the applicability of decision-focused learning. Journal of Artificial Intelligence Research 85.DOI
- [44]Soma T, Yoshida Y (2020) Statistical learning with conditional value at risk. arXiv preprint arXiv:2002.05826 .arxiv.org/abs/2002.05826
- [45]Sun C, Liu L, Li X (2023) Predict-then-calibrate: A new perspective of robust contextual lp. Advances in neural information processing systems 36:17713–17741.arxiv.org/abs/2305.15686
- [46]Tan J, Wang C, Li B, Li Q, Ouyang W, Yin C, Yan J (2020) Equalization loss for long-tailed object recognition. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, 11659–11668 (Computer Vision Foundation / IEEE), URL http://dx.doi.org/10.1109/CVPR42600.2020.01168.
- [47]Tang B, Khalil EB (2023) Pyepo: A pytorch-based end-to-end predict-then-optimize library for linear and integer programming.DOI
- [48]Tang B, Khalil EB (2024) Cave: A cone-aligned approach for fast predict-then-optimize with binary linear programs. Diklina B, ed., Integration of Constraint Programming, Artificial Intelligence, and Operations Research - 21st International Conference, CPAIOR 2024, Uppsala, Sweden, May 28-31, 2024, Proceedings, Part II, volume 14743 of Lecture Notes in Computer Science, 193–210 (Springer).
- [49]Toth P, Vigo D (2002) The vehicle routing problem (SIAM).
- [50]Wang PZ, Chen S, Liang J, Fioretto F, Zhu S (2026) Gen-DFL: Decision-focused generative learning for robust decision making. The Fourteenth International Conference on Learning Representations, URL https://openreview.net/forum?id=GU2I97a3Lm.arxiv.org/abs/2502.05468