betaFeedback

WorksMathematics

Risk-Sensitive Non-Parametric and Kernel Bandits

Abstract

We study non-parametric bandits in which arms are evaluated using Cumulative Prospect Theory (CPT) rather than expected reward. We consider continuous action spaces with arm-dependent noise and propose Non-Parametric CPT Bandits (NPCB\mathtt{NPCB}), an optimistic algorithm that decouples mean estimation from CPT-based decision-making. We prove CPT regret of order O~(T dim⁡E(F,T−2) log⁡N(F,T−2,∥⋅∥∞))\widetilde{\mathcal O}\bigl(\sqrt{T\,\dim_E(\mathcal F,T^{-2})\,\log N(\mathcal F,T^{-2},\|\cdot\|_\infty)}\bigr), where dim⁡E\dim_E is the Eluder dimension of the function class F\mathcal F, TT is the time horizon, and NN is its covering number in the sup norm. This matches the order-wise regret of classical non-parametric bandits, showing that CPT risk sensitivity incurs no additional order-wise regret cost. We extend NPCB\mathtt{NPCB} to unknown noise CDFs, retaining the known-CDF mean-learning term and adding a CDF-learning cost governed by a distributional Eluder dimension. We also study Kernel-CPT bandits and obtain information-gain-based regret bounds for a broad range of kernels, matching the corresponding risk-neutral kernel rates up to logarithmic factors.

Introduction

The Multi-Armed Bandit framework is a fundamental paradigm for sequential decision-making under uncertainty. In the vast majority of this literature [26, 5], the objective is to minimize appropriately defined regret, which depends simply on the mean of the reward distribution associated with the arm pulled. However, in real-world applications—ranging from clinical trials to financial portfolio optimization—the mean does not satisfactorily capture the merits of certain actions, especially when humans are involved in the decision. See [36] for a recent survey.

In such applications, risk plays an important role, and a risk-aware performance measure is often preferred over the average reward. Consequently, significant research has focused on diverse risk measures, including mean-variance (MV) [28], Value-at-Risk (VaR) [3], Conditional Value-at-Risk (CVaR) [3, 32], spectral risk measures [2], and utility-based shortfall risk [3, 19]. While these measures satisfy axioms of coherence or convexity, they often fail to model the nuances of human behavior, such as loss aversion and asymmetric preferences, which are best captured by the celebrated (and Nobel prize-winning) risk measure Cumulative Prospect Theory (CPT) [39]. The failure of standard risk measures is structural. MV penalizes all variance equally, treating rare massive gains and catastrophic losses identically. CVaR only examines the lower tail of the reward distribution, so rare disasters are averaged with ordinary outcomes when their probability is below the chosen tail level, while attractive lottery gains are not its focus. CPT avoids both failures through nonlinear probability weighting — a 1% chance of a fatal crash carries disproportionate psychological weight far exceeding 0.01. This amplification of rare events means CPT simultaneously penalizes hidden disasters and values rare large gains, capturing the two asymmetric behaviors that characterize human decision-making under uncertainty.

To further motivate the necessity of CPT risk, consider the challenge of automated policy design motivated by the Asian Disease Problem [21]. Consider an agent tasked with optimizing a Public Health Score rr (e.g., the number of lives saved). The agent must choose between two policies: a “Safe” protocol guaranteeing that 200 lives are saved (r=200r = 200), and a “Risky” protocol with a 35% chance of saving 600 lives (r=600r = 600) and a 65% chance of saving none (r=0r = 0). A standard risk-neutral agent calculates the averages as μRisky=210>μSafe=200\mu_{\mathrm{Risky}} = 210 > \mu_{\mathrm{Safe}} = 200, therefore identifying the risky protocol as optimal. However, empirical evidence shows that the vast majority of human decision-makers reject this gamble [39] and choose the safe protocol. This reveals a critical insight: the standard method of maximizing the mean reward is misaligned with human ethical and behavioral preferences.

CPT addresses this misalignment by evaluating actions not through a simple average, but through a distorted value Va(f):=∫0∞w(1−Faf(z))dz−∫0∞w(Faf(−z))dz\mathcal{V}_a(f) := \int_{0}^{\infty} w(1-F_a^f(z))\mathrm{d}z - \int_{0}^{\infty} w(F_a^f(-z))\mathrm{d}z, where w(⋅)w(\cdot) is a non-linear weight (distortion) function applied to the cumulative distribution FX(⋅)F_X(\cdot). Note that if w(⋅)w(\cdot) is an identity function, we get the standard average, i.e., V(X)=E(X)\mathcal{V}(X) = \mathbb{E}(X). In the above-mentioned example, VSafe=200\mathcal{V}_{\mathrm{Safe}} = 200 while VRisky=600w(0.35)\mathcal{V}_{\mathrm{Risky}} = 600w(0.35). Hence the safe action is the subjective optimum precisely when w(0.35)<1/3w(0.35) < 1/3. For instance, using the Tversky–Kahneman weighting form with γ=0.5\gamma= 0.5 gives w(0.35)≈0.303w(0.35) \approx0.303, and therefore VmathrmRisky≈182<200=VSafe\mathcal{V}_{\\mathrm{Risky}} \approx182 < 200 = \mathcal{V}_{\mathrm{Safe}}. In general, with distorted weights, the sequence of actions chosen may differ from that under the risk-neutral measure. To demonstrate this distinction empirically, we construct instances in which risk-neutral and risk-sensitive objectives lead to systematically different decisions, with the separation persisting across repeated trials (see, Figure 1 (a)-(Figure 1b)).

Figure 1. Risk preferences and CPT learning. (a)–(b) Population objective scores in Config B (rare loss) and Config E (rare gain). CVaR uses α=0.10\alpha= 0.10, MV uses c=1c = 1, and CPT uses identity utility and Tversky–Kahneman weighting with γ=0.50\gamma= 0.50. (c) Bounded 2D kernel comparison on logarithmic axes: mean cumulative CPT regret ±\pm one standard error over 30 paired trials. Dashed lines are descriptive power-law fits.

Significant research has focused on diverse risk measures in the multi-armed bandit setting. For instance, [34, 45, 16] study the Mean-Variance setup extensively for KK-armed bandits. [44] analyzes Value-at-Risk (VaR) in the context of the total reward sequence. [14, 40, 42, 22] utilize the Conditional Value-at-Risk (CVaR) measure in KK-armed Bandit setting. [23] proposes a distribution-oblivious algorithm for optimizing a linear combination of mean and CVaR, specifically for unbounded reward distributions in the best-arm identification setting. For Kernel bandits, [37] study the standard Upper Confidence Bound (UCB) type algorithms for CVaR and Mean-Variance risk measures and prove near-optimal results. To provide a systematic analysis of such metrics, [11] introduced a unified framework for multi-armed bandits under general risk satisfying a Lipschitz continuity condition with respect to probability distributions (e.g., in the Wasserstein distance). However, the CPT function is generally not Lipschitz continuous [36], often causing the value to fluctuate wildly with small distribution changes. This lack of stability prevents the direct application of existing unified frameworks to CPT.

To the best of our knowledge, bandit optimization with CPT-based risk measures has been limited. Prior works have used CPT-based risk-sensitive measures for simple KK-armed and linear bandits [13], and achieve near-optimal regret bounds. Recently, [TMA^+25] proposes a unified framework for distortion risk measures in the KK-armed bandit setting, covering both monotone and non-monotone distortions and showing that, for certain non-monotone objectives, an optimal policy may require a mixture of arms. However, their analysis is primarily focused on finite KK-armed bandits, whereas our interest is in continuous action spaces and general non-parametric reward functions. Moreover, CPT introduces a distinct nonlinear probability distortion that is not covered by their mixture-based analysis. This leaves a theoretical gap in CPT optimization for non-parametric function classes. This prompts the central question of our work:

Is it possible to learn (general) non-parametric (including Kernel) bandits with CPT risk measure?

In this paper, we answer this question affirmatively. We analyze a learning algorithm, namely Non-Parametric CPT-Bandits (NPCB), that works for general functional bandits with the CPT risk measure. Notably, NPCB is agnostic to (non-parametric) function class and uses the Eluder Dimension [31] to characterize the complexity of the learning task. As a primary instantiation of our framework, we study Kernel bandits and obtain guarantees for Kernel CPT (KCB) algorithm. To the best of our knowledge, this is the first work to study Kernel Bandits (RKHS) under a CPT-based risk measure.

Summary of contributions

Non-parametric CPT bandits: We propose a learning algorithm with CPT-based risk for general, non-parametric bandits, namely NPCB. Our formulation captures continuous action spaces with arm-dependent noise. For this general setup, we derive the regret bound (refer to Theorem 4.2) which matches the order-wise performance of risk-neutral non-parametric bandits [31]. This demonstrates that non-linear human-centric risk measures can be incorporated into infinite-dimensional function learning without incurring a regret penalty.

Kernel CPT bandits: For an RKHS mean-reward class and known arm-dependent noise CDFs, we give kernel algorithms for CPT regret. Our phasewise procedure achieves O~(TγT(λ))\widetilde{O}\left(\sqrt{T\gamma_T(\lambda)}\right) regret, matching the standard risk-neutral dependence on information-gain.

Modular Algorithm: Our algorithms decouple the Learning Module (mean estimation from undistorted rewards) from the Decision Module (CPT valuation). This makes our framework function-class-agnostic, allowing it to be seamlessly applied to any function class (e.g., Linear, RKHS, or Neural Nets), provided a valid confidence set can be constructed.

CPT stability for non-parametric learning: Building on the translation stability of [13], we connect CPT uncertainty to confidence widths for general mean-function classes. Under additive noise, the CPT value is uniformly 2-Lipschitz in the mean, even without Hölder continuity of the weighting function1. This connects CPT regret to the Eluder dimension and covering numbers, and supports our kernel instantiations.

Removing Known CDF assumption: For the case of linear bandits, [20 Section V, Remark 10] require the knowledge of the cumulative distribution function of the noise, remarking that online estimation of the CDF is an open problem left for future work. We learn the noise CDF online using a joint likelihood confidence set. The regret keeps the mean-learning term and adds a cost governed by joint distributional complexities. We bound these complexities for an explicit linear family and its RKHS extension.

Problem Formulation

We consider the sequential decision-making problem over a finite time horizon TT, characterized by a compact and continuous action space A⊆Rd\mathcal{A} \subseteq\mathbb{R}^{d}, which may be infinite. The environment is governed by an unknown true mean-reward function f⋆:A→Rf^{\star}:\mathcal{A} \to\mathbb{R}, which belongs to a known class of non-parametric functions of real-valued functions F\mathcal{F}. The value f⋆(a)f^{\star}(a) represents the expected reward associated with action a∈Aa \in\mathcal{A}. At each step tt, the agent selects an action at∈Aa_t \in\mathcal{A} based on the history of previous observations Dt−1={(aτ,rτ)}τ=1t−1\mathcal{D}_{t-1} = \{(a_\tau, r_\tau)\}_{\tau=1}^{t-1}. The environment then reveals a realized physical reward rt=f∗(at)+ϵtr_t = f^*(a_t) + \epsilon_t.

Unlike the standard bandit formulation, we allow arm-dependent noise. Conditional on the history and the selected arm at=aa_t = a, the noise ϵt\epsilon_t has a fixed distribution FϵaF_{\epsilon_a} with mean zero. For a candidate f∈Ff \in\mathcal{F}, the corresponding reward CDF is Faf(z)=Fϵa(z−f(a))F_a^f(z) = F_{\epsilon_a}(z - f(a)). We assume that these candidate reward distributions are supported in [−M,M][-M, M].

While the agent observes undistorted rewards rtr_t drawn from the distribution defined by FϵaF_{\epsilon_a}, its decision is governed by a subjective distortion of these probabilities. This distortion is parameterized by weight function w:[0,1]→[0,1]\mathsf{w} : [0, 1] \to[0, 1]. We define the weight-distorted reward, denoted by Va(f)\mathcal{V}_a(f), by applying the distortion w\mathsf{w} to the cumulative probabilities of the reward distribution as

Va(f):=∫0∞w(1−Faf(z)) dz−∫0∞w(Faf(−z)) dz\mathcal{V}_a(f) := \int_0^\infty\mathsf{w}\left(1 - F_a^f(z)\right)\,\mathrm{d}z - \int_0^\infty\mathsf{w}\left(F_a^f(-z)\right)\,\mathrm{d}z

The weight function w:[0,1]→[0,1]\mathsf{w} : [0, 1] \to[0, 1] is the behavioral core of CPT. Beyond the minimal conditions w(0)=0\mathsf{w}(0) = 0, w(1)=1\mathsf{w}(1) = 1, and monotone increasing [39, 13], we impose no additional assumptions. Together with the bounded-reward assumption above, w(0)=0\mathsf{w}(0) = 0 ensures that Va(f)\mathcal{V}_a(f) is finite. Several families of w\mathsf{w} satisfying these properties are widely used in behavioral economics, including the Tversky–Kahneman function [39] w(p)=pγ/(pγ+(1−p)γ)1/γ\mathsf{w}(p) = p^\gamma/ (p^\gamma+ (1-p)^\gamma)^{1/\gamma}, the Prelec function [30] w(p)=exp⁡(−(ln⁡1/p)α)\mathsf{w}(p) = \exp(-( \ln1/p)^\alpha), and the Gonzalez–Wu function [15] w(p)=δpγ/(δpγ+(1−p)γ)\mathsf{w}(p) = \delta p^\gamma/ (\delta p^\gamma+ (1-p)^\gamma). Our framework accommodates all of these. A detailed discussion of the behavioral interpretation of w\mathsf{w}, its effect on V\mathcal{V} is provided in Appendix § 14.

Remark 2.1. For simplicity, we use reference point 0, identity utility, and a common probability weight for gains and losses. The theory extends to a fixed reference point x0x_0, separate weights w+\mathsf{w}^{+}, and nondecreasing L±\mathsf{L}^{\pm}-Lipschitz gain/loss utilities. These extensions change only constant factors, as detailed in Appendix § 10.§ 2.

CPT Regret. The agent’s objective is to maximize the cumulative subjective, weight-distorted value of the rewards gathered over the horizon. Formally, under the true mean-reward function f∗f^* and distortion mapping w(⋅)\mathsf{w}(\cdot), we denote the subjectively optimal arm as, a∗∈arg⁡max⁡a∈AVa(f∗)a^* \in\arg\max_{a \in\mathcal{A}} \mathcal{V}_a(f^*). The goal of learning is to maintain a policy that competes with this optimum through sequential interaction with the environment. The performance of the learning policy is evaluated by its cumulative weight-distorted regret (CPT regret), given by

R(T)=∑t=1T(Va∗(f∗)−Vat(f∗)).R(T) = \sum_{t=1}^{T}\left(\mathcal{V}_{a^*}(f^*) - \mathcal{V}_{a_t}(f^*)\right).

Remark 2.2. Designing such an algorithm is challenging because the agent observes samples from the undistorted distribution but must optimize the distorted value Va(f∗)\mathcal{V}_a(f^*). The non-linear transformation w(⋅)\mathsf{w}(\cdot) prevents the estimation of Va\mathcal{V}_a through the sample mean of rtr_t. The distortion can inflate the impact of tail events in FϵaF_{\epsilon_a}, potentially changing the preference order between arms.

Non-parametric CPT bandits (NPCB)

We now present our learning algorithm. Our approach follows the general template of optimistic exploration for general function classes, as discussed by [31], but fundamentally deviates from it in the arm-selection step to accommodate the agent’s behavioral preferences.

Algorithm 1: Non-Parametric CPT Bandits (NPCB)

  1. Input: F\mathcal{F}, known noise CDFs, and {βt}\{\beta_t\}.

  2. for t=1t = 1 to TT do

  3. Set f^tLS∈arg⁡min⁡f∈F∑s<t(rs−f(as))2\hat{f}^{\mathrm{LS}}_t \in\arg\min_{f \in\mathcal{F}} \sum_{s<t}(r_s-f(a_s))^2.

  4. Set Ct={f∈F:∑s<t(f(as)−f^tLS(as))2≤βt}C_t = \{f \in\mathcal{F}: \sum_{s<t}(f(a_s)-\hat{f}^{\mathrm{LS}}_t(a_s))^2 \le\beta_t\}.

  5. Choose at∈arg⁡max⁡a∈Asup⁡f∈CtVa(f)a_t \in\arg\max_{a \in\mathcal{A}} \sup_{f \in C_t} V_a(f).

  6. Observe rtr_t.

  7. end for

References

  1. [1]P. L. A. and S. P. Bhat. A wasserstein distance approach for concentration of empirical risk estimates, 2022. URL: https://arxiv.org/abs/1902.10709.
  2. [2]C. Acerbi. Spectral measures of risk: A coherent representation of subjective risk aversion. Journal of Banking & Finance, 26(7):1505–1518, 2002. DOI: https://doi.org/10.1016/S0378-4266(02)00281-9.
  3. [3]P. Artzner, F. Delbaen, J.-M. Eber, and D. Heath. Coherent measures of risk. Mathematical Finance, 9(3):203–228, 1999. DOI: https://doi.org/10.1111/1467-9965.00068.
  4. [4]P. L. A., C. Jie, M. Fu, S. Marcus, and C. Szepesvári. Cumulative prospect theory meets reinforcement learning: Prediction and control, 2016. URL: https://arxiv.org/abs/1506.02632.
  5. [5]S. Bubeck, N. Cesa-Bianchi, et al. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning, 5(1):1–122, 2012. URL: https://arxiv.org/abs/1204.5721.
  6. [6]D. Baudry, R. Gautron, E. Kaufmann, and O.-A. Maillard. Optimal thompson sampling strategies for support-aware cvar bandits. In International Conference on Machine Learning, 2020. URL: https://api.semanticscholar.org/CorpusID:235826048.
  7. [7]V. Borkar. A sensitivity formula for risk-sensitive cost and the actor–critic algorithm. Systems & Control Letters, 44(5):339–346, 2001. DOI: https://doi.org/10.1016/S0167-6911(01)00152-9.
  8. [8]A. Chateauneuf, J. Eichberger, and S. Grant. Choice under uncertainty with the best and worst in mind: Neo-additive capacities. Journal of Economic Theory, 137(1):538–567, 2007. URL: https://www.sciencedirect.com/science/article/pii/S0022053107000300.
  9. [9]S. R. Chowdhury and A. Gopalan. On kernelized multi-armed bandits. In D. Precup and Y. W. Teh, editors, Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pages 844–853. PMLR, 2017. URL: http://proceedings.mlr.press/v70/chowdhury17a.html.
  10. [10]Y. Chow, M. Ghavamzadeh, L. Janson, and M. Pavone. Risk-constrained reinforcement learning with percentile risk criteria, 2017. URL: https://arxiv.org/abs/1512.01629.
  11. [11]A. Cassel, S. Mannor, and A. Zeevi. A general approach to multi-armed bandits under risk criteria. In S. Bubeck, V. Perchet, and P. Rigollet, editors, Proceedings of the 31st Conference On Learning Theory, volume 75 of Proceedings of Machine Learning Research, pages 1295–1306. PMLR, 06–09 Jul 2018. URL: https://proceedings.mlr.press/v75/cassel18a.html.
  12. [12]J. Q. L. Chang and V. Y. F. Tan. A unifying theory of thompson sampling for continuous risk-averse bandits. In AAAI Conference on Artificial Intelligence, 2021. URL: https://api.semanticscholar.org/CorpusID:237290094.
  13. [13]A. Gopalan, L. Prashanth, M. Fu, and S. Marcus. Weighted bandits or: How bandits learn distorted values that are not expected. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017. URL: https://ojs.aaai.org/index.php/AAAI/article/view/10922.DOI
  14. [14]N. Galichet, M. Sebag, and O. Teytaud. Exploration vs exploitation vs safety: Risk-averse multi-armed bandits. CoRR, abs/1401.1123, 2014. URL: http://arxiv.org/abs/1401.1123.
  15. [15]R. Gonzalez and G. Wu. On the shape of the probability weighting function. Cognitive psychology, 38(1):129–166, 1999.pubmed.ncbi.nlm.nih.gov/10090801
  16. [16]H. Hu, A. Charpentier, M. Ghossoub, and A. Schied. Multiarmed bandits problem under the mean-variance setting, 2024. URL: https://arxiv.org/abs/2212.09192.
  17. [17]K. Huang, S. M. Kakade, J. D. Lee, and Q. Lei. A short note on the relationship of information gain and eluder dimension. CoRR, abs/2107.02377, 2021. URL: https://arxiv.org/abs/2107.02377.
  18. [18]R. A. Howard and J. E. Matheson. Risk-sensitive markov decision processes. Management Science, 18(7):356–369, 1972. URL: http://www.jstor.org/stable/2629352.DOI
  19. [19]Z. Hu and D. Zhang. Utility-based shortfall risk: Efficient computations via monte carlo. Naval Research Logistics (NRL), 65:378 – 392, 2018. URL: https://api.semanticscholar.org/CorpusID:126005263.DOI
  20. [20]R. K. Kolla, P. L. A., A. Gopalan, K. Jagannathan, M. Fu, and S. Marcus. Bandit algorithms to emulate human decision making using probabilistic distortions, 2023. URL: https://arxiv.org/abs/1611.10283.
  21. [21]D. Kahneman. Thinking, Fast and Slow. Farrar, Straus and Giroux, New York: New York, 2011.
  22. [22]R. K. Kolla, P. L. A., and K. P. Jagannathan. Risk-aware multi-armed bandits using conditional value-at-risk. CoRR, abs/1901.00997, 2019. URL: http://arxiv.org/abs/1901.00997.
  23. [23]A. Kagrecha, J. Nair, and K. P. Jagannathan. Distribution oblivious, risk-aware algorithms for multi-armed bandits with unbounded rewards. CoRR, abs/1906.00569, 2019. URL: http://arxiv.org/abs/1906.00569.
  24. [24]D. Kahneman and A. Tversky. Prospect theory: An analysis of decision under risk. Econometrica, 47(2):263–291, 1979. DOI: 10.2307/1914185.DOI
  25. [25]H. Liang and Z. quan Luo. A distribution optimization framework for confidence bounds of risk measures, 2023. URL: https://arxiv.org/abs/2306.07059.
  26. [26]T. Lattimore and C. Szepesvári. Bandit algorithms. Cambridge University Press, 2020.
  27. [27]Z. Li and J. Scarlett. Gaussian process bandit optimization with few batches. In International Conference on Artificial Intelligence and Statistics, pages 92–107. PMLR, 2022. URL: https://proceedings.mlr.press/v151/li22a.html.arxiv.org/abs/2110.07788
  28. [28]H. Markowitz. Portfolio selection. The Journal of Finance, 7(1):77–91, 1952. URL: http://www.jstor.org/stable/2975974.
  29. [30]D. Prelec. The probability weighting function. Econometrica, pages 497–527, 1998. URL: https://www.jstor.org/stable/2998573.DOI
  30. [31]D. Russo and B. V. Roy. Eluder dimension and the sample complexity of optimistic exploration. In C. J. C. Burges, L. Bottou, Z. Ghahramani, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States, pages 2256–2264, 2013. URL: https://proceedings.neurips.cc/paper/2013/hash/41bfd20a38bb1b0bec75acf0845530a7-Abstract.html.
  31. [32]R. T. Rockafellar and S. Uryasev. Optimization of conditional value-at-risk. Journal of Risk, 3:21–41, 2000. URL: https://api.semanticscholar.org/CorpusID:854622.DOI
  32. [33]J. Scarlett, I. Bogunovic, and V. Cevher. Lower bounds on regret for noisy Gaussian process bandit optimization. In Proceedings of the 30th Conference on Learning Theory (COLT), volume 65 of Proceedings of Machine Learning Research, pages 1723–1742. PMLR, 2017. URL: https://proceedings.mlr.press/v65/scarlett17a.html.arxiv.org/abs/1706.00090
  33. [34]A. Sani, A. Lazaric, and R. Munos. Risk-aversion in multi-armed bandits. CoRR, abs/1301.1936, 2013. URL: http://arxiv.org/abs/1301.1936.
  34. [35]M. J. Sobel. The variance of discounted markov decision processes. Journal of Applied Probability, 19(4):794–802, 1982. DOI: 10.2307/3213832.DOI
  35. [36]V. Y. F. Tan, P. L. A., and K. Jagannathan. A survey of risk-aware multi-armed bandits, 2022. URL: https://arxiv.org/abs/2205.05843.
  36. [37]S. Takemori. Distributionally-aware kernelized bandit problems for risk aversion. In K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvári, G. Niu, and S. Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 20933–20959. PMLR, 2022. URL: https://proceedings.mlr.press/v162/takemori22a.html.
  37. [38]A. Tamar, Y. Glassner, and S. Mannor. Optimizing the cvar via sampling, 2014. URL: https://arxiv.org/abs/1404.3862.
  38. [39]A. Tversky and D. Kahneman. Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and uncertainty, 5(4):297–323, 1992. URL: https://doi.org/10.1007/BF00122574.
  39. [40]A. Tamkin, R. Keramati, and E. Brunskill. Distributionally-aware exploration for cvar bandits. In NeurIPS 2019 Workshop on Safety and Robustness on Decision Making, 2019. URL: https://api.semanticscholar.org/CorpusID:219544525.
  40. [41]M. Tatlı, A. Mukherjee, P. L. A., K. Shanmugam, and A. Tajer. Risk-sensitive bandits: Arm mixture optimality and regret-efficient algorithms. In Y. Li, S. Mandt, S. Agrawal, and E. Khan, editors, Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, volume 258 of Proceedings of Machine Learning Research, pages 3871–3879. PMLR, 03–05 May 2025. URL: https://proceedings.mlr.press/v258/tatli25a.html.arxiv.org/abs/2503.08896
  41. [42]C. Tan and P. Weng. Cvar-regret bounds for multi-armed bandits. In Asian Conference on Machine Learning, 2022. URL: https://api.semanticscholar.org/CorpusID:259093003.
  42. [43]S. Vakili, K. Khezeli, and V. Picheny. On information gain and regret bounds in gaussian process bandits. In International Conference on Artificial Intelligence and Statistics, pages 82–90. PMLR, 2021. URL: https://proceedings.mlr.press/v130/vakili21a.html.arxiv.org/abs/2009.06966
  43. [44]S. Vakili and Q. Zhao. Mean-variance and value at risk in multi-armed bandit problems. In 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), pages 1330–1335, 2015. DOI: 10.1109/ALLERTON.2015.7447162.DOI
  44. [45]S. Vakili and Q. Zhao. Risk-averse multi-armed bandit problems under mean-variance measure. CoRR, abs/1604.05257, 2016. URL: http://arxiv.org/abs/1604.05257.
  45. [46]Z. Yang, C. Jin, Z. Wang, M. Wang, and M. I. Jordan. Bridging exploration and general function approximation in reinforcement learning: Provably efficient kernel and neural value iterations. CoRR, abs/2011.04622, 2020. URL: https://arxiv.org/abs/2011.04622.

Paper details

vlnc:W_rf5j3g5c
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

generative AI tools

View statement
Looking for related papers…
Contents