betaFeedback

WorksMathematics

Statistical Foundations of Reinforcement Learning: A Non-Asymptotic Perspective

Abstract

Reinforcement learning (RL) embodies a foundational paradigm for sequential decision-making in the face of uncertainty. The rapid growth of model complexity in emerging applications has significantly amplified the challenges of achieving data-efficient RL, particularly in sample-starved scenarios where data collection may be costly, time-consuming, or even high-stakes. Consequently, understanding and improving the data efficiency of RL algorithms has become a central problem of both fundamental theoretical interest and practical importance. Despite decades of research, the statistical underpinnings of RL remained remarkably underdeveloped until relatively recently, particularly in the finite-sample regimes most relevant to modern practice. Over the past few years, there has been a surge of activity advancing the frontiers of non-asymptotic statistical RL theory, drawing on ideas from high-dimensional statistics and probability, information theory, and statistical learning theory. In this monograph, we present a non-asymptotic treatment for RL based on tabular Markov decision processes, focusing on three canonical sampling mechanisms — RL with a generative model, online RL, and offline RL — together with two algorithmic paradigms: model-based and model-free approaches. Centered around non-asymptotic sample complexity analysis, our treatment emphasizes minimax-optimal algorithm design, particularly methods that attain statistical optimality without significant burn-in costs, as well as sharp statistical characterizations of several widely used RL algorithms. Whenever possible, we provide largely self-contained analyses together with intuitive explanations. We also briefly discuss several extensions, such as robust RL, competitive multi-agent RL, federated RL, uncertainty quantification, and RL from human feedback.

Introduction

Reinforcement learning (RL) (Sutton and Barto, 2018; Bertsekas, 2019), commonly formulated as sequential decision-making and learning in a Markov decision process (MDP), has garnered immense attention in recent years due to its striking empirical success across a wide range of practical domains. By enabling agents to collect data and learn through repeated, trial-and-error interaction with their environments, RL has led to major breakthroughs in applications such as competitive gaming, robotics and control, autonomous driving, e-commerce, power systems, and healthcare, often achieving state-of-the-art performance in complex decision-making tasks (Silver et al., 2017; Mnih et al., 2015; Tang et al., 2025; Kiran et al., 2021; Hu et al., 2018; Bastani et al., 2021; Chen et al., 2022c; Yu et al., 2021; Bai et al., 2025). Beyond these traditional application areas, RL has recently emerged as a cornerstone for fine-tuning and aligning large language models (LLMs). In particular, post-training techniques such as Reinforcement Learning from Human Feedback (RLHF) and Reinforcement Learning with Verifiable Rewards (RLVR) (Guo et al., 2025; Jaech et al., 2024) have proven highly effective in enhancing instruction following, safety, and sophisticated reasoning capabilities, among other things, thereby positioning RL as a key enabler in the rapid advancement of modern generative artificial intelligence (AI).

A central objective of RL is to learn, from a collection of noisy data samples, a decision-making strategy (commonly referred to as a policy in RL) that approximately maximizes long-term rewards in an environment (often quantified through a value function), without direct access to an accurate description of the environment itself. In many modern applications, the state space (i.e., the collection of possible situations an agent may encounter in the environment), the action space (i.e., the set of actions available to the agent at each decision point), and the planning horizon can all be extraordinarily large. As a result, data collection must often be carried out at a substantial scale, which may be prohibitively expensive, time-consuming, or even high-stakes, as encountered in domains such as clinical trials, online advertising, and autonomous systems. These challenges are particularly pronounced in the “sample-starved” regime, where the intrinsic complexity of the environment vastly exceeds the amount of available data, making accurate model estimation infeasible.1 Consequently, a fundamental goal of modern RL theory is to understand how reliable learning can be achieved in such data-limited scenarios. Accomplishing this goal requires developing a precise quantitative understanding of the trade-off between statistical accuracy and sample complexity, as well as designing computationally efficient algorithms that provably attain optimal or near-optimal performance.

Statistical framework and scope

To systematically address the data-efficiency challenges outlined above, this monograph develops a coherent and rigorous statistical framework that accommodates a diverse range of settings. To help orient the reader, we briefly describe three key pillars underlying the topics studied throughout this monograph: the sampling mechanisms, the core algorithmic paradigms, and the performance metrics used to evaluate learning performance.

Data collection mechanisms. The statistical challenges of RL depend fundamentally on how data are collected and made available to the agent. We emphasize three canonical settings that are widely studied in the literature.

  • RL with a generative model. When equipped with access to a generative model or simulator, the agent can query the environment in a fully controlled manner. More specifically, the agent may sample transitions from arbitrary state-action pairs at will, effectively eliminating the need for exploration and isolating the statistical challenges inherent to policy optimization and value estimation. See Chapters 3 and 4.

  • Online RL. In online RL, the agent interacts with the environment sequentially in real time and must adaptively determine its actions based on past observations. Since data can only be gathered through the agent’s own interactions with the environment, one must balance exploration (i.e., acquiring new information about the unknown environment) and exploitation (i.e., leveraging current knowledge of the environment to improve rewards). This fundamental tension leads to the classical exploration-exploitation trade-off, which lies at the heart of online RL. See Chapter 5.

  • Offline RL. In offline RL, the agent is given access only to a static dataset of historical samples — typically logged by one or more (possibly unknown) behavior policies — and needs to learn without any further interaction with the environment. As a result, the agent faces fundamental challenges arising from issues such as limited data coverage and distributional mismatch between the available data and the target policy to be learned. See Chapter 6.

Algorithmic paradigms. Broadly speaking, there are at least two principal algorithmic paradigms in RL: model-based approaches and model-free approaches.

  • Model-based algorithms typically decouple model estimation from policy learning. Namely, one first leverages the available data samples to estimate the unknown environment model and subsequently performs policy optimization by treating the learned model as if it were accurate and applying classical dynamic programming techniques dating back to Richard Bellman (Bellman, (Bellman, 1952; Bertsekas, 2017)). A notable advantage of model-based methods is

their flexibility: once a model has been learned, it can often be reused to solve new downstream tasks without requiring additional interaction with the environment or revisiting the original data samples.

  • In contrast, model-free algorithms seek to learn the optimal value function or policy directly from data, bypassing explicit estimation of the environment’s dynamics and rewards. These algorithms are particularly appealing when constructing or storing an environment model is difficult. Model-free methods are also often more memory-efficient because they avoid storing an explicit, potentially large-dimensional representation of the full model.

Characterizing the sample efficiency of model-based and model-free approaches is a primary theme of this monograph. As a preview, despite their higher memory and computational costs, model-based algorithms have been shown to attain statistically optimal performance across all three canonical sampling mechanisms outlined above, even in the most sample-starved regimes. By contrast, many classical model-free methods, including standard variants of Q-learning, fail to attain optimal sample complexity in some regimes. Whether model-free methods can achieve statistical optimality across the full spectrum of problem regimes without sacrificing their memory advantages remains an important open question.

Evaluation metrics. In this monograph, we evaluate the performance of RL algorithms primarily through the lens of sample complexity, defined within the classical Probably Approximately Correct (PAC) learning framework as the minimum number of samples required to guarantee, with high probability, that the statistical error falls below a prescribed threshold. Depending on the problem goal, the notion of statistical error can be tailored to the specific object of interest. For value estimation, we assess the accuracy of the estimated value function, often measured under the ℓ∞\ell_\infty norm to ensure uniform precision across all initial states. For policy learning, we instead measure the performance gap between the value function induced by the learned policy and that induced by the target policy. More broadly, such sample complexity guarantees provide a precise, quantitative characterization of how statistical accuracy improves as additional data become available.

Another performance metric widely used in online settings is (cumulative) regret, which measures the gap between the cumulative reward that could have been achieved by following an optimal policy and the cumulative reward actually obtained by the learning algorithm over time. Under suitable conditions, regret guarantees can be converted into PAC-style sample complexity guarantees, a connection that we will elaborate on in Chapter § 8.1.

Finally, we also briefly discuss the task of uncertainty quantification (see Chapter 7.4), with the goal of assessing the uncertainty associated with estimated quantities. A common objective is to construct confidence intervals that are both statistically valid and as tight as possible.

Towards non-asymptotic statistical foundations

Despite decades of extensive research, the statistical foundations of RL in finite-sample and finite-time regimes — which carry high operational value in modern practice — remained significantly underdeveloped until the past one to two decades. Early theoretical analyses of core RL algorithms — such as temporal difference learning and Q-learning — were conducted primarily in the asymptotic regime, where the number of samples or iterations tends to infinity while all other problem parameters are held fixed. Although this line of work provided important insights into the convergence behavior of RL algorithms, it offers limited guidance on their data efficiency in modern large-scale applications. As discussed previously, many contemporary RL problems have to deal with enormous state-action spaces and long planning horizons. Treating these problem parameters as fixed constants can therefore fail to capture the statistical bottlenecks in high dimensions.

To overcome these limitations, the past several years have witnessed a flurry of activity aimed at strengthening finite-sample, finite-time theory for RL. These advances draw heavily on ideas from modern development in high-dimensional statistics, high-dimensional probability, information theory, and statistical learning theory. Yet, despite this shift toward non-asymptotic analysis, a complete understanding of the fundamental sample complexity limits remained largely elusive, even for the three canonical data collection mechanisms introduced above. As we will discuss in subsequent chapters, much of the prior literature was, at best, only able to establish sharp or optimal statistical guarantees when the available sample size exceeded an enormous threshold, leaving the sample-starved regime poorly understood until fairly recently. Indeed, even today, sharp and complete statistical characterizations of the performance of RL algorithms are available only in relatively basic settings, while many important problems and settings remain far from fully resolved.

Notably, RL in high-dimensional, sample-limited settings poses significant technical hurdles that call for both refined analysis techniques and new algorithmic ideas. In some sense, the paradigm shift from asymptotic to non-asymptotic RL theory is reminiscent of the rise of high-dimensional statistics in recent decades, where classical large-sample theory proved insufficient and ultimately motivated the development of fundamentally new mathematical techniques and algorithmic paradigms throughout modern statistics.

Roadmap of this monograph

In this monograph, we aim to present a coherent, non-asymptotic framework that covers core algorithmic principles as well as cutting-edge theoretical developments. In particular, we seek to provide sharp sample complexity characterizations for several RL algorithms, benchmarked against minimax lower bounds to assess their statistical optimality or suboptimality.

The bulk of this monograph is organized around the three sampling mechanisms introduced in § 1.1. We devote Chapters 3 and 4 to the generative-model setting, Chapter 5 to online RL, and Chapter 6 to offline RL. In each setting, we present optimal or widely used algorithms alongside sharp statistical guarantees and minimax lower bounds. For the major results, we provide largely self-contained proofs supplemented by intermediate arguments that build intuition. Chapter 7 moves beyond these canonical single-agent formulations and briefly introduces several extensions of growing importance, including distributionally robust RL, multi-agent RL, federated RL, statistical inference, and uncertainty quantification.

To develop these statistical foundations in the cleanest possible settings, we focus predominantly on tabular MDPs, where the state and action spaces are finite, and no low-dimensional function approximation is assumed. Notably, despite decades of study, the fundamental minimax sample complexity limits for tabular MDPs under the three canonical sampling mechanisms remained significantly unresolved for a long time and have only been settled—in several major, though not all, MDP models—very recently. We believe that obtaining a comprehensive understanding of these tabular fundamental limits constitutes an indispensable first step toward elucidating and ultimately achieving optimal statistical efficiency in RL at large, particularly in settings involving function approximation.

Relation to existing RL texts

Several existing books and monographs are particularly relevant to, or complementary with, the present monograph, and we briefly discuss a few of them here.

For introductory material, the foundational textbooks Sutton and Barto (2018), Bertsekas (2019), and Szepesvári (2010) provide excellent and comprehensive introductions to RL for readers with little or no prior background, albeit with much less emphasis on modern statistical foundations. The special cases of multi-armed bandits and contextual bandits have been covered in several books (e.g., Bubeck and Cesa-Bianchi (2012), Slivkins (2019), and Lattimore and Szepesvári (2020)), with Lattimore and Szepesvári (2020) offering perhaps the most comprehensive and up-to-date treatment of modern bandit theory. The monograph Agarwal et al. (2019a) has become a widely used resource for researchers entering RL theory, owing to its broad coverage of contemporary topics. A substantial portion of the present monograph instead focuses on advances in the statistical theory of tabular RL that appeared after Agarwal et al. (2019a) first became available. These include substantial progress in characterizing and attaining sharp statistical limits for RL with a generative model, online RL, and offline RL, as well as in deriving sharp sample complexity guarantees for Q-learning.

We also note the monograph by Foster and Rakhlin (2023), which develops a statistical perspective on interactive decision-making, with a primary focus on the general framework of function approximation. In addition, Zhang (2023) contains chapters on bandits and RL that briefly discuss several modern topics, including linear mixture MDPs and the Bellman–Eluder dimension. The recent book Guo et al. (2026) offers a comprehensive introduction to modern development of continuous RL from the perspective of stochastic control. The emphasis of these three texts, however, differs drastically from that of the present monograph, as none of them centers on the state-of-the-art statistical development in tabular RL.

Moreover, the present monograph grew out of several tutorials and courses taught by some of the authors in recent years, which also led to lecture notes such as Chi et al. (2025b). In comparison, Chi et al. (2025b) offers a bird’s-eye view of the recent statistical and algorithmic advances in RL, including developments in policy optimization, whereas the present monograph focuses more squarely on the statistical foundations of RL, offering a thorough and in-depth treatment accompanied by largely self-contained mathematical proofs.

Notation

For clarity and consistency, we introduce below a set of notation and conventions to be used throughout this monograph. Additional notation specific to individual chapters will be introduced later as needed.

Asymptotic notation. Let X:=(S,A,11−γ,H,1ε,1δ)\mathcal{X} := (S, A, \frac{1}{1-\gamma}, H, \frac{1}{\varepsilon}, \frac{1}{\delta}), where the meanings of these parameters will become clear shortly. The notation f(X)=O(g(X))f(\mathcal{X}) = O(g(\mathcal{X})) or f(X)≲g(X)f(\mathcal{X}) \lesssim g(\mathcal{X}) means there exists a universal constant C1>0C_1 > 0 such that f(X)≤C1g(X)f(\mathcal{X}) \le C_1 g(\mathcal{X}). Similarly, f(X)=Ω(g(X))f(\mathcal{X}) = \Omega(g(\mathcal{X})) or f(X)≳g(X)f(\mathcal{X}) \gtrsim g(\mathcal{X}) means that g(X)=O(f(X))g(\mathcal{X}) = O(f(\mathcal{X})), while f(X)=Θ(g(X))f(\mathcal{X}) = \Theta(g(\mathcal{X})) or f(X)≍g(X)f(\mathcal{X}) \asymp g(\mathcal{X}) indicates that f(X)=O(g(X))f(\mathcal{X}) = O(g(\mathcal{X})) and g(X)=O(f(X))g(\mathcal{X}) = O(f(\mathcal{X})) hold simultaneously. Additionally, the notation O~(⋅)\widetilde{O}(\cdot) (resp. Ω~(⋅)\widetilde{\Omega}(\cdot) and Θ~(⋅)\widetilde{\Theta}(\cdot)) is defined analogously to O(⋅)O(\cdot) (resp. Ω(⋅)\Omega(\cdot) and Θ(⋅)\Theta(\cdot)) but with logarithmic factors suppressed.

Notation

Vectors and matrices. For any vector a=[ai]1≤i≤n∈Rna = [a_i]_{1\le i\le n} \in\mathbb{R}^n, we overload several scalar operators in an entrywise manner. For instance, we write

a:=[ai]1≤i≤n,∣a∣:=[∣ai∣]1≤i≤n,anda2:=[ai2]1≤i≤n.\sqrt{a} := [\sqrt{a_i}]_{1\le i\le n}, \qquad|a| := [|a_i|]_{1\le i\le n}, \qquad\text{and} \qquad a^2 := [a_i^2]_{1\le i\le n}.

For vectors a=[ai]1≤i≤na = [a_i]_{1\le i\le n} and b=[bi]1≤i≤nb = [b_i]_{1\le i\le n}, the notation a≥ba \ge b (resp. a≤ba \le b) means that ai≥bia_i \ge b_i (resp. ai≤bia_i \le b_i) for all 1≤i≤n1 \le i \le n. We use

a∘b:=[aibi]1≤i≤na \circ b := [a_i b_i]_{1\le i\le n}

to represent the Hadamard product of aa and bb. We also let ∥A∥\lVert A\rVert and Tr⁡(A)\operatorname{Tr}(A) denote the spectral norm and the trace of a matrix AA, respectively.

For any finite set S={1,…,S}\mathcal{S} = \{1,\ldots,S\}, we let Δ(S)={p∈RS∣p≥0,1Tp=1}\Delta(\mathcal{S}) = \{p \in\mathbb{R}^{S} \mid p \ge0, \mathbf{1}^{\mathsf{T}}p = 1\} represent the set of probability vectors (i.e., the probability simplex) over S\mathcal{S}. For any matrix PP, we define its ℓ1\ell_1 norm (i.e., the maximum row-wise ℓ1\ell_1 norm) as

∥P∥1:=max⁡i∑j∣Pi,j∣.(1)\lVert P\rVert_1 := \max_i \sum_j |P_{i,j}|. \tag*{(1)}

Probability divergences. For any two probability measures PP and QQ defined on the same measurable space (Ω,F)(\Omega,\mathcal{F}), the total-variation (TV) distance between them is defined as

TV⁡(P,Q):=sup⁡A∈F∣P(A)−Q(A)∣.(2)\operatorname{TV}(P,Q) := \sup_{A\in\mathcal{F}} |P(A)-Q(A)|. \tag*{(2)}

When PP is absolutely continuous w.r.t. QQ, the Kullback-Leibler (KL) divergence and χ2\chi^2 divergence between them are defined respectively by

KL⁡(P∥Q):=∫(log⁡dPdQ)dP,(3)\operatorname{KL}(P\mathbin\Vert Q) := \int\left(\log\frac{\mathrm{d}P}{\mathrm{d}Q}\right)\mathrm{d}P, \tag*{(3)}
χ2(P∥Q):=∫(dPdQ−1)2dQ.(4)\chi^2(P\mathbin\Vert Q) := \int\left(\frac{\mathrm{d}P}{\mathrm{d}Q}-1\right)^2\mathrm{d}Q. \tag*{(4)}

These divergences play a central role in mathematical statistics, information theory, and probability theory as measures of the “statistical distance” between probability distributions; see, e.g., Tsybakov (2009) and Polyanskiy and Wu (2025), for in-depth discussions. These divergence metrics are also related through standard inequalities; for instance, it is well known that (Tsybakov, 2009)

2TV⁡2(P,Q)≤KL⁡(P∥Q)≤χ2(P∥Q).(5)2\operatorname{TV}^2(P,Q) \le\operatorname{KL}(P\mathbin\Vert Q) \le\chi^2(P\mathbin\Vert Q). \tag*{(5)}

Miscellaneous. For any positive integer nn, we denote [n]:={1,…,n}[n] := \{1,\ldots,n\}. We use eie_i to represent the ii-th standard basis vector. The notation 1{⋅}\mathbf{1}\{\cdot\} stands for the indicator function.

Preliminaries

We begin by reviewing the fundamentals of Markov decision processes (MDPs), the canonical framework for developing and analyzing reinforcement learning algorithms. This framework sets the stage for the discussions in the chapters that follow. We focus on two widely studied formulations: discounted infinite-horizon MDPs and finite-horizon episodic MDPs. Additionally, we present core concentration inequalities and techniques for deriving minimax lower bounds for multi-hypothesis testing, which serve as the primary analytical tools for this work.

Discounted infinite-horizon MDPs

We first introduce the basics and notation for discounted infinite-horizon MDPs. More comprehensive introductions can be found in classical references such as Puterman (2014) and Bertsekas (2017).

Basic components

A discounted infinite-horizon MDP can be represented by a tuple M={S,A,P,r,γ}\mathcal{M} = \{\mathcal{S}, \mathcal{A}, P, r, \gamma\}, which models an agent’s sequential interaction with its environment over an unbounded time horizon (see Figure 2.1 for an illustration). Its key components are:

An illustration of an MDP

Figure 2.1. An illustration of an MDP, where an agent interacts sequentially with the environment. At time step tt, the agent observes state sts_t of the environment, selects action ata_t, and receives an immediate reward rtr_t. The environment then transitions to state st+1s_{t+1} according to the transition kernel PP.

  1. S={1,…,S}\mathcal{S} = \{1, \ldots, S\}: a finite state space consisting of SS distinct states. A state can be used to describe, for example, the agent’s current situation (e.g., the position of a self-driving vehicle, the health condition of a patient in a treatment model).

  1. A={1,…,A}\mathcal{A} = \{1, \ldots, A\}: an action space consisting of AA distinct actions available to the agent (e.g., the direction of movement in a navigation task, the choice of medication to prescribe to a patient).

  1. P:S×A→Δ(S)P : \mathcal{S} \times\mathcal{A} \to\Delta(\mathcal{S}): the transition kernel of the MDP, where P(⋅∣s,a)P(\cdot\mid s, a) specifies the probability distribution over the next state when action aa is executed in state ss.

  1. r:S×A→[0,1]r : \mathcal{S} \times\mathcal{A} \to[0, 1]: a deterministic reward function, where r(s,a)r(s, a) indicates the immediate reward received by the agent for taking action aa in state ss. Rewards are normalized to lie

within [0,1][0,1] without loss of generality. To streamline presentation, we assume throughout this monograph that the reward function is fully known to the learner a priori. Note, however, that this assumption is not essential and can often be substantially relaxed.

(v) γ∈[0,1)\gamma\in[0,1): the discount factor, which guarantees that the infinite-horizon cumulative discounted reward is always finite (as we shall see shortly). The quantity 11−γ\frac{1}{1-\gamma} is commonly referred to as the effective horizon of M\mathcal{M}: it is proportional to the number of steps needed for a reward to be discounted to half its value, and hence reflects the time scale over which future rewards remain significant.

Here and throughout, we assume the discounted infinite-horizon MDP to be stationary (or time-homogeneous), meaning that both the transition kernel PP and the reward function rr are time-invariant. In addition, we introduce the following notation for convenience:

Ps,a:=P(⋅∣s,a)∈R1×S.(6)P_{s,a} := P(\cdot\mid s,a) \in\mathbb{R}^{1 \times S}. \tag*{(6)}

Throughout this monograph, we focus on regimes in which the number of states SS, the number of actions AA, and the effective horizon 11−γ\frac{1}{1-\gamma} are all large. This emphasis reflects a common challenge arising in modern RL practice, where algorithms must contend with vast state–action spaces and long horizons.

Policy, value function, Q-function, and occupancy distribution

At each time step, the agent selects an action based on the current state of the environment. Such an action-selection rule is called a policy. Formally, a stationary policy π:S→Δ(A)\pi: \mathcal{S} \to\Delta(\mathcal{A}) is a mapping that assigns to each state s∈Ss \in\mathcal{S} a probability distribution over actions; namely, π(a∣s)\pi(a \mid s) indicates the probability of selecting action aa in state ss. Clearly, this definition allows policies to be randomized. When π\pi is a deterministic policy, we often abuse the notation by letting π(s)\pi(s) represent the action selected by policy π\pi in state ss. We further denote Pπ:S→Δ(S)P_\pi: \mathcal{S} \to\Delta(\mathcal{S}) the equivalent transition probability when action aa is drawn from π(⋅∣s)\pi(\cdot\mid s), namely,

∀s∈S:Pπ(⋅∣s):=Ea∼π(⋅∣s)[P(⋅∣s,a)].(7)\forall s \in\mathcal{S} : \quad P_\pi(\cdot\mid s) := \mathbb{E}_{a \sim\pi(\cdot\mid s)}[P(\cdot\mid s,a)]. \tag*{(7)}

As a remark, for the stationary infinite-horizon MDPs considered here, it is typically sufficient to restrict attention to time-invariant policies.

To evaluate a policy, we introduce two central performance measures (or “utility functions”) in RL: the value function and the QQ-function. Consider a sample trajectory {(st,at)}t≥0\{(s_t,a_t)\}_{t \ge0} generated by the MDP under policy π\pi, which selects action ata_t after observing the state sts_t at time step tt (see Figure 2.2 for an illustration). Note that the trajectory depends on the policy π\pi as the choice of action ata_t affects the next state st+1s_{t+1}. We now define the value function and the Q-function associated with this policy.

An illustration of a trajectory rolled out by an MDP

Figure 2.2. An illustration of a trajectory rolled out by an MDP, where rt=r(st,at)r_t = r(s_t, a_t) is the immediate reward gained at time step tt.

  • Value function. The value function Vπ:S→RV^\pi: \mathcal{S} \to\mathbb{R} of policy π\pi is defined as the expected discounted cumulative reward:

∀s∈S:Vπ(s):=E[∑t=0∞γtr(st,at) | s0=s],(8)\forall s \in\mathcal{S} : \quad V^\pi(s) := \mathbb{E}\left[\sum_{t=0}^{\infty}\gamma^t r(s_t, a_t) \,\middle|\, s_0 = s\right], \tag*{(8)}

References

  1. [1]Li et al. (2024a).

Paper details

vlnc:W_juhvqqgq
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 3

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents