betaFeedback

WorksComputer Science

SliceWise: Traffic-Aware Multi-Agent RL with Temporal Encoding for Joint Slice Admission and Resource Orchestration in O-RAN

Abstract

Ultra-reliable low-latency communication (URLLC) services in mission-critical applications such as industrial automation, remote healthcare, and autonomous systems demand stringent quality-of-service (QoS) guarantees under dynamic, resource-constrained wireless conditions. Open radio access networks (O-RAN) offer a software-defined and programmable foundation for embedding AI-driven control. This letter proposes SliceWise, a traffic-aware multi-agent deep reinforcement learning (MARL) framework for joint admission control, user-slice association, and resource orchestration in O-RAN. The control problem is modeled as a cooperative multi-agent Markov decision process, with long short-term memory (LSTM) networks capturing temporal traffic dynamics and dueling double deep Q-networks (D3QN) ensuring stable learning. Simulation results show that SliceWise substantially reduces delay violations and packet drops while improving slice-level reliability and resource efficiency, offering a scalable orchestration solution for AI-native O-RAN deployments.

Introduction

The proliferation of latency-sensitive and mission-critical applications in 5G and beyond, including industrial automation, remote healthcare, and connected autonomous systems, imposes stringent quality-of-service (QoS) requirements on radio access networks (RAN) [1]. Ultra-reliable low-latency communication (URLLC) demands millisecond-level end-to-end latency and extremely high reliability under dynamic and bursty traffic conditions [2]. Ensuring these guarantees in the presence of limited spectrum resources, heterogeneous service demands, and fluctuating network loads is challenging, particularly in network slicing environments where maintaining service-level agreements (SLAs) across slices during congestion is complex.

Open RAN (O-RAN) provides a disaggregated, software-defined architecture with programmable interfaces that enable fine-grained control of heterogeneous services [3]. The near-real-time RAN Intelligent Controller (near-RT RIC) facilitates data-driven decision-making for admission control, user-slice association, and resource orchestration [4]. While these capabilities position O-RAN as an ideal platform for embedding AI-driven control, existing resource and admission control mechanisms remain inadequate for URLLC. Static bandwidth partitioning, heuristic scheduling, and centralized optimization [5] often lack adaptability to bursty traffic, and recent machine learning (ML) [6] and reinforcement learning (RL) [7, 8] approaches typically treat admission control, slice association, and resource orchestration in isolation, neglecting their interdependence and the distributed nature of O-RAN.

This letter proposes SLICEWISE, a traffic-aware multi-agent deep reinforcement learning (MARL) framework for joint admission control, slice association, and resource orchestration in O-RAN. The problem is modeled as a cooperative multi-agent Markov decision process (MMDP), with agents specialized for each control function. Long short-term memory (LSTM) networks capture temporal traffic dynamics, and dueling double deep Q-networks (D3QN) provide stable and efficient policy learning in high-dimensional, non-stationary environments. The main contributions are as follows:

  1. We propose a unified MARL framework that jointly optimizes admission control, slice association, and resource allocation for O-RAN;

  2. We develop LSTM-based agent designs that exploit temporal features and balance URLLC reliability with resource efficiency;

  3. Extensive simulations show that SLICEWISE improves delay, reliability, and efficiency compared to random, empirical, and partially coordinated RL baselines.

System Model

We consider an O-RAN architecture supporting network slicing for differentiated service provisioning as illustrated in Figure 1. Let M={1,2,…,M}\mathcal{M} = \{1, 2, \ldots, M\} denote the set of user equipments (UEs) served by an O-RAN radio unit (O-RU) under a centralized unit (O-CU), which coordinates with the distributed unit (O-DU) to allocate R∈Z+R \in\mathbb{Z}^{+} physical resource blocks (PRBs).

Integrated admission control and resource orchestration framework in O-RAN architecture

Figure 1. Integrated admission control and resource orchestration framework in O-RAN architecture.

Inter-Slice Resource Partitioning and Admission Control

The system operates over a finite time horizon T={0,1,…,T−1}\mathcal{T} = \{0, 1, \ldots, T-1\}, with each time slot t∈Tt \in\mathcal{T} of fixed duration TsT_s. To support dynamic and SLA-driven traffic management, the RAN is logically partitioned into a set of active slices S={1,2,…,S}\mathcal{S} = \{1, 2, \ldots, S\}, each designed to serve a specific class of traffic. Resource allocation decisions are made at discrete reconfiguration windows, indexed by w∈W={0,1,…,⌊T/Tw⌋−1}w \in\mathcal{W} = \{0, 1, \ldots, \lfloor T/T_w\rfloor- 1\}, where TwT_w denotes the duration (in slots) of each window. The wthw^{\mathrm{th}} reconfiguration window starts at time twt_w and spans the time slots Tw={tw,tw+1,…,tw+Tw−1}\mathcal{T}_w = \{t_w, t_w+1, \ldots, t_w+T_w-1\}, with tw+1=tw+Twt_{w+1} = t_w+T_w to ensure non-overlapping intervals. At the beginning of each window, the near-RT RIC performs user admission control and inter-slice PRB partitioning. Each slice s∈Ss \in\mathcal{S} is assigned a resource budget Rsw∈Z+R_s^w \in\mathbb{Z}^{+}, constrained by the total PRBs, i.e., ∑s∈SRsw≤R\sum_{s\in\mathcal{S}} R_s^w \le R, ∀w∈W\forall w \in\mathcal{W}. Let Matw\mathcal{M}_a^{t_w} denote the set of UEs requesting admission at the start of window ww, and Mbtw\mathcal{M}_b^{t_w} denote the currently admitted users, with Matw∩Mbtw=∅\mathcal{M}_a^{t_w} \cap\mathcal{M}_b^{t_w} = \emptyset and Matw∪Mbtw=M\mathcal{M}_a^{t_w} \cup\mathcal{M}_b^{t_w} = \mathcal{M}. The binary admission decision for UE m∈Matwm \in\mathcal{M}_a^{t_w} is represented as amw∈{0,1}a_m^w \in\{0,1\}, where 1 denotes admission. The complete admission vector is defined as aw=[a1w,a2w,…,a∣Matw∣w]\mathbf{a}_w = [a_1^w, a_2^w, \ldots, a_{|\mathcal{M}_a^{t_w}|}^w]. The set of newly admitted UEs is denoted as Mktw={m∈Matw∣amw=1}\mathcal{M}_k^{t_w} = \{m \in\mathcal{M}_a^{t_w} \mid a_m^w = 1\}, and its cardinality is defined as Mktw=∣Mktw∣M_k^{t_w} = |\mathcal{M}_k^{t_w}|. The set of active UEs is then updated as Mbtw=Mbtw−1∪Mktw\mathcal{M}_b^{t_w} = \mathcal{M}_b^{t_{w-1}} \cup\mathcal{M}_k^{t_w}, and remains unchanged throughout the set of time slots t∈Twt \in\mathcal{T}_w, i.e., Mbt=Mbtw\mathcal{M}_b^t = \mathcal{M}_b^{t_w}. User-slice association is determined by a binary matrix Zw=[z1w,z2w,…,zMbtww]\mathbf{Z}_w = [\mathbf{z}_1^w, \mathbf{z}_2^w, \ldots, \mathbf{z}_{M_b^{t_w}}^w], where zmw=[zm,1w,zm,2w,…,zm,Sw]T\mathbf{z}_m^w = [z_{m,1}^w, z_{m,2}^w, \ldots, z_{m,S}^w]^{\mathsf{T}} assigns UE mm to exactly one slice, i.e., ∑s∈Szm,sw=1\sum_{s\in\mathcal{S}} z_{m,s}^w = 1, ∀w\forall w. Accordingly, the UEs associated with slice ss at window ww are defined as Mstw={m∈Mbtw∣zm,sw=1}\mathcal{M}_s^{t_w} = \{m \in\mathcal{M}_b^{t_w} \mid z_{m,s}^w = 1\} with ⋃s∈SMstw=Mbtw\bigcup_{s\in\mathcal{S}} \mathcal{M}_s^{t_w} = \mathcal{M}_b^{t_w}.

Intra-Slice Resource Allocation and Service Reliability

Each UE m∈Mm \in\mathcal{M} is located at a time-varying distance dm(t)d_m(t) from the O-RU at time slot t∈Tt \in\mathcal{T}. The average path loss βm(t)\beta_m(t) for UE mm at time tt is modeled as a weighted combination of line-of-sight (LoS) and non-LoS components [9] as βm(t)=p⋅FSPLm(t)+(1−p)⋅PLm(t)\beta_m(t) = p \cdot\mathrm{FSPL}_m(t) + (1-p) \cdot\mathrm{PL}_m(t), where p∈[0,1]p \in[0,1] is the LoS probability. The free-space path loss (FSPL) and non-LoS path loss (PL) components are computed as follows:

FSPLm(t)=20log⁡10(dm(t))+20log⁡10(fc)−27.55+η,(1)\mathrm{FSPL}_m(t) = 20\log_{10}(d_m(t)) + 20\log_{10}(f_c) - 27.55 + \eta, \tag*{(1)}
PLm(t)=20log⁡10(dm(t))+20log⁡10(fc)−27.55+ψε,(2)\mathrm{PL}_m(t) = 20\log_{10}(d_m(t)) + 20\log_{10}(f_c) - 27.55 + \psi_\varepsilon, \tag*{(2)}

where fcf_c is the carrier frequency, and η\eta, ψε\psi_\varepsilon represent the excess attenuation for LoS and non-LoS links, respectively. Let τmt(t)\tau_m^t(t) denote the end-to-end transmission delay for UE mm at time tt, which is computed as follows:

τmt(t)=δm(t)Nm(t)+(τmt(t−1)−Ts)+bm(t−1)bm(t).(3)\tau_m^t(t) = \frac{\delta_m(t)N_m(t) + \left(\tau_m^t(t-1)-T_s\right)^{+}b_m(t-1)}{b_m(t)}. \tag*{(3)}

where δm(t)\delta_m(t) is the packet size, Nm(t)N_m(t) is the number of packets, and bm(t)b_m(t) is the achievable data given by:

bm(t)=rm(t)Blog⁡2(1+SINRm(t)),(4)b_m(t) = r_m(t)B\log_2\left(1+\mathrm{SINR}_m(t)\right), \tag*{(4)}

with signal-to-interference-plus-noise ratio (SINR) for UE mm at time tt defined as SINRm(t)=P∣hm(t)∣2rm(t)BN0+Im(t)\mathrm{SINR}_m(t) = \frac{P|\mathbf{h}_m(t)|^2}{r_m(t)BN_0+I_m(t)}, where PP denotes the transmit power of the O-RU, rm(t)r_m(t) is the number of PRBs allocated to UE mm, BB is the bandwidth per PRB, ∣hm(t)∣2|\mathbf{h}_m(t)|^2 is the fading channel gain, N0N_0 is the noise power spectral density, and Im(t)I_m(t) is the interference power. The intra-slice resource constraint ensures that ∑m∈Mstrm(t)≤Rsw\sum_{m\in\mathcal{M}_s^t} r_m(t) \le R_s^w for each slice ss and time t∈Twt \in\mathcal{T}_w. To meet SLA requirements, each UE must satisfy the reliability condition defined as P(τmt(t)≤τmmax⁡)≥λ\mathbb{P}(\tau_m^t(t) \le\tau_m^{\max}) \ge\lambda, ∀m∈Mbt\forall m \in\mathcal{M}_b^t, where τmmax⁡∈{τe,τu}\tau_m^{\max} \in\{\tau_e,\tau_u\} corresponds to latency bounds for enhanced multimedia broadband (eMBB) (τe\tau_e) and URLLC (τu\tau_u) services, and λ∈(0,1)\lambda\in(0,1) is the fulfillment threshold.

Problem Formulation

Slice reliability measures the fraction of admitted UEs whose latency remains within SLA thresholds. It is defined at window ww for slice s∈Ss \in\mathcal{S} as:

Ωs(aw,rm(t),Zw,Rsw)=∏m∈Mstw(P(τmt(t)≤τmmax⁡)−λ),(5)\Omega_s(\mathbf{a}_w,\mathbf{r}_m(t),\mathbf{Z}_w,R_s^w) = \prod_{m\in\mathcal{M}_s^{t_w}} \left(\mathbb{P}\left(\tau_m^t(t) \le\tau_m^{\max}\right)-\lambda\right), \tag*{(5)}

where Mst=Mstw\mathcal{M}_s^t = \mathcal{M}_s^{t_w} for all t∈Twt \in\mathcal{T}_w and gsr(⋅):R→[0,1]g_s^r(\cdot): \mathbb{R} \to[0,1] is a smooth, monotonically increasing function quantifying the satisfaction margin of the latency requirement defined as

gsr(x)=11+exp⁡(−αsr(x+φ)),g_s^r(x) = \frac{1}{1+\exp\left(-\alpha_s^r(x+\varphi)\right)},

where αsr\alpha_s^r controls the slope and φ\varphi adjusts the offset to reflect sensitivity to slice-specific reliability. To ensure temporal reliability, we define a long-term utility that combines instantaneous and discounted future performance, jointly maximizing user admission and slice reliability at time tt:

F1(aw,rm(t),Zw,Rsw)=MktwMatw∏s∈SΩs(aw,rm(t),Zw,Rsw)+E(∑t′=1∞ζt′Γ(aw+⌊t′Tw⌋,rm(t+t′),Zw+⌊t′Tw⌋,Rsw,S)),(6)F_1(\mathbf{a}_w,\mathbf{r}_m(t),\mathbf{Z}_w,R_s^w) = \frac{M_k^{t_w}}{M_a^{t_w}}\prod_{s\in\mathcal{S}}\Omega_s(\mathbf{a}_w,\mathbf{r}_m(t),\mathbf{Z}_w,R_s^w) + \mathbb{E}\left(\sum_{t'=1}^{\infty}\zeta^{t'}\Gamma\left(\mathbf{a}_{w+\lfloor\frac{t'}{T_w}\rfloor},\mathbf{r}_m(t+t'),\mathbf{Z}_{w+\lfloor\frac{t'}{T_w}\rfloor},R_s^w,S\right)\right), \tag*{(6)}

where ζ∈(0,1)\zeta\in(0,1) is a discount factor and Γ(⋅)\Gamma(\cdot) represents the product of admission and slice reliabilities across future windows, expressed as Γ=Mktw+⌊t′/Tw⌋Matw+⌊t′/Tw⌋∏s∈SΩs(⋅)\Gamma= \frac{M_k^{t_{w+\lfloor t'/T_w\rfloor}}}{M_a^{t_{w+\lfloor t'/T_w\rfloor}}}\prod_{s\in\mathcal{S}}\Omega_s(\cdot), t∈Twt \in\mathcal{T}_w. A penalty term captures current and future PRB utilization across admitted users to enforce resource efficiency:

F2(rm(t))=1R∑m∈Mbtrm(t)+E(∑t′=1∞ζt′1R∑m∈Mbt′rm(t+t′)),(7)F_2(\mathbf{r}_m(t)) = \frac{1}{R}\sum_{m\in\mathcal{M}_b^t}r_m(t) + \mathbb{E}\left(\sum_{t'=1}^{\infty}\zeta^{t'}\frac{1}{R}\sum_{m\in\mathcal{M}_b^{t'}}r_m(t+t')\right), \tag*{(7)}

The overall optimization problem is thus formulated as:

max⁡aw,rm(t),Zw,Rsw  F1(aw,rm(t),Zw,Rsw)−νF2(rm(t)),(8)\max_{\mathbf{a}_w,\mathbf{r}_m(t),\mathbf{Z}_w,R_s^w} \; F_1(\mathbf{a}_w,\mathbf{r}_m(t),\mathbf{Z}_w,R_s^w) - \nu F_2(\mathbf{r}_m(t)), \tag*{(8)}

s.t.

∑s∈SRsw≤R,∀w∈W,(9)\sum_{s\in\mathcal{S}} R_s^w \le R,\quad\forall w\in\mathcal{W}, \tag*{(9)}
∑m∈Mstrm(t)≤Rsw,∀t∈Tw,(10)\sum_{m\in\mathcal{M}_s^t} r_m(t) \le R_s^w,\quad\forall t\in\mathcal{T}_w, \tag*{(10)}
Vs(aw,rm(t),Zw,Rsw)≥ϵs,∀s∈S,(11)V_s(\mathbf{a}_w,\mathbf{r}_m(t),\mathbf{Z}_w,R_s^w) \ge\epsilon^s,\quad\forall s\in\mathcal{S}, \tag*{(11)}
∑s∈Szm,sw=1,∀m∈Mbt, ∀t∈Tw.(12)\sum_{s\in\mathcal{S}} z_{m,s}^w = 1,\quad\forall m\in\mathcal{M}_b^t,\ \forall t\in\mathcal{T}_w. \tag*{(12)}

where ν\nu is a scalar balancing the trade-off between performance and efficiency, and ϵs∈(0,1)\epsilon_s \in(0,1) is the minimum reliability threshold for slice ss. The constraint in (11) enforces the expected long-term reliability of each slice, defined as

Vs(aw,rm(t),Zw,Rsw)=E[Ωs(.)+∑ℓ′=1∞ζℓ′Ωs(.)],V_s(\mathbf{a}_w,r_m(t),\mathbf{Z}_w,R_s^w)=\mathbb{E}\left[\Omega_s(.)+\sum_{\ell'=1}^{\infty}\zeta^{\ell'}\Omega_s(.)\right],

with arguments of Ωs(.)\Omega_s(.) implicitly shifting over future windows as in (6).

Methodology & Approach

To coordinate admission, association, and orchestration under dynamic conditions, we model the system as an MMDP and develop SLICEWISE, a DRL framework combining LSTM encoders with a D3QN for temporally aware control.

MMDP Framework and Agent Design

The proposed system consists of a set of agents N={ac,sa,re1,re2,…,reS}\mathcal{N}=\{\mathrm{ac},\mathrm{sa},\mathrm{re}_1,\mathrm{re}_2,\ldots,\mathrm{re}_S\}, where “ac” denotes the admission control agent, “sa” the slice association agent, and “res” the resource orchestrator for slice s∈Ss\in\mathcal{S}. The MMDP is defined by the tuple (N,Senv,{Oi}i∈N,{Ai}i∈N,Penv,{Ri}i∈N,γ)(\mathcal{N},\mathcal{S}_{\mathrm{env}},\{\mathcal{O}_i\}_{i\in\mathcal{N}},\{\mathcal{A}_i\}_{i\in\mathcal{N}},\mathbf{P}_{\mathrm{env}},\{R_i\}_{i\in\mathcal{N}},\gamma), where Senv\mathcal{S}_{\mathrm{env}} represents the global environment state, (Oi)(\mathcal{O}_i) and (Ai)(\mathcal{A}_i) are the observation and action spaces of agent i∈Ni\in\mathcal{N}, Penv\mathbf{P}_{\mathrm{env}} is the transition model, RiR_i is the reward function, and γ∈(0,1)\gamma\in(0,1) is the discount factor. Each agent follows a policy πi:Oi→Ai\pi_i:\mathcal{O}_i\to\mathcal{A}_i, selecting actions Ai∼πi(Ai∣oi)A_i\sim\pi_i(A_i\mid o_i) based on local observations oio_i. Agents receive feedback from the environment through rewards Ri(oi,Ai)R_i(o_i,A_i), guiding policy updates over time.

Observations and Action Spaces

Admission Control Agent

At the beginning of each time window tw∈Twt_w\in\mathcal{T}_w, the admission control agent observes a compact summary of the network’s current load and quality conditions. This includes available bandwidth, recent delay-violation trends, aggregate queue statistics, and average channel quality across both admitted and requesting UEs. Formally, its observation vector is given by:

oactw=[R⋅B,ϑw−1,ϱw−1,ςw−1,1Mbtw−1∑m∈Mbtw−1τmmax⁡,Matw−1,Mbtw−1,1Matw−1∑m∈Matw−1rm(tw−1)B,1Matw−1∑m∈Matw−1SINRmtw−1,1Mbtw−1∑m∈Mbtw−1SINRmtw−1,1Matw−1∑m∈Matw−1τmmax⁡],\mathbf{o}_{\mathrm{ac}}^{t_w}=\left[\mathcal{R}\cdot B,\vartheta_{w-1},\varrho_{w-1},\varsigma_{w-1},\frac{1}{M_b^{t_w-1}}\sum_{m\in\mathcal{M}_b^{t_w-1}}\tau_m^{\max},M_a^{t_w-1},M_b^{t_w-1},\frac{1}{M_a^{t_w-1}}\sum_{m\in\mathcal{M}_a^{t_w-1}}r_m(t_w-1)B,\frac{1}{M_a^{t_w-1}}\sum_{m\in\mathcal{M}_a^{t_w-1}}\mathrm{SINR}_m^{t_w-1},\frac{1}{M_b^{t_w-1}}\sum_{m\in\mathcal{M}_b^{t_w-1}}\mathrm{SINR}_m^{t_w-1},\frac{1}{M_a^{t_w-1}}\sum_{m\in\mathcal{M}_a^{t_w-1}}\tau_m^{\max}\right],

where averaged system delay violation rate within a time window ϑw=1Tw∑t∈Tw∣{m∈Mbt∣1τm(t)≥τmmax⁡}∣Mbt\vartheta_w=\frac{1}{T_w}\sum_{t\in\mathcal{T}_w}\frac{|\{m\in\mathcal{M}_b^t\mid\mathbf{1}_{\tau_m(t)\ge\tau_m^{\max}}\}|}{M_b^t}. Similarly, the average data processing requests from admitted users and new applicants are respectively ϱw=1Tw∑t∈Tw1Mat∑m∈MatNm(t)δm(t)\varrho_w=\frac{1}{T_w}\sum_{t\in\mathcal{T}_w}\frac{1}{M_a^t}\sum_{m\in\mathcal{M}_a^t}N_m(t)\delta_m(t) and ςw=1Tw∑t∈Tw1Mbt∑m∈MbtNm(t)δm(t)\varsigma_w=\frac{1}{T_w}\sum_{t\in\mathcal{T}_w}\frac{1}{M_b^t}\sum_{m\in\mathcal{M}_b^t}N_m(t)\delta_m(t). To maintain system feasibility, the maximum number of admissible UEs per time slot is constrained to Mmax⁡M_{\max}. Based on this observation, the agent selects an action Aac∈Aac={1,2,…,Mmax⁡}A_{\mathrm{ac}}\in\mathcal{A}_{\mathrm{ac}}=\{1,2,\ldots,M_{\max}\}, where each action represents admitting a subset of candidate UEs according to the learned policy that balances SINR conditions, queue occupancy, and latency satisfaction.

Resource Orchestration Agent

The orchestration agent operates at a finer timescale to adjust PRB allocations for each slice ss during every slot t∈Twt\in\mathcal{T}_w. Its observation combines delay bounds, SINR, traffic load, and current slice allocation:

orest=[1Mst−1∑m∈Mst−1τmmax⁡,1Mst−1∑m∈Mst−1SINRmt−1,∑m∈Mstrm(t−1)B,1Mst−1∑m∈Mst−1Nm(t−1)δm(t−1),RswB].\mathbf{o}_{\mathrm{res}}^t=\left[\frac{1}{M_s^{t-1}}\sum_{m\in\mathcal{M}_s^{t-1}}\tau_m^{\max},\frac{1}{M_s^{t-1}}\sum_{m\in\mathcal{M}_s^{t-1}}\mathrm{SINR}_m^{t-1},\sum_{m\in\mathcal{M}_s^t}r_m(t-1)B,\frac{1}{M_s^{t-1}}\sum_{m\in\mathcal{M}_s^{t-1}}N_m(t-1)\delta_m(t-1),R_s^wB\right].

From this observation, the agent selects Arest∈Ares={1,2,…,Nres}A_{\mathrm{res}}^t\in\mathcal{A}_{\mathrm{res}}=\{1,2,\ldots,N_{\mathrm{res}}\}, corresponding to PRB adjustment factors ρst\rho_s^t that scale the slice-level allocation RswR_s^w. The resulting per-UE bandwidth update is:

rm(t)=rm(t−1)+[ρstRswB^m(t)∑m∈MstB^m(t)],m∈Mst.(13)r_m(t)=r_m(t-1)+\left[\rho_s^tR_s^w\frac{\widehat{B}_m(t)}{\sum_{m\in\mathcal{M}_s^t}\widehat{B}_m(t)}\right],\quad m\in\mathcal{M}_s^t. \tag*{(13)}

where B^m(t)=δm(t)Nm(t)+(τmπ(t−1)−Ts)+bm(t−1)τmmax⁡\widehat{B}_m(t)=\frac{\delta_m(t)N_m(t)+(\tau_m^\pi(t-1)-T_s)+b_m(t-1)}{\tau_m^{\max}} estimates per-user bandwidth demand. To ensure resource allocation remains within capacity limits, we employ the minimum adjustment method.

Slice Association Agent

The association agent observes recent traffic sequences of all active users to infer their volatility patterns: osatw=[D1w,D2w,…,DMw,tw]\mathbf{o}_{\mathrm{sa}}^{t_w}=[D_1^w,D_2^w,\ldots,D_{M}^{w,t_w}], where,

Dmw=[Nm(tw−1)δm(tw−1),…,Nm(tw−θTw)δm(tw−θTw)]T.D_m^w=[N_m(t_w-1)\delta_m(t_w-1),\ldots,N_m^{(t_w-\theta T_w)}\delta_m(t_w-\theta T_w)]^{\mathsf{T}}.

The parameter θ\theta defines the number of past time windows considered, thereby controlling the temporal horizon over which traffic variability is captured. The resulting association matrix Zw\mathbf{Z}_w is defined as in Section II.

Reward Functions

The admission control agent’s reward function is formulated to optimize user admissions while ensuring stringent reliability constraints, which mathematically is defined as follows:

Ractw=uMbtwMatw−1∏m∈MbtwGc(τmmax⁡−τ‾m(tw)),(14)R_{\mathrm{ac}}^{t_w}=u\frac{M_b^{t_w}}{M_a^{t_w-1}}\prod_{m\in\mathcal{M}_b^{t_w}}\mathcal{G}_c\left(\tau_m^{\max}-\overline{\tau}_m(t_w)\right), \tag*{(14)}

where function Gc(x)=11+exp⁡(−αc(x+φc))\mathcal{G}_c(x)=\frac{1}{1+\exp(-\alpha_c(x+\varphi_c))}. Average transmission delay over a time slot twt_w is calculated as τ‾m(tw)=1Tw∑t∈Twτmt(t)\overline{\tau}_m(t_w)=\frac{1}{T_w}\sum_{t\in\mathcal{T}_w}\tau_m^t(t) and uu is a scaling factor. For the resource agent in charge of the orchestration of slice ss, the reward at time slot t∈Twt\in\mathcal{T}_w reflects system performance in terms of delay violation and resource efficiency defined as:

Rrest=1Mst∑m∈MstGs(τmmax⁡−τmt(t))−vGu(1Rsw∑m∈Mstrm(t)),(15)R_{\mathrm{res}}^t=\frac{1}{M_s^t}\sum_{m\in\mathcal{M}_s^t}\mathcal{G}_s(\tau_m^{\max}-\tau_m^t(t))-v\mathcal{G}_u\left(\frac{1}{R_s^w}\sum_{m\in\mathcal{M}_s^t}r_m(t)\right), \tag*{(15)}

where Gs(x)=2gs(x)−1\mathcal{G}_s(x)=2g_s(x)-1 and Gu(x)=11+exp⁡(−αu(x+φu))\mathcal{G}_u(x)=\frac{1}{1+\exp(-\alpha_u(x+\varphi_u))}.

Learning and Training Strategy

Training proceeds over multiple episodes of TT time slots. At episode start, UEs are randomly assigned eMBB or URLLC service types with corresponding delay constraints. Traffic dynamics, including packet count Nm(t)N_m(t), packet size δm(t)\delta_m(t), and distance dm(t)d_m(t), evolve via Markov chains with transition matrices Pqg\mathbf{P}_q^g, where q∈{1,2,3}q\in\{1,2,3\} indexes variable type and σ\sigma the traffic profile. Transitions are restricted to adjacent states to ensure smooth variability.

Each D3QN-based agent maintains separate target and online networks, with experience replay buffers of size Sd3qnbufferS_{\mathrm{d3qn}}^{\mathrm{buffer}} storing tuples (oi,Ai,Ri,oi′)(o_i,A_i,R_i,o_i'). Parameters include learning rate

μd3qn\mu_{d3qn}, discount factor γ\gamma, soft update factor λd3qnsoft\lambda^{\mathrm{soft}}_{d3qn}, and batch size Sd3qnBatchS^{\mathrm{Batch}}_{d3qn}. Admission and association agents update at each reconfiguration window, while resource agents update every slot. An ϵ\epsilon-greedy policy with decaying ϵ\epsilon guides exploration.

User–slice association is implemented as an LSTM classifier trained offline on traffic sequences Nm(t)δm(t)N_m(t)\delta_m(t) with volatility1 labels and kept fixed during RL. This preserves a stable mapping from temporal traffic patterns to slice labels, reduces non-stationarity in the control loop, and meets near-RT inference constraints. At runtime, the LSTM outputs slice mappings, followed by inter-slice resource allocation, i.e.,

Rs∈Sw=⌊R∑s∈SMtsw⌋.R^w_{s\in\mathcal{S}}=\left\lfloor\frac{\mathcal{R}}{\sum_{s\in\mathcal{S}}M^w_{t s}}\right\rfloor.

A warm-up period of θTw\theta T_w slots precedes policy execution.

Simulation Results and Analysis

The proposed SLICEWISE framework is implemented and evaluated on a customized Python-based version of the open-source AI-RAN Simulator [10]. The setup emulates a multi-cell O-RAN with a serving O-RU and two interfering O-RUs positioned 2.5 and 3 km apart, all operating on the same 2.5 GHz carrier. Inter-cell interference is incorporated in each UE’s SINR. The total bandwidth under carrier aggregation Btotal∈{350,…,1500}B_{\mathrm{total}} \in\{350,\ldots,1500\} MHz is discretized into R≈⌊Btotal/(Bscs×12)⌋\mathcal{R} \approx\left\lfloor B_{\mathrm{total}}/(B_{\mathrm{scs}} \times12)\right\rfloor PRBs with subcarrier spacing Bscs=15B_{\mathrm{scs}}=15 kHz. Key parameters and learning hyperparameters appear in Table I. Each simulation episode consists of T=100T=100 time slots of duration Ts=0.2T_s=0.2 s. Training is conducted for Ntrain=10,000N_{\mathrm{train}}=10,000 episodes, where the orchestration agent acts at every slot while admission and slice-association agents update once every Tw=20T_w=20 slots. Results are averaged over

ParametersValueParametersValue
dm(t)d_m(t) (m)[400,500][400, 500]MM, SS1515, 44
Nm(t)N_m(t)[50,75][50, 75]δm(t)\delta_m(t) (Mb)[0.003,0.6][0.003, 0.6]
N0N_0 (dBm/Hz)−163-163η\eta, ψξ\psi_{\xi} (dB)1.011.01, 2.232.23
τu\tau_u (s)0.0010.001τe\tau_e (s)0.10.1
γ\gamma0.990.99μd3qn\mu_{d3qn}5×10−55 \times10^{-5}
uu10001000NtrainN_{\mathrm{train}}1000010000
Hd3qnnetH^{\mathrm{net}}_{d3qn}{25,25}\{25,25\}NtestN_{\mathrm{test}}150150
Sd3qnbufferS^{\mathrm{buffer}}_{d3qn}5×1055 \times10^5λd3qnsoft\lambda^{\mathrm{soft}}_{d3qn}5×10−35 \times10^{-3}
Ns1,Ns2,Ns3N_s^1,N_s^2,N_s^310,10,1010,10,10θ,ν\theta,\nu5,0.25,0.2
αr1,αr2,αr3,αr4\alpha_{r1},\alpha_{r2},\alpha_{r3},\alpha_{r4}33,25,20,1633,25,20,16φ\varphi−0.0005-0.0005
αc,αu\alpha_c,\alpha_u20,1020,10φc,φu\varphi_c,\varphi_u−0.0005,−0.6-0.0005,-0.6
HlstmnetH^{\mathrm{net}}_{\mathrm{lstm}}{64,64,64}\{64,64,64\}μlstm\mu_{\mathrm{lstm}}0.00050.0005
Sd3qnBatchS^{\mathrm{Batch}}_{d3qn}256256SlstmBatchS^{\mathrm{Batch}}_{\mathrm{lstm}}256256

Table I. Simulation Parameters.

The network accommodates S=4S=4 active slices and M=15M=15 UEs distributed among them. Each UE’s traffic process is governed by a first-order Markov chain, capturing temporal correlation in both packet arrival count Nm(t)N_m(t) and packet size δm(t)\delta_m(t). For each slice, the instantaneous values of these variables are drawn uniformly from pre-defined intervals, with transitions constrained by a probability of 0.5, allowing either self-transition or progression to the next volatility state. To systematically examine framework adaptability, slices are configured with escalating volatility levels.

Slice 1 (benign) operates with narrow, stable ranges: for eMBB, δm(t)∈[0.01,0.02]\delta_m(t)\in[0.01,0.02] MB and Nm(t)∈[80,240]N_m(t)\in[80,240]; for URLLC, δm(t)∈[0.256,0.32]\delta_m(t)\in[0.256,0.32] Kb and Nm(t)∈[200,400]N_m(t)\in[200,400]. This variability is incrementally expanded across slices. For example, Slice 4 (highly volatile) uses δm(t)∈[0.01,0.06]\delta_m(t)\in[0.01,0.06] MB and Nm(t)∈[80,720]N_m(t)\in[80,720] for eMBB, and δm(t)∈[0.256,0.64]\delta_m(t)\in[0.256,0.64] Kb with Nm(t)∈[200,1600]N_m(t)\in[200,1600] for URLLC. While traditional slice assignment relies on predefined QoS classes, we infer association dynamically from observed traffic volatility, enabling adaptive alignment between user behavior and slice characteristics.

As shown in Figure 2(a)–(Figure 2b), cumulative rewards for admission and orchestration agents increase monotonically, indicating stable convergence. Delay-violation and packet-drop rates (Figure 2(c)–(Figure 2d)) decline rapidly, with the traffic-aware configuration (TA–MAROAC) converging faster and outperforming MARO and MAROAC, confirming the advantage of temporal modeling and joint optimization. The trained policy is tested on unseen multi-cell episodes with variable bandwidth and heterogeneous traffic. Compared with Rand, Empirical, MARO, and MAROAC, SLICEWISE achieves the highest reliability and lowest delay violations (Figure 3(a)–(Figure 3b)). Slice-level results (Figure 3(c)–(Figure 3d)) show sustained URLLC latency compliance even under interference.

Evolution of training metrics

Figure 2. Evolution of training metrics – (a) the reward of admission control agent, (b) the reward of resource allocation agent, (c) the delay violation rate, and (d) packet drop rate.

Comparative evaluation results varying $B_{\mathrm{total}}$ in MHz: (a) system reliability, (b) system delay violation, (c) slice-wise reliability, and (d) slice-wise delay violation

Figure 3. Comparative evaluation results varying BtotalB_{\mathrm{total}} in MHz – (a) system reliability, (b) system delay violation (c) slice-wise reliability, and (d) slice-wise delay violation.

Fig. 4(a) demonstrates that SLICEWISE maintains efficient PRB utilization without over-provisioning. Although the Empirical scheme slightly reduces packet-drop rates (Figure 4(b)), its ECCDF delay profiles in Figure 4(c)–(Figure 4d) exhibit long tails, indicating SLA violations. In contrast, SLICEWISE yields compact delay distributions for both eMBB and URLLC slices, reflecting stable queue dynamics and superior control over latency dispersion. As shown in Figure 5(a), SLICEWISE achieves the best joint optimization of user admission, SLA fulfillment, and resource utilization. The sensitivity analysis with respect to the latency fulfillment threshold λ\lambda (Figure 5(b)) shows that all methods degrade under stricter delay requirements, but SLICEWISE experiences slowest decline showing robustness and adaptability across varying conditions.

Comparative evaluation results varying $B_{\mathrm{total}}$ in MHz: (a) system resource utilization, (b) system packet drop rate, (c) ECCDF of delay of URLLC users, and (d) ECCDF of delay of eMBB users

Figure 4. Comparative evaluation results varying BtotalB_{\mathrm{total}} in MHz – (a) system resource utilization, (b) system packet drop rate (c) ECCDF of delay of URLLC users, and (d) ECCDF of delay of eMBB users.

Resource efficiency evaluation varying $B_{\mathrm{total}}$ in MHz: (a) resource efficiency under various orchestration schemes, (b) resource efficiency of the proposed scheme varying fulfillment threshold $\lambda$

Figure 5. Resource efficiency evaluation varying BtotalB_{\mathrm{total}} in MHz-(a) resource efficiency under various orchestration schemes, (b) resource efficiency of the proposed scheme varying fulfillment threshold λ\lambda.

Conclusion

This paper introduced SLICEWISE a traffic-aware multi-agent reinforcement learning framework that seamlessly integrates resource orchestration and admission control within O-RAN environments. To address the strict slice-specific requirements of URLLC and eMBB traffic, our approach combines a multi-agent deep reinforcement learning model with an LSTM network to dynamically manage user admissions, slice associations, and resource allocations. This dual-model strategy enables real-time optimization of latency, reliability, and resource efficiency while effectively balancing multiple objectives amid evolving network conditions. Notably, the D3QN-based agents in our framework exhibit robust performance, achieving significant reductions in delay violation and packet drop rates compared to baseline methods.

References

  1. [1]P. Rost, C. Mannweiler, D. S. Michalopoulos et al., “Network slicing to enable scalability and flexibility in 5G mobile networks,” IEEE Commun. Mag., vol. 55, no. 5, pp. 72–79, 2017.arxiv.org/abs/1704.02129
  2. [2]R. Han et al., “Dynamic network slice for bursty edge traffic,” IEEE/ACM Transactions on Networking, vol. 32, no. 4, pp. 3142–3157, 2024.
  3. [3]M. Polese, L. Bonati, S. D’Oro et al., “Understanding O-RAN: Architecture, interfaces, algorithms, security, and research challenges,” IEEE Commun. Surveys Tuts., vol. 25, no. 2, pp. 1376–1411, 2023.arxiv.org/abs/2202.01032
  4. [4]H.-T. Thieu, V.-Q. Pham, A. Kak, and N. Choi, “Demystifying the near-real-time RIC: Architecture, operations, and benchmarking insights,” in IEEE Conf. on Comput. Commun. Wkshps. (INFOCOM Wkshps 2023), Hoboken, NJ USA, May 2023.DOI
  5. [5]T. Wang and S. Wang, “Online convex optimization for efficient and robust inter-slice radio resource management,” IEEE Transactions on Communications, vol. 69, no. 9, pp. 6050–6062, 2021.DOI
  6. [6]N. Apostolakis, M. Gramaglia, L. E. Chatzieleftheriou, T. Subramanya, A. Banchs, and H. Sanneck, “ATHENA: machine learning and reasoning for radio resources scheduling in vRAN systems,” IEEE Journal on Selected Areas in Communications, vol. 42, no. 2, pp. 263–279, 2024.DOI
  7. [7]M. Yin, Y. Deng, A. Kak, N. Choi, and T. Han, “PSASlicing: Perpetual SLA-aware reinforcement learning for O-RAN slice management,” in IEEE Global Communications Conference (GLOBECOM 2024), Cape Town, South Africa, Dec. 2024, pp. 4534–4539.DOI
  8. [8]M. Zangooei, M. Golkarifard, M. Rouili, N. Saha, and R. Boutaba, “Flexible RAN slicing in open RAN with constrained multi-agent reinforcement learning,” IEEE Journal on Selected Areas in Communications, vol. 42, no. 2, pp. 280–294, 2024.DOI
  9. [9]K. Hanada, J. Zhang et al., “5G 3GPP-like channel models for outdoor urban microcellular and macrocellular environments,” in IEEE 83rd Vehicular Technology Conference (VTC Spring 2016), 2016, pp. 1–7.
  10. [10]A. Ntung, M. S. Rahman, T. Saha et al., “AI-RAN Simulator: Open-source simulator for AI-native O-RAN resource management,” https://github.com/ntutangyun/ai-ran-sim, 2023, accessed: Oct. 7, 2025.

Paper details

vlnc:W_afqund99
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents