WorksEngineering
Intelligent Carbon Emission-aware Radio Unit Activation Control in Dense Heterogeneous Energy 6G Networks
Abstract
Base stations account for a dominant share of energy consumption and operational carbon emissions in radio access networks. In O-RAN deployments with heterogeneous energy sources, however, minimizing energy consumption alone does not necessarily minimize carbon emissions. In this work, we investigate carbon-aware DRL for RU sleep control. Under the adopted source-matched operational accounting model, renewable-matched RUs are assigned zero direct operational emissions, whereas non-renewable-supplied RUs incur operational emissions. The problem is formulated as a Markov decision process and evaluated using two carbon-aware reward formulations and a conventional energy–QoS reward baseline. We implement and compare TD3, PPO, and a heuristic method under realistic mobility and 3GPP channel models across multiple renewable-matched RU layouts and traffic loads. Simulation results show that TD3 learns more energy-efficient activation policies than PPO, achieving approximately 3%–6% lower energy consumption in representative scenarios. TD3 also achieves 8%–12% lower operational emissions than PPO in scenarios with a low proportion of renewable-matched RUs and up to a 20% reduction relative to the heuristic baseline under moderate renewable availability and high traffic load. Moreover, the joint energy,carbon and QoS reward achieves approximately 12%–28% lower operational emissions than the energy and QoS reward, while maintaining comparable energy consumption.
Introduction
The explosive growth of mobile data traffic has driven the large-scale densification of cellular infrastructures. While dense deployments significantly enhance network capacity and coverage, they simultaneously introduce substantial operational energy demands. Recent reports indicate that Base Stations (BSs) account for approximately 73%–77% of the total energy consumption in mobile networks [1], making them the dominant contributors to both operational expenditure and greenhouse gas emissions. A typical 5G macro BS consumes between 3.3 and 9 kW, with even higher values observed in massive MIMO and mmWave deployments [2]. As networks evolve toward ultra-dense 6G architectures, energy efficiency alone is no longer sufficient; reducing carbon emissions associated with network operation has become a central sustainability objective. Traditional optimization efforts have primarily focused on minimizing instantaneous power consumption. However, energy efficiency does not necessarily translate into carbon efficiency [3]. With the increasing adoption of renewable energy sources at cellular sites, modern radio access networks are progressively transitioning toward hybrid energy supply models, where some infrastructure elements are powered by renewable sources (e.g., solar or wind), while others remain dependent on carbon-intensive grid electricity. In such heterogeneous energy environments, minimizing total energy usage may not lead to minimal carbon emissions. Instead, sustainable network operation requires explicit consideration of the carbon intensity of different energy sources. This shifts the optimization paradigm from purely energy-aware control toward carbon-aware and sustainability-driven orchestration.
O-RAN provides a promising architectural foundation for enabling such sustainability-oriented control. By disaggregating the traditional monolithic BS into Central Unit (CU), Distributed Unit (DU), and RU components, O-RAN introduces programmability and openness through standardized interfaces [4]. Among these components, RUs are responsible for lower-PHY processing and RF transmission, and thus constitute a significant portion of radio access energy consumption [5]. Since RUs are typically dimensioned for peak traffic, placing a subset into sleep mode during lower demand periods provides operational flexibility. In hybrid renewable–grid deployments, such flexibility enables differentiated activation strategies, where renewable-powered RUs can be prioritized over grid-powered ones, facilitating structural carbon reduction beyond conventional energy-saving mechanisms.
The introduction of the RAN Intelligent Controller (RIC) further enhances O-RAN’s capability for sustainability-driven control. The Non Real Time RIC (Non-RT RIC) provides long-term policy guidance and optimization objectives via the A1 interface [6], while the Near Real Time RIC (Near-RT RIC) executes fine-grained radio control decisions through xApps interacting with O-DU and O-RU nodes via the E2 interface [7]. This hierarchical control paradigm naturally supports the integration of sustainability objectives into operational decision-making: long-term carbon-reduction policies can be translated into near-real-time resource management actions. As a result, O-RAN evolves from a flexible architectural framework into a programmable platform capable of supporting intelligent, carbon-aware radio access network operation.
Related Work
BS sleeping [8] has been extensively studied as a means to reduce energy consumption in cellular networks, especially during off-peak hours. Wu et al. [9] proposed two traffic-aware sleep strategies, threshold-based and vacation-based, that jointly optimize BS activation and transmit power, revealing a fundamental tradeoff between energy savings and user delay. In heterogeneous networks, energy savings can be achieved by selectively deactivating underutilized BSs. Zhou et al. [10] formulated a joint user association and BS operation framework to optimize long-term network utility under energy constraints. Their results show that activating only a subset of Small Base Stations (SBSs) based on traffic demand can significantly reduce energy use without degrading service quality. To incorporate renewable energy sources, Gong et al. [11] considered a two-stage optimization scheme for BS sleeping and subcarrier allocation, aiming to minimize non-renewable power consumption. Their approach dynamically adjusts BS states based on both harvested energy availability and user traffic, striking a balance between energy cost and blocking probability. Recent architectural advances enable finer-grained sleep control. Sun et al. [12] investigated BS sleeping in uplink–downlink fully decoupled RANs, jointly optimizing user association, power allocation, and BS activity. Similarly, Wu et al. [13] modeled BS sleep behavior using a Semi-Markov Decision Process (SMDP), enabling dynamic sleep scheduling in macro-femto networks based on user arrival patterns and energy efficiency. In addition to optimization-based methods, traffic prediction has been used to guide sleep decisions. Lin et al. [14] proposed a data-driven BS sleeping strategy based on spatio-temporal traffic forecasting using Graph Neural Network (GNN) and LSTMs. Spatio–temporal correlations are further leveraged in [15], where convolutional-Long Short-term Memory (LSTM) forecasting is coupled with an actor–critic controller to improve robustness under fluctuating traffic.
Recent studies have increasingly adopted DRL-based paradigms to tackle the BS sleep/active control problem, motivated by their ability to handle high-dimensional state spaces, long-term dependencies, and non-linear decision boundaries. Several representative learning-based solutions have been proposed in this context. For instance, [16], the authors propose a Deep Q-Learning Network (DQN)-based framework augmented with an auxiliary action-selection network that filters infeasible actions, thereby improving exploration efficiency and stabilizing training. To explicitly account for switching overheads and service degradation, [17] introduces a modular DRL architecture that combines a Deep Deterministic Policy Gradient (DDPG) policy with a supervised cost estimator and a traffic prediction module. This hybrid design allows the controller to internalize the long-term impact of sleep transitions. In [18], a PPO-based solution is developed to simultaneously coordinate sleep scheduling, cell zooming, user association, and Reconfigurable Intelligent Surface (RIS) configuration, addressing the coupling between discrete and continuous control variables in dense multi-cell environments. Risk-awareness and robustness have also received growing attention. In [19], a digital twin is integrated with a DQN controller to evaluate delay risk prior to action execution, enabling online retraining or feature suppression under abnormal traffic conditions. From a distributed control perspective, [20] models each BS as an autonomous agent and applies a multi-agent PPO with a state-similarity mechanism, jointly optimizing BS sleeping and MIMO antenna activation while mitigating oscillatory behavior.
Beyond pure energy minimization, recent green RAN research increasingly emphasizes where the energy is drawn from and how operational decisions translate into carbon footprint. In renewable energy powered cellular networks, the key objective is often reframed from minimizing total energy to minimizing on-grid energy, since renewable consumption is typically treated as carbon-free and its availability is both stochastic and spatially imbalanced across sites. For example, the paper [21] studies BSs powered by hybrid supplies and jointly optimize traffic management and energy cooperation to reduce grid energy usage while meeting throughput constraints, highlighting that renewable-aware topology/traffic control and energy exchange between sites are complementary levers rather than substitutes. Building on this viewpoint, a line of work investigates joint sleep control and renewable energy sharing in dense deployments. [22] formulate SBS sleep decisions together with renewable energy sharing as a unified Markov Decision Process (MDP), aiming to reduce grid energy consumption and improve renewable utilization under time-varying traffic and harvested energy. To address the curse of dimensionality in Ultra Dense Network (UDN)-scale deployments, they adopt a multi-discrete PPO design that factorizes the combinatorial control space into tractable sub-actions, demonstrating notable gains over schemes that treat sleeping and energy sharing separately. Complementary to terrestrial-only designs, vertical-assisted architectures have also been explored as a sustainability enabler. Paper [23] leverage a High Altitude Platform Station (HAPS) layer to offload traffic and enable more aggressive SBS sleeping, reporting sizeable reductions in grid power consumption while maintaining Quality of Service (QoS). Another strand explicitly models carbon footprint rather than energy alone and shows that operational policies interact with both operational and embodied emissions. [24] analyze network sharing as a mechanism to enable recurrent BS deactivation during low-load periods and quantify carbon-footprint mitigation beyond energy savings, including the effect of switching frequency on equipment degradation and embodied carbon through life-time extension. From an architectural perspective, O-RAN sustainability studies provide foundational power-consumption models and highlight that open and cloud-native O-RAN implementations may shift energy burdens across RU, DU and CU and transport components, motivating careful design of energy-aware control loops. For example, Baldini et al. [25] develop a parametric power model for O-RAN and use real RU deployment datasets to expose how placement and processing/transport assumptions shape end-to-end power trends, thus informing where carbon-aware operation is most impactful.
Contribution
Although the above studies are closely related to sustainable wireless network operation, they differ fundamentally from the focus of this paper. In this work, we do not consider energy sharing, battery dynamics, or joint traffic management. Instead, In this work, we consider an O-RAN system where RUs are powered by heterogeneous energy sources, including renewable energy and the conventional power grid. By explicitly distinguishing renewable-powered RUs, which incur zero operational carbon emissions, from grid-powered RUs, we reformulate the RU control problem as a carbon-aware sequential decision making task. This formulation fundamentally alters the structure of the control problem: RU activation decisions are no longer symmetric, and the same action may lead to different carbon outcomes depending on the underlying energy source. As a result, conventional energy-centric control policies may become suboptimal when carbon reduction is a primary objective. Moreover, unlike [22], which develops a customized multi-discrete PPO framework for large-scale joint control, we conduct a systematic comparison between deterministic off-policy learning and stochastic on-policy learning under the same carbon-aware MDP formulation. This enables us to reveal how different DRL paradigms respond to reward shaping and heterogeneous energy supply in sustainable O-RAN operation. Through extensive simulations, we compare representative policy-gradient and actor–critic methods and analyze their learning behaviors under the proposed sustainability-aware formulation.
The main contributions of this paper are summarized as follows:
We formulate a carbon-aware RU sleep mode control problem by explicitly modeling heterogeneous energy sources, where renewable-powered and grid-powered RUs lead to different carbon implications under identical activation decisions.
We design a sustainability-oriented reward function within the MDP framework that jointly captures network energy consumption, operational carbon emissions, and QoS satisfaction, enabling the learning agent to balance efficiency and environmental impact.
We implement and evaluate representative DRL approaches, including deterministic off-policy (TD3) and stochastic on-policy (PPO) methods, under a unified simulation framework, ensuring fair comparison across learning paradigms.
We provide a comprehensive performance analysis across multiple deployment scenarios, demonstrating that carbon-aware optimization introduces a trade-off between energy efficiency and emissions, and showing that TD3 achieves more stable convergence and consistently better energy–carbon performance compared to PPO and heuristic model.
System Model
We consider an O-RAN-based cellular network comprising a fixed set of RUs, denoted by , and a set of User Equipments (UEs), denoted by .
The RU positions are predetermined and remain unchanged, as shown in Figure 1. Time is divided into slots indexed by .

Figure 1. Proposed heterogeneous energy network
To model heterogeneous electricity-procurement arrangements, the set of RUs is partitioned into two disjoint subsets: (i) renewable-matched RUs, denoted by , whose electricity demand is contractually matched with renewable generation, and (ii) non-renewable-supplied RUs, denoted by , whose electricity demand is assigned the emission factor of the residual non-renewable grid supply. These subsets satisfy
Unlike deployments with on-site renewable generation, we adopt a source-matched, market-based operational accounting model. Electricity consumed by both RU categories may be physically delivered through the same power grid; therefore, the partition does not represent physically separated electricity flows. Instead, it represents different contractual generation attributes assigned to the electricity consumed by the RUs.
The renewable-energy contracts are assumed to provide sufficient matching for the electricity demand of RUs in over the considered control horizon. This is an exogenous scenario assumption rather than a decision made by the DRL agent. Renewable-generation forecasting, electricity procurement, storage operation, curtailment, and energy sharing are outside the scope of this work.
Under the adopted operational accounting boundary, RUs in are assigned zero direct operational emissions at the point of renewable-electricity generation, whereas RUs in are assigned a positive source-specific operational emission factor. This boundary does not include embodied or life-cycle emissions associated with energy-generation infrastructure. The resulting abstraction enables us to isolate the effect of heterogeneous electricity-procurement arrangements on carbon-aware RU activation control.
Each RU is equipped with Physical Resource Blocks (PRBs). Denote by the number of PRBs allocated from RU to UE at time slot . The total PRB usage of RU is
where is the normalized PRB load. The UE association and PRB allocation are handled by the simulation environment. Initially, each UE is associated with its nearest RU. When the serving RU is switched off, the UE is reassigned to the nearest active RU. During UE mobility, a handover to another active candidate RU is performed when the candidate requires fewer PRBs to meet the target rate and has sufficient residual PRB capacity. For each UE, the requested number of PRBs is calculated as the minimum number required to achieve under the current channel condition. If the total number of requested PRBs does not exceed the capacity of RU , each UE receives its requested PRBs. Otherwise, the available PRBs are allocated proportionally among the UEs associated with that RU while satisfying the RU-level PRB capacity. The achieved UE data rates are subsequently recalculated based on the allocated PRBs.
The Signal-to-Noise Ratio (SNR) of UE served by RU at slot is modeled as
where is the transmission power, is the gain of the channel including path loss and fading, is the spectral density of the noise power and is the bandwidth of a PRB. The achievable data rate of UE is then
To guarantee service quality, each UE must satisfy a minimum data rate requirement.
Power Consumption Model
The activation status of RU is described by a binary variable , where if RU is active and if RU is in sleep mode. The power consumption of each RU is composed of three components [16]. The first is fixed power consumption, denoted by , which represents the energy used by signal processing, cooling systems, and power supply units. This component depends solely on the operational mode of the RU. The second component is the load-dependent power consumption, denoted by , primarily attributed to the Power Amplifier (PA). In this work, the load is quantified based on the PRB utilization, as defined in (2). The third component is the transition power , which is incurred only when the RU switches from sleep mode to active mode. By summing these three components, the total power consumption of RU at time slot is given by:
with
Here and are the baseline power levels in active and sleep states, respectively; is the power amplifier efficiency; and captures the overhead due to switching from sleep to active states.
Carbon Emission Model
We consider direct operational emissions associated with the electricity consumed by the RUs. Let [kgCO/kWh] denote the source-specific operational carbon intensity assigned to RU at time slot [26]. Under the adopted market-based accounting boundary, the carbon intensity used in the simulations is
where is the operational carbon intensity assigned to the non-renewable electricity supply. The zero value assigned to renewable-matched RUs represents zero direct emissions at the point of renewable-electricity generation. It does not include embodied or life-cycle emissions associated with energy-generation infrastructure.
Since is measured in watts and the slot duration is measured in seconds, the electricity consumed by RU during time slot is [kWh]. Accordingly, the operational carbon emissions of RU are
The network-wide operational carbon emissions during time slot are
The cumulative operational emissions over the horizon are
The notation retains both RU and time indices so that the general formulation can accommodate RU-specific or time-varying carbon-intensity inputs. In the present simulations, however, we use the two source-specific values in (9) to isolate the effect of heterogeneous electricity procurement on RU activation control.
Problem Formulation
Given the system model, the network operation optimizes RU activity over slots . The objective is to minimize a weighted sum of renewable and non-renewable energy consumption:
s.t.
Here and denote the sets of renewable-energy and non-renewable-energy RUs, respectively. is the instantaneous power consumption of RU given its operational state and traffic load. The scalar controls the relative emphasis between renewable and non-renewable energy consumption. is the downlink rate of UE at slot and must satisfy the QoS requirement . Constraint (15) limits the PRB allocation per RU, while (16) enforces unique RU association for each UE.
Note that explicit carbon-emission equations are omitted from the formulation. Since only non-renewable consumption contributes to carbon emissions, minimizing the weighted energy consumption in (13) naturally encourages deactivation of non-renewable RUs, thereby reducing the carbon footprint.
Markov Decision Process Formulation
In this section, we formulate the RU sleep/active control problem as a MDP, which enables sequential decision-making under dynamic traffic, user mobility, and heterogeneous energy sources. At each time slot, the agent observes the network state, selects a joint RU activation action, and receives a reward that reflects the sustainability and service performance of the network. The environment then evolves according to user mobility, traffic demand variations, and wireless channel dynamics. This formulation allows the agent to learn long-term policies that balance energy efficiency, carbon emission reduction, and QoS satisfaction over time.
Formally, the MDP is defined by the tuple , where denotes the state space, the action space, the state transition probability, and the reward function. The detailed definitions of the state space, action space, and reward function are given as follows.
1) State Space: The state comprises essential information required for policy learning. At each time step , the state includes the real data rates of all UEs, expressed as
which reflects the system’s ability to meet user data rate requirements. The activation states of the RUs from the previous time step are given by
This information captures the temporal coupling between successive RU activation decisions and allows the agent to account for switching behavior.
The PRB utilization levels of the RUs are represented as
which quantify the instantaneous traffic load of each RU. In addition, the spatial positions of the UEs are included as
where denote the coordinates of UE at time . The UE positions directly affect path loss, user association, and achievable data rates, and thus influence RU load distributions. Including spatial information in the state enables the agent to anticipate future handovers, traffic shifts, and coverage-critical regions, rather than reacting solely to instantaneous load conditions.
Overall, the state at time is defined as
All state variables are normalized to comparable ranges to facilitate stable and efficient learning.
2) Action Space: The action represents the operational states of all RUs and is defined as
where each element denotes the operating mode of RU . Specifically, indicates that RU is active, while corresponds to sleep mode. The action is generated by the DRL policy and determines the network-wide RU activation pattern at each time slot.
3) Reward Function: The reward function is designed to guide the agent toward sustainability-aware RU control decisions by balancing energy efficiency, carbon emission reduction, and QoS satisfaction. To evaluate different sustainability objectives, we consider two carbon-aware reward formulations.
a) Energy–Carbon–QoS reward:
where denotes the total network energy consumption at time , represents the corresponding carbon emissions, and is the number of UEs whose data rates fall below the minimum QoS requirement. Each term is normalized to ensure comparable scales and to prevent dominance by any single component. This reward formulation jointly optimizes energy efficiency, carbon reduction, and service reliability.
b) Carbon–QoS-oriented reward:
which focuses explicitly on minimizing operational carbon emissions while maintaining QoS constraints. By removing the explicit energy term, this reward highlights scenarios where carbon reduction, rather than energy minimization, is the dominant objective, particularly in networks with heterogeneous renewable-powered and grid-powered RUs. The parameter in Eq. (13) provides a high-level representation of the relative importance of renewable and non-renewable energy consumption. This trade-off is implemented in the MDP through the reward weights. AG1 penalizes both total energy consumption and carbon emissions, AG2 penalizes only carbon emissions and therefore places the strongest preference on renewable-matched operation. Thus, the two reward formulations correspond to different implementations of the source-weighting principle represented by . And in both reward formulations, QoS constraints are enforced through the penalty term associated with . The temporal accumulation of this penalty discourages persistent QoS violations and promotes stable network operation.
Figure 2. Illustrates the workflow of TD3 training process. Red solid lines indicate critic training based on temporal-difference loss; green dashed lines represent actor updates guided by critic gradients and soft update target network.
Sustainable DRL-Based RU Sleep Control
To solve the formulated MDP, we adopt DRL methods to learn optimal RU activation policies under dynamic traffic variations and heterogeneous energy conditions. In particular, we consider two representative DRL paradigms: deterministic off-policy learning and stochastic on-policy learning.
Off-policy methods, such as TD3, learn the optimal policy using transitions sampled from a replay buffer, which may contain experiences generated by previous policies. This allows for improved sample efficiency and stable learning, as the agent can reuse past experiences and decorrelate data samples. Such properties are particularly advantageous in our setting, where the RU activation problem exhibits a high-dimensional combinatorial action space and strong coupling between decisions across RUs. The ability to leverage historical interactions enables more effective exploration of the action space and refinement of activation patterns over time.
In contrast, on-policy methods, such as PPO, update the policy using only data collected from the current policy, ensuring that the learning process closely follows the latest policy behavior. This results in more stable and theoretically grounded updates, as the policy gradient is computed with respect to the current data distribution. However, this also leads to lower sample efficiency and may limit exploration in complex environments. In the context of RU sleep control, where the reward landscape can be highly non-convex due to QoS constraints and heterogeneous energy sources, on-policy methods may converge faster in early stages but risk settling into suboptimal solutions.
Overall, the comparison between off-policy and on-policy learning provides valuable insights into the trade-offs between exploration, stability, and convergence performance in sustainability-aware RU control. This design enables a systematic evaluation of how different learning paradigms respond to the structural complexity of the problem.
TD3-Based Training for RU Sleep Control
The joint sleep/active control of multiple RUs inherently leads to a high-dimensional and strongly coupled action space. When the number of controllable RUs increases, discrete value-based approaches face an exponential growth of possible action combinations, which severely degrades exploration efficiency and learning stability. To overcome this limitation, we employ a continuous-control formulation based on the TD3 algorithm, which enables learning by relaxing the discrete RU activation decisions into a continuous action space during training.
Continuous Relaxation of RU Activation: In the proposed framework, the actor network parameterized by outputs a continuous action vector
where each element represents the activation tendency of an RU. This continuous representation allows the policy to express fine-grained preferences over RU activation levels, thereby avoiding direct enumeration of all binary activation patterns. Note that, the continuous actor output is interpreted as a proto-action rather than the physical operating state of an RU. Binary actions are executed in the environment and used to train the critics. During the actor update, the critic is evaluated at the continuous proto-action and provides a surrogate gradient over the relaxed action space. This approximation enables deterministic policy-gradient training without explicitly enumerating all joint binary actions. However, the resulting gradient is not the exact gradient of the thresholded binary policy, and approximation errors may arise between the interior of the relaxed action space and its binary vertices.
During interaction with the environment, the continuous actions are converted into executable binary decisions through a deterministic discretization mechanism combined with exploration noise. Specifically, Gaussian noise is added to each action dimension to encourage exploration,
followed by threshold-based discretization;
where denotes the operational state of RU . This hybrid continuous–discrete execution strategy enables efficient exploration in a smooth action space while ensuring compatibility with binary RU sleep/active control.
Critic Learning and Target Estimation: The learning process relies on an experience replay buffer that stores transitions , where the reward reflects energy consumption, carbon emissions, and QoS satisfaction according to the adopted sustainability objective.
To mitigate overestimation bias in value approximation, TD3 employs two independent critic networks, and , both of which estimate the state–action value function.
For each sampled transition, the target Q-value is computed using the minimum of the two target critics,
where and denote the parameters of the target actor and target critics, respectively. The noise term is used for target policy smoothing, which reduces sensitivity to small perturbations in the action space and improves learning robustness.
Each critic network is updated by minimizing the mean squared temporal-difference loss
Actor Update and Delayed Policy Optimization: The actor network is updated using the deterministic policy gradient, which adjusts the policy parameters to maximize the expected Q-value estimated by the critics,
To enhance stability, actor updates are performed less frequently than critic updates, following the delayed policy update strategy of TD3. This separation allows the critics to provide more accurate value estimates before the policy is adjusted.
Target Network Updates: Both the actor and critic target networks are updated using soft updates,
where controls the update rate. This mechanism smooths parameter evolution and further stabilizes training.
PPO-Based Training for RU Sleep Control
To provide a representative stochastic policy-gradient baseline, we implement PPO for the same RU sleep/active control task and environment as the TD3 agent. In our setting, the controller must decide, at every time slot, whether each RU should be active or sleeping, resulting in a multi-dimensional binary action vector . Unlike TD3, which learns a deterministic continuous policy and applies thresholding for execution, PPO directly models the joint RU activation decision as a stochastic policy over binary actions. This design enables on-policy learning with clipped policy updates while preserving the discrete nature of RU switching.
Policy Parameterization for Multi-RU Binary Actions: We adopt an actor-critic architecture with a shared feature extractor and two heads: an actor head producing RU-wise logits and a critic head estimating the state value. Given the observed state , the network outputs
where denotes the logits for all RUs, and is the state-value estimate. The RU activation policy is modeled as a factorized Bernoulli distribution
where is the sigmoid function and is a temperature parameter. Sampling from this distribution yields the RU activation vector , which is executed directly in the environment. This formulation maps naturally to RU sleep scheduling: each dimension corresponds to turning one RU active/sleep, and the stochastic policy allows PPO to explore different activation patterns under the same state.
Algorithm 1: TD3 Training for Sustainability-Aware RU Sleep Control
Input: Initial state ; actor ; critics ; replay buffer ; discount ; noise scales ; update delay ; soft update rate
Output: Trained actor policy for joint RU activation control
for episode do
Reset environment and observe ;
for do
// (1) Continuous activation scores for all RUs
Draw exploration noise ;
Compute activation scores , where ;
Clip to ;
// (2) Discretize to executable RU on/off decisions
Construct RU action vector by ;
// Optionally: ; interpret as the selected active RU set
// (3) Environment interaction
Execute , obtain reward and next state (reward uses the chosen carbon-aware objective);
Store transition into ;
Set ;
// (4) Critic update
Sample a mini-batch from ;
Draw target smoothing noise ;
Compute target scores , clip to ;
Discretize target action by ;
Compute TD target: ;
Update critics by minimizing, for : ;
// (5) Delayed actor and target updates
if then // update every steps
Update actor using deterministic policy gradient: ;
Soft update target networks: ; ;
Exploration Design in RU Scheduling
To avoid premature saturation of Bernoulli probabilities (i.e., or leading to near-zero entropy and poor exploration), we incorporate several exploration mechanisms during training. First, we inject Gaussian noise into the actor logits before sampling,
and apply temperature scaling through to control stochasticity. Both and are annealed over training to transition from exploration to exploitation. Second, an entropy bonus is added to the objective to encourage diverse RU activation decisions at early stages. These mechanisms are particularly important in our scenario because RU control is high-dimensional and strongly coupled via traffic load, user association, and QoS constraints; insufficient exploration may cause the policy to collapse to a fixed RU pattern that fails to adapt to changing UE distributions.
3) Rollout Collection and Advantage Estimation with Episode Boundaries: PPO is trained on on-policy rollouts collected by interacting with the same environment dynamics used by TD3, including mobility, handovers, and the RU-specific energy/carbon models. At each time step, the transition is recorded along with the log-probability and value estimate . We collect rollouts for a fixed number of steps and then perform multiple epochs of PPO updates.
To compute low-variance policy gradients, we use generalized advantage estimation (GAE). Importantly, in our implementation, advantage accumulation is reset at episode boundaries (either environment termination or time-limit truncation), preventing leakage of advantages across independent episodes:
This is crucial for RU scheduling since episodes are time-limited and the environment may reset due to simulation horizon; improper bootstrapping would distort the estimated returns and destabilize training.
4) Clipped PPO Objective for RU Sleep Decisions: Given a batch of rollout data, PPO updates the policy by maximizing the clipped surrogate objective:
where
and is the clipping parameter. The value function is trained with a squared error loss, and an entropy regularizer is used to promote exploration:
where is the estimated return, denotes entropy, and and control the value and entropy terms, respectively. In the RU sleep control context, this objective updates the RU activation probabilities so that actions yielding lower energy/carbon cost under QoS constraints become more likely, while the clipped update prevents abrupt policy shifts that could cause unstable RU switching.
5) Stability Measures and Implementation Details: To improve numerical stability in high-dimensional binary control, we clamp logits to bounded ranges before computing log-probabilities and apply gradient norm clipping during optimization. Moreover, reward normalization is applied per rollout to reduce scale sensitivity and to stabilize the advantage estimates. The actor and critic are optimized jointly using
separate learning rates for the shared backbone/actor head and the critic head. These design choices help mitigate training collapse (e.g., entropy going to zero) that can occur when the policy quickly becomes overconfident in a particular RU activation pattern.
Overall, PPO observes the same state representation as TD3 and outputs a stochastic RU activation vector that is executed directly in the environment. This provides a strong and widely adopted policy-gradient baseline for evaluating the suitability of stochastic versus deterministic control paradigms in sustainability-aware RU sleep scheduling.
Simulation Environment
Simulation Settings
In our simulations, each scenario is modeled as an O-RAN environment where RUs serve UEs within a square area of size . The considered scenarios explicitly differentiate between renewable-powered RUs and grid-powered RUs, enabling the evaluation of sustainability-aware RU sleep control under heterogeneous energy sources. Renewable-powered RUs are assumed to operate with zero operational carbon emissions, while grid-powered RUs incur both energy consumption and carbon emissions according to the adopted energy and carbon models.
In practical O-RAN deployments, the Near-RT RIC is typically hosted at edge cloud or regional edge data centers, with latency budgets ranging from 10 ms to 1 s. In our system model, the inference of the DRL agents is executed within the Near-RT RIC as an xApp, ensuring that RU sleep/active decisions satisfy the real-time control requirements. Model training is performed offline using simulated interaction data, while the trained policies are deployed for online inference. This deployment setting aligns with current O-RAN architectural principles and ensures realistic decision-making latency.
As illustrated in Figure 3, multiple spatial layouts are considered to examine the impact of renewable penetration and spatial distribution. The scenarios include different numbers of renewable-powered RUs (e.g., 2, 4, 6, and 8), as well as different spatial arrangements such as centralized, evenly distributed, and random placements. These layouts are designed to evaluate the robustness of the learned policies under varying renewable availability and spatial heterogeneity.

Figure 3. Spatial deployment scenarios with different renewable RU penetration levels and layouts. Green circles indicate renewable-powered RUs, while gray circles represent grid-powered RUs. These scenarios are designed to evaluate the robustness of RU sleep scheduling policies under varying renewable availability and spatial distributions.
UEs move according to a periodic mobility model with constant speed , where the mean speed is m/s and the standard deviation is m/s. Individual UE speeds are randomly drawn from this range. UEs periodically travel from the boundary of the service area toward the center and back, forming a structured cyclic mobility pattern. This mobility model induces time-varying traffic loads and handovers across RUs, which is critical for evaluating adaptive RU sleep control policies.
The wireless channel model follows the 3GPP Urban Microcell (UMi) specification [27] with both Line of Sight (LOS) and Non-Line of Sight (NLOS) conditions. For LOS links, the path loss is given by
where denotes the distance between RU and UE , is the carrier frequency in GHz, and is the breakpoint distance. For NLOS links, the path loss is modeled as
The LOS probability is defined as
with , and when m.
Each training episode consists of 200 time steps. To account for RU state transition latency and to match practical network operation timescales, each time step corresponds to 1.75 s. This value is motivated by the slow-timescale BS sleep-control model in [16], where the decision interval is selected according to the coherence time of large-scale fading and BS transition latency. In the present work, 1.75 s is used as the interval between two consecutive RU activation decisions and provides time for telemetry collection, policy inference, E2 signaling, and RU actuation. RU sleep/active decisions, energy consumption measurement, and carbon emission accounting are all performed at this temporal granularity. The instantaneous energy and carbon emission values are recorded at each step, while episode-level metrics are obtained by accumulation over the entire episode.
We evaluate two representative DRL approaches for centralized RU sleep control: the deterministic policy-based TD3 and
| Parameter | Value |
| DRL Training Parameters | |
| TD3 actor learning rate () | |
| TD3 critic learning rate () | |
| PPO learning rate () | |
| PPO clipping parameter () | 0.2 |
| GAE parameter () | 0.95 |
| Entropy coefficient () | 0.01 |
| Value loss coefficient () | 0.5 |
| Mini-batch size () | 128 |
| Replay buffer size (TD3 only) | |
| Discount factor () | 0.99 |
| Training episodes () | 2000 |
| Soft update coefficient (TD3, ) | 0.01 |
| Random seed | 42 |
| Network and Channel Parameters | |
| Carrier frequency () | 2 GHz |
| RU height () | 15 m |
| UE height () | 1.7 m |
| Network size () | 1000 m |
| Number of RUs () | 12 |
| Number of UEs () | 40, 80 |
| Noise power density () | dBm/Hz |
| Mobility and Traffic Parameters | |
| Average UE speed () | 2 m/s |
| Std. deviation of UE speed () | 0.5 m/s |
| Minimum data rate () | 6 Mbps |
| Energy and Carbon Model Parameters | |
| Active mode RU power () | 20 W |
| Sleep mode RU power () | 5 W |
| Mode transition power () | 3 W |
| Maximum transmit power () | 1 W (30 dBm) |
| Power amplifier efficiency () | 0.5 |
| Reward weights () | (1, 3, 10) |
| Reward weights () | (1, 2.5) |
Table I. Simulation Parameters
the stochastic policy-gradient-based PPO. Both agents share the same state representation, action space, and environment dynamics to ensure a fair and consistent comparison. To investigate the impact of sustainability-aware optimization, three different reward formulations are considered, summarized in Table II. AG1 jointly accounts for normalized energy consumption, carbon emissions, and QoS satisfaction, enabling a balanced trade-off between efficiency and sustainability. AG2 focuses on carbon-aware optimization by considering carbon emissions and QoS satisfaction only, while AG3 represents a conventional energy-driven design that optimizes energy consumption and QoS without explicitly incorporating carbon emissions [8]. This setup allows us to systematically analyze the interaction and trade-off between energy efficiency and carbon reduction. Detailed network architectures, optimizer configurations and system parameters are summarized in Table I.
| Reward function | Description |
| AG1 | Energy + Carbon + QoS |
| AG2 | Carbon + QoS |
| AG3 | Energy + QoS |
Table II. Comparison of Reward Function Designs
To provide a non-learning baseline, a green-aware hysteresis heuristic is implemented. The heuristic follows five rules. First, any RU serving at least one UE is kept active. Second, an RU is considered idle if it has no associated UEs and its PRB utilisation is below the idle threshold . Third, an idle RU is switched off after it has remained idle for consecutive time steps. Fourth, if the average user satisfaction falls below , or if any active RU has PRB utilisation above the congestion threshold , the heuristic activates nearby sleeping RUs to relieve congestion. When multiple candidate RUs are available, renewable-powered RUs are prioritised before grid-powered RUs. Finally, at least one RU is always kept active as a safety condition.
The idle threshold is set to , the congestion threshold is set to , and the minimum average user satisfaction is set to . The hysteresis duration is , meaning that an idle RU can be switched off immediately once the idle condition is satisfied. To avoid activating too many RUs in one control step, the maximum number of additionally activated RUs is limited to two. The heuristic is evaluated over 50 episodes, with 200 time steps per episode.
Numerical Results
Figure 4 compares the training convergence of TD3 and PPO under the 6-renewable-RU random deployment scenario for both 40-UE and 80-UE traffic loads. Two reward formulations, AG1 and AG2, are evaluated. For the moderate traffic case with 40 UEs, both algorithms converge within approximately the first 1000 episodes. PPO shows faster improvement during the initial training phase, which can be attributed to its stochastic exploration over binary RU activation patterns. However, TD3 improves more steadily and reaches higher final rewards under both reward functions. This suggests that, after sufficient exploration, the deterministic off-policy structure of TD3 is more effective in refining RU activation decisions.

Figure 4. Training convergence comparison between PPO and TD3 under the 6 renewable RU (random layout) scenario. The moving-average episode rewards are shown for 40 UE and 80 UE traffic loads, each evaluated with two independent reward functions (AG1 and AG2).
Under the heavier 80-UE load, the performance gap between TD3 and PPO becomes more pronounced. The increased traffic demand tightens QoS constraints and strengthens the coupling among RU activation, user association, and PRB utilization. In this more complex setting, PPO converges quickly but tends to stabilize in a suboptimal region, whereas TD3 continues to improve over a longer training horizon and achieves higher asymptotic rewards. This demonstrates the advantage of experience replay and twin-critic learning in handling high-dimensional RU sleep control under stringent service requirements.
The reward formulation also affects the convergence behavior. AG1, which jointly considers energy, carbon emissions, and QoS, generally leads to smoother learning because the energy term regularizes excessive RU activation. In contrast, AG2 focuses more directly on carbon reduction, creating a sharper and more asymmetric reward landscape between renewable- and grid-powered RUs. This makes learning more challenging, especially for PPO. TD3 remains comparatively stable across both reward functions, showing stronger robustness to reward shaping and traffic-load variations.
Figure 5 presents the average number of active renewable- and grid-powered RUs under different renewable deployment layouts. A clear operational transition is observed as the number of renewable RUs increases. In low-renewable scenarios, such as the 2-renewable-RU case, learning-based strategies activate nearly all renewable RUs but still rely on grid-powered RUs to satisfy coverage and QoS requirements. As renewable penetration increases, both TD3 and PPO progressively reduce grid-powered activation and shift toward renewable-dominant operation.

Figure 5. Average number of active renewable-powered and grid-powered RUs under different control strategies for various renewable penetration levels and spatial layouts. Each subfigure corresponds to a different deployment scenario. Results are shown for both 40 UEs and 80 UEs.
Among the evaluated methods, TD3 achieves the most compact activation structure. Across all renewable layouts and traffic loads, TD3 activates on average about one fewer RU than PPO and about 2.5 fewer RUs than the heuristic baseline. The difference is particularly visible under the 80-UE case, where PPO tends to keep more grid-powered RUs active to preserve QoS stability. By contrast, TD3 learns a more selective activation policy that better exploits the spatial distribution of renewable-powered RUs. In the 8-renewable-RU scenario under 40 UEs, TD3 nearly eliminates grid-powered RU activation, indicating strong carbon-aware prioritization.
The influence of reward design is also evident from the activation patterns. AG2 encourages stronger renewable utilization than AG1 because carbon emissions are directly penalized while total energy consumption is not explicitly included. As a result, AG2 may activate additional renewable-powered RUs to reduce grid dependence. AG1, on the other hand, balances energy and carbon objectives, leading to a more compact active set. Compared with the heuristic baseline, both DRL methods are more sensitive to renewable placement and traffic distribution, while TD3 provides the most effective carbon-aware reconfiguration.
Figure 6 presents the average network energy consumption for AG1 and AG2. For 40 UEs, the heuristic baseline has the highest energy consumption, remaining above 180 W in low-renewable scenarios. DRL-based methods reduce energy consumption across all renewable configurations, with TD3 consistently outperforming PPO. In the 2-renewable-RU case, TD3 saves approximately 5–10 W compared with PPO, corresponding to about 3%–6% improvement. Similar trends are

Figure 6. Average network energy consumption under different control strategies as a function of the number of renewable RUs for the scenario with 40 (left) and 80 (right) UEs. Results are shown for TD3, PPO with two reward functions (AG1 and AG2) and heuristic baselines.
observed for 80 UEs, where the absolute consumption is higher due to increased traffic demand, but TD3 still achieves typical savings of 5–8 W over PPO.
Reward design also affects energy consumption. AG1 usually consumes less energy than AG2 because it explicitly penalizes total power. AG2, by focusing more strongly on carbon reduction, may activate additional renewable-powered RUs to reduce emissions, leading to slightly higher total energy use. This difference is most visible in moderate-renewable scenarios, where AG1 provides an additional 2–4 W energy reduction.
Figure 7 compares AG1 with AG3, where AG3 includes energy and QoS terms only. AG3 achieves lower energy consumption than AG1 in most scenarios for both traffic loads, as expected, since it does not consider carbon emissions. However, this lower energy use comes at the cost of higher reliance on grid-powered RUs, as shown later in the carbon-emission results. Therefore, AG3 represents a pure energy-minimization policy, while AG1 provides a more sustainable balance between energy efficiency and carbon reduction.

Figure 7. Average network energy consumption under different control strategies as a function of the number of renewable RUs for the scenario with 40 (left) and 80 (right) UEs. Results are shown for TD3, PPO with two reward functions (AG1 and AG3) and heuristic baselines.
Figure 8 reports the average operational carbon emissions under AG1 and AG2. Carbon emissions decrease monotonically as the number of renewable RUs increases, confirming that renewable penetration shifts the network toward low-carbon operation. For 40 UEs, TD3 achieves the lowest emissions across all renewable configurations. In the 2-renewable-RU case, TD3-AG2 reduces emissions by approximately 8%–12% compared with PPO-AG2 and by more than 15% compared with the heuristic baseline. With higher renewable penetration, TD3 maintains a 6%–10% reduction over PPO and over 18% compared with the heuristic strategy.
Figure 8. Average operational carbon emissions under different control strategies as a function of the number of renewable RUs for the scenario with 40 (left) and 80 (right) UEs. The results illustrating how carbon-aware control policies increasingly rely on renewable-powered RUs and reduce grid-dependent emissions as renewable availability grows.
For 80 UEs, emissions increase due to heavier traffic, but the relative ranking remains unchanged. TD3-based policies again achieve the lowest emissions, particularly under 4–6 renewable RUs, where TD3-AG2 obtains about 10% lower emissions than PPO-AG2 and up to 20% lower emissions than the heuristic baseline. AG2 consistently produces lower emissions than AG1 because it directly prioritizes carbon reduction, while AG1 retains energy awareness and therefore offers a more balanced policy.
Figure 9 compares AG1 and AG3 in terms of carbon emissions. AG1 consistently outperforms AG3 across all renewable deployments and traffic loads, reducing emissions by approximately 12%–28%, with an average reduction of about 18%. This improvement comes from explicitly including carbon emissions in the reward, which encourages the agent to prioritize renewable-powered RUs and reduce grid-powered activation. By contrast, AG3 minimizes energy and QoS penalties only, and therefore may select energy-efficient but carbon-intensive RU configurations.
Figure 9. Average operational carbon emissions under different control strategies (AG1 and AG3) as a function of the number of renewable RUs for the scenario with 40 (left) and 80 (right) UEs.
Together, Figs. 7 and 9 demonstrate the trade-off between pure energy minimization and carbon-aware control. AG3 provides the lowest energy consumption, whereas AG1 achieves substantial carbon reduction with only a marginal energy increase. These results confirm that explicit carbon-aware reward design is necessary for sustainable RU sleep control in heterogeneous-energy O-RAN systems.
Table III reports the QoS performance of TD3 and PPO under different renewable-RU deployments and traffic loads. In most scenarios, both algorithms achieve a user satisfaction rate above 95%, indicating that the RU sleep-control policies generally maintain the required 6 Mbps user data rate. TD3 provides more stable QoS performance under the higher traffic load, with satisfaction rates ranging from 95.35% to 99.66%. In comparison, PPO falls below 95% in several cases, particularly under AG3 with six evenly distributed renewable RUs and under AG1 with eight renewable RUs. These results demonstrate that the energy and carbon reductions achieved by TD3 are generally obtained while maintaining a high level of user satisfaction.
| Scenario | Reward | 40 UEs | 40 UEs | 80 UEs | 80 UEs |
| TD3 | PPO | TD3 | PPO | ||
| 2 renewable RUs | AG1 | (99.25%) | (99.83%) | (98.25%) | (97.35%) |
| AG2 | (98.98%) | (99.40%) | (99.45%) | (98.89%) | |
| AG3 | (99.75%) | (98.63%) | (99.66%) | (99.61%) | |
| 4 renewable RUs (corner) | AG1 | (97.95%) | (99.70%) | (98.03%) | (99.84%) |
| AG2 | (99.50%) | (99.75%) | (99.00%) | (99.72%) | |
| AG3 | (99.72%) | (99.43%) | (99.06%) | (98.66%) | |
| 4 renewable RUs (middle) | AG1 | (96.67%) | (97.15%) | (98.61%) | (97.58%) |
| AG2 | (99.25%) | (97.40%) | (98.38%) | (95.01%) | |
| AG3 | (99.43%) | (98.63%) | (98.71%) | (98.66%) | |
| 6 renewable RUs (random) | AG1 | (99.30%) | (99.25%) | (99.05%) | (99.21%) |
| AG2 | (99.77%) | (99.77%) | (98.66%) | (98.86%) | |
| AG3 | (99.50%) | (98.75%) | (98.69%) | (98.58%) | |
| 6 renewable RUs (even) | AG1 | (99.38%) | (97.78%) | (95.35%) | (96.63%) |
| AG2 | (98.73%) | (97.85%) | (98.99%) | (99.50%) | |
| AG3 | (98.40%) | (93.92%) | (99.24%) | (92.66%) | |
| 8 renewable RUs | AG1 | (98.60%) | (98.47%) | (97.64%) | (89.61%) |
| AG2 | (98.95%) | (98.80%) | (97.66%) | (95.91%) | |
| AG3 | (99.28%) | (96.82%) | (99.41%) | (99.50%) |
Table III. QoS SATISFACTION ACHIEVED BY TD3 AND PPO UNDER DIFFERENT RENEWABLE-RU DEPLOYMENT SCENARIOS AND TRAFFIC LOADS.
Conclusion
This paper investigated carbon-aware RU sleep control in O-RAN under heterogeneous energy supply, where renewable-powered RUs are carbon-neutral and grid-powered RUs incur operational emissions. By formulating the problem as a sequential decision-making task and embedding sustainability objectives into reward design, we systematically evaluated TD3, PPO, and heuristic control across multiple renewable penetration levels, spatial layouts, and traffic conditions. The results reveal a clear operational transition from grid-dominant to renewable-dominant activation as renewable availability increases. In structural terms, TD3 activates on average about one fewer RU than PPO and about 2.5 fewer RUs than the heuristic baseline, indicating a more compact and carbon-aware activation pattern. In energy terms, TD3 consistently outperforms PPO, reducing average network energy consumption by 5–10 W in the 40UE case and 5–8 W in the 80-UE case, while AG1 further achieves 2–4 W lower energy consumption than AG2 in most moderate-renewable scenarios. In carbon terms, TD3-AG2 achieves 8%–12% lower emissions than PPO-AG2 in low-renewable scenarios, maintains 6%–10% lower emissions under higher renewable penetration, and reaches up to 20% lower emissions than the heuristic baseline under 80 UEs. In addition, AG1 reduces carbon emissions by approximately 12%–28% relative to AG3, with an average reduction of around 18%, confirming that explicit carbon awareness substantially reshapes RU activation behavior. These findings demonstrate that carbon-aware reward design is essential for sustainable O-RAN operation, and that deterministic off-policy learning provides a more effective solution for large-scale RU control under heterogeneous energy conditions.
References
- [1]J. S. Thompson, S. Fletcher, V. Friderikos, Y. Gao, L. Hanzo, M. R. Nakhai, T. O’Farrell, and P. D. Wells, “Editorial a decade of green radio and the path to “net zero”: A united kingdom perspective,” IEEE Transactions on Green Communications and Networking, vol. 6, no. 2, pp. 657–664, 2022.DOI
- [2]X. Liang, Q. Wang, A. Al-Tahmeesschi, S. B. Chetty, D. Grace, and H. Ahmadi, “Energy consumption of machine learning enhanced open RAN: A comprehensive review,” IEEE Access, vol. 12, pp. 81 889–81 910, 2024.DOI
- [3]H. Ahmadi, M. Rahmani, S. B. Chetty, E. E. Tsoropoulou, H. Arslan, M. Debbah, and T. Quek, “Towards sustainability in 6g and beyond: Challenges and opportunities of open ran,” IEEE Communications Standards Magazine, 2025.arxiv.org/abs/2503.08353
- [4]O-RAN Allience WG1, “O-RAN Architecture Description,” June 2025, accessed: June 2025. [Online]. Available: https://https://specifications.o-ran.org/specifications
- [5]A. I. Abubakar, O. Onireti, Y. Sambo, L. Zhang, G. Ragesh, and M. A. Imran, “Energy efficiency of open radio access network: A survey,” in 2023 IEEE 97th Vehicular Technology Conference (VTC2023-Spring). IEEE, 2023, pp. 1–7.DOI
- [6]O-RAN Allience WG2, “A1 Interface: general aspects and principles 3.0,” June 2025, accessed: June 2025. [Online]. Available: https://https://specifications.o-ran.org/specifications
- [7]O-RAN Allience WG3, “O-RAN E2 General Aspects and Principles (E2GAP) 8.0,” October 2025, accessed: October 2025. [Online]. Available: https://https://specifications.o-ran.org/specifications
- [8]X. Liang, A. Al-Tahmeesschi, S. Chetty, C. Avdar, B. Canberk, and H. Ahmadi, “Scalable machine learning-based approaches for energy saving in densely deployed open ran,” IEEE Transactions on Green Communications and Networking, vol. 10, pp. 2710–2722, 2026.
- [9]J. Wu, S. Zhou, and Z. Niu, “Traffic-aware base station sleeping control and power matching for energy-delay tradeoffs in green cellular networks,” IEEE Transactions on Wireless Communications, vol. 12, no. 8, pp. 4196–4209, 2013.DOI
- [10]T. Zhou, Y. Fu, D. Qin, X. Li, and C. Li, “Joint user association and bs operation for green communications in ultra-dense heterogeneous networks,” IEEE Transactions on Vehicular Technology, vol. 73, no. 2, pp. 2305–2319, 2023.DOI
- [11]J. Gong, J. S. Thompson, S. Zhou, and Z. Niu, “Base station sleeping and resource allocation in renewable energy powered cellular networks,” IEEE Transactions on Communications, vol. 62, no. 11, pp. 3801–3813, 2014.DOI
- [12]Y. Sun, H. Zhou, K. Yu, Y. Xu, B. Qian, and L. X. Cai, “Flexible base station sleeping and resource allocation for green uplink fully-decoupled ran,” IEEE Transactions on Wireless Communications, 2025.DOI
- [13]J. Wu, Y. Li, H. Zhuang, Z. Pan, G. Wang, and Y. Xian, “Smdp-based sleep policy for base stations in heterogeneous cellular networks,” Digital Communications and Networks, vol. 7, no. 1, pp. 120–130, 2021.DOI
- [14]J. Lin, Y. Chen, H. Zheng, M. Ding, P. Cheng, and L. Hanzo, “A data-driven base station sleeping strategy based on traffic prediction,” IEEE Transactions on Network Science and Engineering, vol. 11, no. 6, pp. 5627–5643, 2021.DOI
- [15]Q. Wu, X. Chen, Z. Zhou, L. Chen, and J. Zhang, “Deep reinforcement learning with spatio-temporal traffic forecasting for data-driven base station sleep control,” IEEE/ACM Transactions on Networking, vol. 29, no. 2, pp. 935–948, 2021.arxiv.org/abs/2101.08391
- [16]H. Ju, S. Kim, Y. Kim, and B. Shim, “Energy-efficient ultra-dense network with deep reinforcement learning,” IEEE Transactions on Wireless Communications, vol. 21, no. 8, pp. 6539–6552, 2022.
- [17]J. Ye and Y.-J. A. Zhang, “Drag: Deep reinforcement learning based base station activation in heterogeneous networks,” IEEE Transactions on Mobile Computing, vol. 19, no. 9, pp. 2076–2087, 2019.arxiv.org/abs/1809.02159
- [18]S. Sun, C. Huang, G. Chen, P. Xiao, and R. Tafazolli, “Deep learning-based traffic-aware base station sleep mode and cell zooming strategy in ris-aided multi-cell networks,” IEEE Transactions on Cognitive Communications and Networking, 2024.arxiv.org/abs/2412.18983
- [19]M. Masoudi, E. Soroush, J. Zander, and C. Cavdar, “Digital twin assisted risk-aware sleep mode management using deep q-networks,” IEEE Transactions on Vehicular Technology, vol. 72, no. 1, pp. 1224–1239, 2022.arxiv.org/abs/2208.14380
- [20]Y. Zhen, L. Tao, D. Wu, T. Tang, and R. Wang, “Energy-saving control strategy for ultra-dense network base stations based on multi-agent reinforcement learning,” Digital Communications and Networks, 2024.DOI
- [21]H.-S. Lee and J.-W. Lee, “Adaptive traffic management and energy cooperation in renewable-energy-powered cellular networks,” IEEE Systems Journal, vol. 14, no. 1, pp. 132–143, 2019.DOI
- [22]J. Gan, H. Kou, G. Yang, H. Zhang, Z. Cao, and W. Xu, “Joint sleep control and energy sharing strategy with deep reinforcement learning in green ultra-dense networks,” IEEE Transactions on Green Communications and Networking, 2025.
- [23]M. Salamotmaghdisi, A. Mehrabian, and H. Yanikomeroglus, “Energy sustainability in dense radio access networks via high altitude platform stations,” IEEE Networking Letters, vol. 6, no. 1, pp. 21–25, 2023.
- [24]D. Renga, M. Meo, and L. Nuaymi, “Network sharing: a pathway to sustainability and carbon footprint mitigation in radio access networks,” in 2025 36th International Teletraffic Congress (ITC-36). IEEE, 2025, pp. 1–9.DOI
- [25]G. Baldini, R. Bolla, R. Bruschi, A. Carrega, F. Davoli, C. Lombardo, and R. Rabbani, “Toward sustainable o-ran deployment: An in-depth analysis of power consumption,” IEEE Transactions on Green Communications and Networking, 2024.DOI
- [26]Y. Ding, H. Duan, M. Xie, R. Mao, J. Wang, and W. Zhang, “Carbon emissions and mitigation potentials of 5g base station in china,” Resources, Conservation and Recycling, vol. 182, p. 106339, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S092134492200177XDOI
- [27]3GPP, “Study on channel model for frequencies from 0.5 to 100 ghz,” ETSI, Technical Report TR 138 901, 2017. [Online]. Available: https://www.etsi.org/deliver/etsi_tr/138900_138999/138901.pdf


