betaFeedback

WorksComputer Science

Discovering Programmatic Policies from Reinforcement Learning-Based Traffic Signal Controllers

Abstract

Traffic signal control (TSC) is critical for mitigating urban congestion. Deep reinforcement learning (DRL) has achieved strong performance by learning adaptive policies from traffic observations, but learned neural controllers are often opaque, structurally complex, and difficult to generalize across scenarios. We propose PPL-TSC, a programmatic policy learning framework that distills compact and human-readable TSC policies from DRL teachers. PPL-TSC formulates policy learning as an evolutionary search over phase utility functions (PUFs), which score candidate signal phases and select the highest-utility phase for execution. Each generation consists of four steps: policy generation, policy review, policy calibration, and policy evaluation. In the generation step, a large language model (LLM) acts as a semantics-aware evolutionary operator to propose and refine candidate PUFs. Generated PUFs are screened through explicit checks for syntactic validity, mathematical soundness, and consistency with TSC principles, with invalid ones repaired through iterative feedback. Valid PUFs are then calibrated to improve behavioral fidelity to the DRL teacher, and evaluated using a joint score combining teacher fidelity and traffic-control performance to select candidates for the next generation. Extensive experiments across diverse traffic scenarios and representative DRL-based controllers show that PPL-TSC produces compact and human-readable policies, achieves competitive performance for most teacher families, improves transfer generalization, and substantially reduces computational cost. Code is available at https://github.com/ForrestXie9/PPL-TSC.

Introduction

Traffic signal control (TSC) is a core problem in intelligent transportation systems, with direct implications for urban congestion, travel efficiency, and transportation-related emissions [1, 2]. Classical approaches to TSC include fixed-time control [3, 4, 5], which relies on predetermined timing plans derived from historical demand, and adaptive control [6, 7, 8, 9], which adjusts signal decisions based on real-time traffic observations. While these paradigms have supported decades of deployment, they either lack responsiveness to non-stationary traffic patterns or depend heavily on manually designed control rules and simplified modeling assumptions (e.g., stationary demand) [10]. Designing effective TSC policies that are both adaptive and practically deployable remains a fundamental challenge.

Deep reinforcement learning (DRL) has become a prominent paradigm for adaptive TSC [11, 12, 13]. Through simulator interaction, DRL agents learn neural policies that map traffic states to signal phase decisions, achieving strong empirical performance across diverse traffic networks [14, 15]. Despite this progress, deploying neural controllers in practice remains challenging. First, learned policies lack explicit traffic-control concepts (e.g., queue pressure, downstream congestion), making it difficult to inspect whether their decisions follow established traffic-control principles [16]. Second, policies trained for specific traffic patterns often degrade under distribution shifts [17]. As shown in Table 1, most representative DRL methods remain stable under in-domain traffic-flow variations in Manhattan but show performance degradation when transferred to Los Angeles. Under cross-scenario transfer, methods like EcoLight [18], FRAP [15], and MPLight [19] suffer performance drops, while others (e.g., CoLight [14], TinyLight [20]) are inapplicable due to action-space mismatches (e.g., differing phase definitions and dimensionality), preventing direct policy reuse without architectural adaptation. Addressing such shifts typically requires retraining with additional data and computation. Third, neural controllers impose nontrivial memory and computational costs during real-time inference [20]. Figure 1 compares storage and floating-point operations (FLOPs) per decision. DRL-based methods require substantially more resources than classical rule-based controllers (e.g., fixed-time [3], SOTL [6], and MaxPressure [7]). This resource gap is particularly consequential for deployment on low-resource edge platforms.

TeacherΔ\DeltaATT (%) ↑\uparrowWin Rate (%) ↑\uparrow
In-dom.Cross-scen.In-dom.Cross-scen.
CoLight [14]−0.24±6.47-0.24 \pm6.47N/A50.0%N/A
EcoLight [18]−0.37±6.60-0.37 \pm6.60−51.79±42.43-51.79 \pm42.4350.0%8.0%
FRAP [15]−0.07±3.60-0.07 \pm3.60−20.73±38.45-20.73 \pm38.4565.0%24.0%
MPLight [19]−0.14±5.60-0.14 \pm5.60−2.34±32.67-2.34 \pm32.6745.0%28.0%
TinyLight [20]−62.62±172.58-62.62 \pm172.58N/A50.0%N/A

Table 1. Transfer performance of DRL-based TSC methods under Manhattan traffic-flow shifts and Manhattan-to-Los Angeles transfer. Note: ATT denotes average travel time. Δ\DeltaATT denotes relative ATT improvement under transfer; negative values mean the transferred controller performs worse (i.e., ATT increases). ATT win rate is the fraction of transfer cases in which ATT does not increase. N/A indicates methods not applicable in cross-scenario transfer.

Figure 1. Storage and computational costs of DRL-based and rule-based TSC methods.

These limitations have motivated growing interest in explicit policy learning for TSC, where controllers are represented as rules or programs rather than black-box neural networks [17, 21, 22]. Existing methods construct such policies using Monte Carlo tree search [23] or genetic programming [24], but they typically search from scratch through extensive interaction with the traffic environment. This process is costly in large combinatorial policy spaces and fails to leverage the control knowledge already encoded in DRL teachers. Moreover, these methods provide limited guarantees that the discovered rules remain semantically consistent with established TSC principles.

To address these challenges, we propose PPL-TSC, a unified programmatic policy learning framework that learns explicit and human-readable TSC policies from diverse DRL teachers. Instead of discovering explicit policies through trial-and-error exploration from scratch, PPL-TSC leverages rollout data collected from a pretrained DRL teacher to guide policy learning. We represent the student policy as a phase utility function (PUF), an explicit phase-scoring function that scores candidate phases using phase-level traffic features and selects the highest-scoring phase. Its explicit expression makes the contribution of each term inspectable. Overall, our main contributions are summarized as follows:

  • We propose PPL-TSC, a teacher-guided programmatic policy learning framework for TSC that distills explicit policies from pretrained DRL teachers, reducing reliance on costly policy search from scratch through extensive environment interaction.

  • We formulate policy learning as an iterative generate–review–calibrate–evaluate process over phase utility functions. The process integrates LLM-based program search, explicit validity checks, intersection-specific calibration, and joint evaluation to learn human-readable policies aligned with teacher behavior and TSC principles.

  • We conduct extensive experiments across representative DRL-based TSC methods and diverse traffic scenarios. PPL-TSC learns human-readable student policies that achieve competitive traffic

performance, improve generalization under transfer settings, and substantially reduce storage and computational costs relative to their DRL teachers.

Early TSC systems relied mainly on fixed-time strategies, such as Webster’s method [3], GreenWave [4], and MAXBAND [5], where signal timings are derived from historical demand and fixed during deployment. Adaptive methods later introduced online updates based on real-time traffic observations, including rule-based heuristics such as SOTL [6] and MaxPressure [7], and model-based systems such as SCOOT [8] and SCATS [9]. More recently, DRL has become a prominent paradigm for learning adaptive policies directly from traffic observations, reducing reliance on hand-designed rules and simplified traffic assumptions.

Existing DRL-based TSC methods can be broadly divided into value-based and policy-based approaches. Value-based methods, including FRAP [15], MPLight [19], CoLight [14], and TinyLight [20], estimate the long-term value of each candidate phase and select the phase with the highest value. Policy-based methods, such as A-CATs [25], policy-gradient methods [26], multi-agent actor–critic methods [27], and AttendLight [28], directly learn mappings from traffic states to control actions. Despite their empirical success, these neural controllers remain difficult to interpret, validate, and deploy in resource-constrained settings [17]. These limitations have motivated interest in explicit policy representations for TSC. Early work considered predefined polynomial control forms [29], whose parameters were fitted to mimic trained neural controllers. However, such fixed structures are often too restrictive to capture complex decision logic. More recent methods, such as π\pi-Light [17], GPLight+ [21], and SymLight [22], use program search operators, including Monte Carlo tree search [23] and genetic programming [30], to construct explicit control policies through interaction with the traffic environment. While these methods improve transparency, they typically search from scratch in a large combinatorial policy space, require substantial environment interaction, and make limited use of the knowledge already encoded in strong DRL teachers. In addition, evaluating candidate policies mainly through traffic-control performance does not guarantee that the resulting expressions are mathematically valid or semantically meaningful.

PPL-TSC differs from these approaches by learning programmatic policies directly from pretrained DRL teachers rather than discovering them from scratch. It uses teacher rollout data to guide policy learning and integrates generation, review, calibration, and evaluation, with explicit checks of generated functions and joint evaluation of teacher fidelity and control performance.

Method

We first formalize teacher-guided policy learning for TSC and define the student policy representation. We then describe the four steps of PPL-TSC: generation, review, calibration, and evaluation.

Problem Formulation

We consider a traffic network with MM signalized intersections. At decision step tt, intersection ii observes a local traffic state st(i)s_{t}^{(i)} and selects a signal phase pt(i)p_{t}^{(i)} from a feasible set Pt(i)\mathcal{P}_{t}^{(i)}. A signal phase is a predefined set of compatible, non-conflicting traffic movements that receive the right-of-way concurrently. This feasible set depends on the controller’s action space: phase-selection controllers directly choose among all feasible phases, whereas keep/switch controllers choose between keeping the current phase and switching to the next one, i.e., Pt(i)={pt,cur(i),next⁡(pt,cur(i))}\mathcal{P}_{t}^{(i)} = \{p_{t,\mathrm{cur}}^{(i)}, \operatorname{next}(p_{t,\mathrm{cur}}^{(i)})\}. Definitions of TSC terms are provided in Appendix A.1.1.

In DRL-based TSC, a neural policy πTch\pi^{\mathrm{Tch}}, referred to as the teacher policy, maps traffic states to signal phases and is learned through interaction with the environment. Executing the teacher over the network produces rollout data D=⋃i=1MDi\mathcal{D} = \bigcup_{i=1}^{M} \mathcal{D}_{i}, where Di={(st(i),ptTch,(i))}t=1Ni\mathcal{D}_{i} = \{(s_{t}^{(i)}, p_{t}^{\mathrm{Tch},(i)})\}_{t=1}^{N_{i}} contains the observed state and teacher-selected phase for intersection ii. Background on DRL-based TSC is provided in Appendix A.1.2.

Our objective is to learn a compact and human-readable student policy πStu\pi^{\mathrm{Stu}} that maps the same traffic states to phase decisions, approximates the teacher’s behavior, and maintains competitive traffic-control performance.

Student Policy Representation

We represent the student policy as a phase utility function (PUF) that scores candidate phases. Let xp,t(i)x_{p,t}^{(i)} denote the traffic features used to score candidate phase p∈Pt(i)p \in\mathcal{P}_{t}^{(i)}, constructed from the local state st(i)s_{t}^{(i)} according to the teacher’s input and action interfaces. The utility of phase pp is defined as

Up,t(i)=f(xp,t(i))+βp(i),(1)U_{p,t}^{(i)} = f\left(x_{p,t}^{(i)}\right) + \beta_{p}^{(i)}, \tag*{(1)}

where f(⋅)f(\cdot) is a programmatic expression shared across intersections to capture common decision logic, and βp(i)\beta_{p}^{(i)} is an intersection- and phase-specific bias term for local adaptation. For direct cross-scenario transfer, we retain f(⋅)f(\cdot) and set βp(i)=0\beta_{p}^{(i)} = 0 at each target intersection. For target-specific adaptation, the phase biases can be fine-tuned while keeping f(⋅)f(\cdot) fixed.

The student policy then selects the phase with the highest utility:

ptStu,(i)=arg⁡max⁡p∈Pt(i) Up,t(i).(2)p_{t}^{\mathrm{Stu},(i)} = \underset{p \in\mathcal{P}_{t}^{(i)}}{\arg\max}\ U_{p,t}^{(i)}. \tag*{(2)}

PPL-TSC Framework

PPL-TSC searches for the PUF in Eq. (1) through an evolutionary learning process with four steps: generation, review, calibration, and evaluation, as illustrated in Figure 2. The search proceeds for up to Gmax⁡G_{\max} generations with a population size NpopN_{\mathrm{pop}}.

Figure 2. Overview of PPL-TSC. The framework consists of three modules: data collection, policy learning, and decision making. Data collection gathers rollout samples from a pretrained DRL-based TSC teacher. Policy learning obtains a phase utility function (PUF) through iterative generation, review, calibration, and evaluation. Decision making uses the learned PUF f(x)f(x) with intersection-specific phase bias terms β\beta for real-time signal phase selection.

Policy Generation

The generation module uses an LLM as a program synthesis operator to propose candidate phase-scoring functions f(⋅)f(\cdot) from the teacher rollout data D\mathcal{D} [31, 32]. Its goal is to explore the space of readable phase-scoring rules while keeping generated candidates consistent with the available traffic features and basic TSC principles.

We use two prompt types. The initialization prompt T(Taskinit,Ctxinit,Cons,Out)\mathcal{T}(\mathrm{Task}_{\mathrm{init}}, \mathrm{Ctx}_{\mathrm{init}}, \mathrm{Cons}, \mathrm{Out}) creates the initial population from feature descriptions derived from teacher states. The evolution prompt T(Taskevol,Ctxevol,Cons,Out)\mathcal{T}(\mathrm{Task}_{\mathrm{evol}}, \mathrm{Ctx}_{\mathrm{evol}}, \mathrm{Cons}, \mathrm{Out}) generates offspring by conditioning on parent PUFs and their evaluation feedback. In both cases, Cons specifies syntactic, mathematical, and semantic requirements: generated code must be executable, numerical operations must remain robust under extreme inputs, and feature use should respect TSC principles such as monotonicity with respect to traffic load.

The output specification (i.e., Out) requires a candidate PUF together with a brief design rationale. Detailed example prompts and constraints are provided in Appendices § 7.2 and § 7.3.

Initialization prompts are used until the population reaches NpopN_{\mathrm{pop}}, after which evolution prompts generate offspring from selected parent PUFs.

Policy Review

After receiving candidate PUFs from the generation step, the policy review module checks whether the generated functions satisfy syntactic correctness, mathematical validity, and semantic consistency under the specified constraints. The review checks the generated function ff before calibration; the calibrated biases then adjust the scores of individual phases. This explicit verification is needed because LLM-generated functions may still violate constraints despite prompt guidance [33]. Candidates that fail verification are marked as invalid and returned to the LLM with diagnostic feedback for revision. An example repair prompt is provided in Appendix § 7.2.

Policy Calibration

Policy calibration adjusts the bias terms in Eq. (1) after a candidate PUF passes review. The phase-scoring function f(⋅)f(\cdot) is kept fixed and shared across intersections, while the phase biases are learned separately for each intersection from teacher rollouts. This design improves the student’s agreement with local teacher decisions while preserving the generated rule and avoiding further interaction with the traffic environment.

For intersection ii, let P(i)=⋃t=1NiPt(i)\mathcal{P}^{(i)} = \bigcup_{t=1}^{N_i} \mathcal{P}^{(i)}_t denote the union of candidate phase sets over the rollouts. We assign one bias term βp(i)\beta_p^{(i)} to each phase p∈P(i)p \in\mathcal{P}^{(i)}, collected in β(i)\boldsymbol{\beta}^{(i)}. We optimize β(i)\boldsymbol{\beta}^{(i)} to maximize agreement with teacher decisions on Di\mathcal{D}_i:

max⁡β(i)1Ni∑t=1Ni1(ptStu,(i)=ptTch,(i)),(3)\max_{\boldsymbol{\beta}^{(i)}} \frac{1}{N_i}\sum_{t=1}^{N_i} \mathbf{1}\left(p_t^{\mathrm{Stu},(i)} = p_t^{\mathrm{Tch},(i)}\right), \tag*{(3)}

where 1(⋅)\mathbf{1}(\cdot) is the indicator function, which equals 1 when the student and teacher select the same phase and 0 otherwise. Since this objective function is non-differentiable, we use CMA-ES [34], a robust black-box optimization algorithm, to optimize β(i)\boldsymbol{\beta}^{(i)} independently for each intersection.

Policy Evaluation

Policy evaluation ranks calibrated PUFs according to both teacher fidelity and closed-loop traffic performance. Fidelity measures whether the student reproduces the teacher’s phase choices, while traffic performance assesses whether the student remains effective when deployed in the traffic environment. We combine the two criteria using the joint score

J(U)=w1F(U)+w2P(U),(4)J(U) = w_1F(U) + w_2P(U), \tag*{(4)}

where F(U)F(U) denotes behavioral fidelity, P(U)P(U) denotes traffic performance relative to the teacher, and w1,w2≥0w_1,w_2 \ge0 balance the two terms. In our experiments, we use equal weights, w1=w2=1w_1 = w_2 = 1, for teacher-policy fidelity and downstream control performance.

Behavioral Fidelity F(U)F(U). We measure fidelity on the teacher rollout dataset D\mathcal{D} used in calibration, with student decisions given by Eq. (2). The fidelity score is the fraction of rollout samples on which the student and teacher select the same phase:

F(U)=1∣D∣∑i=1M∑t=1Ni1(ptStu,(i)=ptTch,(i)),(5)F(U) = \frac{1}{|\mathcal{D}|}\sum_{i=1}^{M}\sum_{t=1}^{N_i} \mathbf{1}\left(p_t^{\mathrm{Stu},(i)} = p_t^{\mathrm{Tch},(i)}\right), \tag*{(5)}

where 1(⋅)\mathbf{1}(\cdot) is the indicator function and ∣D∣=∑i=1MNi|\mathcal{D}| = \sum_{i=1}^{M} N_i. This metric lies in [0,1][0,1], with 1 indicating perfect agreement with the teacher on the rollout data.

Control Performance P(U)P(U). We evaluate traffic control performance by deploying the student policy in the same TSC environment used to test the teacher. Let ATTStuATT^{\mathrm{Stu}} and ATTTchATT^{\mathrm{Tch}} denote the average travel times of the student and teacher controllers, respectively. The relative improvement over the teacher is

Δ=ATTTch−ATTStu∣ATTTch∣+ϵ.(6)\Delta= \frac{ATT^{\mathrm{Tch}} - ATT^{\mathrm{Stu}}}{|ATT^{\mathrm{Tch}}| + \epsilon}. \tag*{(6)}

where ϵ=10−4\epsilon= 10^{-4} is used for numerical stability. A positive Δ\Delta indicates that the student achieves a lower average travel time than the teacher, Δ=0\Delta= 0 indicates equal performance, and Δ<0\Delta< 0 indicates worse performance.

We convert this improvement to a bounded performance score in (0,1](0, 1] as

P(U)=exp⁡(min⁡(Δ,0)).(7)P(U) = \exp\left(\min(\Delta, 0)\right). \tag*{(7)}

This score equals 1 when the student matches or outperforms the teacher and decreases smoothly when the student performs worse.

After evaluation, each offspring is compared with its parent using the joint score in Eq. (4). The offspring replaces its parent only if it achieves a higher score; otherwise, the parent is retained. The search terminates when the maximum number of generations Gmax⁡G_{\max} is reached.

Experiments

We evaluate PPL-TSC through six research questions. RQ1 (Performance): Can PPL-TSC match the traffic-control performance of its DRL teacher under identical evaluation settings? RQ2 (Fidelity): How accurately does PPL-TSC reproduce the teacher’s decision behavior? RQ3 (Generalizability): Does PPL-TSC generalize under same-scenario different-traffic-flow and cross-scenario settings? RQ4 (Readability): Can the learned student be expressed as a compact policy consistent with traffic control principles? RQ5 (Resource Efficiency): Is PPL-TSC more resource-efficient than neural teachers? RQ6 (Ablation): How do policy generation, calibration, and evaluation affect the effectiveness of PPL-TSC?

Experimental Setup

Simulator and traffic scenarios. We conduct experiments in CityFlow [35], an open-source microscopic traffic simulator. The benchmark includes single-intersection, corridor, and grid scenarios from seven road networks derived from five real-world urban areas [36]: three 1×11 \times1 Hangzhou networks (Hangzhou1/2/3), a 5×15 \times1 Atlanta corridor, a 3×43 \times4 Jinan grid, a 4×44 \times4 Los Angeles grid, and a 16×316 \times3 Manhattan grid. Jinan and Manhattan are structurally homogeneous, with all non-virtual signalized intersections sharing the same phase and link configurations, whereas Atlanta and Los Angeles are structurally heterogeneous due to multiple intersection templates. Each road network defines a scenario. For each scenario, we use one real-world traffic flow and generate four additional flows by perturbing each vehicle’s departure time with uniform noise in [−60s,60s][-60\text{s}, 60\text{s}].

Teacher controllers and metrics. We use five DRL-based TSC methods as teacher controllers: CoLight [14], FRAP [15], MPLight [19], EcoLight [18], and TinyLight [20]. EcoLight natively uses a keep/switch action interface, so we report EcoLight (native) in this original two-action setting and additionally evaluate an EcoLight (phase-selection) variant in which the student directly scores all feasible phases, consistent with the other teacher families. We evaluate PPL-TSC using average travel time (ATT) for control performance and average behavioral fidelity (ABF) for teacher-policy fidelity. We also report relative ATT improvement (Δ\DeltaATT), defined as the student’s relative ATT reduction over the teacher, and ATT wins, where the student achieves ATT no worse than its teacher.

Performance and Fidelity (RQ1–RQ2)

We first evaluate whether PPL-TSC preserves the control performance and decision behavior of its DRL teachers in the original deployment setting. PPL-TSC (Tailored) learns a separate policy for each teacher–scenario–flow combination, whereas PPL-TSC (Unified) learns a shared programmatic policy for each teacher family across multiple scenarios. Table 2 reports Δ\DeltaATT, ATT wins, and ABF over 35 scenario–flow combinations for each teacher family.

TeacherPPL-TSC (Tailored)PPL-TSC (Unified)
Δ\DeltaATT (%) ↑\uparrowATT wins ↑\uparrowABF ↑\uparrowΔ\DeltaATT (%) ↑\uparrowATT wins ↑\uparrowABF ↑\uparrow
CoLight11.51 ±\pm 23.9817/350.71 ±\pm 0.1716.70 ±\pm 32.9818/350.60 ±\pm 0.22
FRAP11.79 ±\pm 27.1224/350.63 ±\pm 0.1612.07 ±\pm 32.9922/350.58 ±\pm 0.19
MPLight0.54 ±\pm 6.1621/350.73 ±\pm 0.12−0.03 ±\pm 10.9617/350.69 ±\pm 0.13
EcoLight (native)−18.05 ±\pm 41.6615/350.84 ±\pm 0.07−23.27 ±\pm 36.178/350.82 ±\pm 0.07
EcoLight (phase-selection)22.70 ±\pm 27.2330/350.48 ±\pm 0.1120.99 ±\pm 30.6330/350.40 ±\pm 0.12
TinyLight−8.43 ±\pm 38.6021/350.64 ±\pm 0.16−25.19 ±\pm 34.568/350.61 ±\pm 0.12

Table 2. Performance and fidelity of PPL-TSC over 35 scenario–flow combinations.

For RQ1, PPL-TSC achieves competitive closed-loop performance relative to its teachers for most teacher families. Tailored obtains a positive mean Δ\DeltaATT for CoLight, FRAP, MPLight, and EcoLight (phase-selection), while Unified improves CoLight, FRAP, and EcoLight (phase-selection) and remains nearly neutral for MPLight. These results indicate that compact programmatic policies can preserve, and often improve, the teacher’s control quality in the original deployment setting.

The main performance exceptions occur when PPL-TSC learns from TinyLight and EcoLight (native). The student policy distilled from TinyLight underperforms its teacher on average, especially under Unified. TinyLight’s richer phase-level and incoming-lane inputs enlarge the search space, making the default search budget less effective. For EcoLight (native), the two-action keep/switch interface makes closed-loop performance sensitive to accumulated switching errors, leading to weaker ATT results especially in larger networks such as Jinan, Los Angeles, and Manhattan (Appendix § 7.8.2). Controlled diagnostic experiments for these two cases are provided in Appendix § 7.7.1.

For RQ2, PPL-TSC maintains moderate behavioral fidelity to the teacher policies. Tailored consistently achieves higher ABF than Unified across teacher families, indicating that fitting a policy to each teacher–scenario–flow setting tends to better match local phase decisions than using a single policy across multiple scenarios. EcoLight (native) achieves the highest ABF under both Tailored (0.84 ±\pm 0.07) and Unified (0.82 ±\pm 0.07) settings, although its ATT results remain weaker than those of the teacher. Among the phase-selection teachers, MPLight provides the strongest fidelity–performance balance: both variants stay close to the teacher in ATT while achieving high ABF values, suggesting that its decision logic is more amenable to approximation by a compact programmatic policy. Detailed scenario-level results are provided in Appendix § 7.8.2.

Generalizability (RQ3)

We evaluate generalizability under two transfer regimes: (i) same-scenario, different-traffic-flow, where a model trained on one traffic flow is tested on other traffic flows within the same scenario, and (ii) cross-scenario, where a model learned in one scenario is applied to different scenarios. For same-scenario, different-traffic-flow transfer, each scenario contains five traffic flows; each traffic flow is used once as the source flow and evaluated on the other four flows, yielding 20 transfer cases per scenario and 140 cases per teacher. For cross-scenario transfer, each source traffic flow is evaluated on five traffic flows from each of the other six scenarios, yielding 150 transfer cases per source scenario and 1050 cases per teacher. Table 3 reports relative ATT improvements and ATT win counts with respect to the transferred teacher policies.

(a) Same-scenario, different-traffic-flow(b) Cross-scenario
TeacherΔ\DeltaATT (%) ↑\uparrowATT wins ↑\uparrowTeacherΔ\DeltaATT (%) ↑\uparrowATT wins ↑\uparrow
CoLight15.66 ±\pm 38.4961/140EcoLight (native)33.42 ±\pm 77.99938/1050
EcoLight (native)24.51 ±\pm 28.37122/140EcoLight (phase-selection)44.85 ±\pm 34.32954/1050
EcoLight (phase-selection)25.19 ±\pm 31.00120/140FRAP54.22 ±\pm 39.35887/1050
FRAP11.84 ±\pm 36.9689/140MPLight46.04 ±\pm 36.10946/1050
MPLight0.57 ±\pm 11.7955/140
TinyLight−4.48 ±\pm 50.0598/140

Table 3. Generalization under two transfer settings. Higher Δ\DeltaATT and ATT wins are better. CoLight and TinyLight are not applicable to cross-scenario transfer due to architectural limitations.

Table 3 shows different patterns across the two transfer regimes. Under same-scenario, different-flow transfer, PPL-TSC achieves the strongest gains for CoLight, EcoLight (native), EcoLight (phase-selection), and FRAP, indicating that the learned policies generalize beyond the source flow under moderate flow shifts. Cross-scenario transfer is more challenging due to changes in both network topology and traffic patterns. In this setting, programmatic students for EcoLight (native), EcoLight (phase-selection), FRAP, and MPLight achieve substantial improvements over the transferred teachers. These comparisons measure gains over transferred teachers, whose performance can itself deteriorate under distribution shift; they do not establish absolute control quality on the target networks. A separate comparison using MPLight rollouts evaluates PPL-TSC against π-Light-Distill, GPLight+-Distill, and SymLight-Distill under the same teacher-guided protocol (Appendix § 7.8.5). PPL-TSC achieves the lowest mean and median cross-scenario ATT among these methods, providing additional evidence of transfer effectiveness beyond the teacher comparison.

Readability of the Learned Student Policy (RQ4)

We examine the MPLight-derived student, PPL-MPLight, on Hangzhou3. The student uses the teacher’s current-phase and movement-pressure inputs. Figure 3 traces the learned rule, and Table 4 reports its calibrated biases and performance. The learned policy is readable because each term has a clear traffic-control interpretation. Utility increases with both aggregate phase pressure and the maximum directional pressure within a phase, giving priority to phases serving congested movements. A small current-phase retention term discourages unnecessary switching, and the calibrated biases provide phase-specific offsets around the shared pressure-based rule.

Figure 3. Evolution of student (i.e., PPL-MPLight) on Hangzhou3. p^d,t\hat{p}_{d,t} denotes normalized pressure, Rp\mathcal{R}_p the set of served directions, pt,curp_{t,\mathrm{cur}} the current phase, and βp\beta_p the phase bias. The objective value is the joint evaluation score in Eq. (4).

QuantityValue
Phase-0 bias β0\beta_00.42
Phase-1 bias β1\beta_10.42
Phase-2 bias β2\beta_20.24
Phase-3 bias β3\beta_3-0.34
Teacher ATT86.01
Student ATT84.04
MaxPressure ATT103.98
Δ\DeltaATT (%)2.29
ABF0.93
MaxPressure ABF0.81

Table 4. Final phase biases and control performance comparison on Hangzhou3 (single intersection with four phases).

This rule is closely related to MaxPressure [7], a classic pressure-based controller that selects phases using upstream–downstream queue differences. In this example, the student achieves both lower ATT (84.04 vs. 103.98) and higher behavioral fidelity to the teacher (ABF 0.93 vs. 0.81) than MaxPressure (Table 4). The explicit rule thus preserves recognizable pressure-based control logic while improving on MaxPressure in this case.

Resource Efficiency (RQ5)

We evaluate resource efficiency as an additional deployment benefit of the explicit programmatic representation, using two deployment-oriented metrics [22, 20]: mean parameter storage and computational cost. Mean parameter storage is measured in bytes from the number of stored scalar parameters or rule constants, assuming 32-bit values. Computational cost is measured by the estimated floating-point operations (FLOPs) required for one intersection-level signal decision, counting the forward pass of neural teachers or the evaluation of phase utility functions over candidate phases. These metrics quantify parameter storage and policy-scoring operations; executable code, runtime memory, and feature preprocessing incur additional costs. Figure 4 compares DRL teachers and the programmatic students learned by PPL-TSC on a representative Hangzhou3 intersection. The shaded regions for ATmega328P and MSP430G2553 provide illustrative resource-budget references; actual deployment also depends on implementation overhead and the required decision interval.

Figure 4. Parameter storage and policy-scoring FLOPs on a representative Hangzhou3 intersection. Shaded microcontroller regions serve as resource-budget references, not verified deployment limits.

As shown in Figure 4, the programmatic students consistently lie in the low-resource regime. Compared with their DRL teachers, they typically reduce both storage and FLOPs by one to three orders of magnitude. For example, FRAP-derived students reduce mean parameter storage from 5,476 bytes to 47.2 bytes and require an average of 80.0 FLOPs, compared with 183,676 for the teacher. For MPLight, the derived students require only 28.0 bytes and 62.4 FLOPs on average. TinyLight-derived students are also highly compact, achieving storage and FLOP reductions of 98.16% and 95.14%, respectively. Overall, these results establish lower storage and inference costs as an additional practical benefit of the programmatic representation, particularly for resource-constrained deployment.

Ablation Study (RQ6)

To answer RQ6, we perform targeted ablations on three components of PPL-TSC: policy generation, policy calibration, and policy evaluation. The calibration and evaluation ablations use the Tailored setting over 175 tasks: five teacher interfaces (CoLight, FRAP, MPLight, EcoLight (phase-selection), and TinyLight), seven scenarios, and five traffic flows.

We first examine policy generation. Figure 5 compares LLM, GA [21], MCTS [17], and random search under a matched time budget in terms of their ability to produce expressions satisfying three traffic-related semantic constraints: demand monotonicity, zero-demand minimum, and active-phase protection. Definitions of these constraints are provided in Appendix § 7.3. We report the mean overall pass rate, defined as the fraction of expressions satisfying all three constraints simultaneously. LLM achieves the highest observed pass rate on each constraint and on all three jointly.

Figure 5. Average pass rate under three semantic constraints for functions generated by LLM, GA, MCTS, and random search.

We then analyze policy calibration (Table 5(a)). Removing calibration slightly improves ATT-oriented metrics, but reduces ABF from 0.64 to 0.55. This pattern suggests that, in our setting, calibration mainly improves alignment with the teacher’s decisions rather than traffic efficiency directly. Table 5(b) compares variants of the policy objective in Eq. (4). The fidelity-only objective yields the highest ABF but substantially degrades ATT. The performance-only objective improves ATT metrics but reduces ABF. The joint objective provides a compromise between the two single-objective variants: it achieves higher ABF than performance-only evaluation and better ATT outcomes than fidelity-only evaluation.

(a) Policy calibration
VariantΔ\DeltaATT (%) ↑\uparrowATT wins ↑\uparrowABF ↑\uparrow
Full PPL-TSC7.62 ±\pm 28.80113/1750.64±\pm0.14
w/o calibration9.78 ±\pm 34.96116/1750.55±\pm0.18
(b) Policy evaluation
VariantΔ\DeltaATT (%) ↑\uparrowATT wins ↑\uparrowABF ↑\uparrow
Full PPL-TSC7.62 ±\pm 28.80113/1750.64±\pm0.14
Fidelity only-14.07 ±\pm 35.0250/1750.68±\pm0.15
Performance only12.82 ±\pm 28.77116/1750.55±\pm0.18

Table 5. Calibration and evaluation ablations in the Tailored setting over 175 tasks. The EcoLight results use the phase-selection interface, denoted EcoLight (phase-selection).

Conclusion

We present PPL-TSC, a teacher-guided framework for learning human-readable programmatic student policies from DRL-based traffic signal controllers. Across diverse scenarios and teacher families, PPL-TSC achieves competitive closed-loop performance while maintaining moderate behavioral fidelity to its teachers. The learned policies show strong transfer performance under traffic-flow shifts and cross-scenario deployment, and substantially reduce storage and computational costs relative to neural teachers. Future work will extend PPL-TSC to broader TSC settings and evaluate its robustness in more realistic large-scale deployments.

Acknowledgments

This work was supported by the Mainland-Hong Kong Joint Funding Scheme of the Innovation and Technology Commission (Grant No. MHP/038/23) and the General Research Fund of the Research Grants Council of the Hong Kong Special Administrative Region, China (Grant No. 15224423). The authors declare no competing interests.

Appendix

Preliminaries

Term Definitions

Figure A.1. Illustration of a four-approach signalized intersection with eight signal phases. Phase 1 is shown as the active phase, permitting one north-to-south through movement and one south-to-north through movement.

We use a single signalized intersection as an illustrative example to introduce several basic concepts in traffic signal control.

1) Intersection, Approaches, and Lanes. As shown in Fig. A.1, the intersection consists of four approaches corresponding to the cardinal directions: north, east, south, and west. Each approach contains multiple incoming lanes, where vehicles queue before crossing the stop line. In practice, lanes are assigned to different turning purposes, including left-turn, through, and right-turn lanes. Vehicles are discharged from these lanes when the active signal phase permits the corresponding movements.

2) Traffic Movements. A traffic movement represents the trajectory of vehicles from an incoming lane to an outgoing lane. As illustrated by the arrows in Fig. A.1, typical movements include left-turn, through, and right-turn maneuvers. Different movements may conflict if their trajectories intersect within the intersection area, and must therefore be coordinated to ensure safety and efficiency.

3) Phases. A phase is a predefined set of non-conflicting traffic movements that are allowed to proceed simultaneously. In the example in Fig. A.1, eight signal phases are defined to coordinate different movements. During a given phase, vehicles associated with permitted movements receive a green signal, while conflicting movements remain stopped.

4) Phase Switching. Phase switching refers to the transition from one signal phase to another over time. For example, Fig. A.1 shows Phase 1 as the active phase, permitting the north-to-south and south-to-north through movements. At decision step tt, intersection ii observes a local traffic state st(i)s_t^{(i)} and chooses a phase pt(i)p_t^{(i)} from a feasible candidate set Pt(i)\mathcal{P}_t^{(i)}, which may contain all phases or only the current phase and its designated successor. The objective is to choose which phase to activate, or whether to keep or switch the current phase, to improve traffic efficiency.

5) Phase-Level Features. Each candidate phase pp is associated with a set of phase-level features extracted from the current traffic state. Let Lp\mathcal{L}_p denote the set of incoming lanes associated with phase pp, and let Mp\mathcal{M}_p denote the set of movements permitted by phase pp. For each lane ℓ∈Lp\ell\in\mathcal{L}_p or movement m∈Mpm \in\mathcal{M}_p, base measurements such as queue length, waiting time, and vehicle count are available. We construct phase-level features by first mapping lane- or lane-link-level measurements to movement directions, such as north-to-south or east-to-west through movements, and then aggregating them over the movements served by phase pp. This yields phase-level quantities such as pressure, vehicle counts, and waiting-related metrics. We further compute statistical descriptors, including {min⁡,max⁡,mean,std}\{\min,\max,\mathrm{mean},\mathrm{std}\}, to capture distributional characteristics for normalization.

All aggregated and statistical features, together with the phase identifier, are concatenated to form the phase-level feature vector xp,t(i)x_{p,t}^{(i)} for candidate phase pp at intersection ii and decision step tt. This vector serves as the input to the utility function f(⋅)f(\cdot).

DRL-based Traffic Signal Control Methods

Deep reinforcement learning (DRL) has been widely adopted for traffic signal control (TSC) due to its ability to learn adaptive control policies directly from traffic data. In DRL-based TSC, the signal control problem is typically formulated as a Markov Decision Process (MDP), defined by the tuple (S,A,K,r,γ)(\mathcal{S},\mathcal{A},\mathcal{K},r,\gamma), where rr denotes the reward function.

The state st∈Ss_t \in\mathcal{S} represents the current traffic condition at time step tt, which may include features such as queue lengths, vehicle counts, or waiting times on incoming lanes. The action at∈Aa_t \in\mathcal{A} corresponds to the control decision taken by the agent, typically selecting the next signal phase or determining whether to switch the current phase.

After executing an action, the traffic environment transitions to a new state st+1s_{t+1} according to the transition kernel K(st+1∣st,at)\mathcal{K}(s_{t+1} \mid s_t,a_t). The agent then receives a scalar reward rtr_t, obtained from rr, which reflects the quality of the control decision, such as average travel time.

The objective of DRL-based traffic signal control is to learn a policy π(a∣s)\pi(a \mid s) that maximizes the expected discounted return over time:

max⁡πE[∑t=0Tγtrt],\max_{\pi} \mathbb{E}\left[\sum_{t=0}^{T}\gamma^t r_t\right],

where γ∈(0,1)\gamma\in(0,1) is the discount factor and TT is the decision horizon. The learned policy enables the traffic signal controller to adaptively select signal phases based on real-time traffic conditions.

Prompt Templates

This subsection provides representative LLM prompt templates for policy generation and repair in PPL-TSC. Each template follows the structure T(Task,Ctx,Cons,Out)\mathcal{T}(\mathrm{Task},\mathrm{Ctx},\mathrm{Cons},\mathrm{Out}): task specifies the generation objective, context provides traffic features or parent PUFs, constraints encode the syntactic, mathematical, and semantic requirements used to guide generation, and output asks for an executable candidate PUF with a brief rationale. Fig. A.2 illustrates the initialization prompt, evolution prompt, and repair prompt.

Details of Syntactic, Mathematical, and Semantic Constraints

During policy review, each PUF produced by policy generation is explicitly checked against the same constraints encoded in the generation prompt before calibration. To ensure that generated utility

Figure A.2. Illustration of the initialization, evolution, and repair prompt templates used for LLM-based policy generation and revision.

Prompt for Initialization

Task: Discover a student utility function that approximates the teacher model’s decision behavior. The function Compute_utility(state, phase_id) assigns a utility score to each candidate phase; the phase with the highest score is activated. Prioritize diverse and expressive function structures during initialization.

Context: The available input features are {Teacher’s state variables}.

Constrains: The implementation must satisfy on {Syntactic correctness}, {Mathematical validity}, and {Semantic consistency}.

Output: Provide the complete Python function Compute_utility(state, phase_id) that returns a float utility value, followed by a brief explanation (2–3 sentences) describing the design rationale.

Prompt for Evolution

Task: Evolve an improved offspring utility function from the given parent. The function Compute_utility(state, phase_id) assigns a utility score to each candidate phase; the phase with the highest score is activated. The offspring must be structurally distinct from the parent and combine raw features meaningfully to avoid trivial heuristics.

Context: The parent function is {Parent}. Evaluation information is provided in {Eval info}. The available input features are {Teacher’s state variables}.

Requirements: Same as in the initialization prompt.

Output: Same as in the initialization prompt.

Prompt for Repair

Task: Repair an invalid utility function that violates specified constraints. The function Compute_utility(state, phase_id) assigns a utility score to each candidate phase; the phase with the highest score is activated. The repaired function must preserve the original behavioral intent as much as possible while strictly satisfying all constraints and eliminating any invalid operations.

Context: The function to be repaired is {Function}. The identified constraint violations are detailed in {Cons info}. The available input features are {Teacher’s state variables}.

Requirements: Same as in the initialization prompt.

Output: Same as in the initialization prompt.

functions are executable, numerically stable, and consistent with traffic-control principles, we enforce three families of validation checks: syntactic, mathematical, and semantic. An overview is provided in Table A.1.

FamilyCheckDescription
SyntacticParseabilityThe generated function must have valid Python syntax and compile successfully.
SyntacticExecutabilityThe generated function must be loadable and callable at runtime.
SyntacticStructural validityThe generated function must have a valid structure and return a value on every execution path.
MathematicalFinite outputOutput must be a finite real number (no NaN or infinite values).
MathematicalDomain safetyNo invalid operations (e.g., division by zero, invalid logarithms, or square roots of negative inputs).
MathematicalRuntime robustnessNo exceptions under real teacher states and extreme input scenarios (e.g., all-zero, all-max, mixed, single-feature extremes).
SemanticFeature legalityOnly allowed traffic features are accessed.
SemanticDemand monotonicityPhase utility or switching preference varies monotonically with demand according to the action interface.
SemanticZero-demand minimumWith other inputs fixed, setting non-negative demand features to zero must not increase phase utility.
SemanticActive-phase protectionWith demand fixed, the active-phase indicator must not reduce phase utility.

Table A.1. Constraint validation checks.

Syntactic checks. These checks ensure that the generated code is executable in the target pipeline. Each candidate must first pass Python compilation, including a valid function body and valid syntax. It is then loaded into the runtime environment via exec(), where the required entry point compute_utility must exist and be callable. Malformed structures, such as invalid function skeletons or missing return statements, are rejected before acceptance.

Mathematical checks. These checks ensure numerical validity and stability. All outputs must be finite; any NaN, inf, or −inf results in rejection. All operations must respect valid input domains, such as non-zero denominators, positive inputs to logarithms, and non-negative inputs to square roots. The function must also avoid extreme intermediate values that could cause overflow or numerical instability. To test robustness, we evaluate each candidate on phase-level inputs constructed under the following scenarios:

  • Real teacher states: all samples (st(i),ptTch,(i))∈D(s_t^{(i)}, p_t^{\mathrm{Tch},(i)}) \in\mathcal{D}, to ensure stability under the observed data distribution;

  • All-zero: all traffic-measurement features set to zero, with phase indicators kept fixed;

  • All-max: all traffic-measurement features set to their maximum observed values in D\mathcal{D}, with phase indicators kept fixed;

  • Mixed-half: half of the traffic-measurement features set to zero and the other half set to their maximum values;

  • Single-feature-max: each traffic-measurement feature individually set to its maximum observed value, with all remaining features kept at their original sampled values.

Semantic checks. The feature whitelist covers the base traffic measurements pressure, num_vehicle, num_waiting_vehicle, waiting_time, and sum_waiting_time, together with phase_id and current_phase. Candidates may use the defined aggregation and normalization operations on these measurements to construct phase-level features. Access to undefined or disallowed inputs results in rejection.

Demand monotonicity follows the controller’s action interface. For phase-local scoring, utility must be non-decreasing in the demand served by the candidate phase, with other inputs fixed. This applies to phase-selection policies and the original keep/switch student that scores the current and next phases separately. We evaluate this property on sampled real teacher states using multiple scaling factors. For each demand-related feature

x∈{pressure, num_vehicle, num_waiting_vehicle, waiting_time, sum_waiting_time},x \in\{\texttt{pressure},\ \texttt{num\_vehicle},\ \texttt{num\_waiting\_vehicle},\ \texttt{waiting\_time},\ \texttt{sum\_waiting\_time}\},

we generate perturbed states with scaling factors α∈{0.5,0.8,1.0,1.2,1.5}\alpha\in\{0.5, 0.8, 1.0, 1.2, 1.5\}:

xα=α⋅xbase,x_{\alpha} = \alpha\cdot x_{\mathrm{base}},

while keeping all other features unchanged.

For a candidate utility function ff, we evaluate the corresponding phase-level feature vector xp,t,α(i)\mathbf{x}_{p,t,\alpha}^{(i)} for each α\alpha. We sort the perturbed feature values in ascending numerical order and check that utility is non-decreasing in that order:

f(xp,t,αk(i))≥f(xp,t,αj(i)),∀xαk>xαj.f\left(\mathbf{x}_{p,t,\alpha_k}^{(i)}\right) \ge f\left(\mathbf{x}_{p,t,\alpha_j}^{(i)}\right),\quad\forall x_{\alpha_k} > x_{\alpha_j}.

Ordering by feature value rather than scaling factor accommodates signed features such as movement pressure. This check assesses monotonicity at the tested inputs.

For the aligned keep/switch variant in Appendix A.7.1, we instead check the relative preference for switching. Let gg and rr denote green- and red-side demand, and define the switching margin as

m(g,r)=fswitch(g,r)−fkeep(g,r).m(g,r) = f_{\mathrm{switch}}(g,r) - f_{\mathrm{keep}}(g,r).

On sampled teacher states, we vary each demand input using the scaling factors above and hold the other inputs fixed. The margin must be non-increasing in gg and non-decreasing in rr: greater green-side demand favors keeping the current phase, and greater red-side demand favors switching. These checks assess monotonicity at the tested inputs.

Zero-demand minimum. With other inputs fixed, setting non-negative demand features to zero must not increase phase utility. For aligned keep/switch policies, we use the switching-margin checks defined above.

Active-phase protection compares otherwise identical candidate inputs that differ only in whether the phase is currently active. The check verifies that the active-phase indicator is not used to penalize the current phase under comparable demand conditions. This discourages excessive phase switching and reflects the practical need for short-term signal stability.

Decision rule. Candidates that fail any check are returned to the LLM with diagnostic feedback and reviewed again after repair. Only candidates that pass all checks proceed to calibration and evaluation.

Method Complexity Analysis

We analyze the complexity of PPL-TSC in terms of both search-time cost and deployment-time inference cost. Let ∣D∣=∑i=1MNi|\mathcal{D}| = \sum_{i=1}^{M} N_i denote the total number of rollout samples, Bi=max⁡t∣Pt(i)∣B_i = \max_t |\mathcal{P}_t^{(i)}| the maximum feasible phase-set size at intersection ii (all phases for phase-selection controllers and 2 for keep/switch controllers), CfC_f the cost of evaluating f(⋅)f(\cdot) on one phase-level feature vector, and CLLMC_{\mathrm{LLM}} the cost of one LLM prompt-response interaction. We further use Gmax⁡G_{\max} for the maximum number of generations, NpopN_{\mathrm{pop}} for the population size, RR for the number of review inputs per candidate, TiT_i for the number of CMA-ES objective evaluations at intersection ii, Bˉ\bar{B} for the average candidate phase-set size, TsimT_{\mathrm{sim}} for the number of simulator control steps, CenvC_{\mathrm{env}} for one simulator transition, and KrepK_{\mathrm{rep}} for the average number of LLM generations per accepted candidate, including repair calls.

ComponentComplexity
Policy generationO(Gmax⁡NpopKrepCLLM)O(G_{\max}N_{\mathrm{pop}}K_{\mathrm{rep}}C_{\mathrm{LLM}})
Policy reviewO(RCf)O(RC_f) per reviewed candidate
Policy calibrationO(∑i=1MTiNiBiCf)O(\sum_{i=1}^{M} T_iN_iB_iC_f) per candidate
Policy evaluationO(1)O(1) for cached agreement; O(Tsim(MBˉCf+Cenv))O(T_{\mathrm{sim}}(M\bar{B}C_f+C_{\mathrm{env}})) for simulation
DeploymentO(BiCf)O(B_iC_f) per decision step

Table A.2. Complexity of the main steps in PPL-TSC.

Table A.2 summarizes the costs. Across the search, generation requires O(Gmax⁡NpopKrepCLLM)O(G_{\max}N_{\mathrm{pop}}K_{\mathrm{rep}}C_{\mathrm{LLM}}), and review requires O(Gmax⁡NpopKrepRCf)O(G_{\max}N_{\mathrm{pop}}K_{\mathrm{rep}}RC_f), excluding lightweight parsing overhead. LLM latency depends on prompt and response lengths and repair frequency.

Calibration and evaluation. One CMA-ES objective evaluation scores up to BiB_i phases for each of NiN_i samples, costing O(NiBiCf)O(N_iB_iC_f). With TiT_i objective evaluations per intersection, full-search calibration costs O(Gmax⁡Npop∑i=1MTiNiBiCf)O(G_{\max}N_{\mathrm{pop}}\sum_{i=1}^{M}T_iN_iB_iC_f). The final fidelity value on D\mathcal{D} is cached, so its additional evaluation cost is O(1)O(1) per candidate. Closed-loop simulation costs O(Gmax⁡NpopTsim(MBˉCf+Cenv))O(G_{\max}N_{\mathrm{pop}}T_{\mathrm{sim}}(M\bar{B}C_f+C_{\mathrm{env}})) over the search. Simulation is typically the dominant cost in practice, followed by calibration.

Deployment. Scoring at most BiB_i phases costs O(BiCf)O(B_iC_f) per intersection-level decision. The phase-scoring function is shared; each intersection stores its own phase biases.

Limitations

First, PPL-TSC depends on teacher rollouts for behavioral supervision, so teacher quality and state coverage influence the learned policy. Students may inherit undesirable teacher behavior or perform poorly in states underrepresented in the rollouts. Although the joint objective includes closed-loop traffic performance and students can outperform their teachers, this feedback does not eliminate the limitations of teacher-derived data. Future work could expand rollout coverage through student-driven data collection and further refine programmatic policies through closed-loop optimization.

Second, PPL-TSC learns students within a fixed PUF-based policy class. While this class works well for several teacher families, it may be insufficient for teachers whose behavior depends on richer state representations or more complex feature interactions. Future work could explore richer programmatic policy classes that preserve compactness while covering a broader range of teacher behaviors. The verifier checks generated functions on sampled traffic states and constructed test inputs. A function that passes these checks may still behave undesirably in unseen states or different road networks. The predefined semantic checks may not capture all aspects of reasonable control behavior. Applying them to different teachers requires considering the input features, action meanings, and their relationships, and may require adapting the checks accordingly.

Third, the program structure is discovered through LLM-guided generation and refinement, making the final policy dependent on the model’s ability to propose valid and useful candidate expressions. Although LLM-based generation achieves higher semantic-constraint pass rates than GA, MCTS, and random search under a matched time budget, the resulting policy structures may still be suboptimal. Future work could mitigate this dependence by integrating LLM-based synthesis with more systematic local search, stronger verification, and automated repair mechanisms.

ScenarioATT ↓\downarrow TeacherATT ↓\downarrow TailoredATT ↓\downarrow UnifiedABF ↑\uparrow TailoredABF ↑\uparrow Unified
Hangzhou196.00±5.2096.00\pm5.2094.31±3.45\mathbf{94.31\pm3.45}98.63±4.7698.63\pm4.760.66±0.06\mathbf{0.66\pm0.06}0.60±0.060.60\pm0.06
Hangzhou2122.47±4.06\mathbf{122.47\pm4.06}123.52±2.71123.52\pm2.71127.24±3.66127.24\pm3.660.68±0.08\mathbf{0.68\pm0.08}0.62±0.090.62\pm0.09
Hangzhou380.08±1.87\mathbf{80.08\pm1.87}82.39±1.1582.39\pm1.1584.65±2.3784.65\pm2.370.73±0.04\mathbf{0.73\pm0.04}0.71±0.040.71\pm0.04
Atlanta311.00±24.27\mathbf{311.00\pm24.27}311.36±58.31311.36\pm58.31325.75±65.71325.75\pm65.710.73±0.17\mathbf{0.73\pm0.17}0.65±0.140.65\pm0.14
Jinan1109.23±311.541109.23\pm311.54937.99±433.65937.99\pm433.65358.95±27.54\mathbf{358.95\pm27.54}0.60±0.31\mathbf{0.60\pm0.31}0.15±0.040.15\pm0.04
Los Angeles846.30±16.21\mathbf{846.30\pm16.21}853.04±13.30853.04\pm13.30852.89±12.48852.89\pm12.480.95±0.05\mathbf{0.95\pm0.05}0.90±0.090.90\pm0.09
Manhattan1668.92±90.511668.92\pm90.51607.53±104.25607.53\pm104.25502.55±85.26\mathbf{502.55\pm85.26}0.62±0.05\mathbf{0.62\pm0.05}0.56±0.050.56\pm0.05

Table A.3. Scenario-level results for CoLight. Results are averaged over the five traffic flows in each scenario.

Broader Impacts

PPL-TSC may improve the practical deployability of DRL-based traffic signal control by distilling neural controllers into compact programmatic policies. If used responsibly, such policies could help reduce traffic delay and improve control efficiency, with potential downstream benefits for fuel consumption and emissions. The compact programmatic representation may also make controller behavior easier to inspect, maintain, and adapt in resource-constrained deployments.

At the same time, traffic signal control is a safety-critical application, and results on standard simulators do not guarantee safe real-world deployment. Because PPL-TSC is teacher-guided, it may also inherit biases or failure modes from the teacher policy, which could affect different traffic participants unevenly. Real-world use therefore requires careful validation, human oversight, and additional safeguards before deployment.

Licenses for Existing Assets

This work uses existing simulators, traffic scenarios, controller designs, and LLM services only for research evaluation. We credit the corresponding original sources throughout the paper and summarize the asset usage and licensing information below.

  • Traffic simulator. We conduct closed-loop evaluation in CityFlow [35]. CityFlow is publicly released under the Apache License 2.0 according to the CityFlow project repository and documentation.

  • Benchmark scenarios and traffic data. The traffic scenarios used in our experiments are taken from the LibSignal benchmark [36] and related CityFlow-format traffic-signal-control benchmarks. LibSignal is released as an open-source research library.

  • Teacher controller designs. We evaluate against CoLight [14], FRAP [15], MPLight [19], EcoLight [18], and TinyLight [20]. These methods are credited through their original publications. Any publicly available reference implementations or benchmark implementations used to reproduce teacher behavior are used only under their associated repository licenses and terms; this manuscript does not redistribute third-party controller code.

  • LLM services. PPL-TSC uses LLM backbones such as Gemini, Qwen, Claude, GPT, Grok, and MiniMax for candidate policy generation and repair through their official access channels. These services are used according to their respective provider terms of service. We do not redistribute model weights or proprietary service outputs as standalone assets.

  • Generated artifacts. The tables, plots, learned programmatic policies, and analysis outputs reported in this paper are generated by the authors from the experimental pipeline.

Diagnosis of EcoLight (native) and TinyLight

We further investigate the weaker in-domain results obtained when PPL-TSC is applied to EcoLight (native) and TinyLight. We conduct controlled follow-up experiments over 35 tasks (seven scenarios and five traffic flows), changing only the suspected factor while keeping all other training and evaluation settings fixed. Table A.9 reports average behavioral fidelity (ABF) and relative ATT improvement, where higher values are better.

ScenarioATT ↓ABF ↑
TeacherTailoredUnifiedTailoredUnified
Hangzhou1128.94±31.88109.79±19.7699.04±5.850.69±0.160.61±0.23
Hangzhou2150.24±56.75128.03±14.62122.65±1.150.62±0.270.59±0.31
Hangzhou3132.47±110.34134.01±115.0385.05±2.300.72±0.050.58±0.30
Atlanta258.77±25.82333.29±31.27364.11±13.070.75±0.100.66±0.11
Jinan307.61±15.17312.17±2.58316.78±1.920.62±0.020.59±0.03
Los Angeles714.95±74.80442.73±69.32466.25±74.950.47±0.140.45±0.10
Manhattan639.19±423.21232.81±31.61235.63±23.040.56±0.100.56±0.11

Table A.4. Scenario-level results for FRAP. Results are averaged over the five traffic flows in each scenario.

ScenarioATT ↓ABF ↑
TeacherTailoredUnifiedTailoredUnified
Hangzhou1100.83±3.3697.33±6.23100.04±1.620.76±0.070.72±0.07
Hangzhou2121.81±1.32121.97±1.33122.89±2.440.74±0.100.71±0.10
Hangzhou382.72±2.7382.48±2.1083.51±2.420.89±0.040.88±0.04
Atlanta249.45±15.03236.32±5.22241.55±4.610.79±0.110.71±0.15
Jinan297.07±7.04307.66±4.84314.77±6.000.72±0.100.68±0.08
Los Angeles588.16±99.49579.86±61.50544.45±166.220.58±0.040.56±0.15
Manhattan195.42±4.14197.74±6.87198.95±3.510.61±0.060.60±0.06

Table A.5. Scenario-level results for MPLight. Results are averaged over the five traffic flows in each scenario.

ScenarioATT ↓ABF ↑
TeacherTailoredUnifiedTailoredUnified
Hangzhou1205.88±37.67183.19±25.05194.82±15.380.81±0.050.77±0.05
Hangzhou2135.03±0.54134.63±2.11135.92±2.790.91±0.040.90±0.04
Hangzhou398.05±1.7796.31±1.2497.23±1.520.93±0.020.92±0.03
Atlanta311.47±32.78329.75±20.98380.94±38.960.79±0.110.77±0.05
Jinan378.74±3.97765.00±314.15736.17±166.600.84±0.010.81±0.02
Los Angeles640.13±5.03674.28±108.16785.57±116.090.80±0.030.82±0.03
Manhattan1066.33±66.771325.00±16.361340.00±40.560.80±0.010.79±0.01

Table A.6. Scenario-level results for EcoLight (native) under its keep/switch action setting. Results are averaged over the five traffic flows in each scenario.

ScenarioATT ↓ABF ↑
TeacherTailoredUnifiedTailoredUnified
Hangzhou1205.88±42.11126.10±31.32115.29±26.620.43±0.090.29±0.06
Hangzhou2135.03±0.60127.90±3.37124.61±4.310.49±0.030.38±0.08
Hangzhou398.05±1.9883.73±2.5986.99±1.660.65±0.030.62±0.02
Atlanta311.47±36.65358.34±55.18381.22±11.420.56±0.050.48±0.07
Jinan378.74±4.44324.74±9.79320.34±3.630.39±0.020.28±0.03
Los Angeles640.13±5.62485.39±35.87530.55±98.750.44±0.050.41±0.06
Manhattan1066.33±74.65249.80±29.01241.49±33.820.42±0.030.36±0.02

Table A.7. Scenario-level results for EcoLight (phase-selection). Results are averaged over the five traffic flows in each scenario.

ScenarioATT ↓ABF ↑
TeacherTailoredUnifiedTailoredUnified
Hangzhou198.76±5.8197.11±6.78105.11±15.310.67±0.070.57±0.06
Hangzhou2121.84±1.20119.83±0.78123.16±5.120.66±0.080.57±0.09
Hangzhou381.17±1.2281.53±1.1183.86±3.850.82±0.080.70±0.09
Atlanta278.47±12.85359.23±54.13375.47±30.470.69±0.050.65±0.07
Jinan299.54±2.83324.91±89.72397.69±128.980.31±0.110.42±0.10
Los Angeles450.31±43.75517.40±147.30619.48±164.610.73±0.050.72±0.06
Manhattan762.03±532.47562.91±664.601163.20±758.010.58±0.110.63±0.03

Table A.8. Scenario-level results for TinyLight. Results are averaged over the five traffic flows in each scenario.

TeacherConfigurationABF ↑\uparrowΔ\DeltaATT (%) ↑\uparrow
EcoLight (native)Original inputs0.84−18.05-18.05
EcoLight (native)Aligned inputs0.942.71
TinyLightSearch budget (5, 5)0.64−8.43-8.43
TinyLightSearch budget (10, 10)0.711.51

Table A.9. Controlled diagnosis of EcoLight (native) and TinyLight. For EcoLight (native), Original inputs score phases separately using phase-local demand, whereas Aligned inputs include joint green- and red-side demand. The search budget is reported as (population size, number of generations).

Source ScenarioTeacher ATT ↓\downarrowPPL-CoLight ATT ↓\downarrowΔ\DeltaATT (%) ↑\uparrowATT wins
Hangzhou196.74±\pm4.7099.36±\pm2.41−2.96±5.72-2.96\pm5.724/20
Hangzhou2122.85±\pm3.26122.36±\pm1.070.33±$2.61\pm\$2.61 | 6/20
Hangzhou381.04±\pm1.2886.72±\pm1.18−7.04±2.17-7.04\pm2.170/20
Atlanta313.68±\pm20.93402.94±\pm28.94−28.56±6.21-28.56\pm6.210/20
Jinan1114.97±\pm254.41∣∗∗751.00| **751.00\pm474.67474.67**35.48±\pm32.0515/20
Los Angeles846.71±\pm16.73∣∗∗636.83| **636.83\pm188.32188.32**24.57±$23.06\pm\$23.06 | 16/20
Manhattan1660.76±\pm89.71∣∗∗202.42| **202.42\pm8.448.44**87.79±$0.66\pm\$0.66 | 20/20

Table A.10. Supplementary generalization results for CoLight under same-scenario, different-flow transfer. Cross-scenario results are not available in the current benchmark. Rows correspond to the source scenario.

EcoLight (native). EcoLight (native) makes keep/switch decisions using demand from both green and red movements, but the original student scores the current and next phases separately using only the demand each phase serves. An incorrect switch changes the current phase and the decisions available at subsequent steps, allowing errors to accumulate during closed-loop control. Using aligned inputs that provide joint green- and red-side demand, with all other settings unchanged, increases ABF from 0.84 to 0.94 and improves mean Δ\DeltaATT from −18.05%-18.05\% to +2.71%+2.71\%. These results suggest that the input mismatch is a major contributor to the original performance degradation.

TinyLight. TinyLight uses both phase-level and incoming-lane features, giving the student a broader set of inputs to search over. Increasing the population size and number of generations from (5, 5) to (10, 10), with all other settings unchanged, raises ABF from 0.64 to 0.71 and improves mean Δ\DeltaATT from −8.43%-8.43\% to +1.51%+1.51\%. These results suggest that the default budget limits search quality for TinyLight and that a larger budget can improve both fidelity and control performance.

Same-Scenario, Different-Flow
Source ScenarioTeacher ATT ↓\downarrowPPL-FRAP ATT ↓\downarrowΔ\DeltaATT (%) ↑\uparrowATT wins
Hangzhou1126.50±\pm27.48∣∗∗100.03| **100.03\pm2.892.89**18.10±$13.19\pm\$13.19 | 20/20
Hangzhou2142.02±\pm40.37∣∗∗122.95| **122.95\pm1.491.49**8.99±$15.76\pm\$15.76 | 18/20
Hangzhou3134.77±\pm101.22∣∗∗86.61| **86.61\pm1.071.07**12.54±$30.97\pm\$30.97 | 7/20
Atlanta**254.60±\pm16.28∗∗∣389.20** | 389.20\pm25.7925.79−53.10±8.86-53.10\pm8.860/20
Jinan**307.89±\pm13.30∗∗∣319.69** | 319.69\pm1.611.61−4.00±4.01-4.00\pm4.014/20
Los Angeles703.20±\pm77.96∣∗∗409.74| **409.74\pm8.468.46**40.92±\pm7.4820/20
Manhattan675.21±\pm396.23∣∗∗196.57| **196.57\pm4.764.76**59.44±\pm20.0820/20
Cross-Scenario
Source ScenarioTeacher ATT ↓\downarrowPPL-FRAP ATT ↓\downarrowΔ\DeltaATT (%) ↑\uparrowATT wins
Hangzhou1764.29±\pm635.23∣∗∗251.56| **251.56\pm124.11124.11**37.06±\pm37.37114/150
Hangzhou2803.49±\pm584.30∣∗∗252.88| **252.88\pm134.62134.62**50.01±\pm33.28129/150
Hangzhou3914.93±\pm500.84∣∗∗253.10| **253.10\pm119.67119.67**62.16±\pm30.44136/150
Atlanta1342.99±\pm385.91∣∗∗222.45| **222.45\pm160.33160.33**79.14±\pm20.88147/150
Jinan717.59±\pm589.02∣∗∗215.18| **215.18\pm130.26130.26**40.03±\pm44.67104/150
Los Angeles1128.05±\pm548.72∣∗∗199.71| **199.71\pm111.34111.34**66.97±\pm36.88138/150
Manhattan688.96±\pm366.22∣∗∗252.73| **252.73\pm171.47171.47**44.18±\pm46.45119/150

Table A.11. Supplementary generalization results for FRAP. Rows correspond to the source scenario.

Source ScenarioTeacher ATT ↓\downarrowPPL-MPLight ATT ↓\downarrowΔ\DeltaATT (%) ↑\uparrowATT wins
Hangzhou1100.72±\pm5.5198.96±\pm7.211.80±\pm3.1315/20
Hangzhou2122.42±\pm1.72122.19±\pm1.440.18±\pm0.9111/20
Hangzhou382.86±\pm1.2384.43±\pm1.50-1.89±\pm1.482/20
Atlanta251.50±\pm17.14243.17±\pm11.012.87±\pm7.8110/20
Jinan297.09±\pm6.17310.32±\pm1.13-4.50±\pm2.121/20
Los Angeles591.77±\pm55.95537.51±\pm156.138.35±\pm27.9016/20
Manhattan195.72±\pm2.98201.25±\pm7.19-2.80±\pm2.410/20
Source ScenarioTeacher ATT ↓\downarrowPPL-MPLight ATT ↓\downarrowΔ\DeltaATT (%) ↑\uparrowATT wins
Hangzhou1792.01±\pm647.69256.48±\pm155.3940.67±\pm34.82115/150
Hangzhou2798.34±\pm661.12233.04±\pm126.9144.34±\pm33.91124/150
Hangzhou3843.40±\pm531.29240.28±\pm120.1358.53±\pm27.53150/150
Atlanta952.78±\pm522.35215.22±\pm133.9260.49±\pm35.90144/150
Jinan564.21±\pm438.45204.58±\pm129.4338.25±\pm35.77141/150
Los Angeles981.66±\pm481.97185.12±\pm91.2668.68±\pm29.09150/150
Manhattan288.98±\pm240.39231.98±\pm161.9911.29±\pm20.26122/150

Table A.12. Supplementary generalization results for MPLight. Rows correspond to the source scenario. Same-Scenario, Different-Flow; Cross-Scenario.

Source ScenarioTeacher ATT ↓\downarrowPPL-EcoLight ATT ↓\downarrowΔ\DeltaATT (%) ↑\uparrowATT wins
Hangzhou1209.20±\pm35.7598.73±\pm3.1551.52±\pm7.6320/20
Hangzhou2134.42±\pm1.02122.49±\pm1.278.87±\pm1.1520/20
Hangzhou397.86±\pm3.0886.62±\pm1.2111.40±\pm2.9520/20
Atlanta312.77±\pm31.81373.94±\pm5.14-20.60±\pm10.342/20
Jinan379.52±\pm6.98320.75±\pm1.8215.46±\pm1.3920/20
Los Angeles643.40±\pm6.78408.17±\pm9.6136.55±\pm1.7920/20
Manhattan1033.27±\pm46.12328.85±\pm120.4368.38±\pm11.0320/20
Source ScenarioTeacher ATT ↓\downarrowPPL-EcoLight ATT ↓\downarrowΔ\DeltaATT (%) ↑\uparrowATT wins
Hangzhou1773.79±\pm599.72251.53±\pm124.2943.20±\pm33.53150/150
Hangzhou2777.06±\pm599.57247.81±\pm127.9647.92±\pm30.17150/150
Hangzhou3791.48±\pm576.36253.73±\pm122.1850.35±\pm30.28150/150
Atlanta750.53±\pm543.70205.91±\pm121.9360.80±\pm25.66150/150
Jinan537.76±\pm534.79228.62±\pm160.5135.39±\pm30.98139/150
Los Angeles957.33±\pm626.34200.58±\pm111.7060.95±\pm36.34125/150
Manhattan386.32±\pm174.02533.93±\pm362.09-64.69±\pm157.9974/150

Table A.13. Supplementary generalization results for EcoLight (native) under its keep/switch action setting. Rows correspond to the source scenario. Same-Scenario, Different-Flow; Cross-Scenario.

Source ScenarioTeacher ATT ↓\downarrowPPL-EcoLight ATT ↓\downarrowΔ\DeltaATT (%) ↑\uparrowATT wins
Hangzhou1209.20±\pm35.75107.25±\pm12.4347.05±\pm11.1420/20
Hangzhou2134.42±\pm1.02123.58±\pm1.668.06±\pm1.4320/20
Hangzhou397.86±\pm3.0887.07±\pm1.8010.92±\pm3.5920/20
Atlanta312.77±\pm31.81376.35±\pm1.39-21.42±\pm10.780/20
Jinan379.52±\pm6.98324.88±\pm7.6314.34±\pm3.3320/20
Los Angeles643.40±\pm6.78405.74±\pm9.0336.93±\pm1.5720/20
Manhattan1033.27±\pm46.12201.72±\pm8.8480.44±\pm1.2020/20
Source ScenarioTeacher ATT ↓\downarrowPPL-EcoLight ATT ↓\downarrowΔ\DeltaATT (%) ↑\uparrowATT wins
Hangzhou1773.79±\pm599.72335.01±\pm287.5433.77±\pm37.04123/150
Hangzhou2777.06±\pm599.57262.02±\pm151.0145.88±\pm31.28145/150
Hangzhou3791.48±\pm576.36285.34±\pm176.9446.14±\pm33.08140/150
Atlanta750.53±\pm543.70205.71±\pm120.2360.67±\pm25.67150/150
Jinan537.76±\pm534.79220.34±\pm140.7035.58±\pm30.25149/150
Los Angeles957.33±\pm626.34200.30±\pm111.3461.00±\pm36.27125/150
Manhattan386.32±\pm174.02278.49±\pm206.5829.90±\pm31.34122/150

Table A.14. Supplementary generalization results for EcoLight (phase-selection). Rows correspond to the source scenario. Same-Scenario, Different-Flow; Cross-Scenario.

Source ScenarioTeacher ATT ↓\downarrowPPL-TinyLight ATT ↓\downarrowΔ\DeltaATT (%) ↑\uparrowATT wins
Hangzhou199.85±7.3699.85\pm7.3695.99±3.0995.99\pm3.093.56±4.483.56\pm4.4816/20
Hangzhou2121.68±1.44121.68\pm1.44119.59±1.27119.59\pm1.271.71±0.961.71\pm0.9620/20
Hangzhou381.76±1.1081.76\pm1.1081.19±1.3081.19\pm1.300.70±1.300.70\pm1.3015/20
Atlanta276.85±16.05276.85\pm16.05320.72±46.04320.72\pm46.04−16.94±22.24-16.94\pm22.245/20
Jinan299.86±3.14299.86\pm3.14284.36±1.94284.36\pm1.945.16±1.185.16\pm1.1820/20
Los Angeles466.54±39.77466.54\pm39.77490.47±98.35490.47\pm98.35−5.02±17.37-5.02\pm17.3710/20
Manhattan611.26±485.68611.26\pm485.68373.68±330.27373.68\pm330.27−20.54±126.79-20.54\pm126.7912/20

Table A.15. Supplementary generalization results for TinyLight under the same-scenario, different-flow transfer. Cross-scenario results are not available in the current benchmark. Rows correspond to the source scenario.

Additional Experimental Details

Experimental Setup

Teacher Controllers. We use five representative DRL-based TSC methods as teachers. Most of them map intersection-level traffic observations to a phase-selection decision, while EcoLight uses a native keep/switch action interface. They differ substantially in how they represent traffic state and parameterize the control policy.

CoLight [14]. CoLight is a cooperative multi-intersection RL controller that uses graph attention to model interactions among neighboring intersections. For each intersection, the input includes the current phase and lane-level vehicle counts; these local features are further aggregated with features from adjacent intersections through an attention mechanism. The network outputs a value for each candidate phase, and the next phase is selected by taking the maximum-value action.

FRAP [15]. FRAP models TSC from the perspective of phase competition and explicitly exploits the rotational symmetry of intersections. Its input consists of the current phase together with lane-link-level traffic features, which are reorganized into a phase-competition representation describing the demand served by competing phases. The output is a value for each feasible phase.

MPLight [19]. MPLight extends FRAP to large-scale road networks through parameter sharing across intersections and a pressure-based learning objective. Its input includes the current phase and lane-link-level pressure features, where pressure measures the imbalance between upstream and downstream traffic. Similar to FRAP, the network outputs a value for each candidate phase.

EcoLight [18]. EcoLight is a lightweight RL controller designed for budget- and infrastructure-constrained settings. It uses a compact low-dimensional state derived from the current phase and incoming-lane vehicle counts, and formulates control as a binary decision: keep the current phase or switch to the next phase. Its output is therefore a two-dimensional action-value vector over these two choices. In PPL-TSC, these two actions correspond to Pt(i)={pt,cur(i),next⁡(pt,cur(i))}\mathcal{P}_{t}^{(i)} = \{p_{t,\mathrm{cur}}^{(i)}, \operatorname{next}(p_{t,\mathrm{cur}}^{(i)})\}. We report this native keep/switch setting as EcoLight (native), and additionally evaluate an EcoLight (phase-selection) variant in which the student directly scores all feasible phases.

TinyLight [20]. TinyLight is a resource-efficient DRL controller tailored for deployment on devices with extremely limited memory and computation. Its input includes compact phase-level and incoming-lane traffic statistics, such as vehicle count, queue length, waiting time, delay, pressure, and the current phase. The network outputs a value for each feasible phase, and the phase with the highest value is selected for execution.

Implementation details. All experiments are conducted on a machine with an Intel Core i7-10700K CPU, 64 GB RAM, and an NVIDIA GeForce RTX 4080 SUPER GPU with 16 GB VRAM. Teacher controllers are implemented in PyTorch and trained for 128 episodes. For programmatic policy search, PPL-TSC uses Gemini 3.1 Flash with a population size Npop=5N_{\mathrm{pop}} = 5 and a maximum of Gmax=5G_{\mathrm{max}} = 5 generations, unless otherwise stated. For bias calibration, CMA-ES uses a population size of 20 and a maximum of 20 iterations.

Scenario-Level Results

This subsection provides the detailed scenario-level results corresponding to Table 2. Tables A.3–A.8 report results for CoLight, FRAP, MPLight, EcoLight (native), EcoLight (phase-selection), and TinyLight. For each teacher family, we report the mean and standard deviation over the five traffic flows in each scenario. Boldface indicates the best value within each scenario row, with lower values preferred for ATT and higher values preferred for ABF. These tables complement the aggregated summary in the main text by showing how the PPL-TSC (Tailored) and PPL-TSC (Unified) students behave across individual traffic scenarios.

Cross-scenario Generalization Details

This subsection provides supplementary transfer results corresponding to Table 3. Tables A.10–A.15 report transfer results for CoLight, FRAP, MPLight, EcoLight (native), EcoLight (phase-selection), and TinyLight. For each teacher family and each source scenario, we report average ATT under same-scenario, different-flow transfer; for teacher families with applicable settings, we additionally report cross-scenario transfer results. We focus on ATT here because the main goal of RQ3 is to assess whether the learned policy preserves traffic-control performance under distribution shift; compared with fidelity-oriented metrics, ATT provides a more direct measure of transfer effectiveness. Here, “ATT wins” counts the number of transfer test cases in which PPL-TSC improves upon or matches the teacher on ATT. For the same-scenario setting, each row aggregates over 20 transfer test cases; for the cross-scenario setting, each row aggregates over 150 transfer test cases when applicable. Cross-scenario transfer is not applicable to CoLight and TinyLight because their action dimensions vary across scenarios, whereas EcoLight, FRAP, and MPLight support both transfer settings.

Additional Out-of-Distribution Evaluation

We further evaluate PPL-TSC under two targeted out-of-distribution (OOD) disturbances: (i) severe congestion, where traffic demand increases to 1.5×1.5 \times its original level during 40–60% of an episode, and (ii) an accident, simulated by closing all entry lanes of the highest-demand approach during the same period. All policies remain frozen, and each matched teacher–student pair uses the same network, traffic flow, random seed, disturbance, and simulation horizon. Each teacher interface is evaluated over seven scenarios, five traffic flows, and two disturbances, yielding 70 matched cases.

Table A.16: Performance under targeted OOD disturbances. ATT improvement is measured relative to the corresponding teacher; positive values indicate lower student ATT.

Teacher interfaceΔ\DeltaATT (%)ATT wins
CoLight0.6635/70
EcoLight (phase-selection)2.6648/70
EcoLight (native)1.0730/70
FRAP1.2638/70
MPLight1.1940/70
TinyLight-0.1532/70
Overall1.12223/420

Table A.16.

Under the two targeted disturbances, five of six teacher interfaces achieve positive mean improvements, with an overall improvement of 1.12% and 223/420 wins. These results show that PPL-TSC remains broadly competitive with its teachers under severe congestion and accident-induced lane closures.

Comparison with Distill-Adapted Programmatic Baselines

We compare PPL-TSC with three representative programmatic policy-search methods: π\pi-Light [17], GPLight+ [21], and SymLight [22]. Because their original formulations learn through environment interaction rather than teacher rollouts, we adapt them into π\pi-Light-Distill, GPLight+-Distill, and SymLight-Distill for a controlled teacher-guided comparison. These adapted baselines use the same MPLight teacher rollouts, phase interface, search budget, simulator settings, and evaluation protocol as PPL-TSC; SymLight-Distill is reimplemented from its paper because public code is unavailable. No method uses target-scenario search, calibration, or teacher data after transfer.

Each scenario contains five traffic flows, with one policy learned per source flow. For the 42 directed cross-scenario pairs, five source policies are evaluated on five target flows, yielding 25 evaluations per pair and 1,050 per method. For the seven same-scenario pairs, each policy is evaluated on the other four flows, yielding 20 evaluations per pair and 140 per method. This produces 1,190 evaluations per method and 4,760 evaluations in total.

MethodCross-scenario ATT (s) ↓Best pairs ↑Mean rank ↓
PPL-TSC232.51 ± 154.20 (217.01)24/491.94
π-Light-Distill305.99 ± 274.68 (230.98)3/493.21
GPLight+-Distill301.32 ± 286.78 (234.88)11/492.54
SymLight-Distill275.23 ± 249.27 (220.53)11/492.31

Table A.17. Aggregate comparison with distill-adapted programmatic baselines. Cross-scenario ATT is reported as mean ± SD, with the median in parentheses, over the 1,050 off-diagonal evaluations. Best-pair counts and mean ranks use pair-level median ATT over all 49 source–target pairs.

PPL-TSC achieves the lowest cross-scenario mean and median ATT, obtains the best result on 24 of 49 source–target pairs, and has the lowest overall mean rank. Table A.18 provides the complete transfer matrix. Each cell reports median ATT in the order PPL-TSC / π-Light-Distill / GPLight+-Distill / SymLight-Distill; off-diagonal and diagonal cells aggregate 25 cross-scenario and 20 held-out same-scenario evaluations, respectively.

Source / TargetAtlantaHangzhou1Hangzhou2Hangzhou3JinanLos AngelesManhattan
Atlanta239.05 / 236.23 / 236.24 / 235.1298.57 / 100.18 / 100.94 / 98.20121.98 / 122.96 / 122.89 / 122.3285.06 / 85.96 / 86.08 / 85.35310.48 / 972.14 / 451.72 / 327.69446.30 / 494.71 / 473.96 / 523.34216.61 / 551.18 / 284.36 / 217.70
Hangzhou1238.91 / 307.04 / 246.65 / 262.2698.97 / 97.50 / 100.13 / 101.19121.98 / 122.89 / 122.34 / 122.1185.09 / 84.01 / 85.24 / 84.90311.48 / 342.63 / 330.78 / 325.05458.47 / 763.85 / 604.76 / 700.44217.37 / 245.91 / 256.26 / 229.73
Hangzhou2239.32 / 245.45 / 243.67 / 248.4799.13 / 96.18 / 95.20 / 96.18122.21 / 122.07 / 122.25 / 122.1985.33 / 84.66 / 84.35 / 84.66316.33 / 331.47 / 308.03 / 313.59454.33 / 487.48 / 453.35 / 490.67220.93 / 230.42 / 216.49 / 222.03
Hangzhou3246.89 / 245.41 / 244.26 / 308.2595.70 / 100.12 / 93.81 / 95.97122.14 / 122.49 / 121.99 / 122.0384.43 / 85.07 / 83.86 / 83.56316.08 / 321.09 / 309.96 / 303.85470.13 / 484.35 / 542.14 / 763.90210.03 / 221.39 / 224.13 / 218.08
Jinan238.90 / 238.67 / 238.57 / 239.0499.03 / 98.66 / 98.94 / 98.31121.98 / 122.32 / 122.62 / 122.1085.09 / 85.51 / 85.51 / 84.82310.53 / 310.86 / 309.31 / 309.12445.59 / 460.05 / 483.16 / 472.98216.53 / 218.87 / 214.51 / 213.95
Los Angeles240.33 / …96.87 / 102.88 / 106.42 / 100.10122.00 / 123.42 / 123.53 / 123.3085.32 / 86.58 / 91.05 / 86.08313.00 / 941.61 / 339.09 / 335.16466.45 / 490.53 / 461.39 / 466.50217.82 / 226.46 / 215.95 / 221.52
Manhattan238.69 / 262.08 / 244.84 / 261.9698.85 / 98.66 / 97.90 / 96.87122.22 / 122.89 / 122.32 / 122.0084.96 / 86.08 / 85.33 / 85.36313.95 / 340.81 / 328.71 / 325.27453.28 / 466.24 / 461.64 / 482.43213.92 / 216.86 / 209.95 / 207.39

Table A.18. All-to-all transfer comparison with distill-adapted programmatic baselines. Each cell reports median ATT as PPL-TSC / π-Light-Distill / GPLight+-Distill / SymLight-Distill; boldface denotes the lowest value.

Readability Examples Across Teacher Families

Table A.19 extends the MPLight case study in Section 4.4 to all five teacher families and both EcoLight interfaces. The table shows the phase-scoring component fp=f(xp,t(i))f_p = f(x_{p,t}^{(i)}); ATT and ABF are measured using the complete policy Up=fp+βp(i)U_p = f_p + \beta_p^{(i)}. The EcoLight (native) example uses the aligned inputs described in Appendix A.7.1.

Here, IpI_p equals 1 when phase pp is active and 0 otherwise. VpV_p and Vpmax⁡V_p^{\max} denote total and peak vehicle demand, respectively; NpN_p and WpW_p denote queued vehicle count and accumulated waiting time. For brevity, the decision-step index tt and intersection index ii are omitted from the table expressions. The

Teacher / taskPhase-scoring componentTraffic-control interpretationABF ↑ATT (s) ↓
TeacherStudent
CoLight Atlanta / flow 3fp=Vp+0.2IpVp2f_p = V_p + 0.2I_pV_p^2Prioritizes demand; a quadratic bonus retains a busy active phase.0.94339.20317.90
EcoLight (phase-selection) Hangzhou2 / flow 2fp=0.7Vp+0.3Vpmax⁡+0.15Ipf_p = 0.7V_p + 0.3V_p^{\max} + 0.15I_pCombines total and peak demand with an active-phase retention bonus.0.94135.93134.13
EcoLight (native, aligned) Hangzhou2 / flow 2fkeep=gg+1f_{\mathrm{keep}} = \frac{g}{\sqrt{g+1}}; fswitch=rlog⁡(1+r)g+1f_{\mathrm{switch}} = \frac{r\log(1+r)}{g+1}Keeps green for served demand; switches as red-side demand grows relative to green-side demand.0.99135.93135.05
FRAP Atlanta / flow 1fp=∑d∈Kpnd17+0.2Ipf_p = \frac{\sum_{d \in K_p} n_d}{17} + 0.2I_pScores aggregate vehicle demand linearly and adds a fixed bonus for the active phase.0.89255.03245.01
MPLight Hangzhou3 / flow 4fp=PpΣ+Ppmax⁡+0.15Ipf_p = P_p^\Sigma+ P_p^{\max} + 0.15I_pCombines aggregate and bottleneck pressure with a retention bonus, following MaxPressure [7].0.9386.0184.04
TinyLight Hangzhou3 / flow 1fp=PpΣ+0.5log⁡(1+Wp)+0.2Np+0.5Ipf_p = P_p^\Sigma+ 0.5\log(1+W_p) + 0.2N_p + 0.5I_pPrioritizes pressure, queues, and waiting time, with a retention bonus. The logarithm gives additional waiting time progressively less weight.0.8680.7780.63

Table A.19. Representative learned rules and traffic-control interpretations across teacher families. ATT is measured in seconds. Each row corresponds to one scenario–flow task.

pressure terms follow Figure 3. For the set of directions Rp\mathcal{R}_p served by phase pp, PpΣ=∑d∈Rpp^d,tP_p^{\Sigma} = \sum_{d \in\mathcal{R}_p} \hat{p}_{d,t} and Ppmax⁡=max⁡d∈Rpp^d,tP_p^{\max} = \max_{d \in\mathcal{R}_p} \hat{p}_{d,t}, where p^d,t\hat{p}_{d,t} is normalized directional pressure. Before normalization, directional pressure is the upstream vehicle count minus the downstream vehicle count. For EcoLight (native, aligned), gg and rr denote green- and red-side demand, respectively. For FRAP, ndn_d denotes directional vehicle demand, and the constant 17 is the 95th percentile of vehicles_raw in the source data.

LLM Configuration, Cost, and Randomness

The main experiments use Gemini 3.1 Flash with temperature 0.8; top-p and maximum output length use the API defaults, and individual responses contain 407–779 tokens. To evaluate random-seed sensitivity, we conduct five searches for each of five teachers and seven scenarios, giving 175 searches. Within each of the 35 teacher–scenario combinations, we compute the coefficient of variation (CV; standard deviation divided by mean) of the final joint score across the five seeds and then average the CV across combinations. The remaining quantities in Table A.20 are averaged over all 175 searches.

MetricMean
Final joint-score CV2.08%
Token usage per search67.1K
Generation cost per searchUSD 0.0353
LLM API latency per search104.2 s

Table A.20. LLM random-seed sensitivity and discovery cost.

The low average final-score CV indicates limited sensitivity to sampling randomness under this protocol. LLM use is confined to policy discovery: reproducing a search requires the same model and API configuration and may still vary because of provider-side sampling, whereas exported policies execute deterministically as standalone Python code and require no LLM during inference or deployment. Token cost and API latency are provider- and time-dependent and are reported only for the evaluated configuration.

Figure A.3. Parameter sensitivity on MPLight/Los Angeles/flow0. Each heatmap shows the final best score under one LLM backbone, where the score is computed by the policy evaluation objective in Eq. (4). Rows correspond to NpopN_{\mathrm{pop}} and columns to Gmax⁡G_{\max}.

Parameter Analysis

We analyze the sensitivity of PPL-TSC to three configuration factors: the LLM backbone, the population size NpopN_{\mathrm{pop}}, and the number of generations Gmax⁡G_{\max}. Fig. A.3 visualizes the 54 runs on MPLight/Los Angeles/flow0 as per-backbone heatmaps of the final best score, where the score is the policy evaluation objective defined in Eq. (4).

Two observations are clear from the figure. First, the LLM backbone substantially affects the absolute search quality. Gemini 3.1 Flash is the strongest overall, with all nine settings above 0.94 and a peak score of 1.299 at (8, 8). Grok-4.1 and Qwen3.5 form a second tier, reaching 1.254 and 1.231, respectively. By contrast, Claude-Haiku-4.5 and GPT-5.4-Nano are weaker, with best scores of only 0.934 and 1.052.

Second, larger generation budgets are usually more helpful than simply enlarging the population. For five of the six backbones, the best cell lies at Gmax⁡=8G_{\max}=8; the only exception is GPT-5.4-Nano, whose best score appears at (8, 3). The preferred population size is less stable: the best setting uses Npop=8N_{\mathrm{pop}}=8 for MiniMax-M2.7, Gemini 3.1 Flash, GPT-5.4-Nano, and Grok-4.1, but Npop=5N_{\mathrm{pop}}=5 for Claude-Haiku-4.5 and Qwen3.5. Overall, (5, 5) provides a conservative default, while larger-budget settings such as (5, 8) and (8, 8) offer stronger search quality when additional runtime is available.

NeurIPS Paper Checklist

Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction accurately summarize the paper’s core contributions: PPL-TSC, the phase utility function representation, and the generate–review–calibrate–evaluate framework. Their claims on transparency, teacher fidelity, transfer generalization, and computational efficiency are consistent with the experimental results reported in the main paper and appendix.

Guidelines:

  • The answer [N/A] means that the abstract and introduction do not include the claims made in the paper.

  • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A [No] or [N/A] answer to this question will not be perceived well by the reviewers.

  • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

  • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The paper includes a dedicated Limitations subsection in the appendix that discusses key constraints of the current approach, including teacher dependence and LLM-guided search dependence. It also outlines practical implications and future directions to mitigate these limitations.

Guidelines:

  • The answer [N/A] means that the paper has no limitation while the answer [No] means that the paper has limitations, but those are not discussed in the paper.

  • The authors are encouraged to create a separate “Limitations” section in their paper.

  • The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

  • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

  • The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

  • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

  • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

  • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [N/A]

Justification: The paper does not present formal theorems, lemmas, or proof-based theoretical claims. The main contributions are methodological and empirical.

Guidelines:

  • The answer [N/A] means that the paper does not include theoretical results.

  • All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.

  • All assumptions should be clearly stated or referenced in the statement of any theorems.

  • The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

  • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

  • Theorems and Lemmas that the proof relies upon should be properly referenced.

Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The paper and appendix describe the method pipeline, evaluation protocol, teacher settings, key hyperparameters, and implementation choices. The public repository at https://github.com/ForrestXie9/PPL-TSC provides implementation code, traffic networks and flows, configuration templates, and commands for teacher training, trajectory collection, policy learning, and evaluation. Teacher weights, trajectories, and learned policies can be generated using these commands; policy discovery requires user-configured LLM API access.

Guidelines:

  • The answer [N/A] means that the paper does not include experiments.

  • If the paper includes experiments, a [No] answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

  • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

  • Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general, releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

  • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example, (a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The code and traffic networks and flows are publicly available at https://github.com/ForrestXie9/PPL-TSC. The README specifies the required environment and commands for installation, teacher training, trajectory collection, policy learning, and same-scenario and cross-scenario evaluation. Pretrained weights, collected trajectories, and generated policies are not bundled and must be produced using the provided pipeline. LLM-based policy discovery requires API access and may vary with provider-side sampling.

Guidelines:

  • The answer [N/A] means that paper does not include experiments requiring code.

  • Please see the NeurIPS code and data submission guidelines (https://neurips.cc/public/guides/CodeSubmissionPolicy) for more details.

  • While we encourage the release of code and data, we understand that this might not be possible, so [No] is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

  • The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://neurips.cc/public/guides/CodeSubmissionPolicy) for more details.

  • The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

  • The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

  • At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

  • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer) necessary to understand the results?

Answer: [Yes]

Justification: The manuscript specifies the teacher families, scenario setup, optimization choices, and key hyperparameters for policy search and calibration, with additional details in the appendix. The test settings and evaluation metrics used in the main comparisons are also explicitly described.

Guidelines:

  • The answer [N/A] means that the paper does not include experiments.

  • The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

  • The full details can be provided either with the code, in appendix, or as supplemental material.

Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: The paper reports variability using mean and standard deviation (shown as ±\pm values in tables and summarized in appendix tables). The aggregation protocol for these statistics is described alongside the reported results.

Guidelines:

  • The answer [N/A] means that the paper does not include experiments.

  • The authors should answer [Yes] if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

  • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

  • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

  • The assumptions made should be given (e.g., Normally distributed errors).

  • It should be clear whether the error bar is the standard deviation or the standard error of the mean.

  • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

  • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g., negative error rates).

  • If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [No]

Justification: Appendix A.8 specifies the CPU, GPU, memory, training episodes, and search settings. Appendix A.8.7 reports LLM API latency, but per-experiment execution times and the total compute budget are not fully reported.

Guidelines:

  • The answer [N/A] means that the paper does not include experiments.

  • The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

  • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

  • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn’t make it into the paper).

Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?

Answer: [Yes]

Justification: We have reviewed the NeurIPS Code of Ethics and are not aware of any aspect of this work that violates it. The study uses simulated traffic environments and does not involve human-subject experimentation. Guidelines:

  • The answer [N/A] means that the authors have not reviewed the NeurIPS Code of Ethics.

  • If the authors answer [No], they should explain the special circumstances that require a deviation from the Code of Ethics.

  • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: The manuscript includes a dedicated Broader Impacts subsection in the appendix. That discussion covers both potential benefits (e.g., improved traffic efficiency and reduced congestion-related emissions) and potential risks, together with cautionary deployment considerations.

Guidelines:

  • The answer [N/A] means that there is no societal impact of the work performed.

  • If the authors answer [N/A] or [No], they should explain why their work has no societal impact or why the paper does not address societal impact.

  • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

  • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate Deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

  • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

  • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pre-trained language models, image generators, or scraped datasets)?

Answer: [N/A]

Justification: This work does not release high-risk generative models or scraped datasets that would require specialized misuse safeguards. The method targets traffic-signal control in simulation settings.

Guidelines:

  • The answer [N/A] means that the paper poses no such risks.

  • Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

  • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

  • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The original simulator, benchmark, and teacher methods are credited through their original publications, with asset usage described in Appendix § 7.§ 11.7. The public code repository provides third-party attribution in THIRD_PARTY_NOTICES.md and includes license files for the bundled simulator and dependencies. We respect the applicable licenses and terms of use for third-party code, data, and LLM services.

Guidelines:

  • The answer [N/A] means that the paper does not use existing assets.

  • The authors should cite the original paper that produced the code package or dataset.

  • The authors should state which version of the asset is used and, if possible, include a URL.

  • The name of the license (e.g., CC-BY 4.0) should be included for each asset.

  • For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

  • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

  • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

  • If this information is not available online, the authors are encouraged to reach out to the asset’s creators.

New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The released implementation is documented in the repository README at https://github.com/ForrestXie9/PPL-TSC, including installation requirements, supported teachers and student variants, configuration templates, execution commands, and the scope of included assets. Third-party attribution and licensing notes are provided in THIRD_PARTY_NOTICES.md.

Guidelines:

  • The answer [N/A] means that the paper does not release new assets.

  • Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

  • The paper should discuss whether and how consent was obtained from people whose asset is used.

  • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? Answer: [N/A]

Justification: The study does not involve crowdsourcing or human-subject data collection. All experiments are conducted in simulation environments.

Guidelines:

  • The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects.

  • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

  • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [N/A]

Justification: No human participants are involved in this work, so IRB approval and participant-risk disclosure are not applicable.

Guidelines:

  • The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects.

  • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

  • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

  • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigor, or originality of the research, declaration is not required.

Answer: [Yes]

Justification: LLM usage is a core part of the method: the model is used for candidate policy generation and iterative repair in the generate–review–calibrate–evaluate pipeline. The paper describes this role in both the abstract/method section and appendix prompt details.

Guidelines:

  • The answer [N/A] means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

  • Please refer to our LLM policy in the NeurIPS handbook for what should or should not be described.

References

  1. [1]Hua Wei, Guanjie Zheng, Vikash Gayah, and Zhenhui Li. A survey on traffic signal control methods. arXiv preprint arXiv:1904.08117, 2019.arxiv.org/abs/1904.08117
  2. [2]Kan Wu, Jianrong Ding, Jingli Lin, Guanjie Zheng, Yi Sun, Jie Fang, Tu Xu, Yongdong Zhu, and Baojing Gu. Big-data empowered traffic signal control could reduce urban carbon emission. Nature Communications, 16(1):2013, 2025.DOI
  3. [3]Fo Vo Webster. Traffic signal settings. Technical report, 1958.
  4. [4]John DC Little. The synchronization of traffic signals by mixed-integer linear programming. Operations Research, 14(4):568–594, 1966.
  5. [5]John DC Little, Mark D Kelson, and Nathan H Gartner. MAXBAND: A versatile program for setting signals on arteries and triangular networks. 1981.
  6. [6]Seung-Bae Cools, Carlos Gershenson, and Bart D’Hooghe. Self-organizing traffic lights: A realistic simulation. In Advances in Applied Self-Organizing Systems, pages 41–50. Springer, 2007.
  7. [7]Pravin Varaiya. The max-pressure controller for arbitrary networks of signalized intersections. In Advances in Dynamic Network Modeling in Complex Transportation Systems, pages 27–66. Springer, 2013.DOI
  8. [8]PB Hunt, DI Robertson, RD Bretherton, and RI Winton. SCOOT-a traffic responsive method of coordinating signals. Technical report, 1981.
  9. [9]PR Lowrie. SCATS, Sydney co-ordinated adaptive traffic system: A traffic responsive method of controlling urban traffic. 1990.
  10. [10]Hua Wei, Guanjie Zheng, Vikash Gayah, and Zhenhui Li. Recent advances in reinforcement learning for traffic signal control: A survey of models and evaluation. ACM SIGKDD Explorations Newsletter, 22(2):12–18, 2021.DOI
  11. [11]Li Li, Yisheng Lv, and Fei-Yue Wang. Traffic signal timing via deep reinforcement learning. IEEE/CAA Journal of Automatica Sinica, 3(3):247–254, 2016.DOI
  12. [12]Kok-Lim Alvin Yau, Junaid Qadir, Hooi Ling Khoo, Mee Hong Ling, and Peter Komisarczuk. A survey on reinforcement learning models and algorithms for traffic signal control. ACM Computing Surveys (CSUR), 50(3):1–38, 2017.DOI
  13. [13]Feng Xiao, Jiaming Lu, Lu Li, Wenwen Tu, and Chaojing Li. Advances in reinforcement learning for traffic signal control: a review of recent progress. Intelligent Transportation Infrastructure, 4:liaf009, 2025.DOI
  14. [14]Hua Wei, Nan Xu, Huichu Zhang, Guanjie Zheng, Xinshi Zang, Chacha Chen, Weinan Zhang, Yanmin Zhu, Kai Xu, and Zhenhui Li. CoLight: Learning network-level cooperation for traffic signal control. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 1913–1922, 2019.DOI
  15. [15]Guanjie Zheng, Yuanhao Xiong, Xinshi Zhang, Jie Feng, Hua Wei, Huichu Zhang, Yong Li, Kai Xu, and Zhenhui Li. Learning phase competition for traffic signal control. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 1963–1972, 2019.arxiv.org/abs/1905.04722
  16. [16]Yuanyang Zhu, Xiao Yin, and Chunlin Chen. Extracting decision tree from trained deep reinforcement learning in traffic signal control. IEEE Transactions on Computational Social Systems, 10(4):1997–2007, 2022.
  17. [17]Yin Gu, Kai Zhang, Qi Liu, Weibo Gao, Longfei Li, and Jun Zhou. π-Light: Programmatic interpretable reinforcement learning for resource-limited traffic signal control. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 21107–21115, 2024.
  18. [18]Sachin Chauhan, Kashish Bansal, and Rijurekha Sen. EcoLight: Intersection control in developing regions under extreme budget and network constraints. Advances in Neural Information Processing Systems, 33:13027–13037, 2020.
  19. [19]Chacha Chen, Hua Wei, Nan Xu, Guanjie Zheng, Ming Yang, Yuanhao Xiong, Kai Xu, and Zhenhui Li. Toward a thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 3414–3421, 2020.DOI
  20. [20]Dong Xing, Qian Zheng, Qianhui Liu, and Gang Pan. TinyLight: Adaptive traffic signal control on devices with extremely limited resources. arXiv preprint arXiv:2205.00427, 2022.arxiv.org/abs/2205.00427
  21. [21]Xiao-Cheng Liao, Yi Mei, and Mengjie Zhang. GPLight+: a genetic programming method for learning symmetric traffic signal control policy. IEEE Transactions on Evolutionary Computation, 2025.DOI
  22. [22]Xiao-Cheng Liao, Yi Mei, and Mengjie Zhang. SymLight: Exploring interpretable and deployable symbolic policies for traffic signal control. arXiv preprint arXiv:2511.05790, 2025.arxiv.org/abs/2511.05790
  23. [23]Maciej Świechowski, Konrad Godlewski, Bartosz Sawicki, and Jacek Mańdziuk. Monte Carlo tree search: A review of recent modifications and applications. Artificial Intelligence Review, 56(3):2497–2562, 2023.DOI
  24. [24]John R Koza. Genetic programming: A paradigm for genetically breeding populations of computer programs to solve problems, volume 34. Stanford University, Department of Computer Science Stanford, CA, 1990.
  25. [25]Mohammad Aslani, Mohammad Saadi Mesgari, and Marco Wiering. Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events. Transportation Research Part C: Emerging Technologies, 85:732–752, 2017.
  26. [26]Stefano Giovanni Rizzo, Giovanna Vantini, and Sanjay Chawla. Time critic policy gradient methods for traffic signal control in complex and congested scenarios. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1654–1664, 2019.DOI
  27. [27]Tianshu Chu, Jie Wang, Lara Codečà, and Zhaojian Li. Multi-agent deep reinforcement learning for large-scale traffic signal control. IEEE Transactions on Intelligent Transportation Systems, 21(3):1086–1095, 2019.arxiv.org/abs/1903.04527
  28. [28]Afshin Oroojlooy, Mohammadreza Nazari, Davood Hajinezhad, and Jorge Silva. AttendLight: Universal attention-based reinforcement learning model for traffic signal control. Advances in Neural Information Processing Systems, 33:4079–4090, 2020.arxiv.org/abs/2010.05772
  29. [29]James Ault, Josiah P Hanna, and Guni Sharon. Learning an interpretable traffic signal control policy. arXiv preprint arXiv:1912.11023, 2019.arxiv.org/abs/1912.11023
  30. [30]Milad Taleby Ahvanooey, Qianmu Li, Ming Wu, and Shuo Wang. A survey of genetic programming and its applications. KSII Trans. Internet Inf. Syst., 13(4):1765–1794, 2019.
  31. [31]Lindong Xie, Yang Zhang, Zhixian Tang, Edward Chung, Genghui Li, and Zhenkun Wang. Co-evolution of large language models and configuration strategies to enhance surrogate-assisted evolutionary algorithm. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 3321–3332, 2025.DOI
  32. [32]Fei Liu, Xialiang Tong, Mingxuan Yuan, Xi Lin, Fu Luo, Zhenkun Wang, Zhichao Lu, and Qingfu Zhang. Evolution of heuristics: Towards efficient automatic algorithm design using large language model. arXiv preprint arXiv:2401.02051, 2024.arxiv.org/abs/2401.02051
  33. [33]Sathvik Joel, Jie Wu, and Fatemeh Fard. A survey on LLM-based code generation for low-resource and domain-specific programming languages. ACM Transactions on Software Engineering and Methodology, 2024.arxiv.org/abs/2410.03981
  34. [34]Nikolaus Hansen, Sibylle D Müller, and Petros Koumoutsakos. Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES). Evolutionary Computation, 11(1):1–18, 2003.pubmed.ncbi.nlm.nih.gov/12804094
  35. [35]Huichu Zhang, Siyuan Feng, Chang Liu, Yaoyao Ding, Yichen Zhu, Zihan Zhou, Weinan Zhang, Yong Yu, Haiming Jin, and Zhenhui Li. CityFlow: A multi-agent reinforcement learning environment for large scale city traffic scenario. In The World Wide Web Conference, pages 3620–3624. Association for Computing Machinery, 2019. doi: 10.1145/3308558.3314139.DOI
  36. [36]Hao Mei, Xiaoliang Lei, Longchao Da, Bin Shi, and Hua Wei. LibSignal: an open library for traffic signal control. Machine Learning, 113(8):5235–5271, 2024.DOI

Paper details

vlnc:W_4t6fab3m
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents