Highway Congestion Reduction through RL-Based Eulerian Headway Control

Abstract

Connected automated vehicles (CAVs) equipped with adaptive cruise control (ACC) create new opportunities for highway congestion mitigation. Traditional practice relies on Eulerian variable speed limits (VSL) which regulate traffic through roadside signs, but suffer from infrequent updates and limited driver compliance. Recent research explored Lagrangian strategies that directly control individual vehicles, offering high reactivity and compliance, yet in realistic multi-lane settings they depend on drivers' latent lane-change intentions, making robust vehicle-level decisions difficult. Hence, we propose an Eulerian control system optimized through reinforcement learning, that (i) leverages ACC for reactivity and compliance, and (ii) obviates dependence on latent driver intentions by regulating aggregate density near bottlenecks, crucially via headway commands rather than speed commands. We evaluate two variants of our system, time-headway and distance-headway control, in large-scale simulations across a range of traffic conditions. Both variants outperform baselines, improving traffic flow by up to 10.6% over human traffic and 6.7% over traditional VSL. To strengthen evaluation, we propose a novel boundary-aware speed metric addressing a recognized flaw in simulation studies with dynamic vehicle entry and exit. The empirical results, together with our emphasis on deployable system design, suggest a path towards practical, safe, and scalable highway congestion mitigation.

1

Control traffic density through headway

Headway commands set vehicle spacing directly, which makes density — not speed — the quantity the controller actually regulates.

2

Designed for practical deployment

Built on technologies that already exist: production ACC, standard traffic sensing, and low-bandwidth cellular broadcast.

Background

From simulation to the open road

This work continues a line of research on using connected automated vehicles (CAVs) to improve highway flow in mixed traffic. CAV-based coordination has been demonstrated in real-world trials, but has not yet achieved widespread commercial deployment.

Flow (2018)

Cathy Wu, Alex Bayen et al. introduced a framework for applying deep reinforcement learning to mixed-autonomy traffic control.

Distributed control (2021)

Jiaxun Cui, Peter Stone et al., together with our team at GM Research, developed distributed in-vehicle control algorithms on top of that framework.

CIRCLES & the MegaVanderTest (2022)

A multi-institution effort that took the idea out of simulation and onto a live highway.

Human-driven baseline from the distributed-control line of work.

Controlled outflow under distributed in-vehicle control.

CIRCLES: coordinating CAVs to smooth real-world traffic

The vision of CIRCLES was to take the Flow idea out of simulation and into the real world. Over several years, a team headed by Alex Bayen at UC Berkeley — spanning academic institutions, government agencies, and industry partners including GM Research — prepared a large-scale, five-day experiment on I-24 in Tennessee.

The MegaVanderTest was the final open-road test of the project: 100 automated vehicles sent onto I-24 during the morning peak, and one of the largest coordinated open-road traffic-smoothing experiments conducted to date. Analysis found an 8% reduction in average traffic density, and a 52% reduction during congestion formation.

It was also expensive — millions of dollars — and ran under stochastic real-world conditions, which makes it hard to reproduce in a controlled manner. This paper takes the lessons from the MegaVanderTest, proposes a new practical control paradigm, and analyzes it in controlled experiments.

[Lee et al., IEEE Control Systems 2025] · [Wang et al., IEEE Control Systems 2025]

Map of the Nashville area showing the I-24 test road and the target bottleneck — The I-24 test road southeast of Nashville, with the target bottleneck marked.

A row of numbered white and dark SUVs staged in a parking lot for the MegaVanderTest — Part of the 100-vehicle fleet staged for the MegaVanderTest.

The problem

A local merge can trigger a network-wide breakdown

We focus on a high-demand weaving area between Exit 60 and Exit 59 on I-24 westbound — a well-known, recurrent bottleneck during the morning peak. Merging traffic induces slow-downs, lane changes, and stop-and-go waves that reduce flow and increase energy consumption.

Aerial view of I-24 westbound between Exit 60 and Exit 59 — **Target bottleneck.** The high-demand weaving area between Exit 60 and Exit 59 on I-24 westbound, Tennessee.

Velocity contour heatmap of I-24 westbound showing a standing bottleneck at Exit 59 — **Average-speed profile** on the test segment. Red indicates higher density and lower speed. A standing bottleneck forms at Exit 59, from which a queue builds and sends shockwaves upstream. Based on data provided by INRIX, Inc.

Illustration of a four-lane highway where an on-ramp merge creates a density spike and an upstream stop-and-go wave — On a congested multi-lane road, a single on-ramp merge is enough to seed congestion. Merging vehicles create a **density spike** at the bottleneck, which triggers **lane changes**, which produce a **stop-and-go wave** that propagates upstream.

Increasing headway helps when the location and timing of lane changes are predictable. In multi-lane bottlenecks they are not — they depend on hidden driver intentions.

Vehicle-level decisions require access to other drivers' hidden intentions

Diagram contrasting no preemptive headway increase with preemptive headway increase, and the four possible outcomes depending on whether a cut-in occurs

This is the core chicken-and-egg dilemma at the vehicle level. Increasing a vehicle's headway helps only if a neighbour actually cuts in from the congested lane. If no cut-in happens, the enlarged gap is wasted capacity and flow drops.

Choosing the right action therefore depends on predicting the counterfactual behaviour of other drivers, which makes purely in-vehicle, distributed control very challenging.

A robust action would require predicting counterfactual driver behaviour → which motivates density regulation instead.

Our claim

Connectivity and automation can be better utilised

Existing and recently proposed systems each give up something. Our goal is a controller that is responsive, compliant, and intention-agnostic all at once.

Existing systems

Eulerian variable speed limits

Roadside signs · aggregate control

Low responsiveness — signs update only a few times per hour
Inconsistent human compliance
Does not depend on intention prediction

[Papageorgiou 2023, Hegyi 2005, Han 2017]

Recently proposed systems

Lagrangian vehicle control

Direct vehicle actuation

High responsiveness
High compliance
Depends on latent lane-change intentions in multi-lane traffic

[Vinitsky et al. 2023, Cui et al. 2021, Zhang et al. 2023]

Design target: responsive + compliant + intention-agnostic

Headway regulates density more directly than speed

To dodge the intention-prediction problem we go back to the aggregate level and regulate traffic density. Since headway regulates density more directly than speed does, headway is the natural Eulerian control variable.

Time-space diagrams comparing a constant speed limit against a constant time-headway command, showing average speed and density — **Constant speed limit vs. constant time-headway**, applied for identical durations. Top pair: average speed (green is fast, yellow/red is slow). Bottom pair: density (green is sparse, yellow/red is dense). White dotted markers show the control duration and the resulting low-density duration downstream.

Time-headway commands set vehicle spacing directly, sustaining a low-density region for the entire control duration.

With speed commands, low density is preserved only briefly. Generating and maintaining the same effect for longer requires carefully shaping the speed signal over time, in a complex and indirect manner.

This structural simplicity is what motivates headway as the Eulerian control variable.

Proposed system

Sense the segment, infer a headway, broadcast it

Diagram of the analysed scenario: an RL controller receives road-segment measurements and sends headway commands to connected ACC vehicles near the bottleneck — **Eulerian RL-based headway control.** (a) The analysed scenario — an RL controller sends commands to connected-ACC vehicles near the bottleneck based on road-segment measurements. (b) The two command types.

1

Sense

Segment speed and density around the bottleneck.

2

Infer

An RL policy maps that aggregate state to continuous headway actions.

3

Broadcast

Actions go out to connected ACC vehicles in the controlled segments.

4

Actuate

ACC adjusts headway within its existing safeguards.

The policy is Eulerian: decisions are made for fixed road segments around the bottleneck, never for an individual vehicle. There are two command types — time headway (1.5–6 s) and distance headway (0–30 m) — and commands refresh every 2.5 seconds. The low-level actuation is Lagrangian in effect, since moving ACC vehicles execute the command, but the decision never depends on any individual driver's intention.

Safety is assumed to be enforced by production ACC. Commands are constrained to conservative ranges, and we do not claim new safety guarantees.

Practical by design: it runs on the ACC you already have

1

Adaptive cruise control integration

Hooks directly into ACC, adjusting a single headway parameter.

2

Existing sensors & communication

An I-24 MOTION-like sensing system plus low-bandwidth cellular communication.

3

Practical & scalable deployment

One headway command per segment, deployable independently at each location.

4

Built-in ACC safeguards

Headway commands are restricted to ACC-allowed ranges.

In-vehicle ACC controls on the steering wheel and the corresponding instrument-cluster display — The in-vehicle ACC interface used in CIRCLES, where the headway setting is adjusted on the fly.

Because ACC is already standard equipment, no LiDAR arrays or specialised sensors are needed — the test vehicles are essentially production cars. On the infrastructure side we rely on standard traffic sensors for speed and density, plus a simple low-bandwidth broadcast of the desired headway to vehicles near the bottleneck. That is a far lighter requirement than micromanaging every car's speed, acceleration, or lane changes. And because only one headway command is sent per road segment, the approach scales as more automated vehicles join the flow: the same broadcast covers them all, with no extra retrofits and no additional roadside installations.

Method

RL learns headway commands from segment-level traffic fields

We frame control as a Markov decision process and solve it with reinforcement learning.

State

Average speed and density across the upstream, bottleneck, on-ramp, and downstream segments.

Action

Two continuous headway commands for the segments immediately upstream of the bottleneck.

Transition

SUMO microscopic traffic dynamics with IDM car-following.

Reward

An immediate proxy for delay relative to free flow, with delayed entries counted as zero speed.

\[ r_t \;=\; \frac{1}{C}\sum_{i=1}^{N_t}\left(\frac{v_t^{(i)} - v_{\text{free}}\!\left(x_t^{(i)}\right)}{v_{\text{free}}\!\left(x_t^{(i)}\right)}\right)\Delta t \]

where \(N_t\) is the number of vehicles planned to enter the simulation by time \(t\); \(x_t^{(i)}\) and \(v_t^{(i)}\) are the position and speed of vehicle \(i\); \(v_{\text{free}}(x)\) is the local speed limit; \(\Delta t\) is the simulation time step (0.5 s); and \(C\) is a numerical normalisation constant.

Training. Policies are trained with PPO (and SAC) via RLlib in simulation, randomising which vehicles are connected in each episode.

Evaluation

Evaluation setup

Satellite view of the simulated 2 km stretch of I-24 with a single on-ramp — A simulated 2 km stretch of the I-24 merge: four lanes, one on-ramp, and two controlled upstream segments.

~1,000vehicles simulated

500 sepisode length

30random seeds

95%confidence intervals

Swept conditions

Merge inflow: 7 / 14 / 30 veh/min
ACC penetration: 20% / 60% / 100%

Baselines

Human-driven traffic
Traditional VSL
Dynamic RL-based VSL

A boundary-aware metric that penalises delayed entry

Finite-road simulations can be gamed — a previously recognised limitation [Cui et al., AAMAS 2021]. A naive average-speed metric ignores the time vehicles spend waiting upstream before they enter the simulation, so a controller that simply delays entry looks fast while hiding the congestion it created.

Our boundary-aware per-vehicle speed divides the distance travelled by the time elapsed since the planned entry time, so waiting counts as zero speed. Taking the minimum of the exit time and the simulation end handles vehicles that never leave.

A controller can no longer score well merely by delaying vehicles' entry onto the simulated road.

\[ \bar{v}^{(i)} \;=\; \frac{L^{(i)}}{\min\!\left(T_f^{(i)},\, T_{\text{sim}}\right) - T_s^{(i)}} \]

where \(L^{(i)}\) is the distance travelled by vehicle \(i\) within the simulated road; \(T_s^{(i)}\) its planned entry time; \(T_f^{(i)}\) its measured exit time; and \(T_{\text{sim}}\) the total simulation time.

Results

Headway control consistently outperforms speed-based baselines

Maximum gains are reached at 100% connected-ACC penetration: more compliant ACC vehicles means stronger density regulation, which means larger speed gains.

+10.6% average velocity vs. human traffic

+6.7% vs. traditional VSL

+3.4% vs. RL-based VSL

Bar chart (a): average velocity increase across merge inflow rates of 7, 14 and 30 vehicles per minute at 100% connected-ACC penetration, for the four controllers — **Average-velocity improvement.** (a) Across merge inflow rates, at 100% penetration. (b) Across connected-ACC penetration rates, at medium merge flow (14 veh/min). Error bars are 95% confidence intervals. Both headway variants outperform all baselines; we do not claim that either headway variant is universally superior.

Bar chart (b): average velocity increase across connected-ACC penetration rates of 20%, 60% and 100% at medium merge flow, for the four controllers — **Average-velocity improvement.** (a) Across merge inflow rates, at 100% penetration. (b) Across connected-ACC penetration rates, at medium merge flow (14 veh/min). Error bars are 95% confidence intervals. Both headway variants outperform all baselines; we do not claim that either headway variant is universally superior.

These are simulation results. Real-world validation is future work, and the improvement may be more pronounced in more extensive simulations with longer traffic jams and merges.

Summary

Control the traffic density. Let vehicles self-organise.

Sense aggregate traffic

→

Command headway

→

Reduce bottleneck congestion

Contributions

Responsive Eulerian headway control
No dependence on counterfactual lane-change intentions
Uses existing connected-ACC capabilities
Deployment-facing design

Future work

Calibrate the simulation with real-world data
Validate on-road
Combine with ramp metering and dynamic lane assignment

BibTeX

This paper is being presented at IEEE ITSC 2026; the proceedings version is forthcoming. Until it appears, please cite the entry below — it points at the conference and carries the arXiv preprint so readers can still find the paper. We will add the DOI and page numbers here once the proceedings are published.

@inproceedings{veksler2026highway,
  title         = {Highway Congestion Reduction through Reinforcement Learning
                   Based Eulerian Headway Control},
  author        = {Veksler, Yaron and Hornstein, Sharon and Wang, Han and
                   Delle Monache, Maria Laura and Urieli, Daniel},
  booktitle     = {2026 IEEE 29th International Conference on Intelligent Transportation
                   Systems (ITSC)},
  year          = {2026},
  note          = {To appear},
  eprint        = {2412.02520},
  archivePrefix = {arXiv},
  primaryClass  = {cs.MA}
}