Causal reinforcement learning for train scheduling on single-track railway networks

Authors

• Feiyu Yang
• Jixue Liu
• Jiuyong Li
• Lin Liu
• Shengjie Wang
• Wenqing Li
• Shaoquan Ni

Document Type

Journal Article

Publication Date

2025

Subject Area

mode - rail, operations - scheduling

Keywords

Causal reinforcement learning, Railway scheduling, Single-track railway networks

Abstract

Recent advancements in deep reinforcement learning have shown promise for large-scale railway scheduling. By decomposing complex railway networks into infrastructure-defined control units (e.g., sections and stations) managed by intelligent agents, this approach reduces scheduling complexity while enhancing scalability. Crucially, scheduling decisions across these units exhibit complex interdependencies posing a major challenge for existing reinforcement learning methods, particularly in single-track railway networks with spatio-temporal constraints. Achieving optimal scheduling requires understanding and leveraging interaction mechanisms between unit-level behaviors. However, agents managing individual units struggle to capture unobservable interactions, and even determining whether these interactions are discrete, continuous, or hybrid remains a major challenge, posing difficulties in modeling through deterministic variables. To address this, we introduce a latent variable into each agent’s probabilistic decision-making model to capture unobserved interactions, establishing a structural causal model for multi-agent decision-making for the complicated train scheduling task. By inferring latent variables from observed data, we disentangle interdependent decision processes. Each agent integrates a variational autoencoder with an end-to-end causal reinforcement learning framework to enhance collaborative scheduling in single-track networks. Experiments demonstrate state-of-the-art performance, marking the first application of causal modeling in railway scheduling and suggesting new research directions.

Rights

Permission to publish the abstract has been given by Elsevier, copyright remains with them.

Comments

Transportation Research Part C Home Page:

http://www.sciencedirect.com/science/journal/0968090X

Recommended Citation

Yang, F., Liu, J., Li, J., Liu, L., Wang, S., Li, W., & Ni, S. (2025). Causal reinforcement learning for train scheduling on single-track railway networks. Transportation Research Part C: Emerging Technologies, 178, 105215.