Reinforcement Learning (RL) is gaining traction in our increasingly connected world. It excels in uncertain and complex scenarios, aiming to maximize long-term rewards by guiding agents through actions in their environment. RL finds applications in self-driving vessels, traffic optimization, and mastering complex games like Starcraft or Go. Its potential lies in automating tasks prone to errors and accidents, enhancing both efficiency and safety in human labor.
On September 15-16, 2022, the "Friedrich List" faculty of transport and traffic sciences hosted a conference on Reinforcement Learning. The conference was organized by the Reinforcement Learning Group Dresden and was held in a hybrid format, with both online and on-site participation and was attended by over 100 participants from more than 10 countries.
Check out our conference website here .
Ostap Okhrin delivered a keynote "Gumbel Lecture" at the Statistishe Woche 2024 on the topic "Reinforcement Learning in Transportation". Ankit Chaudhari, Saad Sagheer, Haoze Jiang, Shashank Rajput, and Shivachetan Venkateshappa presented at the same place.
Ankit Chaudhari presented work on " Drone-Based Trajectory Data for an All-Traffic-State Inclusive Freeway with Ramps." at the TRC30 and MFTS conferences in Heraklion, Crete, Greece.
Ostap Okhrin gave a keynote speech at the PROBASTAT in Slovakia on „Two-sample testing in reinforcement learning".
Ostap Okhrin gave a talk at SMSA 2024 in Delft on „A Platform-Agnostic Deep Reinforcement Learning Framework for Effective Sim2Real Transfer in Autonomous Driving".
Martin Waltz gave a presentation entitled "Local Path Planning in Transportation using Reinforcement Learning" on the, 1st Symposium On Lifelong Explainable Robot Learning in Nürnberg.
Dianzhao Li paricipated in the 26th IEEE International Conference on Intelligent Transportation Systems (ITSC) where he introduced his work on "Vision-based DRL Autonomous Driving Agent with Sim2Real Transfer".
Ostap Okhrin and Niklas Paulig paricipated in the Statistical Week 2023 at TU Dortmund, where Ostap Okhrin gave a talk on "Two-sample Testing in Reinforcement Learning" and Niklas Paulig presented his work on "Robust Path Following on Rivers Using Bootstrapped Reinforcement Learning".
Chair member Ankit Chaudhari gave an interview at the workshop on 'Integrated Engineering for Future Mobility' in Delhi. The workshop aimed at fostering collaboration and generating innovative research ideas related to urban mobility by using a "Design Thinking" approach.
Martin Waltz paper "Spatial-temporal recurrent reinforcement learning for autonomous ships" has been published in Neural Networks
Ostap Okhrin and Dianzhao Li participated in the 2. Sächsische KI-Kongress des Freistaates Sachsen. This prestigious event brought together more than 250 distinguished guests representing business, science, society, and politics, creating a dynamic platform for discussions on the latest developments and trends in the field of AI. Livestream
The chair recently conducted drone-based traffic data collection effort on the A50 highway in Milan, Italy. With a co-ordinated fleet of 7 Drones flying in a line, traffic was captured over a 1000m road section for 130 minutes. Our team member Ankit Chaudhari oversaw the data collection on site.
Ankit Chaudhari recently participated in a "design thinking based" workshop on 'Integrated Engineering for Future Mobility' organized by the German Centre for Research and Innovation (DWIH) in New Delhi, India.
Fabian Hart's paper "Vessel-following model for inland waterways based on deep reinforcement learning" is accepted for publication in the journal of Ocean Engineering.
Navigating maritime traffic poses multifaceted challenges, including the intricate interplay of environmental factors like wind, waves, and currents, alongside the dynamic presence of other vessels with varying behaviors. These complexities demand precise control strategies that can adapt to real-world conditions, considering factors such as shallow water impacts and spatial restrictions imposed by waterway geometry. Within our research, we aim to address these challenges through modularized frameworks and advanced reinforcement learning techniques, facilitating realistic path planning and robust control of autonomous vessels in diverse maritime environments.
Addressing urban congestion and outdated infrastructure while improving traffic flow and reducing pollution is crucial. Implementing advanced traffic management systems equipped with AI algorithms can optimize flow and reduce congestion through dynamic route planning. We do our best to develop those AI algorithms. Among other tasks, to overcome the challenge of transferring trained autonomous vehicle agents from simulation to reality, we focus on separating agents into perception and control modules, enabling smoother integration into real-world environments. Similarly, by incorporating real-world driving datasets into training reinforcement learning agents, we aim to enhance their behavior and generalization capabilities, ultimately improving autonomous driving systems' performance on actual roads.
Urban air mobility presents unique challenges in managing eVTOL vehicles around vertiports, including congestion, safety, and unpredictable factors like weather and passenger demand. Among others, in our project, we address these complexities by proposing self-organized arrival systems using deep reinforcement learning. Treating each aircraft as an individual agent, we develop a shared policy for decentralized actions based on local information.
The continual development of new algorithms and models is imperative to tackle practical challenges with reinforcement learning. Even established methods like Q-Learning encounter limitations, particularly regarding the maximization operator in target computation. Inherent in the Bellman optimality equation, this operator often inflates Q-values for state-action pairs, impacting subsequent updates. As part of our broader theoretical endeavors in reinforcement learning, we are currently developing a novel suite of estimators tailored to address the max-mu problem, seamlessly integrating them within the framework of deep neural network-based function approximations.
This suite implements several model-free off-policy deep reinforcement learning algorithms for discrete and continuous action spaces in PyTorch.
Fully operational, 2D, Javascript-based web simulator implementing the "Mixed Traffic flow Model" (MTM). This simulation is intended to demonstrate fully two-dimensional but directed traffic flow and visulalize 2D flow models.
This package includes the training and evaluation code under ROS platform for Sim2Real Transfer with Duckiebot for multiple autonomous driving behaviors.
Waltz, M., Okhrin, O., Hart, F. (2024). Knowledge-Based Systems (To Appear)
Waltz, M., Okhrin, O. (2024). Artificial Intelligence 336, 104204
Waltz, M., Okhrin, O., Schultz, M. (2024). Transportation Research Part C: Emerging Technologies, Volume 167, 104806
Hart, F., Okhrin, O., Treiber, M. (2024). Transportation Research Part C: Emerging Technologies, Volume 159, 104486
Year | Name | Degree | Thesis Title |
---|---|---|---|
2023 | Peilin Wang | Master (D.Li) | Autonomous Navigation with Deep Reinforcement Learning in Carla |
2023 | Yuhua Zhu | Master (D.Li) | Autonomous Driving with Deep Reinforcement Learning |
2021 | Paul Ziebarth | Diplom (F.Hart) | Entwicklung eines Reinforcement Learning Agenten zur Realisierung eines Schifffolgemodells |
Ostap Okhrin is the professor for Statistics and Econometrics at the Department of Transportation at the TU Dresden. His expertise lies in mathematical statistics and data science with applications in transportation and economics.
Martin Treiber is a senior expert in traffic flow models including human and automated driving, bicycle, and pedestrian traffic. He also works in traffic data analysis and simulation (traffic-simulation.de, mtreiber.de/mixedTraffic).
Niklas Paulig is a RL Group research associate, with his main field of research being the modeling of autonomous inland vessel traffic based on reinforcement learning methods, and HPC implementations of currently in use algorithms.
Martin Waltz conducted his studies in Industrial Engineering and is now a research associate, with his main research focus being (Deep) Reinforcement Learning.
Ankit Chaudhari is currently working on "Enhancing Traffic-Flow Understanding by Two-Dimensional Microscopic Models". His research interests are traffic flow modelling, traffic simulation, mixed traffic flow, machine learning and reinforcement learning.
Dianzhao Li is a research assistant at RL-Dresden, focusing on the area of trajectory planning for autonomously driving vehicles with reinforcement learning algorithms. He now mixes the human driving datasets with RL in simulated environments to achieve better performance for the vehicles.
Paul Auerbach is a research associate at the Barkhausen Institut and collaborates with RL-Dresden on the simulation and solving of traffic scenarios with the help of reinforcement learning. He aims to transfer the learned RL models to real world model cars.
Gong Chen is a research associate within RL-Dresden, concentrating on applying reinforcement learning to simulate shipping traffic under shallow water conditions.
Saad Sagheer is a Civil Engineer who is currently working as a research associate and collabrates with BAW (Bundesanstalt für Wasserbau). His main field of research is to conduct micro-traffic simulation of inland vessels in Rhine especially Middle Rhine with different water level conditions related to climate change.