Martin Holen

• Holen, Martin; Ruud, Else-Line Malene; Warakagoda, Narada Dilp; Granmo, Ole-Christoffer; Engelstad, Paal E. & Knausgård, Kristian Muri (2022). Towards Using Reinforcement Learning for Autonomous Docking of Unmanned Surface Vehicles. In Iliadis, Lazaros; Jayne, Chrisina; Tefas, Anastasios & Pimenidis, Elias (Ed.), Engineering Applications of Neural Networks. EANN 2022. Communications in Computer and Information Science. Springer Nature. ISSN 978-3-031-08223-8. p. 461–474. doi: 10.1007/978-3-031-08223-8_38. Full text in Research Archive Show summary

  Providing full autonomy to Unmanned Surface Vehicles (USV) is a challenging goal to achieve. Autonomous docking is a subtask that is particularly difficult. The vessel has to distinguish between obstacles and the dock, and the obstacles can be either static or moving. This paper developed a simulator using Reinforcement Learning (RL) to approach the problem. We studied several scenarios for the task of docking a USV in a simulator environment. The scenarios were defined with different sensor inputs and start-stop procedures but a simple shared reward function. The results show that the system solved the task when the IMU (Inertial Measurement Unit) and GNSS (Global Navigation Satellite System) sensors were used to estimate the state, despite the simplicity of the reward function.

• Holen, Martin; Knausgård, Kristian Muri & Goodwin, Morten (2022). Development of a Simulator for Prototyping Reinforcement Learning-Based Autonomous Cars. Informatics. ISSN 2227-9709. 9(2). doi: 10.3390/informatics9020033. Full text in Research Archive Show summary

  Autonomous driving is a research field that has received attention in recent years, with increasing applications of reinforcement learning (RL) algorithms. It is impractical to train an autonomous vehicle thoroughly in the physical space, i.e., the so-called ’real world’; therefore, simulators are used in almost all training of autonomous driving algorithms. There are numerous autonomous driving simulators, very few of which are specifically targeted at RL. RL-based cars are challenging due to the variety of reward functions available. There is a lack of simulators addressing many central RL research tasks within autonomous driving, such as scene understanding, localization and mapping, planning and driving policies, and control, which have diverse requirements and goals. It is, therefore, challenging to prototype new RL projects with different simulators, especially when there is a need to examine several reward functions at once. This paper introduces a modified simulator based on the Udacity simulator, made for autonomous cars using RL. It creates reward functions, along with sensors to create a baseline implementation for RL-based vehicles. The modified simulator also resets the vehicle when it gets stuck or is in a non-terminating loop, making it more reliable. Overall, the paper seeks to make the prototyping of new systems simple, with the testing of different RL-based systems.

• Holen, Martin; Knausgård, Kristian Muri & Goodwin, Morten (2022). An Evaluation of Autonomous Car Simulators and Their Applicability for Supervised and Reinforcement Learning. In Sanfilippo, Filippo; Granmo, Ole-Christoffer; Yildirim Yayilgan, Sule & Bajwa, Imran Sarwar (Ed.), Intelligent Technologies and Applications: 4th International Conference, INTAP 2021, Grimstad, Norway, October 11–13, 2021, Revised Selected Papers. Springer. ISSN 978-3-031-10524-1. p. 367–379. doi: 10.1007/978-3-031-10525-8_29. Show summary

  Recent advancements in the field of Machine Learning have sprouted a renewed interest in the area of autonomous cars. Companies use different techniques to develop autonomous cars, including buying several vehicles to develop Advanced Driver Assistance Systems (ADAS), while others use car simulators. Simulators for autonomous cars include a wide variety of different sensors. Some simulators come free or even open-source, while others require to pay for the simulator itself, server time, or each sensor. The quality and focus of each of these vary, with some having LIDAR scanned roads for highly realistic roads, while others have entirely natural and realistic vehicle dynamics.

• Holen, Martin; Saha, Rupsa; Goodwin, Morten; Omlin, Christian Walter Peter & Sandsmark, Knut Eivind (2020). Road Detection for Reinforcement Learning Based Autonomous Car, ICISS 2020: Proceedings of the 2020 The 3rd International Conference on Information Science and System. ACM Publications. ISSN 978-1-4503-7725-6. p. 67–71. doi: 10.1145/3388176.3388199. Show summary

  Human mistakes in traffic often have terrible consequences. The long-awaited introduction of self-driving vehicles may solve many of the problems with traffic, but much research is still needed before cars are fully autonomous. In this paper, we propose a new Road Detection algorithm using online supervised learning based on a Neural Network architecture. This algorithm is designed to support a Reinforcement Learning algorithm (for example, the standard Proximal Policy Optimization or PPO) by detecting when the car is in an adverse condition. Specifically, the PPO gets a penalty whenever the virtual automobile gets stuck or drives off the road with any of its four wheels. Initial experiments show significantly improved results for PPO when using our Road Detection algorithm, as compared to not using any form of Road Detection. In fact, without this detection algorithm, the vehicle often gets into non-terminating loops (for example, driving into the dividers, getting stuck, or driving into a pit).

View all works in Cristin