Collective efficient deep learning and networked control for multiple collaborative robot systems (DEEPCOBOT)

• Schlanbusch, Siri Marte & Zhou, Jing (2024). Adaptive predictor-based control for a helicopter system with input delays: Design and experiments. Journal of Automation and Intelligence (JAI). ISSN 2949-8554. 3(1), p. 50–56. doi: 10.1016/j.jai.2024.02.001.
• Schlanbusch, Siri Marte & Zhou, Jing (2023). Adaptive quantized control of uncertain nonlinear rigid body systems. Systems & control letters (Print). ISSN 0167-6911. 175. doi: 10.1016/j.sysconle.2023.105513. Full text in Research Archive Show summary

  This paper investigates the attitude tracking control problem for uncertain nonlinear rigid body systems, where both inputs and states are quantized. It is common in networked control systems that sensor and control signals are quantized before they are transmitted via a communication network. An adaptive backstepping control algorithm is developed for a class of uncertain multiple-input multiple-output (MIMO) systems where a class of sector bounded quantizers is considered. It is shown that all the closed-loop signals are ensured uniformly bounded and tracking is achieved. Further, the tracking errors are shown to converge towards a compact set containing the origin and the set can be made small by the choice of the quantization parameters and the control parameters. For illustration of the proposed control scheme, experiments were conducted on a 2 degrees-of-freedom (DOF) helicopter system.

• Sveen, Emil Mühlbradt & Zhou, Jing (2023). Applications of Linear Adaptive Dynamic Programming (ADP) to a Non-linear Four-bar Mechanism. International Conference on Control, Mechatronics and Automation. ISSN 2837-5114. p. 177–182. doi: 10.1109/ICCMA59762.2023.10374822.
• Kumar, Ravi; Dale, Jørgen; Singh, Jayant & Zhou, Jing (2023). Design and Development of an Anthropomorphic Gripper for Service Robotics and Prosthetic Applications. International Conference on Control, Mechatronics and Automation. ISSN 2837-5114. p. 233–238. doi: 10.1109/ICCMA59762.2023.10374949.
• Nama, Ajay Nagendra; Saad, Leila Ben; Beferull-Lozano, Baltasar & Zhou, Jing (2023). Neighborhood Graph Filters Based Graph Convolutional Neural Networks for Multi-Agent Deep Reinforcement Learning. In IEEE, . (Eds.), IECON 2023- 49th Annual Conference of the IEEE Industrial Electronics Society. IEEE (Institute of Electrical and Electronics Engineers). ISSN 979-8-3503-3182-0. doi: 10.1109/IECON51785.2023.10311938. Show summary

  Multi-agent deep reinforcement learning (MADRL), where a group of agents inside multi-agent systems cooperate to achieve a common goal, has been shown useful in many applications such as collaborative robots, autonomous driving or video games involving teams. In this paper, we propose two multi-agent deep reinforcement learning (MADRL) frameworks for value function factorization built by using Graph Convolutional Neural Networks (GCNN) based on neighborhood graph filters (NGFs). These MADRL frameworks are based on the paradigm of centralized training with decentralized execution (CTDE). In this work, we show that the superior stability of the NGFs as compared to standard graph filters leads also to superior performance for the MADRL algorithms. In the first MADRL framework, the NGF-based GCNN is used to predict the local q-value at each of the agents, while in the other one, the NGF-based GCNN is used to mix the local q-values to generate a global Q-value. We have compared the performance of NGF-based GCNNs over state-of-the-art graph neural networks for value function factorization in the MADRL framework for the StarCraft II and Coalition Structure Generation problems. The results show that the proposed MARL frameworks outperform the existing state-of-art architectures.

• Landsverk, Ronny; Zhou, Jing & Hagen, Daniel (2023). Antiswing Control and Trajectory Planning for Offshore Cranes. Proceedings of the Annual Conference of the IEEE Industrial Electronics Society (IECON). ISSN 1553-572X. doi: 10.1109/IECON51785.2023.10312101. Show summary

  Safe handling of heavy payloads in an offshore environment requires careful crane maneuvering to avoid collision with obstacles and other equipment. The residual swing from a hanging payload can ultimately lead to danger and high cost failures if not properly dealt with. This paper investigates an openloop control method for eliminating the payload swing for hanging loads on offshore knuckle-boom cranes. A trajectory tracking method is designed for payload swing suppression for open-loop control and based on the iterative learning algorithm. It is shown that the proposed anti-swing control method guarantees asymptotic convergence of the swing, the angular velocity, and the angular acceleration of the payload using Lyapunov techniques. The simulation results show the superior performance of the proposed anti-swing control method.

• Singh, Jayant; Zhou, Jing & Beferull-Lozano, Baltasar (2023). Enhancing Multi-Agent Reinforcement Learning: Set Function Approximation and Dynamic Policy. Proceedings of the Annual Conference of the IEEE Industrial Electronics Society (IECON). ISSN 1553-572X. Show summary

  While Deep Learning based methods can solve complex problems by employing Neural Networks to act as powerful function approximators, they often suffer from inflexibility in terms of deployment beyond the training scenario and also include irrelevant data priors in the form of an ordered array of input values. This problem is quite evident in the field of MultiAgent Reinforcement Learning (MARL), where most research covers methods that are trained on a fixed number of agents, restricted by the fixed size of the input vector. In this paper, we argue that this is not a reasonable assumption, both in terms of the inflexible amount of environmental information and the restrictive nature of the structure of the information. We explore DeepSets and Set Transformers as two powerful set function approximators to address the problem of cardinality invariance and permutation invariance in the observation space of a reinforcement learning agent. We explain Set-Input Reinforcement Learning (SIRL) in detail and evaluate the performance of the DeepSets and Set Transformer methods through simulated experiments on a challenging multi-agent environment, that otherwise yields sub-optimal policies through traditional function approximation approaches. We demonstrate that both DeepSets and Set Transformer based encoders scale well to increasing the number of agents from training to evaluation.

• Schlanbusch, Siri Marte & Zhou, Jing (2023). Adaptive Control of an Uncertain 2-DOF Helicopter System with Input Delays. Proceedings of the Annual Conference of the IEEE Industrial Electronics Society (IECON). ISSN 1553-572X. Show summary

  In this paper, we consider a 2-degrees-of-freedom (DOF) helicopter system subject to input delays and uncertain system parameters. To address this challenge in control design, we develop an adaptive predictor-feedback control law. The control law is designed to compensate for a known delay while considering the system uncertainty. Stability of the closed-loop system is established, where tracking is achieved. We demonstrate the effectiveness of our proposed control approach through simulations of the helicopter system, where the input delays are compensated in the control-loop.

• Sveen, Emil Mühlbradt & Zhou, Jing (2023). Adaptive Learning Based Motor Control of an Unknown Robot Manipulator. Proceedings of the Annual Conference of the IEEE Industrial Electronics Society (IECON). ISSN 1553-572X. Show summary

  Adaptive dynamic programming, optimal control, robot manipulator, unknown dynamics, noise Abstract—This paper focuses on motor joint control using learning based Adaptive Dynamic Programming (ADP). The dynamics of a 2-link robot manipulator is presented. The system is regarded as a linear system and the optimal control is developed using linear quadratic regulator (LQR) and ADP methods. An off-line, off-policy controller iteration approach is used to find two optimal controllers, one for each motor joint of the robot. The optimal controller is developed to control the joint position without knowing the dynamic model. Simulation results show that two independent optimal linear control policies are learned simultaneously under a common exploration strategy, and the efficiency of ADP is verified.

• Saad, Leila Ben; Nama, Ajay Nagendra & Beferull-Lozano, Baltasar (2022). Neighborhood Graph Neural Networks under Random Perturbations and Quantization Errors , 2022 IEEE 23rd International Workshop on Signal Processing Advances in Wireless Communication (SPAWC). IEEE conference proceedings. ISSN 978-1-6654-9455-7. doi: 10.1109/SPAWC51304.2022.9834020. Show summary

  Graph convolutional neural networks (GCNNs) have emerged as a promising tool in the deep learning community to learn complex hidden relationships of data generated from non-Euclidean domains and represented as graphs. GCNNs are formed by a cascade of layers of graph filters, which replace the classical convolution operation in convolutional neural networks. These graph filters, when operated over real networks, can be subject to random perturbations due to link losses that can be caused by noise, interference and adversarial attacks. In addition, these graph filters are executed by finite-precision processors, which generate numerical quantization errors that may affect their performance. Despite the research works studying the effect of either graph perturbations or quantization in GCNNs, their robustness against both of these problems jointly is still not well investigated and understood. In this paper, we propose a quantized GCNN architecture based on neighborhood graph filters under random graph perturbations. We investigate the stability of such architecture to both random graph perturbations and quantization errors. We prove that the expected error due to quantization and random graph perturbations at the GCNN output is upper-bounded and we show how this bound can be controlled. Numerical experiments are conducted to corroborate our theoretical findings.

• Zhou, Jing & Schlanbusch, Siri Marte (2022). Adaptive Quantized Control of Offshore Underactuated Cranes with Uncertainty. IEEE International Conference on Control and Automation. ISSN 1948-3449. p. 297–302. doi: 10.1109/ICCA54724.2022.9831937. Full text in Research Archive Show summary

  The anti-swing control of offshore cranes presents much more challenges. Most existing controllers for offshore cranes are designed based on linearized dynamics and require the accurate values of the plant parameters. In this paper, an adaptive sliding mode control scheme is investigated for a nonlinear underactuated crane system with unmodeled dynamics. The proposed control method can ensure asymptotic stability and does not need linearization of the complicated nonlinear dynamic equations during controller design and stability analysis. To reduce the communication burden in a network, a uniform quantizer is introduced in the input communication channel. A quantized adaptive sliding mode control scheme is further developed for the underactuated cranes to compensate for the effects of input quantization and uncertain parameters. The proposed controller together with the quantizer ensures the asymptotic stability of the closed-loop system in the sense of signal boundedness and zero stabilization error. Numerical simulations are conducted to illustrate the effectiveness of proposed schemes.

• Singh, Jayant; Zhou, Jing; Beferull-Lozano, Baltasar & Tyapin, Ilya (2022). Learning Cooperative Multi-Agent Policies with Multi-Channel Reward Curriculum Based Q-Learning. Proceedings of the Annual Conference of the IEEE Industrial Electronics Society (IECON). ISSN 1553-572X. doi: 10.1109/IECON49645.2022.9969046.
• Schlanbusch, Siri Marte; Aamo, Ole Morten & Zhou, Jing (2022). Attitude Control of a 2-DOF Helicopter System with Input Quantization and Delay. Proceedings of the Annual Conference of the IEEE Industrial Electronics Society (IECON). ISSN 1553-572X. doi: 10.1109/IECON49645.2022.9968994. Full text in Research Archive Show summary

  In this paper the attitude tracking control problem of a 2 degrees-of-freedom helicopter system with network induced constraints is studied. A predictor feedback control law is developed to compensate a known delay in the communication, where the inputs are quantized before transmitted over the network. Stability of the closed-loop system is established, where tracking is achieved with bounded tracking errors due to the network issues. The developed predictor-based controller is experimentally tested on the helicopter system, where we demonstrate that tracking is achieved in presence of both input delay and quantization.

• Schlanbusch, Siri Marte; Zhou, Jing & Schlanbusch, Rune (2021). Adaptive Backstepping Attitude Control of a Rigid Body with State Quantization, Proceedings of 60th IEEE Conference on Decision and Control. IEEE conference proceedings. ISSN 978-1-7281-1398-2. p. 372–377. doi: 10.1109/CDC45484.2021.9683579. Full text in Research Archive Show summary

  In this paper, the attitude tracking control problem of a rigid body is investigated where the states are quantized. An adaptive backstepping based control scheme is developed and a new approach to stability analysis is developed by constructing a new compensation scheme for the effects of the vector state quantization. It is shown that all closed-loop signals are ensured uniformly bounded and the tracking errors converge to a compact set containing the origin. Experiments on a 2 degrees-of-freedom helicopter system illustrate the proposed control scheme.

View all works in Cristin

• Zhou, Jing; RIPOLL, ÁLVARO ÚBEDA & Jiang, Zhiyu (2023). Active control of the blade leading edge erosion for an onshore wind turbine during rainfall events. Universitetet i Agder.
• Zhou, Jing; Kumar, Ravi; Dale, Jørgen & Singh, Jayant (2023). Design and Development of a low-cost Anthropomorphic Gripper for Service Robotics and Prosthetic Applications. Universitetet i Agder.
• Schlanbusch, Siri Marte & Zhou, Jing (2023). Adaptive Control of Systems with Quantization and Time Delays. University of Agder. ISSN 978-82-8427-128-6. Full text in Research Archive Show summary

  This thesis addresses problems relating to tracking control of nonlinear systems in the presence of quantization and time delays. Motivated by the importance in areas such as networked control systems (NCSs) and digital systems, where the use of a communication network in NCS introduces several constraints to the control system, such as the occurrence of quantization and time delays. Quantization and time delays are of both practical and theoretical importance, and the study of systems where these issues arises is thus of great importance. If the system also has parameters that vary or are uncertain, this will make the control problem more complicated. Adaptive control is one tool to handle such system uncertainty. In this thesis, adaptive backstepping control schemes are proposed to handle uncertainties in the system, and to reduce the effects of quantization. Different control problems are considered where quantization is introduced in the control loop, either at the input, the state or both the input and the state. The quantization introduces difficulties in the controller design and stability analysis due to the limited information and nonlinear characteristics, such as discontinuous phenomena. In the thesis, it is analytically shown how the choice of quantization level affects the tracking performance, and how the stability of the closed-loop system equilibrium can be achieved by choosing proper design parameters. In addition, a predictor feedback control scheme is proposed to compensate for a time delay in the system, where the inputs are quantized at the same time. Experiments on a 2-degrees of freedom (DOF) helicopter system demonstrate the different developed control schemes.

• Sveen, Emil Mühlbradt; Hansen, Rasmus Als & Folgerø, Roy Werner (2023). Robotic Picking in a Cluttered Environment Using Computer Vision. Universitetet i Agder.
• Sveen, Emil Mühlbradt; Sand, Kristoffer; Solberg, Trygve Andre Olsøy & Andersen, Per-Arne (2023). Utilizing Reinforcement Learning and Computer Vision in a Pick-and-Place Operation for Sorting Objects in Motion. Universitetet i Agder.
• Zhou, Jing & Nilsen, Espen (2022). Robot localization. Universitetet i Agder.

View all works in Cristin