Rebekka Olsson Omslandseter

• Omslandseter, Rebekka Olsson; Lei, Jiao & Oommen, John (2023). Pioneering approaches for enhancing the speed of hierarchical LA by ordering the actions. Information Sciences. ISSN 0020-0255. 647, p. 1–17. doi: 10.1016/j.ins.2023.119487. Full text in Research Archive Show summary

  Fixed Structure Stocastic Automata (FSSA), Variable Structure Learning Automata (VSSA), and their discretized versions have been significantly improved by utilizing inexpensive estimates of the actions' reward probabilities. These represent the fastest LA to date. However, the concept of ordering the actions has never been used within the field, and the reason for this is that there is no way to order the actions a priori. The recently-introduced Hierarchical Discrete Pursuit Automaton (HDPA) has an interesting concept of placing two-action LA along the nodes of a tree, implying that the leaves signify an underlying ordering. In this paper, we show that if estimates are available (as in the case of estimator algorithms), these can be used to place the actions at the leaf level to further enhance the convergence capabilities of the overall ensemble of the two-action LAs. This paper contains the design of this HDPA, the proof of this assertion, and the simulation results on benchmark Environments. Based on the results, we believe that it is the fastest and most accurate LA to date. Our position is that it will be very hard to beat its performance, since it has been incorporated all the salient features of the entire field of LA.

• Oommen, John; Omslandseter, Rebekka Olsson & Lei, Jiao (2023). The object migration automata: its field, scope, applications, and future research challenges. Pattern Analysis and Applications. ISSN 1433-7541. 26, p. 917–928. doi: 10.1007/s10044-023-01163-x. Show summary

  Partitioning, in and of itself, is an NP-hard problem. Prior to the Artificial Intelligence (AI)-based solutions, it was solved in the 1970s by optimization-based strategies. However, AI-based solutions appeared in the 1980s in a pioneering way, by using a Learning Automaton (LA)-motivated strategy known as the so-called Object Migrating Automaton (OMA). Although the OMA and its derivatives have been used in numerous applications since then, the basic kernel has remained the same. Because the number of possible partitions in a partitioning problem can be combinatorially exponential and the underlying tasks are NP-hard, the most advanced OMA algorithms could, until recently, only solve issues involving equally sized groups. Due to our recent innovations cited in the body of this paper, the enhanced OMA now also handles non-equally sized groups. Earlier, we had presented in Omslandseter (Pattern Anal Appl, 2023), a comprehensive survey of the state-of-the-art enhancements of the best-known OMA. We believe that these results will be the benchmark for a few decades and that it will be very hard to beat these results. This is a companion paper, intended to augment the contents of Omslandseter (Pattern Anal Appl, 2023). In this paper, we first discuss the OMA’s prior applications, its historical and current innovations, and the OMA-based algorithms’ relevance to societal needs. We also provide well-specified guidelines for future researchers so that they can use them for unresolved tasks, and also develop further advancements.

• Oommen, John; Omslandseter, Rebekka Olsson & Lei, Jiao (2023). Learning automata-based partitioning algorithms for stochastic grouping problems with non-equal partition sizes. Pattern Analysis and Applications. ISSN 1433-7541. 26, p. 751–772. doi: 10.1007/s10044-023-01131-5. Show summary

  It is very fascinating that the principles of Artificial Intelligence, that were proposed many decades ago, have remained to be the foundations of many of the modern-day techniques, and a brief summary of these principles is given in the paper itself. While this truism is valid for problem solving and game playing, in the context of this paper, we emphasize that it is also pertinent for the so-called partitioning problems. In this paper, we consider the general partitioning problem that has been studied for decades. Unlike the heuristic and search strategies, our attention focuses on learning automata (LA) and reinforcement learning methods, and their powerful ability to solve problems in stochastic environments. These render the latter tools to be applicable to various complex tasks. As we shall explain, LA have been employed for partitioning, and in particular, the paradigm of Object Migration Automata (OMA) has offered state-of-the-art adaptive methods for solving grouping and partitioning problems. However, because the number of possible partitions is combinatorially exponential, and of the NP-hardness of the underlying tasks, the existing state-of-the-art OMA algorithms can only solve problems of equal-sized groups. To resolve this, in this paper, we propose two new OMA variants that can solve both equally and unequally sized partitioning problems. The key here is to provide additional information to the algorithms, and in doing so, to expand the solution space. The first variant, the Greatest Common Divisor OMA (GCD-OMA), relaxes the constraint of having equally sized groups by mapping the actions onto a larger state space, and thus permitting it to solve problems where the group sizes have a GCD that is greater than unity. Subsequently, we propose a second variant named the Partition Size Required OMA (PSR-OMA) to make the algorithm more versatile. The PSR-OMA can handle groups of arbitrary sizes when the group sizes are known a priori. We demonstrate the proposed algorithms’ strength in solving stochastic grouping problems through extensive simulations, even with high noise levels. The GCD-OMA and the PSR-OMA represent the current state-of-the-art when it concerns resolving the extremely complex problem of partitioning and can be used in any of the numerous applications in which the OMA itself has been utilized .

• Omslandseter, Rebekka Olsson; Jiao, Lei; Zhang, Xuan; Yazidi, Anis & Oommen, John (2022). The Hierarchical Discrete Pursuit Learning Automaton: A Novel Scheme with Fast Convergence and Epsilon-Optimality. IEEE Transactions on Neural Networks and Learning Systems. ISSN 2162-237X. 35(6), p. 8278–8292. doi: 10.1109/TNNLS.2022.3226538. Full text in Research Archive
• Omslandseter, Rebekka Olsson; Jiao, Lei; Zhang, Xuan; Yazidi, Anis & Oommen, John (2022). The Hierarchical Discrete Learning Automaton Suitable for Environments with Many Actions and High Accuracy Requirements. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 13151, p. 507–518. doi: 10.1007/978-3-030-97546-3_41. Full text in Research Archive Show summary

  Since its early beginning, the paradigm of Learning Automata (LA), has attracted much interest. Over the last decades, new concepts and various improvements have been introduced to increase the LA’s speed and accuracy, including employing probability updating functions, discretizing the probability space, and implementing the “Pursuit” concept. The concept of incorporating “structure” into the ordering of the LA’s actions is one of the latest advancements to the field, leading to the ϵ-optimal Hierarchical Continuous Pursuit LA (HCPA) that has superior performance to other LA variants when the number of actions is large. Although the previously proposed HCPA is powerful, its speed has a handicap when the required action probability of an action is approaching unity. The reason for this slow convergence is that the learning parameter operates in a multiplicative manner within the probability space, making the increment of the action probability smaller as its probability becomes close to unity. Therefore, we propose the novel Hierarchical Discrete Learning Automata (HDPA) in this paper, which does not possess the same impediment as the HCPA. The proposed machine infuse the principle of discretization into the action probability vector’s updating functionality, where this type of updating is invoked recursively at every depth within a hierarchical tree structure and we pursue the best estimated action in all iterations through utilization of the Estimator phenomenon. The proposed machine is ϵ-optimal, and our experimental results demonstrate that the number of iterations required before convergence is significantly reduced for the HDPA, when compared with the HCPA.

• Omslandseter, Rebekka Olsson; Jiao, Lei & Oommen, John (2022). Enhancing the Speed of Hierarchical Learning Automata by Ordering the Actions - A Pioneering Approach. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 13728, p. 775–788. doi: 10.1007/978-3-031-22695-3_54. Show summary

  For the past six decades, the operation of Learning Automata (LA) has involved states and action probabilities. These have been central to “remembering” the quality of the actions chosen during the learning. The latest enhancements have also incorporated estimates of the actions’ reward probabilities. However, a phenomenon that has never been used to-date is that of considering how these actions themselves, can be ordered. Ordering the actions in traditional LA is rather meaningless unless one resorts to invoking the theory of Random Races [1]. However, we show that such an ordering makes sense if the automata operate hierarchically, within a tree, with the actions being placed at the leaves. In this paper, we shall show that when the LA are arranged “in a tree formation”, and when the learning is achieved within such a tree, the hierarchical LA has a superior performance if the actions located at the leaves of the tree are arranged suitably. While this concept can be incorporated in any hierarchical LA, we demonstrate its power for the most recent machine, i.e., the Hierarchical Discretized Pursuit Automaton (HDPA). These strategies can also be included in the Hierarchical Continuous Pursuit Automaton (HCPA), and to both of these which utilize traditional Maximum Likelihood (ML) or Bayesian estimates [2]. The experimental results presented here are very impressive, and so, if we consider the chronology of LA from FSSA through VSSA, the Estimator schemes, and the recent hierarchical LA, our modest claim is that the inclusion of the ADE represents the state-of-the-art which is not easily surpassed.

• Omslandseter, Rebekka Olsson; Lei, Jiao; Liu, Yuanwei & Oommen, John (2022). User grouping and power allocation in NOMA systems: a novel semi-supervised reinforcement learning-based solution. Pattern Analysis and Applications. ISSN 1433-7541. doi: 10.1007/s10044-022-01091-2. Full text in Research Archive Show summary

  In this paper, we present a pioneering solution to the problem of user grouping and power allocation in non-orthogonal multiple access (NOMA) systems. The problem is highly pertinent because NOMA is a well-recognized technique for future mobile radio systems. The salient and difficult issues associated with NOMA systems involve the task of grouping users together into the pre-specified time slots, which are augmented with the question of determining how much power should be allocated to the respective users. This problem is, in and of itself, NP-hard. Our solution is the first reported reinforcement learning (RL)-based solution, which attempts to resolve parts of this issue. In particular, we invoke the object migration automaton (OMA) and one of its variants to resolve the grouping in NOMA systems. Furthermore, unlike the solutions reported in the literature, we do not assume prior knowledge of the channels’ distributions, nor of their coefficients, to achieve the grouping/partitioning. Thereafter, we use the consequent groupings to heuristically infer the power allocation. The simulation results that we have obtained confirm that our learning scheme can follow the dynamics of the channel coefficients efficiently, and that the solution is able to resolve the issue dynamically.

• Omslandseter, Rebekka Olsson; Jiao, Lei & Oommen, John (2021). Object Migration Automata for Non-equal Partitioning Problems with Known Partition Sizes. IFIP Advances in Information and Communication Technology. ISSN 1868-4238. doi: 10.1007/978-3-030-79150-6_11. Full text in Research Archive
• Omslandseter, Rebekka Olsson; Jiao, Lei & Oommen, John (2021). A Learning-Automata Based Solution for Non-equal Partitioning: Partitions with Common GCD Sizes. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 12799, p. 227–239. doi: 10.1007/978-3-030-79463-7_19. Full text in Research Archive
• Omslandseter, Rebekka Olsson; Lei, Jiao; Liu, Yuanwei & Oommen, John (2020). User Grouping and Power Allocation in NOMA Systems: A Reinforcement Learning-Based Solution, Trends in Artificial Intelligence Theory and Applications. Artificial Intelligence Practices.. Springer Nature. ISSN 978-3-030-55789-8. p. 299–311. doi: 10.1007/978-3-030-55789-8_27. Full text in Research Archive
• Omslandseter, Rebekka Olsson; Lei, Jiao & Haglund, Magne Arild (2019). Field Measurements and Parameter Calibrations of Propagation Model for Digital Audio Broadcasting in Norway. IEEE Vehicular Technology Conference (VTC). ISSN 1090-3038. doi: 10.1109/VTCFall.2018.8690746. Full text in Research Archive

View all works in Cristin

• Omslandseter, Rebekka Olsson (2023). On the Theory and Applications of Hierarchical Learning Automata and Object Migration Automata. Universitetet i Agder. ISSN 978-82-8427-161-3. Full text in Research Archive Show summary

  The paradigm of Artificial Intelligence (AI) and the sub-group of Machine Learning (ML) have attracted exponential interest in our society in recent years. The domain of ML contains numerous methods, and it is desirable (and in one sense, mandatory) that these methods are applicable and valuable to real-life challenges. Learning Automata (LA) is an intriguing and classical direction within ML. In LA, non-human agents can find optimal solutions to various problems through the concept of learning. The LA instances learn through Agent-Environment interactions, where advantageous behavior is rewarded, and disadvantageous behavior is penalized. Consequently, the agent eventually, and hopefully, learns the optimal action from a set of actions. LA has served as a foundation for Reinforcement Learning (RL), and the field of LA has been studied for decades. However, many improvements can still be made to render these algorithms to be even more convenient and effective. In this dissertation, we record our research contributions to the design and applications within the field of LA. Our research includes novel improvements to the domain of hierarchical LA, major advancements to the family of Object Migration Automata (OMA) algorithms, and a novel application of LA, where it was utilized to solve challenges in a mobile radio communication system. More specifically, we introduced the novel Hierarchical Discrete Pursuit Automaton (HDPA), which significantly improved the state of the art in terms of effectiveness for problems with high accuracy requirements, and we mathematically proved its ϵ-optimality. In addition, we proposed the Action Distribution Enhanced (ADE) approach to hierarchical LA schemes which significantly reduced the number of iterations required before the machines reached convergence. The existing schemes in the OMA paradigm, which are able to solve partitioning problems, could only solve problems with equally-sized partitions. Therefore, we proposed two novel methods that could handle unequally-sized partitions. In addition, we rigorously summarized the OMA domain, outlined its potential benefits to society, and listed further development cases for future researchers in the field. With regard to applications, we proposed an OMA-based approach to the grouping and power allocation in Non-orthogonal Multiple Access (NOMA) systems, demonstrating the applicability of the OMA and its advantage in solving fairly complicated stochastic problems. The details of these contributions and their published scientific impacts will be summarized in this dissertation, before we present some of the research contributions in their entirety.

View all works in Cristin