Daniel Biermann

• Biermann, Daniel; Goodwin, Morten & Granmo, Ole-Christoffer (2023). Transfer Learning Through Knowledge-Infused Representations with Contextual Experts. In Maglogiannis, Ilias; Iliadis, L.; MacIntyre, J. & Dominguez, Manuel (Ed.), Artificial Intelligence Applications and Innovations. Springer. ISSN 978-3-031-34106-9. doi: 10.1007/978-3-031-34107-6_45.
• Aaby, Pernille; Biermann, Daniel; Yazidi, Anis; Borges Moreno e Mello, Gustavo & Palumbo, Fabrizio (2023). Exploring Multilingual Word Embedding Alignments in BERT Models: A Case Study of English and Norwegian. In Bramer, Max & Stahl, Frederic (Ed.), Artificial Intelligence XL: 43rd SGAI International Conference on Artificial Intelligence, AI 2023. Springer. ISSN 978-3-031-47993-9. p. 47–58. doi: 10.1007/978-3-031-47994-6_4.
• Biermann, Daniel; Goodwin, Morten & Granmo, Ole-Christoffer (2023). Knowledge Infused Representations Through Combination of Expert Knowledge and Original Input. In Zouganeli, Evi; Yazidi, Anis; Borges Moreno e Mello, Gustavo & Lind, Pedro (Ed.), Nordic Artificial Intelligence Research and Development: 4th Symposium of the Norwegian AI Society, NAIS 2022, Oslo, Norway, May 31-June 1, 2022, Revised Selected Papers. Springer Nature. ISSN 9783031170294. p. 3–15. doi: 10.1007/978-3-031-17030-0_1. Show summary

  Sophisticated applications in natural language processing, such as conversational agents, often need to be able to generalize across a range of different tasks to generate natural-feeling language. In this paper, we introduce a model that aims to improve generalizability with regard to different tasks by combining the original input with the output of a task-specific expert. Through a combination mechanism, we create a new representation that has been enriched with the information given by the expert. These enriched representations then serve as input to a downstream model. We test three different combination mechanisms in two combination paradigms and evaluate the performance of the new enriched representation in a simple encoder-decoder model. We show that even very simple combination mechanisms are able to significantly improve performance of the downstream model. This means that the encoded expert information is transported through the new enriched input representation, leading to a beneficial impact on performance within the task domain. This opens the way for exciting future endeavors such as testing performance on different task domains and the combination of multiple experts.

• Lien, Henrik; Biermann, Daniel; Palumbo, Fabrizio & Goodwin, Morten (2022). An Exploration of Semi-supervised Text Classification. In Iliadis, Lazaros; Jayne, Chrisina; Tefas, Anastasios & Pimenidis, Elias (Ed.), Engineering Applications of Neural Networks - 23rd International Conference, EAAAI/EANN 2022. Springer Nature. ISSN 978-3-031-08222-1. p. 477–488. doi: 10.1007/978-3-031-08223-8_39. Full text in Research Archive Show summary

  Good performance in supervised text classification is usually obtained with the use of large amounts of labeled training data. However, obtaining labeled data is often expensive and time-consuming. To overcome these limitations, researchers have developed Semi-Supervised learning (SSL) algorithms exploiting the use of unlabeled data, which are generally easy and free to access. With SSL, unlabeled and labeled data are combined to outperform Supervised-Learning algorithms. However, setting up SSL neural networks for text classification is cumbersome and frequently based on a trial and error process. We show that the hyperparameter configuration significantly impacts SSL performance, and the learning rate is the most influential parameter. Additionally, increasing model size also improves SSL performance, particularly when less pre-processing data are available. Interestingly, as opposed to feed-forward models, recurrent models generally reach a performance threshold as pre-processing data size increases. This article expands the knowledge on hyperparameters and model size in relation to SSL application in text classification. This work supports the use of SSL work in future NLP projects by optimizing model design and potentially lowering training time, particularly if time-restricted.

View all works in Cristin