Explainability in AI and RL (known as XAI and XRL) becomes increasingly important. In our paper we investigate several possibilities to replace complex “black box” deep reinforcement learning (DRL) models by intrinsically interpretable decision trees (DTs) which require orders of magnitudes fewer parameters. A highlight of our paper is that we find on seven classic control RL problems that the DTs achieve similar reward as the DRL models, sometimes even surpassing the reward of the DRL models. The key to this success is an iterative sampling method that we have developed. In our work, we present and compare three different methods of collecting samples to train DTs from DRL agents. We test our approaches on seven problems including all classic control environments from Open AI Gym), LunarLander, and the CartPole-SwingUp challenge. Our iterative approach combining exploration of DTs and DRL agent’s predictions, in particular, is able to generate shallow, understandable, oblique DTs that solve the challenges and even outperform the DRL agents they were trained from. Additionally we demonstrate how, given their simpler structure and fewer parameters, DTs allow for inspection and insights, and offer higher degrees of explainability. The investigation carried out by Niklas Fabig constitutes very interesting and brand-new research in this field, which has now led to winning the 3rd place in the Steinmüller Engineering Award 2023. His supervisor Wolfgang Konen was deeply impressed by the solid, comprehensive and innovative work done by Niklas Fabig and congratulates him warmly. It should be noted, that the master thesis was conducted in the Corona years 2021 - 2022 and so the supervision had to be fully online. Nevertheless, the result of the work and the motivation of Niklas Fabig was by no means less than if the supervision had taken place in presence. View from Certosa di Pontignano As previously announced, last week I had the pleasure to present our joint work with our partners from Ruhr-University Bochum on explainable reinforcement learning at the 8th Annual Conference on machine Learning, Optimization and Data science (LOD). The presentation sparked interesting questions and lead to inspiring discussions in the enchanting ambiance of the medieval monastery in Tuscany. Group picture of the participants
To readers interested in explainable AI and understandable reinforcement learning in particular, we recommend to take a look at our open-access article.
We contributed with an extended abstract and our scientific poster “Finding the Relevant Samples for Decision Trees in Reinforcement Learning” presented during Tuesday’s poster session. The opportunity for fruitful discussions and interactions with fellow PhD students from the Dataninja project was much appreciated!
or to using AI methods in this area, it should be thoroughly investigated and evaluated via simulations, whether AI can learn (e.g.´, by reinforcment learning, RL) to power up and shut down a nuclear reactor and how well such an approach meets the safety requirements. This was exactly the task of Niklas Fabig's master thesis which he conducted under the supervision of Prof. Dr. Wolfgang Konen and PhD-candidate Raphael Engelhardt as part of our (RL)^3-project as part of https://dataninja.nrw/. The works uses as a starting point a Java-based NPP simulation tool from Prof. Dr. Benjamin Weyers, University Trier (screenshot example in image). Niklas Fabig constructed first a Java-Python bridge and then conducted over 2000 RL simulation experiments under various settings. He could show that RL algorithms can learn the power-up procedure yielding high returns, but much more research is needed to reliably meet the safety requirements.
As the conference was held in conjunction with the Advanced Course & Symposium on Artificial Intelligence & Neuroscience (ACAIN), we could profit from a very stimulating interdisciplinary environment with talks, tutorials, and posters covering topics reaching from the biology of neuronal development to implementation details of different deep learning frameworks.
We are looking forward to LOD 2023!
CIOP News
We are delighted to announce that our article “Iterative Oblique Decision Trees Deliver Explainable RL Models” was accepted and is now part of MDPI Algorithms special issue “Advancements in Reinforcement Learning Algorithms”.
The second edition of the Dataninja Spring-School was held from 8th to 10th of May 2023 in Bielefeld and as a hybrid event. We had the honor and pleasure to attend talks and tutorials from renowned researchers and aspiring young scientists.
The operation of nuclear power plants (NPPs) is one of the most safety-critical tasks in industry. Pri
The second edition of the annual Dataninja-Retreat took place this September in Tecklenburg. The different Dataninja projects presented their proceedings, we had the pleasure of attending a lecture by Prof. Xiaoyi Jiang, and fresh PhD graduates of the KI-Starter-project kindly shared experiences and some tips from their recently concluded PhD-journey. Last but not least it was of course a very pleasant and rare occasion for seeing each other offline, exchanging ideas, experiences, struggles, and successes.