Adam D. Tupper

Selected publications

1. IVCNZ

   Evaluating Learned State Representations for Atari

   Tupper, Adam, and Neshatian, Kourosh

   In 2020 35th International Conference on Image and Vision Computing New Zealand (IVCNZ) 2020

   Abs PDF Slides

   Deep reinforcement learning, the combination of deep learning and reinforcement learning, has enabled the training of agents that can solve complex tasks from visual inputs. However, these methods often require prohibitive amounts of computation to obtain successful results. To improve learning efficiency, there has been a renewed focus on separating state representation and policy learning. In this paper, we investigate the quality of state representations learned by different types of autoencoders, a popular class of neural networks used for representation learning. We assess not only the quality of the representations learned by undercomplete, variational, and disentangled variational autoencoders, but also how the quality of the learned representations is affected by changes in representation size. To accomplish this, we also present a new method for evaluating learned state representations for Atari games using the Atari Annotated RAM Interface. Our findings highlight differences in the quality of state representations learned by different types of autoencoders and their robustness to reduction in representation size. Our results also demonstrate the advantage of using more sophisticated evaluation methods over assessing reconstruction quality.

2. GECCO

   Evolving Neural Network Agents to Play Atari Games with Compact State Representations

   Tupper, Adam, and Neshatian, Kourosh

   In Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion 2020

   Abs PDF Supp Code Poster

   Recent success in solving hard reinforcement learning problems can be partly credited to the use of deep neural networks, which can extract high-level features and learn compact state representations from high-dimensional inputs, such as images. However, the large networks required to learn both state representation and policy using this approach limit the effectiveness and benefits of neuroevolution methods that have proven effective at solving simpler problems in the past. One potential solution to this problem is to separate state representation and policy learning and only apply neuroevolution to the latter. We extend research following this approach by evolving small policy networks for Atari games using NEAT, that learn from compact state representations provided by the recently released Atari Annotated RAM Interface (Atari ARI). Our results show that it is possible to evolve agents that exceed expert human performance using these compact state representations, and that, for some games, successful policy networks can be evolved that contain only a few or even no hidden nodes.