Over the past few weeks I developed a small roll-a-ball project and took some close looks into its compoments. I tested tools that could extract the projects assets or scripts. I managed to fetch the ingame score of the project from the Windows registry. Now its time to explain why – its time to introduce this topics‘ main objective.
In this (anniversary edition) blog, I present the big picture of „Reinforcement Learning in Unity“.
Anniversary
Reinforcement Learning is about AI learning to play games and by now, I have an idea on how it works.
52 weeks ago I wrote my first blog entry about the book Reinforcement Learning by Richard Sutton. The basic theory as explained in the book hold true to this day, but its method were outdated. So I moved on into practice, and analyzed the MarI/O project by SethBling. To extend my knowledge about algorithms I moved on to TensorFlow and tested the most modern approaches to Reinforcement Learning by going through a few examples. Then, I started my own project, which applied the concept of MarI/O onto Super Metroid. I finished the project after many weeks of development – and spent more weeks running tests. I decided to leave retro games behind and move on to modern Unity games. And that is where I stand now, aiming to create a framework, as shown in Figure 1.
I never made any real plans along this way. I always worked on one step until I figured out the next one. Like a forest, the field of Artificial Intelligence is a system of many elements. If we want to cross the forest, we do not need to go past every tree. The ‚perfect‘ route might cover as many or as little trees as possible (depending on the goal), but that does no mean that every other path is bad. Walking our own paths through the forest could turn out as a ‚good‘ route, which really is fair enough. I know that my path is not perfect by any means, but I am still happy with it.
Big picture
My goal is to create a framework that puts a RL bot into any given Unity game. This framework serves as an API between the AI, the game and the user, as shown in Figure 2. To keep things simple, the framework will be a Windows application.
I see two main use cases for this framework. One the one hand, users could watch AI overcoming obstacles in games for entertainment, while on the other hand, game developers could test new features by observing and AI’s actions.
That is the basic concept. In two weeks, some design may follow. In between, an exercise.
