Autor: Martin Haring

‚mario-neat.lua‘ is the core of MarI/O, handling everything.

All other scripts act as helper scripts. Their sole purpose is to provide functions to this script (and to keep the project structured).

This blog is a breakdown of the contents of the mario-neat script; it takes a closer look on its functions and functionality.

Also, a conclusion of how the behavior of the scripts could be changed will be given.

The MarI/O project (https://github.com/mam91/Neat-Genetic-Mario) is structured into five Lua files and a pool of neural network states. The five Lua files in question are labeled ‘config’, ‘game’, ‘mario-neat’, ‘mathFunctions’ and ‘spritelist’. In this blog, I will take a closer look on four of them – ‘mario-neat’ will be next week’s topic due to its immense size.

The main idea of this project was to develop an AI similar to MarI/O, which has the purpose of mastering the game “Super Mario World” completely on its own – by interacting and learning. At first, I thought it would be sufficient if I managed to apply the AI to other emulated games.

I want to use this blog to glance over what was accomplished so far and what the next steps will be.

MarI/O was programmed in the Lua programming language.

“Lua is a powerful, efficient, lightweight, embeddable scripting language. It supports procedural programming, object-oriented programming, functional programming, data-driven programming, and data description.” (https://www.lua.org/about.html)

Every Artificial Neural Network (ANN) has its own design for solving its problem most efficiently.

Artificial Neurons

ANNs imitate the human brain. Therefore, they consist of many artificial neurons.
Each neuron is designed to receive an input, process that input (and adjust its weights) and sends the processed input as output to the next neuron, as shown in figure 1.

Neural Network Topologies

The topology of an Artificial Neural Network (ANN) describes how all neurons in the network are connected.

The MIT Journal “Evolving Neural Networks through Augmenting Topologies”, published by Kenneth O. Stanley and Risto Miikkulainen in 2002, claims that NeuroEvolution of Augmenting Topologies (aka NEAT) outperforms the best fixed-topology methods due to

• using a crossover method for different topologies,
• ‘speciation‘ (taking care the networks’ structure and structuring), and
• constant incremental growing from a minimal structure.

Since this is a high-level claim, this blog will give a theoretical introduction to these concepts. (Future blogs might extend on this one.) There are many subcategories in the field of Artificial Intelligence. For this topic, two of them are important: Artificial Neural Networks and Genetic Algorithms.

MarI/O, the inspiration of this project, was installed and tested.

The MarI/O AI is a collection of scripts, which contain the logic of the AI, and states, which save the progress of the AI. Using the Lua scripting language, it is possible to implement the AI into the so-called BizHawk emulator, which is able to emulate multiple platforms while also allowing for scripting with Lua. BizHawk is typically known in the speedrunning scene, where people try to write scripts that play a game perfect to the frame, also known as TAS (tool-assisted speedrun).

The training process of the AI is simple: the AI tries to get as far as possible by giving input to the emulator. This is done with a so-called fitness-value, which is incremented when the AI is moving without hitting anything and when the AI collects coins or scores otherwise. The higher this value gets, the better the AI performed.

The book ‘Reinforcement Learning’ by Richard S. Sutton lead to two important conclusions.

First, a reinforcement learning AI requires with four key elements:

• Policy – the way of work. A set of rules.
• Reward function – a sign for the AI that it did something right
• Value function – determines how much an action leads to the reward function
• Goal – the desired result.

 

In the book, there are three key techniques that could be used to implement an AI:

• Dynamic Programming
• Monte Carlo Algorithms
• Temporal Differential Algorithms

However, since the book was published in 2009 these practices might be outdated.

This idea is inspired by MarI/O, an AI that uses a neural network and evolutionary computation to learn to play the Game Boy game ‘Super Mario World’ on an emulator; solely by interacting with the game.

In my research, I aim to build a similar AI. In the first step, the AI should be able to access controls and to make inputs by itself. In the next step, the AI should learn to play a game – Super Mario World seems to be very fitting. After the AI manages to clear a few levels, the AI should also be applied to other emulated Game Boy games.

This will be achieved by doing research on the topic, using both online sources and primary literature. The AI will be developed using the IntelliJ IDE and the Python programming language.