Temporal difference control within a dynamic environment

The aim of this paper is to investigate reinforcement learning, specifically the use of Temporal Difference learning methods for the generation of player character movement, within a dynamic, digital game environment. Using a variation of the classic arcade game Pac-Man, the Sarsa and Sarsa(λ) algorithms have been utilised for the control of a Pac-Man game agent, with results indicating that the chosen learning algorithms are successful in achieving the underlying objectives of the game agent. However, a number of trade-offs between the objectives of the game agent must be made during the selection of parameter values for the learning algorithms. In the experiments presented herein, the incorporation of a priori game information into the chosen learning algorithms has shown an improvement in the performance of the game agent in terms of both the score obtained and time taken per game.