This project aims to explore reinforcement learning as a strategy to guide an ml-agent in a mixed reality application. Mixed reality applications allow virtual content to interact with an environment as detected by sensors, such as a smartphone camera. This poses the challenge of finding algorithms that can cope with a variety of uncontrollable environments. Machine learning provides solution to adapt and overcome this challenge. In this project reinforcement learning is used to provide the framework for defining the behavior of a Non-Player-Character. The NPC can be trained by providing it with rewards when it achieves the desired goal and completes a task correctly. This is showcased by a mixed reality mobile game for Android where a user can play fetch with a dog in a virtual scene placed on a real-world environment.
ARCore, Machine Learning, Unity, C#, Tensorflow, Unity ML Agents Toolkit