Camping at the location of the orbs is a solid strategy: the player must collect orbs to win (imagine if Pac-Man’s ghosts simply lingered near the entrances to every corner of the map). It also makes the game less fun. Players no longer experience an exciting chase. Instead, the AI could set an unpredictable ambush. Trachel and Peyrot say their goal is “not to create superhuman bots – that wouldn’t be fun and engaging for a novice gamer – but rather to find ways to incorporate machine learning into the AI tools of game already used in production”.
This might sound boring for gamers looking for better AI. Still, the machine-learning techniques shown by Trachel and Peyrot remain useful for tuning difficulty even when the enemies players face in the finished game aren’t using it. Julien Togeliusco-founder and research director at Model.ai, has spent nearly five years using AI to test games. Modl.ai uses bots to hunt for graphical glitches, find flaws in world geometry, and detect situations that make it impossible to win.
“You can tell us what kind of failure state you’re interested in. And then, basically, it works. You submit a job, and it works based on what you want to explore,” Togelius explains. “And of course, we can put them together for you and provide a report, showing here where you seem to be having problems, and so on.”
Modl.ai’s test bots use machine learning to adapt to each game tested, although its current implementation limits these adaptations to each specific title. Togelius says the company is prototyping adding deep learning that will train bot behavior across multiple games. Once used, Modl.ai’s bots will learn to mimic the behavior of real players, which should more effectively reveal any issues players would find.
For true machine learning, game engines need a revolution
When it comes to difficulty, machine learning can be both a problem and a solution. But creating a fair and fun challenge isn’t the only hurdle faced by developers who want to use machine learning in games. The issues run deeper – so deep, in fact, that they may force a rethink in how games are built.
Performance is an obstacle. Machine learning requires a lot of training data to get valid results, and that data can only be acquired by playing a game thousands or tens of thousands of times (although bots can lighten the load, a tactic used by Trachel and Peyrot to build their demo). And once the training data is collected, the resulting model can become cumbersome to run in real time.
“Yes, performance is clearly an issue, especially with large ML models that process frames for every tick of the game clock,” Trachel and Peyrot said in an email. “In our case, to avoid performance issues, we used a small neural network that only inferred at specific times in the game.” Evolving into the massive open-world environments that modern gamers have come to expect is another matter altogether.
Togelius says the way modern game engines work exacerbates the problem. Machine learning, he says, “will necessarily be slow because game engines aren’t designed for it. One of the many reasons why we don’t see more interesting modern AI in games is that Unreal and Unity and all their ilk are basically terrible – anti-AI in many ways.
Animation is another issue. Most modern game engines expect animations to be strictly defined frame by frame. This works well when the animators know for sure how the characters in the game will behave, but an AI controlled by machine learning can behave in ways that the animators did not expect. Designers can work around this problem with a physics-based approach to animationbut this puts even more pressure on the performance of a game console or computer hardware and comes with its own development challenges.
In short, the developers face a monster of their own. Game engines are designed to use behavior trees and prescribed actions to create AI-controlled NPC worlds that work well even on lean hardware. But as machine learning gains momentum, these classic solutions will have to be reconsidered.
“If you go talk to a machine learning researcher who doesn’t know game design, they’ll say, ‘Why don’t you use new stuff and get NPCs that are more realistic and fit the way you play? “, and so on,” Togelius says. “But you can’t just plug that into an existing game. You have to rethink what the game even is.”
