NVIDIA Developers are Using StarCraft to Train Machine Learning AIs

Training artificial intelligence (AI) systems these days is all about creating simulations that can operate at machine speeds so that the training time can be compressed. Rather than creating all simulations from scratch, developers have started to use video games to help with the training. Case in point is this story of NVIDIA developers using StarCraft to learn about training AIs at scale. The skills the AIs pick up likely don’t have much direct real-world implications unless we have a War Of The Worlds event, but games are complex. They are designed to be played by humans and not only have high levels of difficulty, they are also defined and—with strategy games in particular—they have a ton of moving parts that need to by dynamically managed.

Of course, using any combat strategy game would have direct implications for AIs used for defense so there are some more direct military applications for this (why did my office Alexa suddenly decide to play the theme from Terminator I wonder?). But mostly these efforts are to think through ways to speed training for complex problems in general, which could be applied to an ever-broader list of potential markets and related tasks.

Let’s talk about game-based AI training simulations this week.

Dealing with Compound Complexity

In the real world, AIs have to interact with a lot of things that aren’t connected like people, animals, weather, and a lot of legacy hardware. In addition, the mobile platforms many of the AIs will inhabit will have to interact with other AIs which may, or may not, be connected to them. Even if they are connected different goals, priorities, and decision processes may come into play. For instance, an AI delivery system from one company may be trained to avoid a collision by veering left where one from a rival company might veer right which, at speed, would almost always result in a head to head collision if these platforms were approaching each other directly. If they are connected, they can possibly work this out before a catastrophic event, but—as noted above—most things they are likely to share the road or sidewalk with won’t be connected for years, if not decades.

This means we need to train these mobile AIs on how to deal with the unknown in a way that will assure the survival of the AI and everything around them. As they become more connected, working collaboratively to avoid problems and danger will become more common place but training them remains problematic because you don’t want living things as collateral damage.

So, the path forward is initially through simulation because in a computer program there is relative safety from real terminal outcomes or injuries. But creating simulations is expensive and the number of people that know how to train AIs is still relatively small. So, games suddenly become interesting because they are simulations by nature, they already exist, and many represent millions of dollars in development just to make them seem more real.

That reality, which often includes advanced physics allows them to become an existing training resource and the complexity of a game like StarCraft should translate into ways of building skills for AI operated robots doing complex tasks like being a waiter or a bus boy. The training with StarCraft had an AI agent controlling hundreds of StarCraft units and learning from the experience.

The Creation of An AI Benchmark

This kind of scenario is also used in benchmarking software to highlight the relative performance of PC hardware. Benchmarking products like 3DMark use games as a way to compare PCs to each other and a similar benchmark for AIs will likely come out of this effort which is now called the StarCraft Multi-Agent Challenge.

Wrapping Up

We have a need for tools to train AIs and one that has emerged is using high-level PC games to do the training and use the related experience to both develop better AIs and to develop better AI measurement tools. In the near-term future, your AIs may hall have IQ-like scores developed by using computer games, much like they are used with GPUs, as benchmarks. But, even more important, the use of games in this way—games that can operate at machine speeds—should massively increase the AI advancements that are taken through this game simulation experience.

Now think about what this would do to NPCs? (Non-Player Characters) in a game. There is a very real possibility that these developments would not only create computerized opponents that are nearly unbeatable but also NPCs that will behave far more like real people. This will in turn create even more realistic training scenarios and likely significantly speed up AI training.

The future continues to advance towards us at blazing speeds.