| Over the past few years, AI researchers have made great strides in developing algorithms that can perform well in challenging games. However, when it comes to real‐time strategy (RTS) games, the enormous state spaces involved make it difficult for many researchers to develop ef‐ ficient algorithms for RTS AI. In this study, we used Proximal Policy Optimization (PPO), along with several other improvements, to address the problem of large state spaces in MicroRTS. Our improvements included invalid action masking, KL penalty, and roll back, which helped to make our algorithm more efficient. We also used Recurrent Proximal Policy Optimization to address the issue of partial observation, and Parameterized Reward Shaping to deal with the volatile environment in RTS games. As a result of these improvements, our AI performed better than most of the AIs provided by the MicroRTS. To further improve the performance of our AI, we suggest three potential solutions. First, better techniques are needed to solve the problem of partial observation space. Second, optimizing parameter settings could help to further improve the performance of our AI. Finally, combining human‐designed bots with PPO could potentially lead to even better results. |
