CMA-TWEANN: Efficient optimization of neural networks via self-adaptation and seamless augmentation

Neuroevolutionary algorithms are successful methods for optimizing neural networks, especially for learning a neural policy (controller) in reinforcement learning tasks. Their significant advantage over gradient-based algorithms is the capability to search network topology as well as connection weights. However, state-of-the-art topology evolving methods are known to be inefficient compared to weight evolving methods with an appropriately hand-tuned topology. This paper introduces a novel efficient algorithm called CMA-TWEANN for evolving both topology and weights. Its high efficiency is achieved by introducing efficient topological mutation operators and integrating a state-of-the-art function optimization algorithm for weight optimization. Experiments on benchmark reinforcement learning tasks demonstrate that CMA-TWEANN solves tasks significantly faster than existing topology evolving methods. Furthermore, it outperforms weight evolving techniques even when they are equipped with a hand-tuned topology. Additional experiments reveal how and why CMA-TWEANN is the best performing weight evolving method.