Autonomous Network Defence using Reinforcement Learning

Thesis for MSc Information Security Degree

Here is the PDF version

Abstract

Network defence becomes more challenging because of the dramatic increase in network size and the growing shortage of cyber security professionals. In contrast, current adversaries (e.g., Advanced Persistent Threat attackers) keep evolving their capabilities through information gathering and the development of defence evasion tools. A promising research direction to solve such problems is Artificial Intelligence for Security. Because of the success of Reinforcement Learning in games, it is possible to extend its applications to network security. The current works mainly focus on automated penetrating testing and intelligent intrusion detection systems. One of the limitations of these works is that their environments lack competitive interactions between attackers and defenders. Therefore, we investigate the capabilities of an autonomous network defender who can actively select actions to mitigate the impact of adversaries in an interactive cyber environment named CybORG. The main contributions of this work are extending the performance of prior adversarial strategies, training novel autonomous agents, and evaluating performance. Our results indicate that the hierarchical Reinforcement Learning method can successfully defend against multiple types of adversaries over varying lengths of time with high performance and robustness.