Scalable Cyber-Physical Simulation for Automated Cyber Agent Training

Modern cyber-physical systems (CPS) are highly complex systems-of-systems, in which understanding the breadth and severity of cyberattacks is highly challenging. As cyberattacks and defensive operations become increasingly automated, there is a greater need to understand the complexities of interactions between the cyber and physical worlds. A scalable, detailed simulation platform will provide a means of developing and evaluating automated techniques within these complex systems. We will leverage the Cyber-Physical Systems Wind Tunnel (CPSWT), a model-based framework for rapidly synthesizing heterogeneous simulation integration, to develop a standard method for assessing cyber operations within these systems. We will leverage the Cyber AI Gym Environment (CAGE), an APL developed tool, to assess the feasibility of Reinforcement Learning (RL) within the integrative simulation environment, demonstrating the ability to select and target attacks based on the secondary observables generated in non-cyber simulations. The overarching project goal is to increase adaptability of automated learning solutions by:

• Showing that we can learn offensive and defensive maneuvers simultaneously within realistic simulation environments,
• Showing that we can customize, deploy, and validate general RL policies to highly realistic modeling and simulation (and to the real-world using hardware-in-the-loop simulation), and
• Showing that high-fidelity simulation provides a viable approach towards better understanding and validating cyber effects.