Machine Learning for Complex Cyber-Physical Systems

This dissertation presents novel applications of machine learning techniques in enhancing the security and efficiency of complex cyber-physical systems such as power grids and magnetic navigation. The work focuses on four key areas: cyber defense for power grids using deep reinforcement learning (RL), preferential resource allocation for power grid defense, heterogeneous RL strategies for defending against cyber-attacks, and detecting weak signals in magnetic navigation with random forests. After the introduction chapter, the second chapter investigates the application of deep Q-learning to defend power grids from cyber-attacks. Simulating the dynamic interaction between attackers and defenders is crucial, and stochastic game theory with RL offers a strong framework. Current approaches using conventional Q-learning are limited to small systems and struggle with handling cascading failure timings and the stochastic nature of attacks. The proposed method is demonstrated on the W\&W 6-bus and IEEE 30-bus systems, showing its scalability and effectiveness compared to traditional RL approaches. Chapter three introduces a framework for preferential cyber defense, applying a reinforcement learning approach combined with mixed-integer programming (MIP). The results show that this framework effectively optimizes resource allocation to protect power grids from attacks, even when facing fluctuating renewable energy sources. The model adapts to the inherent uncertainties in renewable power generation, ensuring optimal defense strategies that satisfy predefined preferences. The dissertation then explores heterogeneous RL for power grid defense while also comparing deep Q-learning and soft actor-critic (SAC) algorithms. The research demonstrates that dividing the action space and employing specialized agents can significantly reduce training times while achieving comparable or superior performance. Specialized agents focusing on subsets of actions, such as curtailment or topology switching, outperformed agents exploring the entire action space, highlighting the importance of targeted strategies in large, complex environments. Lastly, the work addresses detecting weak signals in magnetic navigation using random forests. By analyzing flight data from multiple sensors, the random forest model successfully detects weak magnetic anomalies, which are crucial for navigation in GPS-denied environments. The results show that random forests outperform other methods such as k-nearest neighbors (KNN) and decision trees, providing a robust framework for precise positioning and signal filtering.