PNNL

Abstract

This article presents a reinforcement learning based control framework for adaptive generator protection in wide-area power systems. Out-of-step (OOS) generator tripping control is one of the most effective approaches to mitigating risks of a system-wide black-out following any severe disturbance. Tra- ditional protection schemes utilize rule-based mechanisms that fail to adapt to changing operating conditions. With the recent advances in deep reinforcement learning (DRL), the primary objective of our proposed methodology is to learn a DRL agent that: (a) can timely identify and isolate the affected generators after any potential disturbance and thereby maintain system stability, and (b) can adapt in unseen scenarios. But learning to identify an optimal set of generators for bulk power systems under various operating conditions is prohibitive due to: (a) the combinatorial nature of the problem, (b) the exponential increase of action space, and (c) ultra-selectivity of the generator trip- action. To address these key challenges, we utilized the concept of action masks integrating system physics in the learning process, thereby blocking unnecessary actions in the exploration phase of the policy training, where the action masks are learned in conjunction with the DRL policy. In the policy part, we utilized a derivative-free parallel augmented random search (PARS)-based DRL algorithm, which is fast and highly scalable. Finally, we validated the proposed methodology with IEEE 300-bus systems