Research Projects

Flood disasters can have devastating effects on communities, causing loss of lives and damage to properties. In recent years, there has been a growing interest in using unmanned aerial vehicles (UAVs) for flood relief operations. These UAVs can explore the flooded areas and gather critical information in a limited time-frame, thereby aiding in rescue and recovery efforts. To ensure effective use of UAVs during flood disasters, we propose a multi-UAV based solution that utilizes deep reinforcement learning (Deep RL) algorithms to learn autonomous multi-UAV control policies. These algorithms can be either centralized (DQN) or decentralized (Dec-DQN) in nature. However, standard Deep RL algorithms face challenges in the early exploration phases of policy learning due to low reward accumulation and divergence. To address these issues, we propose integrating water-flow estimation algorithms, such as D8 and D-infinity (DINF), with the Deep RL methods. The water-flow estimation algorithms use terrain elevation information to formulate directed exploration strategies that improve the performance of the Deep RL methods. For discrete action space algorithms (DQN), we modify the random action selection strategy with a guided exploration strategy. In this strategy, early exploration relies more on D8-induced actions rather than random actions. For continuous action space problems, we adopt the DINF flow estimation algorithm that modifies the target actor of the DDPG algorithm to boost the policy gradient in the correct direction. Centralized training paradigms have limited practical usage when the information is vastly distributed throughout the environment. Therefore, we suggest adopting a decentralized training and execution framework, where UAVs can be trained individually. In this framework, information sharing among agents can be enabled to boost policy learning. To avoid UAVs clustering together at low-elevation regions, we propose the Path Scatter strategy to increase the overall coverage. In addition, once the critical regions have been discovered in the flooded area, we plan to develop an algorithm to find the serviceable paths to reach such critical regions for evacuating trapped persons. The multi-UAV system can help monitor the continuously changing flood environment while the evacuation operations are underway. Finally, the Deep RL models developed in this project will be integrated with the ground control station of the UAVs in order to create a proof-of-concept prototype system for demonstrating the capabilities of autonomous multi-UAV area coverage. The prototype system will also be field tested to establish its practical viability. Overall, the proposed multi-UAV based solution with Deep RL algorithms and water-flow estimation algorithms can significantly aid the search and rescue mission during floods.