Research Projects

Accurate hydrodynamic modeling is essential for reliable control and mission performance of Autonomous Underwater Vehicles (AUVs), particularly in tasks requiring precise navigation near seabed or obstacles. Traditional methods for estimating hydrodynamic coefficients rely on either Computational Fluid Dynamics (CFD), which is computationally expensive, or towing tank experiments that require specialized infrastructure. While free-running trials offer a practical alternative, the data they generate often lacks sufficient excitation of the AUV dynamics, resulting in poor or biased parameter estimation. This project proposes a novel framework for information-rich system identification using reinforcement learning (RL) to generate trajectories that excite the vehicle’s dynamics. Instead of manually designed maneuvers, an RL agent learns control actions that maximize the entropy of visited states, thereby promoting diverse and informative exploration of the AUV’s operational space. The hypothesis is that an exploration-aware policy can autonomously generate trajectories that are more suitable for identifying hydrodynamic coefficients than standard maneuvers. Training occurs within a high-fidelity simulation environment that captures full 6-DOF dynamics, actuator models (including control fins and thruster), sensor emulation (IMU, DVL, pressure), and ROS 2 interfacing. Domain randomization is employed to expose the agent to variability in parameters and noise, supporting robust policy transfer to real AUV hardware. The learned policy is later fine-tuned on the real platform using safe exploration techniques. Data collected through these trajectories is processed using a Bayesian Linear Regression (BLR) approach. This allows not only estimation of damping coefficients but also quantification of uncertainty and identifiability. Posterior variance highlights which parameters are confidently estimated and which require further excitation. If necessary, the RL agent is retrained with new constraints or reward structures to target under-excited dynamics. Key experiments include: - Training RL agents in simulation to generate entropy-maximizing trajectories - Deploying the trained policy on a real AUV for data collection - Performing Bayesian system identification from both simulated and real datasets - Evaluating model accuracy by comparing identified parameters with known ground truth - Demonstrating safe policy transfer from simulation to water This work will provide a generalizable, data-driven methodology for underwater system identification, reducing dependency on expensive experimental facilities and advancing the autonomy and adaptability of AUV platforms.