BlueROV2 Autonomy

BlueROV2 Standalone package is a complete, fault tolerant autonomous underwater software system, based on UUV Simulator and BlueROV2 ROS simulation base packages, extended excessively by Vanderbilt University, Institute of Software Integrated Systems in the DARPA Assured Autonomy project.

This package is capable of simulating an autonomous BlueROV2 UUV in an underwater environment, also prepared to operate on the real ROV hardware as well.

The software is based on ROS Kinetic, and the Gazebo, an open source 3D robotics simulator.

The system is divided into hardware, which is the UUV itself including thrusters, actuators, CPU, etc., and software, which includes the autonomy engine and sensing-to-actuation pipeline. The autonomy engine consists of a world model which describes the UUV’s operating environment, including the occupancy grid-based pipeline map, obstacle map and other smaller maps for navigation and guidance. The mission model describes the UUV mission with the help of a mission file (what to execute) and a mission server (how and when to execute). The system model is a representation of the UUV operational state, either nominal or degraded, and any identified hazards such as low battery or obstacle detected. The autonomy manager implements the higher-level autonomy in the system, including the mission execution, and LEC-based fault-adaptive operation.

For detailed information about the BlueROV2 Behaviour Tree based autonomy and the fault detection topic please check our paper: https://www.mdpi.com/1424-8220/21/18/6089

For the autonomy manager check 4.2.1. Autonomy Manager in the paper. The top level BlueROV root node has parallel children:

  • topics2bb: Reads ROS topics, and update the input for the tree
  • mission_server: Reads mission file, and controls the mission subtree
  • DDLEC: FDIR subsystem
  • Obstacle Avoidance: This node creates the final HSD command for the UUV based on the available HSD commands from mission and obstacle ROS nodes - always active
  • Priorities
    • Failsafes: Battery low, RTH, Pipe lost, obstacle standoff, emergency brake, geofence etc.
    • Mission: Pipe Tracking, Waypoint following, Loiter subtrees

BlueROV2 simulation in headless mode (ALC Toolchain)

The video below presents the BlueROV2 package, running the simulation through the ALC Toolchain. In the ActivityDefinition the user can change the internal parameters like mission file, obstacle avoidance, contingency management details, fault detection and control reallocation, etc. The simulation is running realtime, and it can be visualized through the toolchain using RViz in a browser window. The video demonstrates a “pipe tracing” autonomous mission. The default view is “TopDownOrtho”, the random generated pipeline is the blue line, what the UUV follows. At the top, the grayscale waterfall image is the simulated VU side scan sonar image.

BlueROV2 simulation running from IDE (ALC Toolchain)

In the ALC Toolchain / IDE (VSCode), user can open terminal windows and run commands in each launch the simulation and visualization. These are equvivalent to running the sim on the host machine or running it on the real hardware, like a companion computer with a Pixhawk or BeagleBone Blue. The video below presents a tutorial, how to run a waypoint following mission.

Vandy Side Scan Sonar with waterfall image visualization

The video below presents a Simulated SSS based on Gazebo laser ray sensor. The top greyscale image is the side scan output, with a shipwreck, a pipeline and the nadir gap in the middle. The RGB image under it is the generated Ground Truth data for LEC training purposes. Green is the seabed/background, red is pipeline, blue is other (eg. shipwreck), black is unknown / no data (eg. acoustic shadow).

MiDaS - monocular depth estimation integration

Using a single camera image fused with the FLS range a point cloud is calculated from the estimated depth map. This video presents an underwater simulation with obstacles in murky water, using an NVidia Jetson Nano for the realtime MiDaS estimation. In the RViz view, the top three windows are:

  • RAW noisy, small resolution onboard camera image (from Gazebo),
  • MiDaS RAW output
  • FLS fused depth map output

In the main window the 3D pointcloud is visible, generated from the FLS fused MiDaS depth map. The estimation is optimized for obstacles, when they present on both the FLS and the camera data.

Selective Classification Assurance Monitor for FDIR (Degradation Detector / DDLEC)

BlueROV2 has an FDIR system, described in the paper above.

The video below presents the system in operation in a waypoint following mission. The UUV starts the mission in nominal state, but at t=45s, a severe thruster degradation occures. The LEC detects the degradation (first plot: degradation classification, second plot: degraded efficiency), and the AM (bottom two plots) assures the right classification. Using that assured LEC classification the FDIR system makes a control reallocation, and the UUV resumes mission.

Compared to the paper above the AM has been updated to Selective Classification and eliminates most of the false positive failure classifications.

Real-Time Reachability

The RTReach node is always running in the BlueROV simulation. It warns the behavior true if the commanded heading leads to an unsafe situation for the uuv (collision with obstacle or nogo zone). When RTReach reports UNSAFE condition based on the obstacles, the UUV is commanded to “emergency stop”. Thereafter, the UUV moves straight up to the surface.

The high level autonomy is continuously monitoring the RtReach outputs, and calculates the long term RtReach value (outputs fused together) to make high level decisions (like Return To Home, Skip Waypoint) based on the projected safety / risk scores.