The ability to operate for long periods of time and then return back safely - is a critical feature for Unmanned Underwater Vehicles (UUVs) in many important applications such as subsea inspection, remote surveillance, and seasonal monitoring. A major challenge for a robot in such long-term missions is to estimate its location accurately since GPS signals cannot penetrate the ocean's surface, and Wi-Fi or radio communication infrastructures are not available underwater. Using a dedicated surface vessel for acoustic referencing or coming up to the water surface for GPS signals are power hungry, computationally expensive, and often impossible (in stealth applications). This project makes scientific and engineering advances by using a novel optics-based framework and on-board AI technologies to solve this problem. The proposed algorithms and systems allows underwater robots to estimate their location with GPS-quality accuracy without ever resurfacing. More importantly, these features enables long-term autonomous navigation and safe recovery of underwater robots without the need for dedicated surface vessels for acoustic guidance.

Overall, the outcomes of this project contributes to the long-term marine ecosystem monitoring and ocean climate observatory research as well as in remote stealth mission execution for defense applications. This project is supported by the the NSF Foundational Research in Robotics (FRR) program. We are working on this project in collaboration with the FOCUS lab (Dr. Koppal) and APRIL lab (Dr. Shin) at UF.

This project advocates a novel solution to the foundational problem of underwater robot localization and navigation by introducing the notion of `optical homing and penning'. This new optics-based framework incorporates three sets of novel technologies for (a) distant UUV (Unmanned Underwater Vehicle) positioning with blue-green laser speckles, (b) accurate 3D orientation measurements from coded bokeh spectrums, and (c) GPS-quality pose estimates by a directionally-controlled adaptive LIDAR. The combined optical sensory system will be deployable from specialized buoys acting as floating lighthouses. An intelligent visual SLAM system will also be developed for robust state estimation in deep waters when no lighthouse beacons are visible. For feasibility analysis and assessment, this project will formalize real-world deployment strategies on two UUV platforms through comprehensive ocean trials in the northern Gulf of Mexico and the Atlantic Ocean.

Underwater caves are challenging environments that are crucial for water resource management. We are investigating the automatic labeling of different cave formations (e.g., speleothems: stalactites, stalagmites, columns) together with navigational aids such as arrows and cookies, and developing an autonomous caveline-following system with a VSLAM system for visual servoing. Such autonomous operations inside caves will potentially lead to high-definition photorealistic map generation and more accurate volumetric models.

Caveline detection and following is paramount as robot navigation guidance in autonomous cave mapping missions. In the Caveline Project, we present a novel Vision Transformer (ViT)-based learning pipeline for fast caveline detection in images from underwater caves. We formulate a weakly supervised approach that facilitates a rapid model adaptation to data from new location with very few ground truth labels. We validate the utility and effectiveness of such weak supervision in three different cave locations: USA, Mexico, and Spain. Our proposed model, CL-ViT, balances the robustness-efficiency trade-off, and ensures good generalization performance while offering 10+ FPS on single-board Jetson TX2s.

•  Paper
 Demo
•  Project Page

Monocular depth estimation is challenging in low-light underwater scenes, particularly on computationally constrained devices. In the UDepth project, we demonstrate that it is possible to achieve state-of-the-art depth estimation performance while ensuring a small computational footprint. While the full model offers over 66 FPS inference rates on a single GPU , our domain projection for coarse depth prediction runs at 51.5 FPS rates on single-board Jetson TX2s.

•  Paper
•  Code
•  Project Page

We are working on developing robust sensing and estimation capabilities of on-device cameras in thermal, acoustic, and spectral domains. In particular, our focus is on wearable cameras used by firefighters and deployed in various SAR applications.
We are hiring UF undergraduate students in this project.

FUnIE-GAN is a GAN-based model for fast underwater image enhancement. It can be used as a visual filter in the robot autonomy pipeline for improved perception in noisy low-light conditions underwater. In addition to SOTA performance, it offers over 48+ FPS inference on Jetson Xavier and 25+ FPS on TX2 devices.

•  Paper
•  Code

Light scattering and attenuation underwater cause range-and-wavelength-dependent non-linear distortions that severely affect machine vision despite often using high-end cameras. Physics-based approxiations with prior knowledge and/or learning-based image enhancement models can help restore the input image qualities, which in turn improve visual perception. However, such visual filtering is challenging without any prior knowledge on embedded robotic systems. We are working with the FOCUS Laboratory and other external collaborators on these problems for a variety of degraded settings. The goal of these projects is to design adaptable solutions for combating degraded machine vision by harnessing the power of online learning and deep reinforcement learning.

Despite recent advancements of interactive vision APIs and AutoML technologies, there are no universal platforms or criteria to measure the goodness of visual sensing conditions underwater to extrapolate the performance bounds of visual perception algorithms. Our current work attempts to address these issues for real-time underwater robot vision.
  More details: coming soon...

An essential capability of visually-guided robots is to identify interesting and salient objects in images for accurate scene parsing and to make important operational decisions. Our work on saliency-guided visual attention modeling (SVAM) develops robust and efficient solutions for saliency estimation by combining the power of bottom-up and top-down deep visual learning. Our proposed model, named SVAM-Net, integrates deep visual features at various scales and semantics for effective SOD in natural underwater images. SVAM-Net incorporates two spatial attention modules to jointly learn coarse-level and fine-level semantic features for accurate salient object detection in underwater imagery. It provides SOTA scores on benchmark datasets, exhibits better generalization performance on challenging test cases than existing approaches, and achieves fast end-to-end run-time on single-board machines.

•  Paper
•  Paper
•  Code

While SVAM is a class-agnostic approach, we are also working on the problem of class-aware visual attention modeling by semantic scene parsing. We emperically demonstrated that a general-purpose solution of spatial attention modeling can facilitate over 45% faster processing in robot perception tasks such as salient ROI enhancement, image super-resolution, and uninformed visual search. We wre currently developing efficient deep visual models to achieve the desired performance bounds.

Not all desired behavior of a robot can be modeled as tractable optimization problems or scripted by traditional robot programming paradigms. Our research attempts to identify such problems and subsequently design practical solutions by using learning from demonstration (LfD) techniques on the TurtleBot-4 platform.

Over 54% of the US population thinks that drones and UAVs (unmanned aerial vehicles) should not be allowed to fly in residential areas as it undermines the ability to assess context and measure trust. Such growing concerns are pervasive across cyberspace toward numerous other human-centric robots and intelligent systems. We are trying to address these issues by devising effective technological and/or educational solutions to ensure transparency and trust. We are exploring various forms of implicit and explicit human-robot interaction for companion robots (eg, Piaggio Fast Forward, Mabu, Staaker, Skydio, Pepper) in manufacturing, health care, and entertainment industry. With a broader goal of ensuring safe and effective human-robot cooperation in various application-specific use cases, we are extending our prior work to define and quantify these interactions and implement other socially-compliant features for companion robots.

Focusing on the Florida coastlines, we are working closely with the Center for Coastal Solutions (CCS) and Warren B. Nelms IoT Institute to develop technological solutions to address the practicalities of important subsea applications such as monitoring water quality, surveying seabed or seagrass habitats, and farming artificial reefs. We are exploring deployable systems for both passive sensing and prediction (of hazards or salient events) as well as coordinated active tracking by autonomous mobile robots.
We are hiring SURF UGs in this project

we are further exploring thermal imaging and sonar imaging modalities to formulate improved techniques that will facilitate useful augmented visuals in autonomous exploration, manned or unmanned rescue operations, and other remote monitoring applications. We are particularly focusing on low-power aerial surveillance cameras of SAR drones deployed in adverse conditions.
  More details: coming soon...

We are conducting several projects that deal with developing low-cost low-power robotic systems and software infrastructures for enabling autonomous and semi-autonomous underwater robots to work alongside human divers in subsea structure inspection and tracking invasive fish (eg, lion fish, jewel fish).
  More details: coming soon...

In particular, we are developing a long-term distributed sensing infrastructure for offshore aquaculture. Our goal is to enable smart feature integration in low-cost aquaculture farms using a distributed IoT-based scalable ecosystem.

With a rapid growth of coastal cities, alarming levels of light pollution are causing irrecoverable damages to its natural habitats. We are developing a platform to accurately measure light pollution and its consequences.