CNN based dense underwater 3D scene reconstruction by transfer learning using bubble database

11/21/2018
by   Kazuto Ichimaru, et al.
1

Dense 3D shape acquisition of swimming human or live fish is an important research topic for sports, biological science and so on. For this purpose, active stereo sensor is usually used in the air, however it cannot be applied to the underwater environment because of refraction, strong light attenuation and severe interference of bubbles. Passive stereo is a simple solution for capturing dynamic scenes at underwater environment, however the shape with textureless surfaces or irregular reflections cannot be recovered. Recently, the stereo camera pair with a pattern projector for adding artificial textures on the objects is proposed. However, to use the system for underwater environment, several problems should be compensated, i.e., disturbance by fluctuation and bubbles. Simple solution is to use convolutional neural network for stereo to cancel the effects of bubbles and/or water fluctuation. Since it is not easy to train CNN with small size of database with large variation, we develop a special bubble generation device to efficiently create real bubble database of multiple size and density. In addition, we propose a transfer learning technique for multi-scale CNN to effectively remove bubbles and projected-patterns on the object. Further, we develop a real system and actually captured live swimming human, which has not been done before. Experiments are conducted to show the effectiveness of our method compared with the state of the art techniques.

READ FULL TEXT

page 2

page 3

page 4

page 5

page 7

page 8

research
08/25/2018

Multi-scale CNN stereo and pattern removal technique for underwater active stereo system

Demands on capturing dynamic scenes of underwater environments are rapid...
research
05/23/2019

Underwater Stereo using Refraction-free Image Synthesized from Light Field Camera

There is a strong demand on capturing underwater scenes without distorti...
research
11/09/2012

3D Surface Reconstruction of Underwater Objects

In this paper, we propose a novel technique to reconstruct 3D surface of...
research
04/15/2015

Tracking Live Fish from Low-Contrast and Low-Frame-Rate Stereo Videos

Non-extractive fish abundance estimation with the aid of visual analysis...
research
09/05/2021

Underwater 3D Reconstruction Using Light Fields

Underwater 3D reconstruction is challenging due to the refraction of lig...
research
04/07/2021

Predictive 3D Sonar Mapping of Underwater Environments via Object-specific Bayesian Inference

Recent work has achieved dense 3D reconstruction with wide-aperture imag...
research
07/20/2020

Fusing Concurrent Orthogonal Wide-aperture Sonar Images for Dense Underwater 3D Reconstruction

We propose a novel approach to handling the ambiguity in elevation angle...

Please sign up or login with your details

Forgot password? Click here to reset