Marine Snow Removal Benchmarking Dataset

1 Introduction

Image restoration has been one of the main topics of computer vision for decades

. In the current era of deep learning, the quality of image restoration for various tasks have been significantly improved under a sufficient number of pairs of ground-truth and degraded images. Therefore, the current focus on image restoration will be for those taken under extreme situations such as underwater, satellite, and medical images

[1, 6, 23, 30, 31, 22, 14, 20, 12, 37, 39, 26, 29, 25, 32, 3]. This problem is called extreme image restoration herein. Extreme images often have an untypical degradation beyond that of a Gaussian model, and the numbers of available images for training and modifying restoration algorithms are also limited. As a result, extreme image restoration is still a challenging problem.

In this paper, we focus on underwater image restoration, the main challenge of which is mostly due to the fact that we generally have no ground-truth images. This makes it difficult to calculate the objective qualities of the restored images and measure the restoration performance. Moreover, the lack of image pairs causes a difficulty in training a neural network (if we consider using a deep neural network for restoration) because the loss functions of the image restoration typically contain some objective image quality metrics such as mean squared error (MSE) and structured similarity index (SSIM)

[38]

. Hence, unsupervised restoration methods, including linear and nonlinear filtering, are still popular in underwater image restoration

[6, 23, 2].

Several studies for underwater image restoration with deep neural networks have been proposed [30, 31, 22, 14, 20]. Most focus on removing the color shift from underwater images. That is, the blueish underwater images are restored by enhancing the red and green channels. Conventional underwater image enhancement methods using deep neural networks tackle the problem on the lack of large-scale underwater image datasets in various ways. For example, the studies in [31, 4, 30, 14] synthetically generate degraded images from clean images taken on the ground by simulating the degradation processes of absorption and scattering. Moreover, [20, 24, 15]

uses a generative adversarial network. Although a number of challenges to improving the restoration performance still exist, some promising results have been presented. However, such studies omit another main source of degradation for underwater images:

Marine snow.

In this paper, we consider a marine snow removal (MSR) problem for underwater images by constructing a large-scale dataset containing underwater image pairs, as shown in Fig. 1. Marine snow is a typical underwater image degradation, examples of which are shown in Fig. 2. Marine snow artifacts are visible in digital images when we take an underwater image using a flashlight. Small particles in the scene, e.g., organic matter and sand, cause marine snow. The particles reflect the light from the flashlight, and the reflected light will be captured by the photosensors. Marine snow is often annoying in underwater photography because it affects the overall image quality. However, the particles are unevenly distributed in the scene, and the degradation process does not have a typical statistical property. Therefore, standard methods for marine snow removal are still limited to simple median-filter-based approaches [5, 13, 21]. Moreover, as with standard underwater image restoration, we do not have a pair of ground-truth and degraded image pairs to calculate the objective image qualities and tune the parameters of the restoration algorithms.

We propose marine snow synthesis methods for the above-mentioned problems of MSR. In other words, we intentionally append the marine snow artifacts to real underwater images and generate a pair of ground-truth images (i.e., clean underwater images) and degraded images (those with marine snow artifacts). The key for synthesizing artifacts is to model real marine snow in digital images as accurately as possible. We discover that many marine snow artifacts can be mathematically modeled through two typical pixel value distributions. By utilizing this fact, we can synthetically generate many pairs of underwater images with/without marine snow artifacts.

We also propose two marine snow removal tasks and develop datasets corresponding to these tasks. This Marine Snow Removal Benchmarking Dataset (MSRB Dataset) is the first attempt in this field and is publicly available online¹¹1https://github.com/ychtanaka/marine-snow. We also present the first MSR benchmarking results using the MSRB Dataset: We compare the objective performance of marine snow removal among median filters and a deep neural network. Real MSR results as well as the limitations are also described.

The remainder of this paper is organized as follows: Section 2 reviews several related studies for underwater image restoration. Two representative marine snow models that we discovered are presented in Section 3 along with their synthesizing method. The MSRB Dataset specifications are introduced in Section 4 and include two tasks. The first MSR benchmarking results with objective image quality metrics are shown in Section C. Real MSR results and limitations are described in Section 6. Finally, we provide some concluding remarks in Section 7.

2 Related Studies

There have been few studies on modeling marine snow in digital images. The seminal study on modeling marine snow is described in [7, 8]. A marine snow artifact is modeled using a Gaussian function in which the artifact is less transparent in its center than its surrounding area. This method is better than the traditional salt and pepper noise; however, it does not simulate the actual shapes of marine snow artifacts. As a result, the center part of marine snow becomes excessively thick and the resulting images are unrealistic.

MSR is also an underrepresented problem in underwater image restoration. Here, we briefly review some existing approaches.

A widely used method for MSR is a median filter (MF) [9, 19]. If marine snow artifacts can be assumed sufficiently small (typically $1$ – $3$ pixels in diameter), MF is expected to work because small marine snow artifacts can be considered to have a salt-and-pepper effect. However, it significantly blurs the entire image, particularly when we use a large filter kernel size.

Specific to MSR, a modified version of a MF is proposed in [5, 17]. This method applies the MF selectively if the target pixel has a higher intensity than the surrounding pixels. However, it is still difficult to remove large marine snow artifacts.

A few MSR methods for video sequences have also been proposed. They utilize the fact that marine snow artifacts continuously move in consecutive video frames. In [16], background modeling is used to remove marine snow artifacts from a static scene. Marine snow artifacts are tracked and a customized MF is applied to the detected artifacts in [13]. However, they cannot be used for a single underwater image.

Importantly, all methods above are based on a model-based approach, and there have been no methods to remove marine snow artifacts based on deep neural networks. This is mainly due to a lack of high-quality datasets, as mentioned in Section 1.

In the following, we address the contributions of this study; marine snow models and the dataset specifications.

3 Marine Snow Models

There are various sources of marine snow and it is impractical to estimate the sources of all particles from a single underwater image. Instead of the estimation of the sources, we model the pixel value distributions of marine snow artifacts from observations of underwater images.

3.1 Real Marine Snow Examples

First, we start by showing marine snow examples in real underwater images. Figs. 3(a) and (c) show enlarged portions of representative marine snow artifacts cropped from real underwater images. Although they look similar, their pixel value distributions are slightly different.

Taking a closer look, the 3D plots of Figs. 3(a) and (c) are shown in Figs. 3(b) and (d), respectively. As clearly observed, they do not have a shape like a Gaussian function in contrast to the conventional assumption in [8, 7]. Rather than a Gaussian function, these 3D plots are similar to elliptic conical frusta, i.e., sliced elliptic cones. Furthermore, the top surfaces of the frusta have different characteristics between Figs. 3

(b) and (d). In our preliminary observation, most marine snow artifacts can be classified into these two representative shapes. In the following, we present the mathematical models of marine snow artifacts that reflect the above-mentioned observations.

3.2 Marine Snow Model 1: Highland Type

The first marine snow model is the Highland Type (type H). The type H marine snow corresponds to Fig. 3(a) and can be modeled as an elliptic conical frustum with a rough surface.

For modeling type H marine snow, first suppose that we have two ellipses $f_{1} (m, n)$ and $f_{2} (m, n)$ , both centered at the coordinates $(k, l)$ , where $m$ and $n$ represent the row and column indices of an image, respectively. As shown in Fig. 4, we assume the focal points of $f_{1} (m, n)$ and $f_{2} (m, n)$ are located on the horizontal axis, i.e., the ellipses are wider, for simplicity. Let $a_{i}$ and $b_{i}$ be the semi-major and semi-minor axes of $f_{i} (m, n)$ , respectively. Suppose that $f_{1} (m, n)$ is larger than $f_{2} (m, n)$ , i.e., $a_{1} > a_{2}$ and $b_{1} > b_{2}$ and the eccentricities of $f_{1} (m, n)$ and $f_{2} (m, n)$ are identical, i.e., $\frac{a_{1}}{b_{1}} = \frac{a_{2}}{b_{2}}$ .

Figure 4: Ellipses used for marine snow synthesis.

For notational simplicity, $(k, l)$ is set to $(0, 0)$ hereafter. The $(m, n)$ th pixel value of the type H marine snow $h (m, n)$ is formulated as follows:

h (m, n) = ⎧ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎩ \begin{matrix} c + ϵ & \frac{m^{2}}{a_{2}^{2}} + \frac{n^{2}}{b_{2}^{2}} \leq 1 c \frac{d (p, q_{1})}{d (q_{1}, q_{2})} + ϵ & 1 < \frac{m^{2}}{a_{2}^{2}} + \frac{n^{2}}{b_{2}^{2}} \land \frac{m^{2}}{a_{1}^{2}} + \frac{n^{2}}{b_{1}^{2}} \leq 1 0 & otherwise, \end{matrix}

(1)

where $c$ is a constant that determines the transparency of the marine snow, $d (\cdot, \cdot)$ calculates the Euclidean distance between two points, and $ϵ$ is a small perturbation that mimics a rough surface. Furthermore, $p := [m, n]^{⊤}$ and $q_{i}$ ( $i \in {1, 2}$ ) is the intersection of the ellipse $f_{i} (m, n)$ and the straight line crossing $(m, n)$ , as illustrated in Fig. 4.

An example of the synthesized type H marine snow artifacts is shown in Fig. 5(a) along with its 3D plot in Fig. 5(b). See the similarity in Figs. 3(a) and (b). For creating a dataset, the major axis of $h (m, n)$ is randomly rotated and the transparency $c$ is also randomly chosen. These specifications are shown later in Section 4.

3.3 Marine Snow Model 2: Volcanic Crater Type

The second marine snow model is the volcanic crater type (type V). An example of type V marine snow is shown in Figs. 3(c) and (d). This can be modeled by a modified version of the type H with an overshooted top edge of the frustrum.

For the type V marine snow, we consider another ellipse $f_{3} (m, n)$ , where $a_{3}$ and $b_{3}$ are its semi-major and semi-minor axes, respectively. It is located inside $f_{2} (m, n)$ , as illustrated in Fig. 4. As for the type H, focal points of all ellipses are assumed to be located along the horizontal axis and their eccentricities are identical, i.e., $\frac{a_{1}}{b_{1}} = \frac{a_{2}}{b_{2}} = \frac{a_{3}}{b_{3}}$ .

Based on the type H function $h (m, n)$ , the $(m, n)$ th pixel value of the type V marine snow $v (m, n)$ is formulated as $v (m, n) = h (m, n) + g (m, n)$ , where $g (m, n)$ is the rim term defined as

(2)

where $c_{r}$ is the constant for the maximum rim height and $q_{3}$ indicates the coordinates of the intersection of the ellipse $f_{3} (m, n)$ and the straight line crossing $(m, n)$ .

The synthesized type V marine snow artifact is shown in Figs. 5(c) and (d), which are similar to the real versions in Figs. 3(c) and (d).

3.4 Synthesizing Images with Marine Snow

By using the marine snow models introduced in the previous subsection, we synthesize the marine snow with real underwater images.

Let $U \in R^{H \times W \times 3}$ be an original underwater image. First, we randomly select a target pixel coordinate $(k, l)$ to append the marine snow. Second, we randomly select the major and minor axes of the ellipses $a_{i}$ and $b_{i}$ . In addition, the direction of the major axis is randomly set. Third, for each channel, type H or V marine snow is independently generated, where the parameters $c$ and $c_{r}$ are independently set. We denote an image with one marine snow artifact as $M$ , where the elements in $M$ are all-zero except the marine snow artifact. Finally, $U$ and $M$ are combined as follows to yield the synthesized image $I$ :

I_{k, l, t} = {\begin{matrix} M_{k, l, t} + G_{k, l, t} & M_{k, l, t} \geq 0 U_{k, l, t} & otherwise \end{matrix}

(3)

where $G \in R^{H \times W \times 3}$ is the Gaussian filtered version of $U$ that reflects blur owing to marine snow.

We iterate the process until a predetermined number of marine snow artifacts are appended.

4 Dataset Specifications

In this section, we present the specifications of synthesized marine snow artifacts in the MSRB Dataset. It is designed for two tasks:

MSR Task 1: Removal of small-sized marine snow artifacts.
MSR Task 2: Removal of various-sized marine snow artifacts.

Clearly, Task 2 is more difficult than Task 1.

First, detailed parameters and setups shared in both tasks are presented. Second, we introduce MSR Tasks with corresponding synthesized images.

$*$ $b_{1}$ is obtained from the eccentricity of $f_{2} (m, n)$
Parameter	Setting
$c$ in (1)	$U {10, 80}$
$c_{r}$ in (2)	$U {1, 40}$
Gaussian filter size in (3)	$(2 r + 1) \times (2 r + 1)$ where $r \sim U (1, 6)$
$a_{2}$	${\begin{matrix} U (1, 3) & for small marine snow artifacts U (3, 16) & for large marine snow artifacts \end{matrix}$
$b_{2}$	${\begin{matrix} U (0.3 a_{2}, a_{2}) & for small marine snow artifacts% U (0.8 a_{2}, a_{2}) & for large marine snow artifacts \end{matrix}$
$a_{1}$	$U (a_{2}, 2 a_{2})^{*}$
$a_{3}$	$U (0.5 a_{2}, a_{2})^{†}$
$†$ $b_{3}$ is obtained from the eccentricity of $f_{2} (m, n)$

Table 1: Parameter settings for marine snow synthesis, where

U (v_{min}, v_{max})

is a continuous uniform distribution between

v_{min}

and

v_{max}

4.1 General Setup

Each sub-dataset corresponding to a MSR Task contains $2, 300$ training image pairs and $400$ test image pairs, all having a pixel resolution of $384 \times 384$ . An image pair contains one original underwater image and one image containing synthesized marine snow artifacts. All original images are collected from flickr²²2https://www.flickr.com/ under a Creative Commons Attribution-NonCommercial-ShareAlike 2.0 Generic (CC BY-NC-SA 2.0) License.

Each synthesized image contains $N$ marine snow particles where $N$ is chosen from $U {100, 600}$ , in which $U {v_{min}, v_{max}}$ is a discrete uniform distribution between $v_{min}$ and $v_{max}$

. In each synthesized image, type H and V marine snow particles are randomly generated with a probability of

0.7

for type H and

0.3

for type V, according to our preliminary observations. The other representative parameters introduced in the previous section are listed in Table 3. Most of the parameters are determined according to our preliminary observations of real underwater images with marine snow artifacts.

4.2 MSR Task 1: Removing Small-Sized Marine Snow Artifacts

We introduce the first MSR Task: Removal of small-sized marine snow artifacts. In Task 1, the maximum width/height of the artifacts is restricted to $6$ pixels, which correspond to roughly $1.6$ % of the image width/height. Task 1 is designed for underwater scenes where particles are relatively far from the photosensors. Because the task is relatively simple, the conventional MF approach is expected to be successfully applied (at least partially).

Examples of test image pairs in MSR Task 1 are shown in Fig. 12. As clearly visualized, the synthesized images have various sized (but small) marine snow artifacts.

4.3 MSR Task 2: Removing Various-Sized Marine Snow Artifacts

The second MSR task is designed for underwater scenes containing particles in various distances. As a result, the synthesized images for Task 2 have small- and large-sized marine snow artifacts. For large-sized artifacts, we set the largest width/height of marine snow to $32$ pixels, which corresponds to $8.3$ % compared to the image width. Furthermore, the probabilities of small- and large-sized artifacts are set to $0.7$ and $0.3$ , respectively.

Training image examples for Task 2 are also shown in Fig. 12. The images have various-sized artifacts and the degraded areas are larger than the images in Task 1.

In the following, we show the benchmarking results of MSR as well as the real MSR.

5 MSR Benchmarking Results

In this section, we present the first benchmarking performance for MSR Tasks 1 and 2. The methods used for the benchmarking are 1) MF [19], 2) adaptive MF [5], and 3) U-Net [28]. The kernel size of MFs is set to $3 \times 3$ or $5 \times 5$ pixels. Thanks to the MSRB Dataset, we can train a deep neural network for removing (synthesized) marine snow. For a benchmarking purpose, we train U-Net [28] using the dataset. The parameter settings are slightly different from the original and are shown in the appendix along with the network architecture.

The MSR results for Tasks 1 and 2 are shown in Figs. 7 and 8, respectively. As clearly observed, MF oversmoothes the images, which result in blur even for regions without marine snow artifacts. The adaptive MF suppresses blur; however, many artifacts are not removed mainly owing to the threshold-based algorithm. Furthermore, both MFs do not operate successfully for large marine snow artifacts in Task 2. By contrast, U-Net successfully removes marine snow for both tasks. Even for Task 2, it is able to suppress the large marine snow artifacts because of its multiresolution structure and the large number of trainable parameters.

For an objective comparison, we compute the average PSNRs and SSIMs over the test datasets. We summarize the results in Table 2. The objective measure indicates that U-Net is superior to the MFs for both MSR Tasks. Note that all MFs present smaller PSNRs/SSIMs than those of the synthesized images because they filtered areas without marine snow. By contrast, U-Net achieves a better objective quality than the synthesized images, which implies the effect of a deep neural network for MSR.

	Task 1		Task 2
Method	PSNR	SSIM	PSNR	SSIM
MF (3 $\times$ 3)	28.55	0.846	22.81	0.770
MF (5 $\times$ 5)	25.98	0.711	21.93	0.645
Adaptive MF (3 $\times$ 3)	29.88	0.910	23.35	0.842
Adaptive MF (5 $\times$ 5)	28.08	0.861	22.83	0.794
U-Net	36.82	0.978	30.95	0.932
Synthesized image	32.20	0.945	23.83	0.876

Table 2: Average PSNRs and SSIMs over the test dataset.

6 Real Marine Snow Removal Examples

We construct the MSRB Dataset to mimic real marine snow artifacts. In this section, we show the real MSR results and present limitations of the dataset.

6.1 Real Marine Snow Removal

Fig. 9 shows MSR results for some real images. We use MF, adaptive MF, and U-Net as in the previous section. U-Net is trained using MSRB Task 2 Dataset.

As clearly observed, MFs cannot suppress marine snow artifacts because of their various sizes and different transparencies. U-Net presents a better restoration image quality than MFs: It (partly) suppresses artifacts having both small and large sizes. This implies the effectiveness of deep-learning-based methods trained using the MSRB Dataset for real MSR.

Note that thick marine snow artifacts remain in the restored images. The reason for this could be two-fold: First, U-Net is not specifically designed for MSR although we slightly customized its structure. A neural network designed specifically for MSR would improve the performance. Second, our proposed model classifies marine snow artifacts into two representative types. Marine snow artifacts not fitted to these types are not well removed. This limitation is described in the next section.

6.2 Limitations

Because our MSR benchmarking dataset is based on synthesizing marine snow artificially into real underwater images, some limitations exist. First, the dataset is designed for removing marine snow artifacts and does not aim at achieving other underwater image enhancements such as color correction. The design of a simultaneous restoration of MSR and color correction is an interesting area of future study.

Although U-Net trained using the MSRB Dataset presents promising results, the dataset itself is imperfect. Specifically, real MSR performances are occasionally limited if MSR artifacts are not fitted to our marine snow models. Examples are shown in Fig. 10, which compares real underwater images and their restoration results using U-Net. Typically, U-Net fails to suppress artifacts when marine snow artifacts are larger and denser than our MSR benchmarking dataset. Moreover, if the lighting conditions differ from the dataset, the current version of U-Net will fail. Through a future study, this can be improved by carefully updating the MSRB Dataset to reflect various underwater scenes.

Figure 10: Limitations. Top: Original images. Bottom: Restoration results by U-Net.

7 Conclusions

In this paper, the Marine Snow Removal Benchmarking Dataset was proposed. We mathematically modeled two representative marine snow artifacts and synthesized them in real underwater images. Two MSR tasks, the removal of small-sized and various-sized marine snow artifacts were also proposed. The first benchmarking results were also shown, which revealed the effectiveness of a deep neural network for MSR compared to median filter-based methods. MSR for real underwater images indicates that our MSRB Dataset contributes to real MSR problems. Our future studies will include improving the marine snow models as well as designing deep neural networks specifically designed for MSR.

8 Acknowledgements

This work was supported in part by JST PRESTO under grant JPMJPR1935, JSPS KAKENHI under grant 20H02145.

Appendix A U-Net Structure

The structure of U-Net used for our marine snow removal experiments are illustrated in Fig. 11. This is basically the same as the original U-Net but we slightly changed the number of convolution layers and that of channels in each layer. Also, we initialize the weights of convolution layers by the method in [18]

. The hyperparameters used are summarized in Table

Appendix B Additional Examples from Marine Snow Removal Benchmarking Dataset

Fig. 12 shows additional examples from Marine Snow Removal Benchmarking Dataset (MSRB Dataset) both for MSR Tasks 1 and 2.

Appendix C Additional MSR Benchmarking Results

Figs. 13 and 14 compare the additional MSR results for test images from MSRB Dataset. As in the main article, we used median filter, adaptive median filter, and U-Net.

Parameter	Setting
Learning rate	0.001
Number of epochs	30
Weight initializing method	[18]
Initialized bias values	0

Table 3: Parameter settings for U-Net.

Figure 11: Structure of U-Net for marine snow removal.

References

[1] Derya Akkaynak and Tali Treibitz. A revised underwater image formation model. In
Proceedings of the IEEE conference on computer vision and pattern recognition
, pages 6723–6732, 2018.
[2] Derya Akkaynak and Tali Treibitz. Sea-thru: A method for removing water from underwater images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1682–1691, 2019.
[3] Saeed Anwar and Chongyi Li. Diving deeper into underwater image enhancement: A survey. Signal Processing: Image Communication, 89:115978, Nov. 2020.
[4] Saeed Anwar, Chongyi Li, and Fatih Porikli. Deep Underwater Image Enhancement. arXiv:1807.03528, July 2018.
[5] Soma Banerjee, Gautam Sanyal, Shatadal Ghosh, Ranjit Ray, and Nath Shome, Shome. Elimination of Marine Snow effect from underwater image - An adaptive probabilistic approach. In 2014 IEEE Students’ Conference on Electrical, Electronics and Computer Science, pages 1–4, Mar. 2014.
[6] Dana Berman, Tali Treibitz, and Shai Avidan. Diving into haze-lines: Color restoration of underwater images. In Proc. British Machine Vision Conference (BMVC), volume 1, 2017.
[7] Matthieu Boffety and Frédéric Galland. Phenomenological marine snow model for optical underwater image simulation: applications to color restoration. In 2012 Oceans-Yeosu, pages 1–6, 2012.
[8] Matthieu Boffety, Frédéric Galland, and Anne-Gaëlle Allais. Color image simulation for underwater optics. Applied optics, 51(23):5633–5642, 2012.
[9] David RK Brownrigg. The weighted median filter. Communications of the ACM, 27(8):807–818, 1984.
[10] Stanley H. Chan, Xiran Wang, and Omar A. Elgendy. Plug-and-play admm for image restoration: Fixed-point convergence and applications. IEEE Transactions on Computational Imaging, 3(1):84–98, 2016.
[11] Chang Chen, Zhiwei Xiong, Xinmei Tian, and Feng Wu. Deep boosting for image denoising. In Proceedings of the European Conference on Computer Vision (ECCV), pages 3–18, 2018.
[12] Xueyun Chen, Shiming Xiang, Cheng-Lin Liu, and Chun-Hong Pan.
Vehicle detection in satellite images by hybrid deep convolutional neural networks.
IEEE Geoscience and remote sensing letters, 11(10):1797–1801, 2014.
[13] Boguslaw Cyganek and Karol Gongola. Real-time marine snow noise removal from underwater video sequences. Journal of Electronic Imaging, 27(4):043002, July 2018.
[14] Akshay Dudhane, Praful Hambarde, Prashant Patil, and Subrahmanyam Murala. Deep underwater image restoration and beyond. IEEE Signal Processing Letters, 27:675–679, 2020.
[15] Cameron Fabbri, Md Jahidul Islam, and Junaed Sattar. Enhancing Underwater Imagery Using Generative Adversarial Networks. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 7159–7165, May 2018.
[16] Fahimeh Farhadifard, Martin Radolko, and Uwe Freiherr von Lukas. Marine Snow Detection and Removal: Underwater Image Restoration Using Background Modeling. Václav Skala - UNION Agency, 2017.
[17] Fahimeh Farhadifard, Martin Radolko, and Uwe von Lukas. Single image marine snow removal based on a supervised median filtering scheme. In 12th International Conference on Computer Vision Theory and Applications, VISIGRAPP 2017. Proceedings. Vol.4: VISAPP, pages 280–287, 2017.
[18] K. He, X. Zhang, S. Ren, and J. Sun.
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification.
In 2015 IEEE International Conference on Computer Vision (ICCV), pages 1026–1034, Dec. 2015.
[19] Thomas Huang, GJTGY Yang, and Greory Tang. A fast two-dimensional median filtering algorithm. IEEE Transactions on Acoustics, Speech, and Signal Processing, 27(1):13–18, 1979.
[20] Qin Jiang, Yang Chen, Guoyu Wang, and Tingting Ji. A novel deep neural network for noise removal from underwater image. Signal Processing: Image Communication, 87:115921, 2020.
[21] Michał Koziarski and Bogusław Cyganek. Marine Snow Removal Using a Fully Convolutional 3D Neural Network Combined with an Adaptive Median Filter. In Pattern Recognition and Information Forensics, pages 16–25, 2019.
[22] Chongyi Li, Saeed Anwar, and Fatih Porikli. Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recognition, 98:107038, 2020.
[23] Chongyi Li, Chunle Guo, Wenqi Ren, Runmin Cong, Junhui Hou, Sam Kwong, and Dacheng Tao. An underwater image enhancement benchmark dataset and beyond. IEEE Transactions on Image Processing, 29:4376–4389, 2020.
[24] Jie Li, Katherine A. Skinner, Ryan M. Eustice, and Matthew Johnson-Roberson. WaterGAN: Unsupervised Generative Network to Enable Real-Time Color Correction of Monocular Underwater Images. IEEE Robotics and Automation Letters, 3(1):387–394, Jan. 2018.
[25] Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen Awm Van Der Laak, Bram Van Ginneken, and Clara I Sánchez. A survey on deep learning in medical image analysis. Medical image analysis, 42:60–88, 2017.
[26] Emmanuel Maggiori, Yuliya Tarabalka, Guillaume Charpiat, and Pierre Alliez. Convolutional neural networks for large-scale remote-sensing image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(2):645–657, 2016.
[27] Yaniv Romano, Michael Elad, and Peyman Milanfar. The little engine that could: Regularization by denoising (red). SIAM Journal on Imaging Sciences, 10(4):1804–1844, 2017.
[28] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, pages 234–241, Cham, 2015.
[29] Dinggang Shen, Guorong Wu, and Heung-Il Suk. Deep learning in medical image analysis. Annual review of biomedical engineering, 19:221–248, 2017.
[30] Takumi Ueda, Koki Yamada, and Yuichi Tanaka. Underwater image synthesis from rgb-d images and its application to deep underwater image restoration. In Proc. 2019 IEEE International Conference on Image Processing (ICIP), pages 2115–2119. IEEE, 2019.
[31] Yang Wang, Jing Zhang, Yang Cao, and Zengfu Wang. A deep CNN method for underwater image enhancement. In 2017 IEEE International Conference on Image Processing (ICIP), pages 1382–1386, Sept. 2017.
[32] Martin Weigert, Uwe Schmidt, Tobias Boothe, Andreas Müller, Alexandr Dibrov, Akanksha Jain, Benjamin Wilhelm, Deborah Schmidt, Coleman Broaddus, Siân Culley, et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nature methods, 15(12):1090–1097, 2018.
[33] Junyuan Xie, Linli Xu, and Enhong Chen. Image denoising and inpainting with deep neural networks. Advances in neural information processing systems, 25:341–349, 2012.
[34] Li Xu, Jimmy SJ Ren, Ce Liu, and Jiaya Jia. Deep convolutional neural network for image deconvolution. In Proceedings of the 27th International Conference on Neural Information Processing Systems-Volume 1, pages 1790–1798, 2014.
[35] Raymond A. Yeh, Chen Chen, Teck Yian Lim, Alexander G Schwing, Mark Hasegawa-Johnson, and Minh N. Do.
Semantic image inpainting with deep generative models.
In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5485–5493, 2017.
[36] Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising. IEEE Transactions on Image Processing, 26(7):3142–3155, July 2017.
[37] Liangpei Zhang, Lefei Zhang, and Bo Du. Deep learning for remote sensing data: A technical tutorial on the state of the art. IEEE Geoscience and Remote Sensing Magazine, 4(2):22–40, 2016.
[38] Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, Apr. 2004.
[39] Xiao Xiang Zhu, Devis Tuia, Lichao Mou, Gui-Song Xia, Liangpei Zhang, Feng Xu, and Friedrich Fraundorfer. Deep learning in remote sensing: A comprehensive review and list of resources. IEEE Geoscience and Remote Sensing Magazine, 5(4):8–36, 2017.

Marine Snow Removal Benchmarking Dataset

Underwater Image Restoration via Contrastive Learning and a Real-world Dataset

Interactive Removal and Ground Truth for Difficult Shadow Scenes

Underwater Image Haze Removal and Color Correction with an Underwater-ready Dark Channel Prior

Parallax Effect Free Mosaicing of Underwater Video Sequence Based on Texture Features

GTAV-NightRain: Photometric Realistic Large-scale Dataset for Night-time Rain Streak Removal

The CSIRO Crown-of-Thorn Starfish Detection Dataset

Underwater autonomous mapping and characterization of marine debris in urban water bodies

Code Repositories

marine-snow

1 Introduction

2 Related Studies

3 Marine Snow Models

3.1 Real Marine Snow Examples

3.2 Marine Snow Model 1: Highland Type

3.3 Marine Snow Model 2: Volcanic Crater Type

3.4 Synthesizing Images with Marine Snow

4 Dataset Specifications

4.1 General Setup

4.2 MSR Task 1: Removing Small-Sized Marine Snow Artifacts

4.3 MSR Task 2: Removing Various-Sized Marine Snow Artifacts

5 MSR Benchmarking Results

6 Real Marine Snow Removal Examples

6.1 Real Marine Snow Removal

6.2 Limitations

7 Conclusions

8 Acknowledgements

Appendix A U-Net Structure

Appendix B Additional Examples from Marine Snow Removal Benchmarking Dataset

Appendix C Additional MSR Benchmarking Results

References

Marine Snow Removal Benchmarking Dataset

Related Research

Underwater Image Restoration via Contrastive Learning and a Real-world Dataset

Interactive Removal and Ground Truth for Difficult Shadow Scenes

Underwater Image Haze Removal and Color Correction with an Underwater-ready Dark Channel Prior

Parallax Effect Free Mosaicing of Underwater Video Sequence Based on Texture Features

GTAV-NightRain: Photometric Realistic Large-scale Dataset for Night-time Rain Streak Removal

The CSIRO Crown-of-Thorn Starfish Detection Dataset

Underwater autonomous mapping and characterization of marine debris in urban water bodies

Code Repositories

marine-snow

1 Introduction

2 Related Studies

3 Marine Snow Models

3.1 Real Marine Snow Examples

3.2 Marine Snow Model 1: Highland Type

3.3 Marine Snow Model 2: Volcanic Crater Type

3.4 Synthesizing Images with Marine Snow

4 Dataset Specifications

4.1 General Setup

4.2 MSR Task 1: Removing Small-Sized Marine Snow Artifacts

4.3 MSR Task 2: Removing Various-Sized Marine Snow Artifacts

5 MSR Benchmarking Results

6 Real Marine Snow Removal Examples

6.1 Real Marine Snow Removal

6.2 Limitations

7 Conclusions

8 Acknowledgements

Appendix A U-Net Structure

Appendix B Additional Examples from Marine Snow Removal Benchmarking Dataset

Appendix C Additional MSR Benchmarking Results

References