Adaptive and Implicit Regularization for Matrix Completion

08/11/2022
by   Zhemin Li, et al.
33

The explicit low-rank regularization, e.g., nuclear norm regularization, has been widely used in imaging sciences. However, it has been found that implicit regularization outperforms explicit ones in various image processing tasks. Another issue is that the fixed explicit regularization limits the applicability to broad images since different images favor different features captured by different explicit regularizations. As such, this paper proposes a new adaptive and implicit low-rank regularization that captures the low-rank prior dynamically from the training data. The core of our new adaptive and implicit low-rank regularization is parameterizing the Laplacian matrix in the Dirichlet energy-based regularization, which we call the regularization AIR. Theoretically, we show that the adaptive regularization of AIR enhances the implicit regularization and vanishes at the end of training. We validate AIR's effectiveness on various benchmark tasks, indicating that the AIR is particularly favorable for the scenarios when the missing entries are non-uniform. The code can be found at https://github.com/lizhemin15/AIR-Net.

READ FULL TEXT

page 9

page 11

page 12

10/12/2021

AIR-Net: Adaptive and Implicit Regularization Neural Network for Matrix Completion

Conventionally, the matrix completion (MC) model aims to recover a matri...
07/29/2020

A regularized deep matrix factorized model of matrix completion for image restoration

It has been an important approach of using matrix completion to perform ...
09/24/2016

Max-Norm Optimization for Robust Matrix Recovery

This paper studies the matrix completion problem under arbitrary samplin...
04/20/2016

Depth Image Inpainting: Improving Low Rank Matrix Completion with Low Gradient Regularization

We consider the case of inpainting single depth images. Without correspo...
06/27/2012

Efficient and Practical Stochastic Subgradient Descent for Nuclear Norm Regularization

We describe novel subgradient methods for a broad class of matrix optimi...
01/24/2018

Matrix Completion with Nonconvex Regularization: Spectral Operators and Scalable Algorithms

In this paper, we study the popularly dubbed matrix completion problem, ...
06/17/2021

Adaptive Low-Rank Regularization with Damping Sequences to Restrict Lazy Weights in Deep Networks

Overfitting is one of the critical problems in deep neural networks. Man...