RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning

by   Kang Liao, et al.

The wide-angle lens shows appealing applications in VR technologies, but it introduces severe radial distortion into its captured image. To recover the realistic scene, previous works devote to rectifying the content of the wide-angle image. However, such a rectification solution inevitably distorts the image boundary, which potentially changes related geometric distributions and misleads the current vision perception models. In this work, we explore constructing a win-win representation on both content and boundary by contributing a new learning model, i.e., Rectangling Rectification Network (RecRecNet). In particular, we propose a thin-plate spline (TPS) module to formulate the non-linear and non-rigid transformation for rectangling images. By learning the control points on the rectified image, our model can flexibly warp the source structure to the target domain and achieves an end-to-end unsupervised deformation. To relieve the complexity of structure approximation, we then inspire our RecRecNet to learn the gradual deformation rules with a DoF (Degree of Freedom)-based curriculum learning. By increasing the DoF in each curriculum stage, namely, from similarity transformation (4-DoF) to homography transformation (8-DoF), the network is capable of investigating more detailed deformations, offering fast convergence on the final rectangling task. Experiments show the superiority of our solution over the compared methods on both quantitative and qualitative evaluations. The code and dataset will be made available.


page 3

page 4

page 6

page 7

page 8

page 13

page 14

page 15


Deep Rectangling for Image Stitching: A Learning Baseline

Stitched images provide a wide field-of-view (FoV) but suffer from unple...

Unsupervised Medical Image Alignment with Curriculum Learning

We explore different curriculum learning methods for training convolutio...

TartanCalib: Iterative Wide-Angle Lens Calibration using Adaptive SubPixel Refinement of AprilTags

Wide-angle cameras are uniquely positioned for mobile robots, by virtue ...

FisheyeEX: Polar Outpainting for Extending the FoV of Fisheye Lens

Fisheye lens gains increasing applications in computational photography ...

DarSwin: Distortion Aware Radial Swin Transformer

Wide-angle lenses are commonly used in perception tasks requiring a larg...

Generating Animatable 3D Cartoon Faces from Single Portraits

With the booming of virtual reality (VR) technology, there is a growing ...

Learning Edge-Preserved Image Stitching from Large-Baseline Deep Homography

Image stitching is a classical and crucial technique in computer vision,...

Please sign up or login with your details

Forgot password? Click here to reset