End-to-end Full Projector Compensation

07/30/2020
by   Bingyao Huang, et al.
2

Full projector compensation aims to modify a projector input image to compensate for both geometric and photometric disturbance of the projection surface. Traditional methods usually solve the two parts separately and may suffer from suboptimal solutions. In this paper, we propose the first end-to-end differentiable solution, named CompenNeSt++, to solve the two problems jointly. First, we propose a novel geometric correction subnet, named WarpingNet, which is designed with a cascaded coarse-to-fine structure to learn the sampling grid directly from sampling images. Second, we propose a novel photometric compensation subnet, named CompenNeSt, which is designed with a siamese architecture to capture the photometric interactions between the projection surface and the projected images, and to use such information to compensate the geometrically corrected images. By concatenating WarpingNet with CompenNeSt, CompenNeSt++ accomplishes full projector compensation and is end-to-end trainable. Third, to improve practicability, we propose a novel synthetic data-based pre-training strategy to significantly reduce the number of training images and training time. Moreover, we construct the first setup-independent full compensation benchmark to facilitate future studies. In thorough experiments, our method shows clear advantages over prior art with promising compensation quality and meanwhile being practically convenient.

READ FULL TEXT

page 1

page 5

page 6

page 9

page 11

page 12

page 13

page 14

08/17/2019

CompenNet++: End-to-end Full Projector Compensation

Full projector compensation aims to modify a projector input image such ...
04/08/2019

End-to-end Projector Photometric Compensation

Projector photometric compensation aims to modify a projector input imag...
03/06/2020

DeLTra: Deep Light Transport for Projector-Camera Systems

In projector-camera systems, light transport models the propagation from...
06/09/2021

NeRF in detail: Learning to sample for view synthesis

Neural radiance fields (NeRF) methods have demonstrated impressive novel...
05/03/2019

Processing Megapixel Images with Deep Attention-Sampling Models

Existing deep architectures cannot operate on very large signals such as...
04/03/2019

Random Projection in Neural Episodic Control

End-to-end deep reinforcement learning has enabled agents to learn with ...
07/27/2018

AXNet: ApproXimate computing using an end-to-end trainable neural network

Neural network based approximate computing is a universal architecture p...