DIME-Net: Neural Network-Based Dynamic Intrinsic Parameter Rectification for Cameras with Optical Image Stabilization System

03/20/2023
by   Shu-Hao Yeh, et al.
0

Optical Image Stabilization (OIS) system in mobile devices reduces image blurring by steering lens to compensate for hand jitters. However, OIS changes intrinsic camera parameters (i.e. K matrix) dynamically which hinders accurate camera pose estimation or 3D reconstruction. Here we propose a novel neural network-based approach that estimates K matrix in real-time so that pose estimation or scene reconstruction can be run at camera native resolution for the highest accuracy on mobile devices. Our network design takes gratified projection model discrepancy feature and 3D point positions as inputs and employs a Multi-Layer Perceptron (MLP) to approximate f_K manifold. We also design a unique training scheme for this network by introducing a Back propagated PnP (BPnP) layer so that reprojection error can be adopted as the loss function. The training process utilizes precise calibration patterns for capturing accurate f_K manifold but the trained network can be used anywhere. We name the proposed Dynamic Intrinsic Manifold Estimation network as DIME-Net and have it implemented and tested on three different mobile devices. In all cases, DIME-Net can reduce reprojection error by at least 64% indicating that our design is successful.

READ FULL TEXT

page 1

page 5

research
09/24/2020

3D Object Localization Using 2D Estimates for Computer Vision Applications

A technique for object localization based on pose estimation and camera ...
research
02/21/2023

SU-Net: Pose estimation network for non-cooperative spacecraft on-orbit

Spacecraft pose estimation plays a vital role in many on-orbit space mis...
research
05/14/2019

American Sign Language Alphabet Recognition using Deep Learning

Tremendous headway has been made in the field of 3D hand pose estimation...
research
05/26/2021

How to Calibrate Your Event Camera

We propose a generic event camera calibration framework using image reco...
research
03/22/2022

Ray3D: ray-based 3D human pose estimation for monocular absolute 3D localization

In this paper, we propose a novel monocular ray-based 3D (Ray3D) absolut...
research
04/13/2018

Pose estimation of a single circle using default intrinsic calibration

Circular markers are planar markers which offer great performances for d...
research
09/27/2016

De-noising, Stabilizing and Completing 3D Reconstructions On-the-go using Plane Priors

Creating 3D maps on robots and other mobile devices has become a reality...

Please sign up or login with your details

Forgot password? Click here to reset