Towards Spectral Estimation from a Single RGB Image in the Wild

12/03/2018
by   Berk Kaya, et al.
0

In contrast to the current literature, we address the problem of estimating the spectrum from a single common trichromatic RGB image obtained under unconstrained settings (e.g. unknown camera parameters, unknown scene radiance, unknown scene contents). For this we use a reference spectrum as provided by a hyperspectral image camera, and propose efficient deep learning solutions for sensitivity function estimation and spectral reconstruction from a single RGB image. We further expand the concept of spectral reconstruction such that to work for RGB images taken in the wild and propose a solution based on a convolutional network conditioned on the estimated sensitivity function. Besides the proposed solutions, we study also generic and sensitivity specialized models and discuss their limitations. We achieve state-of-the-art competitive results on the standard example-based spectral reconstruction benchmarks: ICVL, CAVE, NUS and NTIRE. Moreover, our experiments show that, for the first time, accurate spectral estimation from a single RGB image in the wild is within our reach.

READ FULL TEXT

page 1

page 2

page 4

page 6

page 7

page 8

research
05/07/2020

NTIRE 2020 Challenge on Spectral Reconstruction from an RGB Image

This paper reviews the second challenge on spectral reconstruction from ...
research
06/01/2021

Separated-Spectral-Distribution Estimation Based on Bayesian Inference with Single RGB Camera

In this paper, we propose a novel method for separately estimating spect...
research
08/15/2021

Semantic-embedded Unsupervised Spectral Reconstruction from Single RGB Images in the Wild

This paper investigates the problem of reconstructing hyperspectral (HS)...
research
12/18/2020

Spectral Reflectance Estimation Using Projector with Unknown Spectral Power Distribution

A lighting-based multispectral imaging system using an RGB camera and a ...
research
08/28/2019

BioFaceNet: Deep Biophysical Face Image Interpretation

In this paper we present BioFaceNet, a deep CNN that learns to decompose...
research
04/04/2023

Learning to Recover Spectral Reflectance from RGB Images

This paper tackles spectral reflectance recovery (SRR) from RGB images. ...
research
03/05/2018

Spectral reflectance estimation from one RGB image using self-interreflections in a concave object

Light interreflections occurring in a concave object generate a color gr...

Please sign up or login with your details

Forgot password? Click here to reset