Neural Imaging Pipelines - the Scourge or Hope of Forensics?

02/27/2019
by   Pawel Korus, et al.
10

Forensic analysis of digital photographs relies on intrinsic statistical traces introduced at the time of their acquisition or subsequent editing. Such traces are often removed by post-processing (e.g., down-sampling and re-compression applied upon distribution in the Web) which inhibits reliable provenance analysis. Increasing adoption of computational methods within digital cameras further complicates the process and renders explicit mathematical modeling infeasible. While this trend challenges forensic analysis even in near-acquisition conditions, it also creates new opportunities. This paper explores end-to-end optimization of the entire image acquisition and distribution workflow to facilitate reliable forensic analysis at the end of the distribution channel, where state-of-the-art forensic techniques fail. We demonstrate that a neural network can be trained to replace the entire photo development pipeline, and jointly optimized for high-fidelity photo rendering and reliable provenance analysis. Such optimized neural imaging pipeline allowed us to increase image manipulation detection accuracy from approx. 45 to over 90 to digital watermarks, which facilitate subsequent manipulation detection. Analysis of performance trade-offs indicates that most of the gains can be obtained with only minor distortion. The findings encourage further research towards building more reliable imaging pipelines with explicit provenance-guaranteeing properties.

READ FULL TEXT

page 5

page 6

page 8

page 9

page 11

page 16

page 19

page 20

research
12/04/2018

Content Authentication for Neural Imaging Pipelines: End-to-end Optimization of Photo Provenance in Complex Distribution Channels

Forensic analysis of digital photo provenance relies on intrinsic traces...
research
04/04/2020

Empirical Evaluation of PRNU Fingerprint Variation for Mismatched Imaging Pipelines

We assess the variability of PRNU-based camera fingerprints with mismatc...
research
03/05/2021

Effects of Image Compression on Face Image Manipulation Detection: A Case Study on Facial Retouching

In the past years, numerous methods have been introduced to reliably det...
research
08/30/2021

Learning JPEG Compression Artifacts for Image Manipulation Detection and Localization

Detecting and localizing image manipulation are necessary to counter mal...
research
11/10/2020

End-to-end optimized image compression for machines, a study

An increasing share of image and video content is analyzed by machines r...
research
07/31/2023

DRAW: Defending Camera-shooted RAW against Image Manipulation

RAW files are the initial measurement of scene radiance widely used in m...

Please sign up or login with your details

Forgot password? Click here to reset