DeepAI
Log In Sign Up

Learning Adaptive Sampling and Reconstruction for Volume Visualization

07/20/2020
by   Sebastian Weiss, et al.
0

A central challenge in data visualization is to understand which data samples are required to generate an image of a data set in which the relevant information is encoded. In this work, we make a first step towards answering the question of whether an artificial neural network can predict where to sample the data with higher or lower density, by learning of correspondences between the data, the sampling patterns and the generated images. We introduce a novel neural rendering pipeline, which is trained end-to-end to generate a sparse adaptive sampling structure from a given low-resolution input image, and reconstructs a high-resolution image from the sparse set of samples. For the first time, to the best of our knowledge, we demonstrate that the selection of structures that are relevant for the final visual representation can be jointly learned together with the reconstruction of this representation from these structures. Therefore, we introduce differentiable sampling and reconstruction stages, which can leverage back-propagation based on supervised losses solely on the final image. We shed light on the adaptive sampling patterns generated by the network pipeline and analyze its use for volume visualization including isosurface and direct volume rendering.

READ FULL TEXT

page 1

page 8

page 10

page 11

page 12

page 15

page 16

10/14/2022

Deep Learning based Super-Resolution for Medical Volume Visualization with Direct Volume Rendering

Modern-day display systems demand high-quality rendering. However, rende...
07/21/2009

Image Sampling with Quasicrystals

We investigate the use of quasicrystals in image sampling. Quasicrystals...
06/15/2019

Volumetric Isosurface Rendering with Deep Learning-Based Super-Resolution

Rendering an accurate image of an isosurface in a volumetric field typic...
06/09/2021

Deep Direct Volume Rendering: Learning Visual Feature Mappings From Exemplary Images

Volume Rendering is an important technique for visualizing three-dimensi...
09/14/2020

VC-Net: Deep Volume-Composition Networks for Segmentation and Visualization of Highly Sparse and Noisy Image Data

The motivation of our work is to present a new visualization-guided comp...
07/21/2022

AdaNeRF: Adaptive Sampling for Real-time Rendering of Neural Radiance Fields

Novel view synthesis has recently been revolutionized by learning neural...
09/27/2022

Im2Oil: Stroke-Based Oil Painting Rendering with Linearly Controllable Fineness Via Adaptive Sampling

This paper proposes a novel stroke-based rendering (SBR) method that tra...