An Energy-Efficient Edge Computing Paradigm for Convolution-based Image Upsampling

07/15/2021
by   Ian Colbert, et al.
0

A novel energy-efficient edge computing paradigm is proposed for real-time deep learning-based image upsampling applications. State-of-the-art deep learning solutions for image upsampling are currently trained using either resize or sub-pixel convolution to learn kernels that generate high fidelity images with minimal artifacts. However, performing inference with these learned convolution kernels requires memory-intensive feature map transformations that dominate time and energy costs in real-time applications. To alleviate this pressure on memory bandwidth, we confine the use of resize or sub-pixel convolution to training in the cloud by transforming learned convolution kernels to deconvolution kernels before deploying them for inference as a functionally equivalent deconvolution. These kernel transformations, intended as a one-time cost when shifting from training to inference, enable a systems designer to use each algorithm in their optimal context by preserving the image fidelity learned when training in the cloud while minimizing data transfer penalties during inference at the edge. We also explore existing variants of deconvolution inference algorithms and introduce a novel variant for consideration. We analyze and compare the inference properties of convolution-based upsampling algorithms using a quantitative model of incurred time and energy costs and show that using deconvolution for inference at the edge improves both system latency and energy efficiency when compared to their sub-pixel or resize convolution counterparts.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/10/2017

Checkerboard artifact free sub-pixel convolution: A note on sub-pixel convolution, resize convolution and convolution resize

The most prominent problem associated with the deconvolution layer is th...
research
11/30/2017

Properties on n-dimensional convolution for image deconvolution

Convolution system is linear and time invariant, and can describe the op...
research
06/28/2022

LiteCON: An All-Photonic Neuromorphic Accelerator for Energy-efficient Deep Learning (Preprint)

Deep learning is highly pervasive in today's data-intensive era. In part...
research
01/18/2022

Hardware-Efficient Deconvolution-Based GAN for Edge Computing

Generative Adversarial Networks (GAN) are cutting-edge algorithms for ge...
research
05/06/2020

AutoScale: Optimizing Energy Efficiency of End-to-End Edge Inference under Stochastic Variance

Deep learning inference is increasingly run at the edge. As the programm...
research
07/25/2019

HUGE2: a Highly Untangled Generative-model Engine for Edge-computing

As a type of prominent studies in deep learning, generative models have ...
research
06/24/2023

Deep learning-based deconvolution for interferometric radio transient reconstruction

Radio astronomy is currently thriving with new large ground-based radio ...

Please sign up or login with your details

Forgot password? Click here to reset