DeepAI AI Chat
Log In Sign Up

SplitSR: An End-to-End Approach to Super-Resolution on Mobile Devices

01/20/2021
by   Xin Liu, et al.
UNIVERSITY OF TORONTO
University of Washington
63

Super-resolution (SR) is a coveted image processing technique for mobile apps ranging from the basic camera apps to mobile health. Existing SR algorithms rely on deep learning models with significant memory requirements, so they have yet to be deployed on mobile devices and instead operate in the cloud to achieve feasible inference time. This shortcoming prevents existing SR methods from being used in applications that require near real-time latency. In this work, we demonstrate state-of-the-art latency and accuracy for on-device super-resolution using a novel hybrid architecture called SplitSR and a novel lightweight residual block called SplitSRBlock. The SplitSRBlock supports channel-splitting, allowing the residual blocks to retain spatial information while reducing the computation in the channel dimension. SplitSR has a hybrid design consisting of standard convolutional blocks and lightweight residual blocks, allowing people to tune SplitSR for their computational budget. We evaluate our system on a low-end ARM CPU, demonstrating both higher accuracy and up to 5 times faster inference than previous approaches. We then deploy our model onto a smartphone in an app called ZoomSR to demonstrate the first-ever instance of on-device, deep learning-based SR. We conducted a user study with 15 participants to have them assess the perceived quality of images that were post-processed by SplitSR. Relative to bilinear interpolation – the existing standard for on-device SR – participants showed a statistically significant preference when looking at both images (Z=-9.270, p<0.01) and text (Z=-6.486, p<0.01).

READ FULL TEXT

page 6

page 14

page 15

12/15/2022

NAWQ-SR: A Hybrid-Precision NPU Engine for Efficient On-Device Super-Resolution

In recent years, image and video delivery systems have begun integrating...
11/09/2020

MPRNet: Multi-Path Residual Network for Lightweight Image Super Resolution

Lightweight super resolution networks have extremely importance for real...
12/16/2018

Efficient Super Resolution Using Binarized Neural Network

Deep convolutional neural networks (DCNNs) have recently demonstrated hi...
08/21/2019

MobiSR: Efficient On-Device Super-Resolution through Heterogeneous Mobile Processors

In recent years, convolutional networks have demonstrated unprecedented ...
11/20/2020

On-Device Text Image Super Resolution

Recent research on super-resolution (SR) has witnessed major development...
03/16/2023

A High-Performance Accelerator for Super-Resolution Processing on Embedded GPU

Recent years have witnessed impressive progress in super-resolution (SR)...
05/17/2021

Real-Time Video Super-Resolution on Smartphones with Deep Learning, Mobile AI 2021 Challenge: Report

Video super-resolution has recently become one of the most important mob...

Code Repositories

SplitSR

SplitSR: An End-to-End Approach to Super-Resolution on Mobile Devices (Unofficial Implementation)


view repo