DeepAI AI Chat
Log In Sign Up

SplitSR: An End-to-End Approach to Super-Resolution on Mobile Devices

by   Xin Liu, et al.
University of Washington

Super-resolution (SR) is a coveted image processing technique for mobile apps ranging from the basic camera apps to mobile health. Existing SR algorithms rely on deep learning models with significant memory requirements, so they have yet to be deployed on mobile devices and instead operate in the cloud to achieve feasible inference time. This shortcoming prevents existing SR methods from being used in applications that require near real-time latency. In this work, we demonstrate state-of-the-art latency and accuracy for on-device super-resolution using a novel hybrid architecture called SplitSR and a novel lightweight residual block called SplitSRBlock. The SplitSRBlock supports channel-splitting, allowing the residual blocks to retain spatial information while reducing the computation in the channel dimension. SplitSR has a hybrid design consisting of standard convolutional blocks and lightweight residual blocks, allowing people to tune SplitSR for their computational budget. We evaluate our system on a low-end ARM CPU, demonstrating both higher accuracy and up to 5 times faster inference than previous approaches. We then deploy our model onto a smartphone in an app called ZoomSR to demonstrate the first-ever instance of on-device, deep learning-based SR. We conducted a user study with 15 participants to have them assess the perceived quality of images that were post-processed by SplitSR. Relative to bilinear interpolation – the existing standard for on-device SR – participants showed a statistically significant preference when looking at both images (Z=-9.270, p<0.01) and text (Z=-6.486, p<0.01).


page 6

page 14

page 15


NAWQ-SR: A Hybrid-Precision NPU Engine for Efficient On-Device Super-Resolution

In recent years, image and video delivery systems have begun integrating...

MPRNet: Multi-Path Residual Network for Lightweight Image Super Resolution

Lightweight super resolution networks have extremely importance for real...

Efficient Super Resolution Using Binarized Neural Network

Deep convolutional neural networks (DCNNs) have recently demonstrated hi...

MobiSR: Efficient On-Device Super-Resolution through Heterogeneous Mobile Processors

In recent years, convolutional networks have demonstrated unprecedented ...

On-Device Text Image Super Resolution

Recent research on super-resolution (SR) has witnessed major development...

A High-Performance Accelerator for Super-Resolution Processing on Embedded GPU

Recent years have witnessed impressive progress in super-resolution (SR)...

Real-Time Video Super-Resolution on Smartphones with Deep Learning, Mobile AI 2021 Challenge: Report

Video super-resolution has recently become one of the most important mob...

Code Repositories


SplitSR: An End-to-End Approach to Super-Resolution on Mobile Devices (Unofficial Implementation)

view repo