Local-Global Temporal Difference Learning for Satellite Video Super-Resolution

by   Yi Xiao, et al.

Optical-flow-based and kernel-based approaches have been widely explored for temporal compensation in satellite video super-resolution (VSR). However, these techniques involve high computational consumption and are prone to fail under complex motions. In this paper, we proposed to exploit the well-defined temporal difference for efficient and robust temporal compensation. To fully utilize the temporal information within frames, we separately modeled the short-term and long-term temporal discrepancy since they provide distinctive complementary properties. Specifically, a short-term temporal difference module is designed to extract local motion representations from residual maps between adjacent frames, which provides more clues for accurate texture representation. Meanwhile, the global dependency in the entire frame sequence is explored via long-term difference learning. The differences between forward and backward segments are incorporated and activated to modulate the temporal feature, resulting in holistic global compensation. Besides, we further proposed a difference compensation unit to enrich the interaction between the spatial distribution of the target frame and compensated results, which helps maintain spatial consistency while refining the features to avoid misalignment. Extensive objective and subjective evaluation of five mainstream satellite videos demonstrates that the proposed method performs favorably for satellite VSR. Code will be available at <https://github.com/XY-boy/TDMVSR>


page 1

page 2

page 4

page 5

page 8

page 9

page 10

page 13


Temporal Modulation Network for Controllable Space-Time Video Super-Resolution

Space-time video super-resolution (STVSR) aims to increase the spatial a...

Look Back and Forth: Video Super-Resolution with Explicit Temporal Difference Modeling

Temporal modeling is crucial for video super-resolution. Most of the vid...

TDN: Temporal Difference Networks for Efficient Action Recognition

Temporal modeling still remains challenging for action recognition in vi...

Temporal Consistency Learning of inter-frames for Video Super-Resolution

Video super-resolution (VSR) is a task that aims to reconstruct high-res...

EFENet: Reference-based Video Super-Resolution with Enhanced Flow Estimation

In this paper, we consider the problem of reference-based video super-re...

Boosting Video Super Resolution with Patch-Based Temporal Redundancy Optimization

The success of existing video super-resolution (VSR) algorithms stems ma...

FTM: A Frame-level Timeline Modeling Method for Temporal Graph Representation Learning

Learning representations for graph-structured data is essential for grap...

Please sign up or login with your details

Forgot password? Click here to reset