DeepAI AI Chat
Log In Sign Up

CARAFE++: Unified Content-Aware ReAssembly of FEatures

by   Jiaqi Wang, et al.
Nanyang Technological University
The Chinese University of Hong Kong

Feature reassembly, i.e. feature downsampling and upsampling, is a key operation in a number of modern convolutional network architectures, e.g., residual networks and feature pyramids. Its design is critical for dense prediction tasks such as object detection and semantic/instance segmentation. In this work, we propose unified Content-Aware ReAssembly of FEatures (CARAFE++), a universal, lightweight and highly effective operator to fulfill this goal. CARAFE++ has several appealing properties: (1) Unlike conventional methods such as pooling and interpolation that only exploit sub-pixel neighborhood, CARAFE++ aggregates contextual information within a large receptive field. (2) Instead of using a fixed kernel for all samples (e.g. convolution and deconvolution), CARAFE++ generates adaptive kernels on-the-fly to enable instance-specific content-aware handling. (3) CARAFE++ introduces little computational overhead and can be readily integrated into modern network architectures. We conduct comprehensive evaluations on standard benchmarks in object detection, instance/semantic segmentation and image inpainting. CARAFE++ shows consistent and substantial gains across all the tasks (2.5 APmask, 1.94 overhead. It shows great potential to serve as a strong building block for modern deep networks.


page 1

page 3

page 4

page 6

page 7

page 9

page 10

page 12


CARAFE: Content-Aware ReAssembly of FEatures

Feature upsampling is a key operation in a number of modern convolutiona...

MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features

In this work, we tackle the problem of instance segmentation, the task o...

SpatialFlow: Bridging All Tasks for Panoptic Segmentation

The newly proposed panoptic segmentation task, which aims to encompass t...

Tug-of-War: Observations on Unified Content Handling

Modern applications and Operating Systems vary greatly with respect to h...

SAPA: Similarity-Aware Point Affiliation for Feature Upsampling

We introduce point affiliation into feature upsampling, a notion that de...

Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Network

One of recent trends [30, 31, 14] in network architec- ture design is st...

Path Aggregation Network for Instance Segmentation

The way that information propagates in neural networks is of great impor...