detrtensorflow
Tensorflow implementation of DETR : Object Detection with Transformers
view repo
We present a new method that views object detection as a direct set prediction problem. Our approach streamlines the detection pipeline, effectively removing the need for many handdesigned components like a nonmaximum suppression procedure or anchor generation that explicitly encode our prior knowledge about the task. The main ingredients of the new framework, called DEtection TRansformer or DETR, are a setbased global loss that forces unique predictions via bipartite matching, and a transformer encoderdecoder architecture. Given a fixed small set of learned object queries, DETR reasons about the relations of the objects and the global image context to directly output the final set of predictions in parallel. The new model is conceptually simple and does not require a specialized library, unlike many other modern detectors. DETR demonstrates accuracy and runtime performance on par with the wellestablished and highlyoptimized Faster RCNN baseline on the challenging COCO object detection dataset. Moreover, DETR can be easily generalized to produce panoptic segmentation in a unified manner. We show that it significantly outperforms competitive baselines. Training code and pretrained models are available at https://github.com/facebookresearch/detr.
READ FULL TEXT VIEW PDFTensorflow implementation of DETR : Object Detection with Transformers
The PyTorch reimplement of the official DETR.
TF 2.x implementation of DETR (EndtoEnd Object Detection with Transformers, ECCV 2020).
bla bla lba
None
The goal of object detection is to predict a set of bounding boxes and category labels for each object of interest. Modern detectors address this set prediction task in an indirect way, by defining surrogate regression and classification problems on a large set of proposals [37, 5], anchors [23], or window centers [53, 46]
. Their performances are significantly influenced by postprocessing steps to collapse nearduplicate predictions, by the design of the anchor sets and by the heuristics that assign target boxes to anchors
[52]. To simplify these pipelines, we propose a direct set prediction approach to bypass the surrogate tasks. This endtoend philosophy has led to significant advances in complex structured prediction tasks such as machine translation or speech recognition, but not yet in object detection: previous attempts [43, 16, 4, 39] either add other forms of prior knowledge, or have not proven to be competitive with strong baselines on challenging benchmarks. This paper aims to bridge this gap.We streamline the training pipeline by viewing object detection as a direct set prediction problem. We adopt an encoderdecoder architecture based on transformers [47], a popular architecture for sequence prediction. The selfattention mechanisms of transformers, which explicitly model all pairwise interactions between elements in a sequence, make these architectures particularly suitable for specific constraints of set prediction such as removing duplicate predictions.
Our DEtection TRansformer (DETR, see Figure 1
) predicts all objects at once, and is trained endtoend with a set loss function which performs bipartite matching between predicted and groundtruth objects. DETR simplifies the detection pipeline by dropping multiple handdesigned components that encode prior knowledge, like spatial anchors or nonmaximal suppression. Unlike most existing detection methods, DETR doesn’t require any customized layers, and thus can be reproduced easily in any framework that contains standard CNN and transformer classes.
^{1}^{1}1In our work we use standard implementations of Transformers [47] and ResNet [15]backbones from standard deep learning libraries.
.Compared to most previous work on direct set prediction, the main features of DETR are the conjunction of the bipartite matching loss and transformers with (nonautoregressive) parallel decoding [29, 12, 10, 8]. In contrast, previous work focused on autoregressive decoding with RNNs [43, 41, 30, 36, 42]. Our matching loss function uniquely assigns a prediction to a ground truth object, and is invariant to a permutation of predicted objects, so we can emit them in parallel.
We evaluate DETR on one of the most popular object detection datasets, COCO [24], against a very competitive Faster RCNN baseline [37]. Faster RCNN has undergone many design iterations and its performance was greatly improved since the original publication. Our experiments show that our new model achieves comparable performances. More precisely, DETR demonstrates significantly better performance on large objects, a result likely enabled by the nonlocal computations of the transformer. It obtains, however, lower performances on small objects. We expect that future work will improve this aspect in the same way the development of FPN [22] did for Faster RCNN.
Training settings for DETR differ from standard object detectors in multiple ways. The new model requires extralong training schedule and benefits from auxiliary decoding losses in the transformer. We thoroughly explore what components are crucial for the demonstrated performance.
The design ethos of DETR easily extend to more complex tasks. In our experiments, we show that a simple segmentation head trained on top of a pretrained DETR outperfoms competitive baselines on Panoptic Segmentation [19], a challenging pixellevel recognition task that has recently gained popularity.
Our work build on prior work in several domains: bipartite matching losses for set prediction, encoderdecoder architectures based on the transformer, parallel decoding, and object detection methods.
There is no canonical deep learning model to directly predict sets. The basic set prediction task is multilabel classification (see e.g., [40, 33]
for references in the context of computer vision) for which the baseline approach, onevsrest, does not apply to problems such as detection where there is an underlying structure between elements (i.e., nearidentical boxes). The first difficulty in these tasks is to avoid nearduplicates. Most current detectors use postprocessings such as nonmaximal suppression to address this issue, but direct set prediction are postprocessingfree. They need global inference schemes that model interactions between all predicted elements to avoid redundancy. For constantsize set prediction, dense fully connected networks
[9]are sufficient but costly. A general approach is to use autoregressive sequence models such as recurrent neural networks
[48]. In all cases, the loss function should be invariant by a permutation of the predictions. The usual solution is to design a loss based on the Hungarian algorithm [20], to find a bipartite matching between groundtruth and prediction. This enforces permutationinvariance, and guarantees that each target element has a unique match. We follow the bipartite matching loss approach. In contrast to most prior work however, we step away from autoregressive models and use transformers with parallel decoding, which we describe below.
Transformers were introduced by Vaswani et al. [47] as a new attentionbased building block for machine translation. Attention mechanisms [2] are neural network layers that aggregate information from the entire input sequence. Transformers introduced selfattention layers, which, similarly to NonLocal Neural Networks [49]
, scan through each element of a sequence and update it by aggregating information from the whole sequence. One of the main advantages of attentionbased models is their global computations and perfect memory, which makes them more suitable than RNNs on long sequences. Transformers are now replacing RNNs in many problems in natural language processing, speech processing and computer vision
[8, 27, 45, 34, 31].Transformers were first used in autoregressive models, following early sequencetosequence models [44], generating output tokens one by one. However, the prohibitive inference cost (proportional to output length, and hard to batch) lead to the development of parallel sequence generation, in the domains of audio [29], machine translation [12, 10], word representation learning [8], and more recently speech recognition [6]. We also combine transformers and parallel decoding for their suitable tradeoff between computational cost and the ability to perform the global computations required for set prediction.
Most modern object detection methods make predictions relative to some initial guesses. Twostage detectors [37, 5] predict boxes w.r.t. proposals, whereas singlestage methods make predictions w.r.t. anchors [23] or a grid of possible object centers [53, 46]. Recent work [52] demonstrate that the final performance of these systems heavily depends on the exact way these initial guesses are set. In our model we are able to remove this handcrafted process and streamline the detection process by directly predicting the set of detections with absolute box prediction w.r.t. the input image rather than an anchor.
Several object detectors [9, 25, 35] used the bipartite matching loss. However, in these early deep learning models, the relation between different prediction was modeled with convolutional or fullyconnected layers only and a handdesigned NMS postprocessing can improve their performance. More recent detectors [37, 23, 53] use nonunique assignment rules between ground truth and predictions together with an NMS.
Learnable NMS methods [16, 4] and relation networks [17] explicitly model relations between different predictions with attention. Using direct set losses, they do not require any postprocessing steps. However, these methods employ additional handcrafted context features like proposal box coordinates to model relations between detections efficiently, while we look for solutions that reduce the prior knowledge encoded in the model.
Closest to our approach are endtoend set predictions for object detection [43] and instance segmentation [41, 30, 36, 42]. Similarly to us, they use bipartitematching losses with encoderdecoder architectures based on CNN activations to directly produce a set of bounding boxes. These approaches, however, were only evaluated on small datasets and not against modern baselines. In particular, they are based on autoregressive models (more precisely RNNs), so they do not leverage the recent transformers with parallel decoding.
Two ingredients are essential for direct set predictions in detection: (1) a set prediction loss that forces unique matching between predicted and ground truth boxes; (2) an architecture that predicts (in a single pass) a set of objects and models their relation. We describe our architecture in detail in Figure 2.
DETR infers a fixedsize set of predictions, in a single pass through the decoder, where is set to be significantly larger than the typical number of objects in an image. One of the main difficulties of training is to score predicted objects (class, position, size) with respect to the ground truth. Our loss produces an optimal bipartite matching between predicted and ground truth objects, and then optimize objectspecific (bounding box) losses.
Let us denote by the ground truth set of objects, and the set of predictions. Assuming is larger than the number of objects in the image, we consider also as a set of size padded with (no object). To find a bipartite matching between these two sets we search for a permutation of elements with the lowest cost:
(1) 
where is a pairwise matching cost between ground truth and a prediction with index . This optimal assignment is computed efficiently with the Hungarian algorithm, following prior work (e.g. [43]).
The matching cost takes into account both the class prediction and the similarity of predicted and ground truth boxes. Each element of the ground truth set can be seen as a where is the target class label (which may be ) and
is a vector that defines ground truth box center coordinates and its height and width relative to the image size. For the prediction with index
we define probability of class
as and the predicted box as . With these notations we define as .This procedure of finding matching plays the same role as the heuristic assignment rules used to match proposal [37] or anchors [22] to ground truth objects in modern detectors. The main difference is that we need to find onetoone matching for direct set prediction without duplicates.
The second step is to compute the loss function, the Hungarian loss for all pairs matched in the previous step. We define the loss similarly to the losses of common object detectors, i.e. a linear combination of a negative loglikelihood for class prediction and a box loss defined later:
(2) 
where is the optimal assignment computed in the first step (1). In practice, we downweight the logprobability term when by a factor to account for class imbalance. This is analogous to how Faster RCNN training procedure balances positive/negative proposals by subsampling [37]. Notice that the matching cost between an object and doesn’t depend on the prediction, which means that in that case the cost is a constant. In the matching cost we use probabilities instead of logprobabilities. This makes the class prediction term commensurable to (described below), and we observed better empirical performances.
The second part of the matching cost and the Hungarian loss is that scores the bounding boxes. Unlike many detectors that do box predictions as a w.r.t. some initial guesses, we make box predictions directly. While such approach simplify the implementation it poses an issue with relative scaling of the loss. The most commonlyused loss will have different scales for small and large boxes even if their relative errors are similar. To mitigate this issue we use a linear combination of the loss and the generalized IoU loss [38] that is scaleinvariant. Overall, our box loss is defined as where
are hyperparameters. These two losses are normalized by the number of objects inside the batch.
The overall DETR architecture is surprisingly simple and depicted in Figure 2. It contains three main components, which we describe below: a CNN backbone to extract a compact feature representation, an encoderdecoder transformer, and a simple feed forward network (FFN) that makes the final detection prediction.
Unlike many modern detectors, DETR can be implemented in any deep learning framework that provides a common CNN backbone and a transformer architecture implementation with just a few hundred lines. Inference code for DETR can be implemented in less than 50 lines in PyTorch
[32]. We hope that the simplicity of our method will attract new researchers to the detection community.Starting from the initial image (with color channels^{2}^{2}2The input images are batched together, applying 0padding adequately to ensure they all have the same dimensions as the largest image of the batch.), a conventional CNN backbone generates a lowerresolution activation map . Typical values we use are and .
First, a 1x1 convolution reduces the channel dimension of the highlevel activation map from to a smaller dimension . creating a new feature map . The encoder expects a sequence as input, hence we collapse the spatial dimensions of into one dimension, resulting in a feature map. Each encoder layer has a standard architecture and consists of a multihead selfattention module and a feed forward network (FFN). Since the transformer architecture is permutationinvariant, we supplement it with fixed positional encodings [31, 3] that are added to the input of each attention layer. We defer to the supplementary material the detailed definition of the architecture, which follows the one described in [47].
The decoder follows the standard architecture of the transformer, transforming embeddings of size using multiheaded self and encoderdecoder attention mechanisms. The difference with the original transformer is that our model decodes the objects in parallel at each decoder layer, while Vaswani et al. [47] use an autoregressive model that predicts the output sequence one element at a time. We refer the reader unfamiliar with the concepts to the supplementary material. Since the decoder is also permutationinvariant, the input embeddings must be different to produce different results. These input embeddings are learnt positional encodings that we refer to as object queries, and similarly to the encoder, we add them to the input of each attention layer. The object queries are transformed into an output embedding by the decoder. They are then independently decoded into box coordinates and class labels by a feed forward network (described in the next subsection), resulting final predictions. Using self and encoderdecoder attention over these embeddings, the model globally reasons about all objects together using pairwise relations between them, while being able to use the whole image as context.
The final prediction is computed by a 3layer perceptron with ReLU activation function and hidden dimension
, and a linear projection layer. The FFN predicts the normalized center coordinates, height and width of the box w.r.t. the input image, and the linear layer predicts the class label using a softmax function. Since we predict a fixedsize set of bounding boxes, where is usually much larger than the actual number of objects of interest in an image, an additional special class label is used to represent that no object is detected within a slot. This class plays a similar role to the “background” class in the standard object detection approaches.We found helpful to use auxiliary losses [1] in decoder during training, especially to help the model output the correct number of objects of each class. We add prediction FFNs and Hungarian loss after each decoder layer. All predictions FFNs share their parameters. We use an additional shared layernorm to normalize the input to the prediction FFNs from different decoder layers.
We show that DETR achieves competitive results compared to Faster RCNN in quantitative evaluation on COCO. Then, we provide a detailed ablation study of the architecture and loss, with insights and qualitative results. Finally, to show that DETR is a versatile and extensible model, we present results on panoptic segmentation, training only a small extension on a fixed DETR model. We provide code and pretrained models to reproduce our experiments at https://github.com/facebookresearch/detr.
We perform experiments on COCO 2017 detection and panoptic segmentation datasets [24, 18]
, containing 118k training images and 5k validation images. Each image is annotated with bounding boxes and panoptic segmentation. There are 7 instances per image on average, up to 63 instances in a single image in training set, ranging from small to large on the same images. If not specified, we report AP as bbox AP, the integral metric over multiple thresholds. For comparison with Faster RCNN we report validation AP at the last training epoch, for ablations we report median over validation results from the last 10 epochs.
We train DETR with AdamW [26] setting the initial transformer’s learning rate to , the backbone’s to , and weight decay to . All transformer weights are initialized with Xavier init [11]
, and the backbone is with ImageNetpretrained ResNet model
[15] from torchvision with frozen batchnorm layers. We report results with two different backbones: a ResNet50 and a ResNet101. The corresponding models are called respectively DETR and DETRR101. Following [21], we also increase the feature resolution by adding a dilation to the last stage of the backbone and removing a stride from the first convolution of this stage. The corresponding models are called respectively DETRDC5 and DETRDC5R101 (dilated C5 stage). This modification increases the resolution by a factor of two, thus improving performance for small objects, at the cost of a 16x higher cost in the selfattentions of the encoder, leading to an overall 2x increase in computational cost. A full comparison of FLOPs of these models and Faster RCNN is given in Table
1.We use scale augmentation, resizing the input images such that the shortest side is at least 480 and at most 800 pixels while the longest at most 1333 [50]. To help learning global relationships through the selfattention of the encoder, we also apply random crop augmentations during training, improving the performance by approximately 1 AP. Specifically, a train image is cropped with probability 0.5 to a random rectangular patch which is then resized again to 8001333. The transformer is trained with default dropout of 0.1. At inference time, some slots predict empty class. To optimize for AP, we override the prediction of these slots with the second highest scoring class, using the corresponding confidence. This improves AP by 2 points compared to filtering out empty slots. Other training hyperparameters can be found in section 0.A.4. For our ablation experiments we use training schedule of 300 epochs with a learning rate drop by a factor of 10 after 200 epochs, where a single epoch is a pass over all training images once. Training the baseline model for 300 epochs on 16 V100 GPUs takes 3 days, with 4 images per GPU (hence a total batch size of 64). For the longer schedule used to compare with Faster RCNN we train for 500 epochs with learning rate drop after 400 epochs. This schedule adds 1.5 AP compared to the shorter schedule.
Model  GFLOPS/FPS  #params  AP  AP_{50}  AP_{75}  AP_{S}  AP_{M}  AP_{L} 

Faster RCNNDC5  320/16  166M  39.0  60.5  42.3  21.4  43.5  52.5 
Faster RCNNFPN  180/26  42M  40.2  61.0  43.8  24.2  43.5  52.0 
Faster RCNNR101FPN  246/20  60M  42.0  62.5  45.9  25.2  45.6  54.6 
Faster RCNNDC5+  320/16  166M  41.1  61.4  44.3  22.9  45.9  55.0 
Faster RCNNFPN+  180/26  42M  42.0  62.1  45.5  26.6  45.4  53.4 
Faster RCNNR101FPN+  246/20  60M  44.0  63.9  47.8  27.2  48.1  56.0 
DETR  86/28  41M  42.0  62.4  44.2  20.5  45.8  61.1 
DETRDC5  187/12  41M  43.3  63.1  45.9  22.5  47.3  61.1 
DETRR101  152/20  60M  43.5  63.8  46.4  21.9  48.0  61.8 
DETRDC5R101  253/10  60M  44.9  64.7  47.7  23.7  49.5  62.3 
Transformers are typically trained with Adam or Adagrad optimizers with very long training schedules and dropout, and this is true for DETR as well. Faster RCNN, however, is trained with SGD with minimal data augmentation and we are not aware of successful applications of Adam or dropout. Despite these differences we attempt to make a Faster RCNN baseline stronger. To align it with DETR, we add generalized IoU [38] to the box loss, the same random crop augmentation and long training known to improve results [13]. Results are presented in Table 1. In the top section we show Faster RCNN results from Detectron2 Model Zoo [50] for models trained with the 3x schedule. In the middle section we show results (with a “+”) for the same models but trained with the 9x schedule (109 epochs) and the described enhancements, which in total adds 12 AP. In the last section of Table 1 we show the results for multiple DETR models. To be comparable in the number of parameters we choose a model with 6 transformer and 6 decoder layers of width 256 with 8 attention heads. Like Faster RCNN with FPN this model has 41.3M parameters, out of which 23.5M are in ResNet50, and 17.8M are in the transformer. Even though both Faster RCNN and DETR are still likely to further improve with longer training, we can conclude that DETR can be competitive with Faster RCNN with the same number of parameters, achieving 42 AP on the COCO val subset. The way DETR achieves this is by improving AP_{L} (+7.8), however note that the model is still lagging behind in AP_{S} (5.5). DETRDC5 with the same number of parameters and similar FLOP count has higher AP, but is still significantly behind in AP_{S} too. Faster RCNN and DETR with ResNet101 backbone show comparable results as well.
Attention mechanisms in the transformer decoder are the key components which model relations between feature representations of different detections. In our ablation analysis, we explore how other components of our architecture and loss influence the final performance. For the study we choose ResNet50based DETR model with 6 encoder, 6 decoder layers and width 256. The model has 41.3M parameters, achieves 40.6 and 42.0 AP on short and long schedules respectively, and runs at 28 FPS, similarly to Faster RCNNFPN with the same backbone.
We evaluate the importance of global imagelevel selfattention by changing the number of encoder layers (Table 2). Without encoder layers, overall AP drops by 3.9 points, with a more significant drop of 6.0 AP on large objects. We hypothesize that, by using global scene reasoning, the encoder is important for disentangling objects. In Figure 3, we visualize the attention maps of the last encoder layer of a trained model, focusing on a few points in the image. The encoder seems to separate instances already, which likely simplifies object extraction and localization for the decoder.
#layers  GFLOPS/FPS  #params  AP  AP_{50}  AP_{S}  AP_{M}  AP_{L} 

0  76/28  33.4M  36.7  57.4  16.8  39.6  54.2 
3  81/25  37.4M  40.1  60.6  18.5  43.8  58.6 
6  86/23  41.3M  40.6  61.6  19.9  44.3  60.2 
12  95/20  49.2M  41.6  62.1  19.8  44.9  61.9 
We apply auxiliary losses after each decoding layer (see Section 3.2.5), hence, the prediction FFNs are trained by design to predict objects out of the outputs of every decoder layer. We analyze the importance of each decoder layer by evaluating the objects that would be predicted at each stage of the decoding (Fig. 5). Both AP and AP_{50} improve after every layer, totalling into a very significant +8.2/9.5 AP improvement between the first and the last layer. With its setbased loss, DETR does not need NMS by design. To verify this we run a standard NMS procedure with default parameters [50] for the outputs after each decoder. NMS improves performance for the predictions from the first decoder. This can be explained by the fact that a single decoding layer of the transformer is not able to compute any crosscorrelations between the output elements, and thus it is prone to making multiple predictions for the same object. In the second and subsequent layers, the selfattention mechanism over the activations allows the model to inhibit duplicate predictions. We observe that the improvement brought by NMS diminishes as depth increases. At the last layers, we observe a small loss in AP as NMS incorrectly removes true positive predictions.
Similarly to visualizing encoder attention, we visualize decoder attentions in Fig. 6, coloring attention maps for each predicted object in different colors. We observe that decoder attention is fairly local, meaning that it mostly attends to object extremities such as heads or legs. We hypothesise that after the encoder has separated instances via global attention, the decoder only needs to attend to the extremities to extract the class and object boundaries.
FFN inside tranformers can be seen as convolutional layers, making encoder similar to attention augmented convolutional networks [3]. We attempt to remove it completely leaving only attention in the transformer layers. By reducing the number of network parameters from 41.3M to 28.7M, leaving only 10.8M in the transformer, performance drops by 2.3 AP, we thus conclude that FFN are important for achieving good results.
There are two kinds of positional encodings in our model: spatial positional encodings and output positional encodings (object queries). We experiment with various combinations of fixed and learned encodings, results can be found in table 3. Output positional encodings are required and cannot be removed, so we experiment with either passing them once at decoder input or adding to queries at every decoder attention layer. In the first experiment we completely remove spatial positional encodings and pass output positional encodings at input and, interestingly, the model still achieves more than 32 AP, losing 7.8 AP to the baseline. Then, we pass fixed sine spatial positional encodings and the output encodings at input once, as in the original transformer [47], and find that this leads to 1.4 AP drop compared to passing the positional encodings directly in attention. Learned spatial encodings passed to the attentions give similar results. Surprisingly, we find that not passing any spatial encodings in the encoder only leads to a minor AP drop of 1.3 AP. When we pass the encodings to the attentions, they are shared across all layers, and the output encodings (object queries) are always learned.
spatial pos. enc.  output pos. enc.  

encoder  decoder  decoder  AP  AP_{50}  
none  none  learned at input  32.8  7.8  55.2  6.5 
sine at input  sine at input  learned at input  39.2  1.4  60.0  1.6 
learned at attn.  learned at attn.  learned at attn.  39.6  1.0  60.7  0.9 
none  sine at attn.  learned at attn.  39.3  1.3  60.3  1.4 
sine at attn.  sine at attn.  learned at attn.  40.6    61.6   
Given these ablations, we conclude that transformer components: the global selfattention in encoder, FFN, multiple decoder layers, and positional encodings, all significantly contribute to the final object detection performance.
To evaluate the importance of different components of the matching cost and the loss, we train several models turning them on and off. There are three components to the loss: classification loss, bounding box distance loss, and GIoU [38] loss. The classification loss is essential for training and cannot be turned off, so we train a model without bounding box distance loss, and a model without the GIoU loss, and compare with baseline, trained with all three losses. Results are presented in table 4.
class  GIoU  AP  AP_{50}  AP_{S}  AP_{M}  AP_{L}  

✓  ✓  35.8  4.8  57.3  4.4  13.7  39.8  57.9  
✓  ✓  39.9  0.7  61.6  0  19.9  43.2  57.9  
✓  ✓  ✓  40.6    61.6    19.9  44.3  60.2 
GIoU loss on its own accounts for most of the model performance, losing only 0.7 AP to the baseline with combined losses. Using without GIoU shows poor results. We only studied simple ablations of different losses (using the same weighting every time), but other means of combining them may achieve different results.
In Fig. 7 we visualize the boxes predicted by different slots for all images in COCO 2017 val set. DETR learns different specialization for each query slot. We observe that each slot has several modes of operation focusing on different areas and box sizes. In particular, all slots have the mode for predicting imagewide boxes (visible as the red dots aligned in the middle of the plot). We hypothesize that this is related to the distribution of objects in COCO.
Some classes in COCO are not well represented with many instances of the same class in the same image. For example, there is no image with more than 13 giraffes in the training set. We create a synthetic image^{3}^{3}3Base picture credit: https://www.piqsels.com/en/publicdomainphotojzlwu to verify the generalization ability of DETR (see Figure 5). Our model is able to find all 24 giraffes on the image which is clearly out of distribution. This experiment confirms that there is no strong classspecialization in each object query.
Panoptic segmentation [19] has recently attracted a lot of attention from the computer vision community. Similarly to the extension of Faster RCNN [37] to Mask RCNN [14], DETR can be naturally extended by adding a mask head on top of the decoder outputs. In this section we demonstrate that such a head can be used to produce panoptic segmentation [19] by treating stuff and thing classes in a unified way. We perform our experiments on the panoptic annotations of the COCO dataset that has 53 stuff categories in addition to 80 things categories.
We train DETR to predict boxes around both stuff and things classes on COCO, using the same recipe. Predicting boxes is required for the training to be possible, since the Hungarian matching is computed using distances between boxes. We also add a mask head which predicts a binary mask for each of the predicted boxes, see Figure 8. It takes as input the output of transformer decoder for each object and computes multihead (with heads) attention scores of this embedding over the output of the encoder, generating attention heatmaps per object in a small resolution. To make the final prediction and increase the resolution, an FPNlike architecture is used. We describe the architecture in more details in the supplement. The final resolution of the masks has stride 4 and each mask is supervised independently using the DICE/F1 loss [28] and Focal loss [23].
The mask head can be trained either jointly, or in a two steps process, where we train DETR for boxes only, then freeze all the weights and train only the mask head for 25 epochs. Experimentally, these two approaches give similar results, we report results using the latter method since it results in a shorter total wallclock time training.
To predict the final panoptic segmentation we simply use an argmax over the mask scores at each pixel, and assign the corresponding categories to the resulting masks. This procedure guarantees that the final masks have no overlaps and, therefore, DETR does not require a heuristic [19] that is often used to align different masks.
We train DETR, DETRDC5 and DETRR101 models following the recipe for bounding box detection to predict boxes around stuff and things classes in COCO dataset. The new mask head is trained for 25 epochs (see supplementary for details). During inference we first filter out the detection with a confidence below 85%, then compute the perpixel argmax to determine in which mask each pixel belongs. We then collapse different mask predictions of the same stuff category in one, and filter the empty ones (less than 4 pixels).
Model  Backbone  PQ  SQ  RQ  AP  

PanopticFPN++  R50  42.4  79.3  51.6  49.2  82.4  58.8  32.3  74.8  40.6  37.7 
UPSnet  R50  42.5  78.0  52.5  48.6  79.4  59.6  33.4  75.9  41.7  34.3 
UPSnetM  R50  43.0  79.1  52.8  48.9  79.7  59.7  34.1  78.2  42.3  34.3 
PanopticFPN++  R101  44.1  79.5  53.3  51.0  83.2  60.6  33.6  74.0  42.1  39.7 
DETR  R50  43.4  79.3  53.8  48.2  79.8  59.5  36.3  78.5  45.3  31.1 
DETRDC5  R50  44.6  79.8  55.0  49.4  80.5  60.6  37.3  78.7  46.5  31.9 
DETRR101  R101  45.1  79.9  55.5  50.5  80.9  61.7  37.0  78.5  46.0  33.0 
Qualitative results are shown in Figure 9. In table 5 we compare our unified panoptic segmenation approach with several established methods that treat things and stuff differently. We report the Panoptic Quality (PQ) and the breakdown on things () and stuff (). We also report the mask AP (computed on the things classes), before any panoptic posttreatment (in our case, before taking the pixelwise argmax). We show that DETR outperforms published results on COCOval 2017, as well as our strong PanopticFPN baseline (trained with same dataaugmentation as DETR, for fair comparison). The result breakdown shows that DETR is especially dominant on stuff classes, and we hypothesize that the global reasoning allowed by the encoder attention is the key element to this result. For things class, despite a severe deficit of up to 8 mAP compared to the baselines on the mask AP computation, DETR obtains competitive . We also evaluated our method on the test set of the COCO dataset, and obtained 46 PQ. We hope that our approach will inspire the exploration of fully unified models for panoptic segmentation in future work.
We presented DETR, a new design for object detection systems based on transformers and bipartite matching loss for direct set prediction. The approach achieves comparable results to an optimized Faster RCNN baseline on the challenging COCO dataset. DETR is straightforward to implement and has a flexible architecture that is easily extensible to panoptic segmentation, with competitive results. In addition, it achieves significantly better performance on large objects than Faster RCNN, likely thanks to the processing of global information performed by the selfattention.
This new design for detectors also comes with new challenges, in particular regarding training, optimization and performances on small objects. Current detectors required several years of improvements to cope with similar issues, and we expect future work to successfully address them for DETR.
We thank Sainbayar Sukhbaatar, Piotr Bojanowski, Natalia Neverova, David LopezPaz, Guillaume Lample, Danielle Rothermel, Ross Girshick, Xinlei Chen and the whole Facebook AI Research Paris team for discussions and advices without which this work would not be possible.
AlRfou, R., Choe, D., Constant, N., Guo, M., Jones, L.: Characterlevel language modeling with deeper selfattention. In: AAAI Conference on Artificial Intelligence (2019)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: ICLR (2015)
Milletari, F., Navab, N., Ahmadi, S.A.: Vnet: Fully convolutional neural networks for volumetric medical image segmentation. In: 3DV (2016)
Since our model is based on the Transformer architecture, we remind here the general form of attention mechanisms we use for exhaustivity. The attention mechanism follows [47], except for the details of positional encodings (see Equation 8) that follows [7].
The general form of multihead attention with heads of dimension is a function with the following signature (using
, and giving matrix/tensors sizes in underbrace)
(3) 
where is the query sequence of length , is the keyvalue sequence of length (with the same number of channels for simplicity of exposition), is the weight tensor to compute the socalled query, key and value embeddings, and is a projection matrix. The output is the same size as the query sequence. To fix the vocabulary before giving details, multihead selfattention (mhsattn) is the special case , i.e.
(4) 
The multihead attention is simply the concatenation of single attention heads followed by a projection with . The common practice [47]
is to use residual connections, dropout and layer normalization. In other words, denoting
and the concatenation of attention heads, we have(5)  
(6) 
where [;] denotes concatenation on the channel axis.
An attention head with weight tensor , denoted by , depends on additional positional encoding and . It starts by computing socalled query, key and value embeddings after adding the query and key positional encodings [7]:
(7) 
where is the concatenation of . The attention weights are then computed based on the softmax of dot products between queries and keys, so that each element of the query sequence attends to all elements of the keyvalue sequence ( is a query index and a keyvalue index):
(8) 
In our case, the positional encodings may be learnt or fixed, but are shared across all attention layers for a given query/keyvalue sequence, so we do not explicitly write them as parameters of the attention. We give more details on their exact value when describing the encoder and the decoder. The final output is the aggregation of values weighted by attention weights: The th row is given by .
The original transformer alternates multihead attention and socalled FFN layers [47], which are effectively multilayer 1x1 convolutions, which have input and output channels in our case. The FFN we consider is composed of twolayers of 1x1 convolutions with ReLU activations. There is also a residual connection/dropout/layernorm after the two layers, similarly to equation 6.
For completeness, we present in detail the losses used in our approach. All losses are normalized by the number of objects inside the batch. Extra care must be taken for distributed training: since each GPU receives a subbatch, it is not sufficient to normalize by the number of objects in the local batch, since in general the subbatches are not balanced across GPUs. Instead, it is important to normalize by the total number of objects in all subbatches.
Similarly to [41, 36], we use a soft version of Intersection over Union in our loss, together with a loss on :
(9) 
where are hyperparameters and is the generalized IoU [38]:
(10) 
means “area”, and the union and intersection of box coordinates are used as shorthands for the boxes themselves. The areas of unions or intersections are computed by of the linear functions of and , which makes the loss sufficiently wellbehaved for stochastic gradients. means the largest box containing (the areas involving are also computed based on of linear functions of the box coordinates).
The DICE coefficient is closely related to the Intersection over Union. If we denote by
the raw mask logits prediction of the model, and
the binary target mask, the loss is defined as:(11) 
where
is the sigmoid function. This loss is normalized by the number of objects.
The detailed description of the transformer used in DETR, with positional encodings passed at every attention layer, is given in Fig. 10. Image features from the CNN backbone are passed through the transformer encoder, together with spatial positional encoding that are added to queries and keys at every multihead selfattention layer. Then, the decoder receives queries (initially set to zero), output positional encoding (object queries), and encoder memory, and produces the final set of predicted class labels and bounding boxes through multiple multihead selfattention and decoderencoder attention. The first selfattention layer in the first decoder layer can be skipped.
Every selfattention in the encoder has complexity : is the cost of computing a single query/key/value embeddings (and ), while is the cost of computing the attention weights for one head. Other computations are negligible. In the decoder, each selfattention is in , and crossattention between encoder and decoder is in , which is much lower than the encoder since in practice.
Given that the FLOPS for Faster RCNN depends on the number of proposals in the image, we report the average number of FLOPS for the first 100 images in the COCO 2017 validation set. We compute the FLOPS with the tool flop_count_operators from Detectron2 [50]. We use it without modifications for Detectron2 models, and extend it to take batch matrix multiply (bmm) into account for DETR models.
We train DETR using AdamW [26] with improved weight decay handling, set to
. We also apply gradient clipping, with a maximal gradient norm of
. The backbone and the transformers are treated slightly differently, we now discuss the details for both.ImageNet pretrained backbone ResNet50 is imported from Torchvision, discarding the last classification layer. Backbone batch normalization weights and statistics are frozen during training, following widely adopted practice in object detection. We finetune the backbone using learning rate of
. We observe that having the backbone learning rate roughly an order of magnitude smaller than the rest of the network is important to stabilize training, especially in the first few epochs.We train the transformer with a learning rate of . Additive dropout of is applied after every multihead attention and FFN before layer normalization. The weights are randomly initialized with Xavier initialization.
We use linear combination of and GIoU losses for bounding box regression with and weights respectively. All models were trained with decoder query slots.
Our enhanced FasterRCNN+ baselines use GIoU [38] loss along with the standard loss for bounding box regression. We performed a grid search to find the best weights for the losses and the final models use only GIoU loss with weights and for box and proposal regression tasks respectively. For the baselines we adopt the same data augmentation as used in DETR and train it with 9 schedule (approximately 109 epochs). All other settings are identical to the same models in the Detectron2 model zoo [50].
Encoder activations are associated with corresponding spatial positions of image features. In our model we use a fixed absolute encoding to represent these spatial positions. We adopt a generalization of the original Transformer [47] encoding to the 2D case [31]. Specifically, for both spatial coordinates of each embedding we independently use sine and cosine functions with different frequencies. We then concatenate them to get the final channel positional encoding.
Some extra qualitative results for the panoptic prediction of the DETRR101 model are shown in Fig.11.


By design, DETR cannot predict more objects than it has query slots, i.e. 100 in our experiments. In this section, we analyze the behavior of DETR when approaching this limit. We select a canonical square image of a given class, repeat it on a grid, and compute the percentage of instances that are missed by the model. To test the model with less than 100 instances, we randomly mask some of the cells. This ensures that the absolute size of the objects is the same no matter how many are visible. To account for the randomness in the masking, we repeat the experiment 100 times with different masks. The results are shown in Fig.12. The behavior is similar across classes, and while the model detects all instances when up to 50 are visible, it then starts saturating and misses more and more instances. Notably, when the image contains all 100 instances, the model only detects 30 on average, which is less than if the image contains only 50 instances that are all detected. The counterintuitive behavior of the model is likely because the images and the detections are far from the training distribution.
Note that this test is a test of generalization outofdistribution by design, since there are very few example images with a lot of instances of a single class. It is difficult to disentangle, from the experiment, two types of outofdomain generalization: the image itself vs the number of object per class. But since few to no COCO images contain only a lot of objects of the same class, this type of experiment represents our best effort to understand whether query objects overfit the label and position distribution of the dataset. Overall, the experiments suggests that the model does not overfit on these distributions since it yields nearperfect detections up to 50 objects.
To demonstrate the simplicity of the approach, we include inference code with PyTorch and Torchvision libraries in Listing 0.A.6. The code runs with Python 3.6+, PyTorch 1.4 and Torchvision 0.5. Note that it does not support batching, hence it is suitable only for inference or training with DistributedDataParallel with one image per GPU. Also note that for clarity, this code uses learnt positional encodings in the encoder instead of fixed, and positional encodings are added to the input only instead of at each transformer layer. Making these changes requires going beyond PyTorch implementation of transformers, which hampers readability. The entire code to reproduce the experiments will be made available before the conference.
[ht] [fontsize=, linenos=true, python3=true]pythonpython/detr.py