DeepAI AI Chat
Log In Sign Up

Connecting Language and Vision for Natural Language-Based Vehicle Retrieval

by   Shuai Bai, et al.

Vehicle search is one basic task for the efficient traffic management in terms of the AI City. Most existing practices focus on the image-based vehicle matching, including vehicle re-identification and vehicle tracking. In this paper, we apply one new modality, i.e., the language description, to search the vehicle of interest and explore the potential of this task in the real-world scenario. The natural language-based vehicle search poses one new challenge of fine-grained understanding of both vision and language modalities. To connect language and vision, we propose to jointly train the state-of-the-art vision models with the transformer-based language model in an end-to-end manner. Except for the network structure design and the training strategy, several optimization objectives are also re-visited in this work. The qualitative and quantitative experiments verify the effectiveness of the proposed method. Our proposed method has achieved the 1st place on the 5th AI City Challenge, yielding competitive performance 18.69 We hope this work can pave the way for the future study on using language description effectively and efficiently for real-world vehicle retrieval systems. The code will be available at


page 3

page 4

page 6

page 7


Symmetric Network with Spatial Relationship Modeling for Natural Language-based Vehicle Retrieval

Natural language (NL) based vehicle retrieval aims to search specific ve...

All You Can Embed: Natural Language based Vehicle Retrieval with Spatio-Temporal Transformers

Combining Natural Language with Vision represents a unique and interesti...

End-to-end trainable network for degraded license plate detection via vehicle-plate relation mining

License plate detection is the first and essential step of the license p...

SBNet: Segmentation-based Network for Natural Language-based Vehicle Search

Natural language-based vehicle retrieval is a task to find a target vehi...

Dual Embedding Expansion for Vehicle Re-identification

Vehicle re-identification plays a crucial role in the management of tran...

V^2L: Leveraging Vision and Vision-language Models into Large-scale Product Retrieval

Product retrieval is of great importance in the ecommerce domain. This p...

Multi-Domain Learning and Identity Mining for Vehicle Re-Identification

This paper introduces our solution for the Track2 in AI City Challenge 2...