DeepAI AI Chat
Log In Sign Up

Object Detection for Understanding Assembly Instruction Using Context-aware Data Augmentation and Cascade Mask R-CNN

by   Joosoon Lee, et al.

Understanding assembly instruction has the potential to enhance the robot s task planning ability and enables advanced robotic applications. To recognize the key components from the 2D assembly instruction image, We mainly focus on segmenting the speech bubble area, which contains lots of information about instructions. For this, We applied Cascade Mask R-CNN and developed a context-aware data augmentation scheme for speech bubble segmentation, which randomly combines images cuts by considering the context of assembly instructions. We showed that the proposed augmentation scheme achieves a better segmentation performance compared to the existing augmentation algorithm by increasing the diversity of trainable data while considering the distribution of components locations. Also, we showed that deep learning can be useful to understand assembly instruction by detecting the essential objects in the assembly instruction, such as tools and parts.


page 2

page 3


Assembly Planning by Recognizing a Graphical Instruction Manual

This paper proposes a robot assembly planning method by automatically re...

BinBert: Binary Code Understanding with a Fine-tunable and Execution-aware Transformer

A recent trend in binary code analysis promotes the use of neural soluti...

Context-Aware Data Augmentation for LIDAR 3D Object Detection

For 3D object detection, labeling lidar point cloud is difficult, so dat...

Scale-aware Automatic Augmentation for Object Detection

We propose Scale-aware AutoAug to learn data augmentation policies for o...

Part-Aware Data Augmentation for 3D Object Detection in Point Cloud

Data augmentation has greatly contributed to improving the performance i...

Defining Pathway Assembly and Exploring its Applications

How do we estimate the probability of an abundant objects' formation, wi...

PalmTree: Learning an Assembly Language Model for Instruction Embedding

Deep learning has demonstrated its strengths in numerous binary analysis...