GoferBot: A Visual Guided Human-Robot Collaborative Assembly System

04/18/2023
by   Zheyu Zhuang, et al.
0

The current transformation towards smart manufacturing has led to a growing demand for human-robot collaboration (HRC) in the manufacturing process. Perceiving and understanding the human co-worker's behaviour introduces challenges for collaborative robots to efficiently and effectively perform tasks in unstructured and dynamic environments. Integrating recent data-driven machine vision capabilities into HRC systems is a logical next step in addressing these challenges. However, in these cases, off-the-shelf components struggle due to generalisation limitations. Real-world evaluation is required in order to fully appreciate the maturity and robustness of these approaches. Furthermore, understanding the pure-vision aspects is a crucial first step before combining multiple modalities in order to understand the limitations. In this paper, we propose GoferBot, a novel vision-based semantic HRC system for a real-world assembly task. It is composed of a visual servoing module that reaches and grasps assembly parts in an unstructured multi-instance and dynamic environment, an action recognition module that performs human action prediction for implicit communication, and a visual handover module that uses the perceptual understanding of human behaviour to produce an intuitive and efficient collaborative assembly experience. GoferBot is a novel assembly system that seamlessly integrates all sub-modules by utilising implicit semantic information purely from visual perception.

READ FULL TEXT

page 1

page 4

research
06/09/2023

How Object Information Improves Skeleton-based Human Action Recognition in Assembly Tasks

As the use of collaborative robots (cobots) in industrial manufacturing ...
research
07/18/2023

Fusing Hand and Body Skeletons for Human Action Recognition in Assembly

As collaborative robots (cobots) continue to gain popularity in industri...
research
06/01/2020

Robots and COVID-19: Challenges in integrating robots for collaborative automation

Objective: The status of human-robot collaboration for assembly applicat...
research
07/08/2020

Mastering the working sequence in human-robot collaborative assembly based on reinforcement learning

A long-standing goal of the Human-Robot Collaboration (HRC) in manufactu...
research
03/15/2023

Classification of Primitive Manufacturing Tasks from Filtered Event Data

Collaborative robots are increasingly present in industry to support hum...
research
09/06/2020

A Hierarchical Architecture for Human-Robot Cooperation Processes

In this paper we propose FlexHRC+, a hierarchical human-robot cooperatio...
research
08/08/2020

Human Robot Collaborative Assembly Planning: An Answer Set Programming Approach

For planning an assembly of a product from a given set of parts, robots ...

Please sign up or login with your details

Forgot password? Click here to reset