Fruit Detection, Segmentation and 3D Visualisation of Environments in Apple Orchards

by   Hanwen Kang, et al.

Robotic harvesting of fruits in orchards is a challenging task, since high density and overlapping of fruits and branches can heavily impact the success rate of robotic harvesting. Therefore, the vision system is demanded to provide comprehensive information of the working environment to guide the manipulator and gripping system to successful detach the target fruits. In this study, a deep learning based one-stage detector DaSNet-V2 is developed to perform the multi-task vision sensing in the working environment of apple orchards. DaSNet-V2 combines the detection and instance segmentation of fruits and semantic segmentation of branch into a single network architecture. Meanwhile, a light-weight backbone network LW-net is utilised in the DaSNet-V2 model to improve the computational efficiency of the model. In the experiment, DaSNet-V2 is tested and evaluated on the RGB-D images of the orchard. From the experiment results, DaSNet-V2 with lightweight backbone achieves 0.844, 0.858, and 0.795 on the F 1 score of the detection, and mean intersection of union on the instance segmentation of fruits and semantic segmentation of branches, respectively. To provide a direct-viewing of the working environment in orchards, the obtained sensing results are illustrated by 3D visualisation . The robustness and efficiency of the DaSNet-V2 in detection and segmentation are validated by the experiments in the real-environment of apple orchard.


page 6

page 12

page 14


Visual Perception and Modelling in Unstructured Orchard for Apple Harvesting Robots

Vision perception and modelling are the essential tasks of robotic harve...


We present Panoptic-DeepLab, a bottom-up and single-shot approach for pa...

Single-Shot Panoptic Segmentation

We present a novel end-to-end single-shot method that segments countable...

Deep Neural Network Pruning for Nuclei Instance Segmentation in Hematoxylin Eosin-Stained Histological Images

Recently, pruning deep neural networks (DNNs) has received a lot of atte...

MuraNet: Multi-task Floor Plan Recognition with Relation Attention

The recognition of information in floor plan data requires the use of de...

Embedded Vision for Self-Driving on Forest Roads

Forest roads in Romania are unique natural wildlife sites used for recre...

Real-time Strawberry Detection Based on Improved YOLOv5s Architecture for Robotic Harvesting in open-field environment

This study proposed a YOLOv5-based custom object detection model to dete...

Please sign up or login with your details

Forgot password? Click here to reset