Detection and Longitudinal Tracking of Pigmented Skin Lesions in 3D Total-Body Skin Textured Meshes

05/02/2021
by   Mengliu Zhao, et al.
4

We present an automated approach to detect and longitudinally track skin lesions on 3D total-body skin surfaces scans. The acquired 3D mesh of the subject is unwrapped to a 2D texture image, where a trained region convolutional neural network (R-CNN) localizes the lesions within the 2D domain. These detected skin lesions are mapped back to the 3D surface of the subject and, for subjects imaged multiple times, the anatomical correspondences among pairs of meshes and the geodesic distances among lesions are leveraged in our longitudinal lesion tracking algorithm. We evaluated the proposed approach using three sources of data. Firstly, we augmented the 3D meshes of human subjects from the public FAUST dataset with a variety of poses, textures, and images of lesions. Secondly, using a handheld structured light 3D scanner, we imaged a mannequin with multiple synthetic skin lesions at selected location and with varying shapes, sizes, and colours. Finally, we used 3DBodyTex, a publicly available dataset composed of 3D scans imaging the colored (textured) skin of 200 human subjects. We manually annotated locations that appeared to the human eye to contain a pigmented skin lesion as well as tracked a subset of lesions occurring on the same subject imaged in different poses. Our results, on test subjects annotated by three human annotators, suggest that the trained R-CNN detects lesions at a similar performance level as the human annotators. Our lesion tracking algorithm achieves an average accuracy of 80 different poses. As there currently is no other large-scale publicly available dataset of 3D total-body skin lesions, we publicly release the 10 mannequin meshes and over 25,000 3DBodyTex manual annotations, which we hope will further research on total-body skin lesion analysis.

READ FULL TEXT

page 1

page 3

page 4

page 6

page 8

page 9

research
07/18/2023

Skin Lesion Correspondence Localization in Total Body Photography

Longitudinal tracking of skin lesions - finding correspondence, changes ...
research
05/14/2022

Monitoring of Pigmented Skin Lesions Using 3D Whole Body Imaging

Modern data-driven machine learning research that enables revolutionary ...
research
12/04/2016

Skin Cancer Detection and Tracking using Data Synthesis and Deep Learning

Dense object detection and temporal tracking are needed across applicati...
research
05/22/2023

DermSynth3D: Synthesis of in-the-wild Annotated Dermatology Images

In recent years, deep learning (DL) has shown great potential in the fie...
research
03/28/2018

The HAM10000 Dataset: A Large Collection of Multi-Source Dermatoscopic Images of Common Pigmented Skin Lesions

Training of neural networks for automated diagnosis of pigmented skin le...
research
12/28/2020

Lesion Net – Skin Lesion Segmentation Using Coordinate Convolution and Deep Residual Units

Skin lesions segmentation is an important step in the process of automat...
research
12/09/2020

Deep Lesion Tracker: Monitoring Lesions in 4D Longitudinal Imaging Studies

Monitoring treatment response in longitudinal studies plays an important...

Please sign up or login with your details

Forgot password? Click here to reset