How many images do I need? Understanding how sample size per class affects deep learning model performance metrics for balanced designs in autonomous wildlife monitoring

by   Saleh Shahinfar, et al.

Deep learning (DL) algorithms are the state of the art in automated classification of wildlife camera trap images. The challenge is that the ecologist cannot know in advance how many images per species they need to collect for model training in order to achieve their desired classification accuracy. In fact there is limited empirical evidence in the context of camera trapping to demonstrate that increasing sample size will lead to improved accuracy. In this study we explore in depth the issues of deep learning model performance for progressively increasing per class (species) sample sizes. We also provide ecologists with an approximation formula to estimate how many images per animal species they need for certain accuracy level a priori. This will help ecologists for optimal allocation of resources, work and efficient study design. In order to investigate the effect of number of training images; seven training sets with 10, 20, 50, 150, 500, 1000 images per class were designed. Six deep learning architectures namely ResNet-18, ResNet-50, ResNet-152, DnsNet-121, DnsNet-161, and DnsNet-201 were trained and tested on a common exclusive testing set of 250 images per class. The whole experiment was repeated on three similar datasets from Australia, Africa and North America and the results were compared. Simple regression equations for use by practitioners to approximate model performance metrics are provided. Generalized additive models (GAM) are shown to be effective in modelling DL performance metrics based on the number of training images per class, tuning scheme and dataset. Key-words: Camera Traps, Deep Learning, Ecological Informatics, Generalised Additive Models, Learning Curves, Predictive Modelling, Wildlife.


page 5

page 18

page 19

page 20

page 21

page 23

page 24

page 25


A first step towards automated species recognition from camera trap images of mammals using AI in a European temperate forest

Camera traps are used worldwide to monitor wildlife. Despite the increas...

Choosing an Appropriate Platform and Workflow for Processing Camera Trap Data using Artificial Intelligence

Camera traps have transformed how ecologists study wildlife species dist...

Human Blastocyst Classification after In Vitro Fertilization Using Deep Learning

Embryo quality assessment after in vitro fertilization (IVF) is primaril...

Bag of Tricks for Long-Tail Visual Recognition of Animal Species in Camera Trap Images

Camera traps are a strategy for monitoring wildlife that collects a larg...

Unravelling Small Sample Size Problems in the Deep Learning World

The growth and success of deep learning approaches can be attributed to ...

Instability in clinical risk stratification models using deep learning

While it has been well known in the ML community that deep learning mode...

CWD30: A Comprehensive and Holistic Dataset for Crop Weed Recognition in Precision Agriculture

The growing demand for precision agriculture necessitates efficient and ...

Please sign up or login with your details

Forgot password? Click here to reset