IchthyNet: An Ensemble Method for the Classification of In Situ Marine Zooplankton Shadowgraph Images

This study explores the use of machine learning for the automated classification of the ten most abundant groups of marine organisms (in the size range of 5–12 cm) plus marine snow found in the ecosystem of the U.S. east coast. Images used in this process were collected using a shadowgraph imaging system on a towed, undulating platform capable of collecting continuous imagery over large spatiotemporal scales. As a large quantity (29,818,917) of images was collected, the task of locating and identifying all imaged organisms could not be efficiently achieved by human analysis alone. Several tows of data were collected off the coast of Delaware Bay. The resulting images were then cleaned, segmented into regions of interest (ROIs), and fed through three convolutional neural networks (CNNs): VGG-16, ResNet-50, and a custom model created to find more high-level features in this dataset. These three models were used in a Random Forest Classifier-based ensemble approach to reach the best identification fidelity. The networks were trained on a training set of 187,000 ROIs augmented with random rotations and pixel intensity thresholding to increase data variability and evaluated against two datasets. While the performance of each individual model is examined, the best approach is to use the ensemble, which performed with an F1-score of 98% and an area under the curve (AUC) of 99% on both test datasets while its accuracy, precision, and recall fluctuated between 97% and 98%.