Counting on Uncertainty: Obtaining Fish Counts from Machine Learning Decisions

Abstract

Most questions in the Fish4Knowledge project are related to the ability to count fish, fish species or events using video analysis Chap. 2. Automatic video analysis however brings uncertainty due to False Positive/False Negative classifications, which makes determining the counts based on automatic video analysis difficult. Automatic video analysis software is often able to express a measure of uncertainty for a single detection/recognition using a similarity score, indicating how certain the software is about a single decision. Logistic Regression allows us to combine these similarity scores to compute an estimated count, outperforming the estimates of the underlying machine learning methods used for the original video analysis. We show this works both for the two-class and multi-class problem. Furthermore, we identify potential pitfalls where ensuring a correct sampling procedure is essential. The error in the estimated species counts using Logistic Regression was on average around 15 on a set of 11,585 fish images, while the machine methods had an average error of 800. The key to understanding this huge improvement in accuracy is that we use Logistic Regression over the individual classifications to estimate the size of the whole population, rather than the species classification of any individual observation.