Image Classification Using Contrastive Language-Image Pre-training: Application to Aerial Views of Power Line Infrastructures

Abstract

This article evaluates the use of CLIP, a contrastive language-image pre-training methodology, for analyzing aerial images of power line infrastructures. Companies record videos using drones and helicopters to assess the health status of the infrastructures, resulting in hours of unlabeled video. This study proposes a semi-supervised approach that combines natural language processing and image understanding to learn a common representation of images and text. A small set of images labeled based on criteria such as transmission tower type, camera angle view, and background were used to fine-tune CLIP for generating domain-specific embeddings. Results show that this approach achieved an F1 score of over 96% for detecting transmission towers, which could be used to automatically classify unlabeled aerial images as the first step in maintenance data pipelines for predictive detection of anomalies in components, presence of nests or plants, etc.