Deep Neural Network–Based Detection and Verification of Microelectronic Images

Abstract

The safety and integrity of complex electronic devices depend on their electronic components, many of which traverse a complex global supply chain before reaching the device manufacturer. Ensuring that these components are correct and legitimate is a significant challenge, especially given the billions of electronic devices that we depend upon. One possible approach is to use computer vision algorithms to analyze images of electronic components—either installed on printed circuit boards or in isolation—to try to automatically spot incorrect or suspicious parts or other potential problems. Such an automatic approach could be especially helpful for large-scale collections of devices, for which manual inspection would be prohibitively expensive. In this paper, we consider two specific problems in this challenging area of microelectronic device inspection: (i) electronic component detection and (ii) electronic component verification. First, we introduce a technique for locating integrated circuits (ICs) on printed circuit boards (PCBs). We apply modern computer vision algorithms, specifically deep learning with convolutional neural networks, to this problem, but find that the small and cluttered nature of electronic components is a significant challenge. We introduce techniques to help overcome this challenge. Second, we consider the problem of component verification: given a pair of IC images, we try to determine if they are the same part or not, ignoring variations caused both by imaging conditions and by expected manufacturing variations across legitimate instances of the same part. We learn a deep feature representation automatically for this problem by showing the algorithm pairs of known similar parts and different parts during training. We evaluate these techniques on large-scale datasets of PCB and IC images we collected from the web.