Border and security agencies often rely on the Automatic Identification System (AIS) to gather intelligence and information on the current situation in their waters. However, they cannot always rely on the ship's AIS transceiver to operate properly. Using Convolutional Neural Networks (CNNs) for the detection of vessels in images and Field Programmable Gate Arrays (FPGA) platforms for deployment, it is possible to deploy a solution that is both faster than Graphical Processing Units (GPUs) and much more power efficient. This solution provides authorities with more tools to secure their ports. In this article, we propose an object detection solution capable of identifying individual ships from images as well as its Register Transfer Level (RTL) design that offers a speedup on inference time by a factor of 4 as well as consuming almost 50 times less power than the traditional GPU solution.