Visual inspection and maintenance ofindustrialpipes using robots represent an emerging application in Oil & Gas and refinery facilities. In this domain, we present a pose tracking system based on a single camera sensor to localize an unmanned aerial vehicle operating over a pipe-rack to carry out inspection activities. We propose a unified framework based on the Lie group SE(3) which allows the simultaneous estimation of the pose of the UAV along with some parameters of the pipe-rack model. Numerical simulations have been performed to demonstrate the effectiveness of the proposed approach.