A galvo-based variable boresight imaging system (VBIS) can quickly switch viewing direction by deflecting the two mirrors in the galvo. A novel physical parameter imaging model of VBIS, which essentially reveals the optical imaging process corresponding to any boresight, is presented to better support 3-D vision applications. The light deflection process in the galvo is subtly modeled with two Householder matrices. The entire imaging model is achieved by cascading the ordinary camera model and the galvo model in the form of a serial of matrix multiplications. In addition, a calibration method for determining the parameters in the imaging model is presented. The work in this article generalizes the ordinary camera imaging model to VBIS in a very concise way. Experimental results show that the reprojection residual of the calibration algorithm is 0.775 pixels, and the standard deviation is 0.470 pixels. As exemplary applications, the VBIS-based pose estimation and 3-D image stitching methods are proposed and verified.