We propose a methodology for cross matching color face images and Short Wave Infrared (SWIR) face images reliably and accurately. We first adopt a recently designed image encoding and matching technique which is capable to encode face images in both visible and SWIR spectral bands. Encoding is performed in two steps. Images are initially filtered with a bank of Gabor filters. Then three local operators: Simplified Weber Local Descriptor and Local Binary Pattern applied to magnitude of filtered images and Generalized Local Binary Pattern applied to the phase are involved to create histogram-like feature templates. The distance between two encoded face images is measured by symmetric I-divergence. The encoding and matching methods are demonstrated on long range SWIR data matched against close range visible images. A considerable performance improvement is observed compared to the results by FaceIt G8. To further enhance performance we propose an adaptive score normalization approach. We demonstrate that significant performance improvement is achieved with a small training set. Matching scores obtained by the proposed normalized method and by FaceIt G8 are fused to result in further performance improvement.