This article presents a spectrum-based local image descriptor, namely, Radial Fourier Analysis (RFA) image descriptor. The RFA descriptor uses Fourier transform to convert the image gradients in the local region of a keypoint from spatial domain to frequency domain. The transformed gradient frequencies are then analysed to obtain the principle description within the local region. The principle description is represented by low frequency Fourier coefficients and they are directly used in the descriptor to represent the keypoint. Experimental results using a series of performance evaluation procedures showed that RFA descriptor demonstrates higher performances against different image variations comparing to benchmark local image descriptors. The results also indicated that the RFA descriptor is particularly reliable when used on the images, which are degraded by blurring and JPG compression. According to the performance evaluations presented in this paper, spectral analysis shows strong potential for local image description. It paves the way for future research in alternative spectrum-based techniques such as Wavelet transform to precisely analyse local image patches.