We propose a two-phase parametric registration algorithm for retinal images. Our algorithm focuses on dealing with the geometric transformation and the intensity transformation in the retinal image registration problem. In the first phase, we efficiently detect only one pair of feature points in the source and the target retinal images to estimate a translation transformation and get a warped source image. In the second phase, we estimate both the intensity and the geometric transformations between the target image and the warped source image by fitting parametric expressions. The displacement field is generated by a super fast and accurate coarse-to-fine elastic registration algorithm—local all-pass filters algorithm (LAP). At each iteration of the LAP, the elastic displacement field and the intensity difference take turns being fitted by two different low-order polynomial functions. The fitting steps are performed by solving linear systems of equations efficiently. Experiments on real retinal image datasets demonstrated the high accuracy and computational efficiency of the proposed retinal image registration method.