Abstract
In image registration the optimal transformation parameters of a given transformation model are typically obtained by minimizing a cost function. Stochastic gradient descent (SGD) is an efficient optimization algorithm for image registration. In SGD optimization, stochastic approximations of the cost function derivative are used in each iteration to update the transformation parameters. The stochastic approximation error leads to large variance in the parameters. To enforce convergence nonetheless, SGD methods are typically implemented in combination with a gradually decreasing update step size. However, selecting a proper sequence of step sizes is a major challenge in practice. An alternative strategy in numerical optimization is to use a constant step size and enforce convergence by averaging the parameters obtained by SGD over several iterations. It was proven mathematically that the highest possible rate of convergence is achieved in this way. Inspired by this work, we propose an averaged SGD (Avg-SGD) method for efficient image registration. In the Avg-SGD approach, a constant step size is used, in combination with an exponentially weighted iterate averaging scheme. Experiments on 3D lung CT scans demonstrate the effectiveness of the Avg-SGD method in terms of convergence rate, accuracy and precision.
W. Sun—This work was in part supported by the National Natural Science Foundation of China No. U1301251.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Klein, S., Staring, M., Pluim, J.P.W.: Evaluation of optimization methods for nonrigid medical image registration using mutual information and B-splines. IEEE Trans. Image Process. 16(12), 2879–2890 (2007)
Viola, P., Wells III, W.M.: Alignment by maximization of mutual information. Int. J. Comput. Vis. 24(2), 137–154 (1997)
Sun, W., Poot, D.H., Smal, I., Yang, X., Niessen, W.J., Klein, S.: Stochastic optimization with randomized smoothing for image registration. Med. Image Anal. 35, 146–158 (2017)
Sun, W., Niessen, W.J., Klein, S.: Randomly perturbed B-splines for nonrigid image registration. IEEE Trans. Pattern Anal. Mach. Intell. 39(7), 1401–1413 (2017)
Kushner, H.J., Yin, G.: Stochastic Approximation and Recursive Algorithms and Applications, vol. 35. Springer, New York (2003). https://doi.org/10.1007/b97441
Klein, S., Pluim, J.P.W., Staring, M., Viergever, M.A.: Adaptive stochastic gradient descent optimisation for image registration. Int. J. Comput. Vis. 81(3), 227–239 (2009)
Qiao, Y., van Lew, B., Lelieveldt, B.P., Staring, M.: Fast automatic step size estimation for gradient descent optimization of image registration. IEEE Trans. Med. Imaging 35(2), 391–403 (2016)
Bottou, L., Le Cun, Y.: On-line learning for very large data sets. Appl. Stoch. Models. Bus. Ind. 21(2), 137–151 (2005)
Bordes, A., Bottou, L., Gallinari, P.: SGD-QN: careful quasi-Newton stochastic gradient descent. J. Mach. Learn. Res. 10, 1737–1754 (2009)
Xu, W.: Towards optimal one pass large scale learning with averaged stochastic gradient descent. arXiv preprint arXiv:1107.2490 (2011)
Polyak, B.T., Juditsky, A.B.: Acceleration of stochastic approximation by averaging. SIAM J. Control Optim. 30(4), 838–855 (1992)
Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P.: Multimodality image registration by maximization of mutual information. IEEE Trans. Med. Imaging 16(2), 187–198 (1997)
Ruppert, D.: Efficient estimations from a slowly convergent Robbins-Monro process. Technical report, Cornell University Operations Research and Industrial Engineering (1988)
Yin, G.: Stochastic approximation via averaging: the Polyak’s approach revisited. In: Pflug, G., Dieter, U. (eds.) Simulation and Optimization. Lecture Notes in Economics and Mathematical Systems, vol. 374, pp. 119–134. Springer, Heidelberg (1992). https://doi.org/10.1007/978-3-642-48914-3_9
Klein, S., Staring, M., Murphy, K., Viergever, M.A., Pluim, J.P.W.: Elastix: a toolbox for intensity-based medical image registration. IEEE Trans. Med. Imaging 29(1), 196–205 (2010)
Castillo, R., Castillo, E., Guerra, R., Johnson, V., McPhail, T., Garg, A., Guerrero, T.: A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets. Phys. Med. Biol. 54(7), 1849–1870 (2009)
Papież, B.W., Heinrich, M.P., Fehrenbach, J., Risser, L., Schnabel, J.A.: An implicit sliding-motion preserving regularisation via bilateral filtering for deformable image registration. Med. Image Anal. 18(8), 1299–1311 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Sun, W., Poot, D.H.J., Yang, X., Niessen, W.J., Klein, S. (2018). Averaged Stochastic Optimization for Medical Image Registration Based on Variance Reduction. In: Klein, S., Staring, M., Durrleman, S., Sommer, S. (eds) Biomedical Image Registration. WBIR 2018. Lecture Notes in Computer Science(), vol 10883. Springer, Cham. https://doi.org/10.1007/978-3-319-92258-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-92258-4_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-92257-7
Online ISBN: 978-3-319-92258-4
eBook Packages: Computer ScienceComputer Science (R0)