Abstract
The acquisition of the planar images of the same object may be considerably different due to viewpoint dependencies, which influences the shape extraction, hence possibly making the curves partially visible and often accompanied by perspective distortions. In this paper, we propose a new contour alignment system relating to the special affine transformations that contain rotations and stretches, useful for describing planar contours which move in three-dimensional space and which are far enough away from the camera. The registration system that we suggest here includes a first optimization step relating to the dataset concerned. It consists in optimizing the number of correspondence points N between the curves to be registered. This is achieved by minimizing the conditioning of the correspondence matrix which is obtained by matching the re-sampling points by the equi-affine length of the two curves. This correspondence matrix is calculated for all the pairs of curves of the dataset by varying N. After extracting the optimal value of N, the estimation of the special affine transformation between a given couple of curves is realized by the pseudo-inverse of the correspondence matrix in the \(N_{0}\) resolution. This approach allows both providing the best accuracy and stabilizing the results of registration. We evaluate and compare our algorithm with other existing methods under different shape variations including noise, missing parts, and articulated deformations. The experiments are conducted on several known datasets.
























Similar content being viewed by others
References
Adamek T, O’Connor NE (2004) A multiscale representation method for nonrigid shapes with a single closed contour. IEEE Trans Circuits Syst Video Technol 14:742–753
Alajlan N, Kamel MS, Freeman GH (2008) Geometry-based image retrieval in binary image databases. IEEE Trans Pattern Anal Mach Intell 30:1003–1013
Arbter K, Snyder WE, Burkhardt H, Hirzinger G (1990) Application of affine-invariant Fourier descriptors to recognition of 3-D objects. IEEE Trans Pattern Anal Mach Intell 12:640–647
Bachelder IA, Ullman S (1992) Contour matching using local affine transformations. Massachusetts Inst of Tech Cambridge Artificial Intelligence Lab
Bai X, Yang X, Latecki LJ, Liu W, Tu Z (2009) Learning context-sensitive shape similarity by graph transduction. IEEE Trans Pattern Anal Mach Intell 32:861–874
BenKhlifa A, Ghorbel F (2019) An almost complete curvature scale space representation: Euclidean case. Signal Process Image Commun, pp 32–43
Benzinou A, Nasreddine K, Khalil M, Faour G (2014) An optimal elastic partial shape matching via shape geodesics. In: 2014 IEEE international conference on image processing, pp 4742–4746
Bruckstein AM, Katzir N, Lindenbaum M, Porat M (1992) Similarity-invariant signatures for partially occluded planar shapes. Int J Comput Vis 7:271–285
Bryner D, Klassen E, Le H, Srivastava A (2013) 2D affine and projective shape analysis. IEEE Trans Pattern Anal Mach Intell 36:998–1011
Cao X, Fan J, Dong P, Ahmad S, Yap PT, Shen D (2020) Image registration using machine and deep learning. In: Handbook of medical image computing and computer assisted intervention, 319–342
Cao X, Yang J, Zhang J, Nie D, Kim M, Wang Q, Shen D (2017) Deformable image registration based on similarity-steered CNN regression. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 300-308
Cao Y, Zhang Z, Czogiel I, Dryden I, Wang S (2011) 2D nonrigid partial shape matching using MCMC and contour subdivision. CVPR, pp 2345–2352
Chaker F, Bannour MT, Ghorbel F (2007) Contour retrieval and matching by affine invariant fourier descriptors. MVA7, pp 291–294
Chen L, Feris R, Turk M (2008) Efficient partial shape matching using smith-waterman algorithm. In 2008 IEEE computer society conference on computer vision and pattern recognition workshops, pp 1–6
Crimmins TR (1982) A complete set of Fourier descriptors for two-dimensional shapes. IEEE Trans. Syst Man Cybern 12:848–855
Cui M, Femiani J, Hu J, Wonka P, Razdan A (2009) Curve matching for open 2D curves. Pattern Recognit. Lett. 30:1–10
Cyganski D (1987) An affine transformation invariant curvature function. In: 1st international conference on computer vision
Cyganski D, Vaz RF (1995) A linear signal decomposition approach to affine invariant contour identification. Pattern Recognit. 28:1845–1853
Cyganski D, Cott TA, Orr JA, Dodson RJ (1988) Object identification and orientation estimation from contours based on an affine invariant curvature. Intell Robots Comput Vis VI:33–39
Daliri MR, Torre V (2008) Robust symbolic representation for shape recognition and retrieval. Pattern Recognit 41:1782–1798
De Vos BD, Berendsen F, Viergever MA, Staring M, Isgum I (2017) End-to-end unsupervised deformable image registration with a convolutional neural network. In: Deep learning in medical image analysis and multimodal learning for clinical decision support, pp 204–212
De Vos BD, Berendsen F, Viergever MA, Sokooti H, Staring M, Isgum I (2019) A deep learning framework for unsupervised affine and deformable image registration. Med Image Anal 52:128–143
Demirci MF (2010) Efficient shape retrieval under partial matching. In 2010 20th International Conference on Pattern Recognition, 3057-3060
Domokos C, Kato Z (2010) Parametric estimation of affine deformations of planar shapes. Pattern Recogn 43:569–578
Donoser M, Riemenschneider H, Bischof H (2009) Efficient partial shape matching of outer contours. In: Asian conference on computer vision, pp 281–292
Dosovitskiy A, Fischer P, Ilg E, Hausser P, Hazirbas C, Golkov V, Brox T (2015) Flownet: Learning optical flow with convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp 2758–2766
Egozi A, Keller Y, Guterman H (2010) Improving shape retrieval by spectral matching and meta similarity. IEEE Trans Image Process 19:1319–1327
El Rube I, Ahmed M, Kamel M (2005) Wavelet approximation-based affine invariant shape representation functions. IEEE Trans Pattern Anal Mach Intell 28:323–327
El-ghazal A, Basir O, Belkasim S (2009) Farthest point distance: a new shape signature for Fourier descriptors. Signal Process. Image Commun. 24:572–586
El-ghazal A, Basir O, Belkasim S (2012) Invariant curvature-based Fourier shape descriptors. Journal of Visual Communication and Image Representation, 622-633
Elghoul S, Ghorbel F (2021) Fast global SA (2, R) shape registration based on invertible invariant descriptor. Signal Process Image Commun 90:116058
Elghoul S, Ghorbel F (2018) An efficient 2D curve matching algorithm under affine transformations. In: VISIGRAPP, pp 474–480
Eppenhof KA, Lafarge MW, Moeskops P, Veta M, Pluim JP (2018) Deformable image registration using convolutional neural networks. Image Processing, In Medical Imaging
Felzenszwalb PF, Schwartz JD (2007) Hierarchical matching of deformable shapes. In: 2007 IEEE conference on computer vision and pattern recognition, pp 1–8
Ferrante E, Paragios N (2017) Slice-to-volume medical image registration: a survey. Med Image Anal 39:101–123
Ferrante E, Dokania PK, Silva RM, Paragios N (2018) Weakly supervised learning of metric aggregations for deformable image registration. IEEE J Biomed Health Inf 23:1374–1384
Fu H, Tian Z, Ran M, Fan M (2013) Novel affine-invariant curve descriptor for curve matching and occluded object recognition. IET Comput Vis, pp 279–292
Garg R, Bg VK, Carneiro G, Reid I (2016) Unsupervised CNN for single view depth estimation: geometry to the rescue. In: European conference on computer vision, pp 740–756
Genovese A, Piuri V, Scotti F (2014) Palmprint biometrics. In: Touchless palmprint recognition systems, pp 49–109
Ghorbel F (1994) A complete invariant description for gray-level images by the harmonic analysis approach. Pattern Recognit. Lett. 15:1043–1051
Ghorbel F (1998) Towards a unitary formulation for invariant image description: application to image coding. Ann Telecommun 53:242–260
Gopalan R, Turaga P, Chellappa R (2010) Articulation-invariant representation of non-planar shapes. In European Conference on Computer Vision, 286-299
Gope C, Kehtarnavaz N, Hillman G, Wursig B (2005) An affine invariant curve matching method for photo-identification of marine mammals. Pattern Recognit 38:125–132
Hu MK (1962) Visual pattern recognition by moment invariants. IRE Trans Inf Theory, 179–187
Hu MK (1962) Visual pattern recognition by moment invariants. IRE Trans Inf Theory 8:179–187
Hu N (2011) Centroaffine space curves with constant curvatures and homogeneous surfaces. J Geometry 102:103–114
Hu R, Jia W, Ling H, Huang D (2012) Multiscale distance matrix for fast plant leaf recognition. IEEE Trans. Image Process. 21:4667–4672
Huang X, Wang B, Zhang L (2005) A new scheme for extraction of affine invariant descriptor and affine motion estimation based on independent component analysis. Pattern Recognit Lett 28:1244–1255
Hu Y, Modat M, Gibson E, Ghavami N, Bonmati E, Moore CM, Vercauteren T (2018) Label-driven weakly-supervised learning for multimodal deformable image registration. In 2018 IEEE 15th International Symposium on Biomedical Imaging, 1070-1074
Huttenlocher DP, Kedem K (1990) Computing the minimum Hausdorff distance for point sets under translation. In: Proceedings of the sixth annual symposium on computational geometry, pp 340–349
Ilg E, Mayer N, Saikia T, Keuper M, Dosovitskiy A, Brox T (2017) Flownet 2.0: Evolution of optical flow estimation with deep networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2462–2470
Jaderberg M, Simonyan K, Zisserman A (2015) Spatial transformer networks. Adv Neural Inf Process Syst 2:2017–2025
Jia Q, Fan X, Liu Y, Li H, Luo Z, Guo H (2016) Hierarchical projective invariant contexts for shape recognition. Pattern Recognit 52:358–374
Jin Q, Yan P (1992) A new method of extracting invariants under affine transform. In: 11th IAPR international conference on pattern recognition, pp 742–745
Joo H, Jeong Y, Duchenne O, Ko SY, Kweon IS (2009) Graph-based robust shape matching for robotic application. In: 2009 IEEE international conference on robotics and automation, pp 1207–1213
Joshi K, Patel MI (2020) Recent advances in local feature detector and descriptor: a literature survey. Int J Multimed Inf Retr 9:1–17
Khalil MI, Bayoumi M (2001) A dyadic wavelet affine invariant function for 2D shape recognition. IEEE Trans Pattern Anal Mach Intell 23:1152–1164
Khotanzad A, Hong YH (1990) Invariant image recognition by Zernike moments. IEEE Trans Pattern Analy Mach Intell 12:489–497
Krebs J, Mansi T, Delingette H, Zhang L, Ghesu FC, Miao S, Kamen A (2017) Robust non-rigid registration through agent-based action learning. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 344-352
Krotosky SJ, Trivedi M (2007) Mutual information based registration of multimodal stereo videos for person tracking. Computer Vision and Image Understanding, 270-287
Kun Z, Xiao M, Xinguo L (2019) Shape matching based on multi-scale invariant features. IEEE Access 7:115637–115649
Laiche N, Larabi S (2018) Shape retrieval through normalized B-splines curves. Multimedia Tools Appl 77:13891–13921
Lamdan Y, Schwartz JT, Wolfson HJ (1990) Affine invariant model-based object recognition. IEEE Trans Robot Autom 6:578–589
Latecki LJ, Lakamper R, Eckhardt T (2000) Shape descriptors for non-rigid shapes with a single closed contour. In: Proceedings IEEE conference on computer vision and pattern recognition, pp 424–429
Latecki LJ, Lakaemper R, Wolter D (2005) Optimal partial shape similarity. Image Vis Comput 23:227–236
Latecki LJ, Megalooikonomou V, Wang Q, Yu D (2007) An elastic partial shape matching technique. Pattern Recognit 40:3069–3080
Liao R, Miao S, Tournemire P, Grbic S, Kamen A, Mansi T, Comaniciu D (2017) An artificial agent for robust image registration. In Proceedings of the AAAI Conference on Artificial Intelligence
Lin WS, Fang CH (2007) Synthesized affine invariant function for 2D shape recognition. Pattern Recognit. 40:1921–1928
Ling H, Jacobs DW (2007) Shape classification using the inner-distance. IEEE Trans Pattern Anal Mach Intell 29:286–299
Ling H, Okada K (2007) An efficient earth mover’s distance algorithm for robust histogram comparison. IEEE Trans Pattern Anal Mach Intell 29:840–853
Liu H (2014) Curves in affine and semi-Euclidean Spaces. Results Math 65:235–249
Mai F, Chang CQ, Hung YS (2011) A subspace approach for matching 2D shapes under affine distortions. Pattern Recognit 44:210–221
Mai F, Chang CQ, Hung YS (2010) Affine-invariant shape matching and recognition under partial occlusion. In: 2010 IEEE international conference on image processing, pp 4605–4608
Marvaniya S, Gupta R, Mittal A (2018) Adaptive locally affine-invariant shape matching. Mach Vis Appl 29:553–572
Michel D, Oikonomidis I, Argyros A (2011) Scale invariant and deformation tolerant partial shape matching. Image and Vision Computing, 459-469
Mokhtarian F, Abbasi S (2001) Affine curvature scale space with affine length parametrisation. Pattern Anal 4:1–8
Morel JM, Yu G (2009) ASIFT: A new framework for fully affine invariant image comparison. SIAM J Imaging Sci 2:438–469
Mori G, Belongie S, Malik J (2005) Efficient shape matching using shape contexts. IEEE Trans Pattern Anal Mach Intell 27:1832–1837
Moyou M, Rangarajan A, Corring J, Peter AM (2019) A Grassmannian graph approach to affine invariant feature matching. IEEE Trans Image Process, pp 3374–3387
Nomizu K, Katsumi N, Sasaki T (1994) Affine differential geometry: geometry of affine immersions. Cambridge University Press, Cambridge
Olver PJ (2010) Moving frames and differential invariants in centro-affine geometry. Lobachevskii J Math 31:77–89
Olver PJ (2015) Modern developments in the theory and applications of moving frames. London Math Soc Impact150 Stories 1:14–50
Orrite C, Herrero JE (2004) Shape matching of partially occluded curves invariant under projective transformation. Comput Vis Image Understand 93:34–64
Osowski S (2002) Fourier and wavelet descriptors for shape recognition using neural networks a comparative study. Pattern Recognit 35:1949–1957
Raviv D, Kimmel R (2015) Affine invariant geometry for non-rigid shapes. Int J Comput Vis 111:1–11
Reiss TH (1991) The revised fundamental theorem of moment invariants. IEEE Transactions on Pattern Analysis and Machine Intelligence, 830-834
Rodriguez M, Facciolo G, Gioi RGV, Muse P, Delon J (2020) Robust estimation of local affine maps and its applications to image matching. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 1342–1351
Rui Y, She AC, Huang TS (1997) A modified Fourier descriptor for shape matching in MARS. In: Image databases and multi-media search, pp 165–177
Saber E, Xu Y, Tekalp AM (2005) Partial shape recognition by sub-matrix matching for partial matching guided image labeling. Pattern Recognit 38:1560–1573
Sebastian TB, Philip N, Benjamin B (2004) Recognition of shapes by editing their shock graphs. IEEE Trans Pattern Anal Mach Intell 26:550–571
Shan Y, Sawhney HS, Matei B, Kumar R (2006) Shapeme histogram projection and matching for partial object recognition. IEEE Trans Pattern Anal Mach Intell 28:568–577
Sharma A, Horaud R, Mateus D (2021) 3D shape registration using spectral graph embedding and probabilistic matching. ArXiv preprint
Shekar BH, Pilar B, Kittler J (2015) An unification of inner distance shape context and local binary pattern for shape representation and classification. In: Proceedings of the 2nd international conference on perception and machine intelligence, pp 46–55
Shu X, Wu XJ (2011) A novel contour descriptor for 2D shape matching and its application to image retrieval. Image Vis Comput 29:286–294
Soderkvist O (2001) Computer vision classification of leaves from Swedish trees
Sokic E, Konjicija S (2014) Novel fourier descriptor based on complex coordinates shape signature. In: 2014 12th international workshop on content-based multimedia indexing (CBMI), pp 1–4
Spivak M (1975) A comprehensive introduction to differential geometry. Publish or Perish, Incorporated
Stergios C, Mihir S, Maria V, Guillaume C, Marie-Pierre R, Stavroula M, Nikos P (2018) Linear and deformable image registration with 3D convolutional neural networks. In: Image analysis for moving organ, breast, and thoracic images, pp 13–22
Temlyakov A, Munsell BC, Waggoner JW, Wang S (2010) Two perceptually motivated strategies for shape classification. In: 2010 IEEE computer society conference on computer vision and pattern recognition, pp 2289–2296
Thies J, Zollhofer M, Stamminger M, Theobalt C, Niener M (2016) Face2face: Real-time face capture and reenactment of rgb videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2387-2395
Tieng QM, Boles W (1995) An application of wavelet-based affine-invariant representation. Pattern Recognit. Lett. 16:1287–1296
Turney JL, Trevor NM, Richard A (1985) Recognizing partially occluded parts. IEEE Trans Pattern Anal Mach Intell 4:410–421
Tu Z, Yuille AL (2004) Shape matching and recognition using generative models and informative features. In: European conference on computer vision, pp 195–209
Wang J, Bai X, You X, Liu W, Latecki LJ (2012) Shape matching and classification using height functions. Pattern Recognit Lett 33:134–143
Wang Z, Xu G, Cheng Y, Guo R, Wang Z (2018) A curvature salience descriptor for full and partial shape matching. Multimedia Tools Appl 77:27405–27426
Wang W, Yan X, Wang Z, Shi J (2017) A robust affine invariant point extraction algorithm for image registration. In: 2017 10th international symposium on computational intelligence and design, pp 79–82
Wu J, Rehg JM (2010) Centrist: A visual descriptor for scene categorization. IEEE Trans. Pattern Anal. Mach. Intell. 33:1489–1501
Xu C, Liu J, Tang X (2008) 2D shape matching by contour flexibility. IEEE Trans Pattern Anal Mach Intell 31:180–186
Yang Y, Yu Y (2020) Moving frames and differential invariants on fully affine planar curves. Bull Malays Math Sci Soc 43:3229–3258
Yang J, Wang H, Yuan J, Li Y, Liu J (2016) Invariant multi-scale descriptor for shape representation, matching and retrieval. Comput Vis Image Understand 145:43–58
Yang C, Wei H, Yu Q (2018) A novel method for 2D nonrigid partial shape matching. Neurocomputing 275:1160–1176
Yang X, Koknar-Tezel S, Latecki LJ (2009) Locally constrained diffusion process on locally densified distance spaces with applications to shape retrieval. In: 2009 IEEE conference on computer vision and pattern recognition, pp 357–364
Yang G, Li R, Liu Y, Wang J (2021) A robust nonrigid point set registration framework based on global and intrinsic topological constraints. Vis Comput 1–21
Yang C, Wei H, Yu Q (2016) Multiscale triangular centroid distance for shape-based plant leaf recognition. EGAI, pp 269–276
Yang C, Yu Q (2019) Multiscale Fourier descriptor based on triangular features for shape retrieval. Signal Process Image Commun, pp 110–119
Yang Y, Yu Y (2018) Affine Maurer Cartan invariants and their applications in self-affine fractals. Fractals
Ye Y, Bruzzone L, Shan J, Bovolo F, Zhu Q (2019) Fast and robust matching for multimodal remote sensing image registration. IEEE Trans Geosci Remote Sens 57:9059–9070
You X, Tang Y (2007) Wavelet-based approach to character skeleton. IEEE Transactions on Image Processing, 1220-1231
Zhang GM, Chu J (2011) Recognizing partially occluded object from a line drawing. J Comput 6:1740–1747
Zhang D, Lu G (2005) Study and evaluation of different Fourier methods for image retrieval. Image Vis. Comput. 23:33–49
Zhang T, Li J, Jia W, Sun J, Yang H (2018) Fast and robust occluded face detection in ATM surveillance. Pattern Recognit Lett 107:33–40
Zhang Y, Cui J, Wang Z, Kang J, Min Y (2020) Leaf image recognition based on bag of features. Appl Sci 10:5177
Zhang D, Lu G (2002) A comparative study of Fourier descriptors for shape representation and retrieval. In: 5th Asian conference on computer vision
Zhang G, Xu J, Liu J (2015) A new method for recognition partially occluded curved objects under affine transformation. In 2015 10th International Conference on Intelligent Systems and Knowledge Engineering (ISKE), 456-461
Zhao C, Chan S, Cham WK, Chu LM (2015) Plant identification using leaf shapes a pattern counting approach. Pattern Recognit 48:3203–3215
Zuliani M, Bhagavathy S, Manjunath BS, Kenney CS (2004) Affine-invariant curve matching. In: 2004 international conference on image processing, pp 3041–3044
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Elghoul, S., Ghorbel, F. A fast and robust affine-invariant method for shape registration under partial occlusion. Int J Multimed Info Retr 11, 39–59 (2022). https://doi.org/10.1007/s13735-021-00224-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13735-021-00224-3