Supervised learning for parameterized Koopmans–Beckmann’s graph matching☆

https://doi.org/10.1016/j.patrec.2020.12.012Get rights and content

Highlights

  • •

    Discusses a novel graph matching model, i.e., parameterized Koopmans–Beckmann’s graph matching (KBGMw).

  • •

    Proposes a supervised learning method for KBGMw.

  • •

    Shows the performances of the proposed method and several state-of-the-art graph matching methods.

  • •

    Summarizes the advantages and disadvantages of the proposed method.

Abstract

In this paper, we discuss a novel graph matching problem, namely the parameterized Koopmans–Beckmann’s graph matching (KBGMw). KBGMw is defined by a weighted linear combination of a series of Koopmans–Beckmann’s graph matching. First, we show that KBGMw can be taken as a special case of the parameterized Lawler’s graph matching, subject to certain conditions. Second, based on structured SVM, we propose a supervised learning method for automatically estimating the parameters of KBGMw. Experimental results on both synthetic and real image matching data sets show that the proposed method achieves relatively better performances, even superior to some deep learning methods.

Section snippets

Instruction

Graph matching involves establishing correspondences between the nodes of two graphs, usually with the consideration of graph structures. It is a fundamental problem closely related to many famous problems, including, for instance, MAP estimation of Markov random field [25] and quadratic assignment problem [12]. Moreover, it plays a central role in many practical applications also, such as visual object tracking [31], human activity recognition [33], 2D or 3D reconstruction [23], and image

Literature review and preliminaries

Graph matching is in general an NP-hard problem such that many computationally efficient methods have been proposed to seek inexact solutions. For example, the graduated assignment method [13], spectral methods [9], [18], and reweighted random walks [6] are some typical methods for LGM; the path following method [35] for KBGM; the GNCCP based methods [20], [32] for both LGM and KBGM. The key point of these approximate methods is to relax the discrete graph matching problem to be a continuous

Problem formulation

We define KBGMw on a set of adjacency matrices as follows,minX∈PFKBGMw(G,X;w)=∑l=1mwl∥ADl−XAMlXT∥F2s.t.∀l∈{1,2,…,m},wl≥0,∑l=1mwl=1 where ADl and AMl are respectively denote the lth adjacency matrices of graphs GD and GM, w=[w1,…,wm]T, wl measures the weight of the lth KBGM.

KBGMw can be seen as a generalization of KBGM, which allows considering different kinds of graph adjacency matrices in a single unifying model. Mathematically it can achieve performance better than, or at least equivalent to

Experimental results and analysis

We evaluate the performance of the proposed method and several peer methods both on synthetic and real-world image data sets. The peer methods involved in our experiments are as follows.

  • - SM [18]: The most classic spectral method.

  • - SMAC [27]: A spectral method with affine constraints.

  • - PGM [9]: A spectral method with a probabilistic matching scheme.

  • - IPFP [19]: The integer projected fixed point method, which solves graph matching problem iteratively by optimizing its first-order Taylor

Conclusion

In this paper, we propose a supervised learning method for the parameterized Koopmans–Beckmann’s graph matching problem (KBGMw). Firstly, we show that KBGMw can be taken as a special case of the parameterized Lawler’s graph matching (LGMw), subject to certain conditions. Then, based on the structured SVM, we propose a supervised learning method to automatically estimate its parameters. We evaluate the proposed method on two synthetic data sets with rotation and noise transformations, and two

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was supported in part by the National Key Research and Development Plan of China (Grants 2016YFC0300801 and 2017YFB1300202), and in part by the National Natural Science Foundation of China (NSFC) (Grants 61503383, 61633009, and U1613213).

References (36)

  • A. Egozi et al.

    A probabilistic approach to spectral graph matching

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2013)
  • M. Everingham et al.

    The pascal visual object classes (VOC) challenge

    Int. J. Comput. Vis.

    (2010)
  • S. Francesc

    A general model to define the substitution, insertion and deletion graph edit costs based on an embedded space

    Pattern Recognit. Lett.

    (2020)
  • S. Gold et al.

    A graduated assignment algorithm for graph matching

    IEEE Trans. Pattern Anal. Mach. Intell.

    (1996)
  • P.E. Hart et al.

    A formal basis for the heuristic determination of minimum cost paths

    IEEE Trans. Syst. Sci. Cybern.

    (1968)
  • J.E. Kelley

    The cutting-plane method for solving convex programs

    J. Soc. Ind. Appl. Math.

    (1960)
  • T.C. Koopmans et al.

    Assignment problems and the location of economic activities

    Econometrica

    (1957)
  • E.L. Lawler

    The quadratic assignment problem

    Manag. Sci.

    (1963)

    • Cited by (3)

    ☆

    Editor: Andrea F. Abate

    View full text
    © 2020 Elsevier B.V. All rights reserved.