Skip to main content
SpringerLink
Log in
Book cover

Asian Conference on Computer Vision

ACCV 2016: Computer Vision – ACCV 2016 pp 369–384Cite as

Sparse Gradient Pursuit for Robust Visual Analysis

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10111))

Abstract

Many high-dimensional data analysis problems, such as clustering and classification, usually involve the minimization of a Laplacian regularization, which is equivalent to minimize square errors of the gradient on a graph, i.e., the disparity among the adjacent nodes in a data graph. However, the Laplacian criterion usually preserves the locally homogeneous data structure but suppresses the discrimination among samples across clusters, which accordingly leads to undesirable confusion among similar observations belonging to different clusters. In this paper, we propose a novel criterion, named Sparse Gradient Pursuit (SGP), to simultaneously preserve the within-class homogeneity and the between-class discrimination for unsupervised data clustering. In addition, we show that the proposed SGP criterion is generic and can be extended to handle semi-supervised learning problems by incorporating the label information into the data graph. Though this unified semi-supervised learning model leads to a nonconvex optimization problem, we develop a new numerical scheme for the SGP related nonconvex optimization problem and analyze the convergence property of the proposed algorithm under mild conditions. Extensive experiments demonstrate that the proposed algorithm performs favorably against the state-of-the-art unsupervised and semi-supervised methods.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    For simplicity, we adopt the self-representation in this paper, but the proposed method is still valid with other dictionaries [13, 14].

  2. 2.

    The Textures can be downloaded from: http://sipi.usc.edu/database/database.cgi?volume=textures.

References

  1. Von Luxburg, U.: A tutorial on spectral clustering. Stat. Comput. 17, 395–416 (2007)

    Article  MathSciNet  Google Scholar 

  2. Shi, J., Malik, J.: Normalized cuts and image segmentation. TPAMI 22, 888–905 (2000)

    Article  Google Scholar 

  3. Hu, H., Lin, Z., Feng, J., Zhou, J.: Smooth representation clustering. In: CVPR, pp. 3834–3841 (2014)

    Google Scholar 

  4. Liu, R., Lin, Z., Su, Z.: Learning markov random walks for robust subspace clustering and estimation. Neural Netw. 59, 1–15 (2014)

    Article  MATH  Google Scholar 

  5. He, X., Cai, D., Shao, Y., Bao, H., Han, J.: Laplacian regularized Gaussian mixture model for data clustering. TKDE 23, 1406–1418 (2011)

    Google Scholar 

  6. Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15, 1373–1396 (2003)

    Article  MATH  Google Scholar 

  7. Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)

    Article  Google Scholar 

  8. Yin, M., Gao, J., Lin, Z.: Laplacian regularized low-rank representation and its applications. TPAMI 38, 504–517 (2016)

    Article  Google Scholar 

  9. Belkin, M., Niyogi, P., Sindhwani, V.: Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. JMLR 7, 2399–2434 (2006)

    MathSciNet  MATH  Google Scholar 

  10. Gao, S., Tsang, I.W.H., Chia, L.T.: Laplacian sparse coding, hypergraph laplacian sparse coding, and applications. TPAMI 35, 92–104 (2013)

    Article  Google Scholar 

  11. He, X., Yan, S., Hu, Y., Niyogi, P., Zhang, H.J.: Face recognition using Laplacianfaces. TPAMI 27, 328–340 (2005)

    Article  Google Scholar 

  12. Yang, Y., Wang, Z., Yang, J., Han, J., Huang, T.S.: Regularized l1-graph for data clustering. In: BMVC (2014)

    Google Scholar 

  13. Protter, M., Elad, M.: Image sequence denoising via sparse and redundant representations. TIP 18, 27–35 (2009)

    MathSciNet  Google Scholar 

  14. Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online dictionary learning for sparse coding. In: ICML, pp. 689–696 (2009)

    Google Scholar 

  15. Chapelle, O., Schölkopf, B., Zien, A.: Semi-supervised Learning. MIT Press, Cambridge (2006)

    Book  Google Scholar 

  16. Zhu, X.: Semi-supervised learning. In: Encyclopedia of Machine Learning, pp. 892–897 (2011)

    Google Scholar 

  17. Zhu, X., Ghahramani, Z., Lafferty, J., et al.: Semi-supervised learning using Gaussian fields and harmonic functions. In: ICML, vol. 3, pp. 912–919 (2003)

    Google Scholar 

  18. Zhuang, L., Gao, H., Lin, Z., Ma, Y., Zhang, X., Yu, N.: Non-negative low rank and sparse graph for semi-supervised learning. In: CVPR, pp. 2328–2335 (2012)

    Google Scholar 

  19. Li, S., Fu, Y.: Low-rank coding with b-matching constraint for semi-supervised classification. In: IJCAI (2013)

    Google Scholar 

  20. Joachims, T., et al.: Transductive learning via spectral graph partitioning. In: ICML, vol. 3, pp. 290–297 (2003)

    Google Scholar 

  21. Belkin, M., Matveeva, I., Niyogi, P.: Regularization and semi-supervised learning on large graphs. In: Shawe-Taylor, J., Singer, Y. (eds.) COLT 2004. LNCS, vol. 3120, pp. 624–638. Springer, Heidelberg (2004). doi:10.1007/978-3-540-27819-1_43

    Chapter  Google Scholar 

  22. Yan, S., Wang, H.: Semi-supervised learning by sparse representation. In: SDM, pp. 792–801 (2009)

    Google Scholar 

  23. Li, C.G., Lin, Z., Zhang, H., Guo, J.: Learning semi-supervised representation towards a unified optimization framework for semi-supervised learning. In: ICCV, pp. 2767–2775 (2015)

    Google Scholar 

  24. Wright, J., Ganesh, A., Rao, S., Peng, Y., Ma, Y.: Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. In: NIPS, pp. 2080–2088 (2009)

    Google Scholar 

  25. Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., Ma, Y.: Robust recovery of subspace structures by low-rank representation. TPAMI 35, 171–184 (2013)

    Article  Google Scholar 

  26. Li, Z., Liu, J., Tang, J., Lu, H.: Robust structured subspace learning for data representation. TPAMI 37, 2085–2098 (2015)

    Article  Google Scholar 

  27. Zheng, M., Bu, J., Chen, C., Wang, C., Zhang, L., Qiu, G., Cai, D.: Graph regularized sparse coding for image representation. TIP 20, 1327–1336 (2011)

    MathSciNet  Google Scholar 

  28. Liu, R., Lin, Z., Su, Z.: Linearized alternating direction method with parallel splitting and adaptive penalty for separable convex programs in machine learning. In: ACML, pp. 116–132 (2013)

    Google Scholar 

  29. Elhamifar, E., Vidal, R.: Sparse subspace clustering: algorithm, theory, and applications. TPAMI 35, 2765–2781 (2013)

    Article  Google Scholar 

  30. Nasihatkon, B., Hartley, R.: Graph connectivity in sparse subspace clustering. In: CVPR, pp. 2137–2144 (2011)

    Google Scholar 

  31. Tang, K., Dunson, D.B., Su, Z., Liu, R., Zhang, J., Dong, J.: Subspace segmentation by dense block and sparse representation. Neural Netw. 75, 66–76 (2016)

    Article  Google Scholar 

  32. Bresson, X., Laurent, T., Uminsky, D., von Brecht, J.: Multiclass total variation clustering. In: NIPS, pp. 1421–1429 (2013)

    Google Scholar 

  33. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2011)

    Article  MATH  Google Scholar 

  34. Lin, Z., Liu, R., Su, Z.: Linearized alternating direction method with adaptive penalty for low-rank representation. In: NIPS, pp. 612–620 (2011)

    Google Scholar 

  35. Yuan, G., Ghanem, B.: l0tv: a new method for image restoration in the presence of impulse noise. In: CVPR, pp. 5369–5377 (2015)

    Google Scholar 

  36. Liu, G., Lin, Z., Yu, Y.: Robust subspace segmentation by low-rank representation. In: ICML, pp. 663–670 (2010)

    Google Scholar 

  37. Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression (PIE) database. In: AFG, pp. 46–51 (2002)

    Google Scholar 

  38. Bartels, R.H., Stewart, G.W.: Solution of the matrix equation AX+XB=C. CACM 15, 820–826 (1972)

    Article  Google Scholar 

  39. Hull, J.J.: A database for handwritten text recognition research. TPAMI 16, 550–554 (1994)

    Article  Google Scholar 

  40. Tang, K., Liu, R., Su, Z., Zhang, J.: Structure-constrained low-rank representation. TNNLS 25, 2167–2179 (2014)

    Google Scholar 

  41. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146, 459–494 (2014)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgement

Risheng Liu is supported by National Natural Science Foundation of China (NSFC) (Nos. 61300086, 61432003, 61672125), Fundamental Research Funds for the Central Universities (No. DUT15QY15), and the Hong Kong Scholar Program (No. XJ2015008). Zhixun Su is supported by NSFC (No. 61572099) and National Science and Technology Major Project (Nos. ZX20140419, 2014ZX04001011).

Author information

Authors and Affiliations

  1. Dalian University of Technology, Dalian, China

    Jiangxin Dong, Risheng Liu, Yiyang Wang, Xindong Zhang & Zhixun Su

  2. Liaoning Normal University, Dalian, China

    Kewei Tang

Authors
  1. Jiangxin Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Risheng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kewei Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yiyang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xindong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhixun Su
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Risheng Liu .

Editor information

Editors and Affiliations

  1. National Tsing Hua University, Hsinchu, Taiwan

    Shang-Hong Lai

  2. Graz University of Technology, Graz, Austria

    Vincent Lepetit

  3. Drexel University, Philadelphia, Pennsylvania, USA

    Ko Nishino

  4. The University of Tokyo, Tokyo, Japan

    Yoichi Sato

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 2488 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Dong, J., Liu, R., Tang, K., Wang, Y., Zhang, X., Su, Z. (2017). Sparse Gradient Pursuit for Robust Visual Analysis. In: Lai, SH., Lepetit, V., Nishino, K., Sato, Y. (eds) Computer Vision – ACCV 2016. ACCV 2016. Lecture Notes in Computer Science(), vol 10111. Springer, Cham. https://doi.org/10.1007/978-3-319-54181-5_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-54181-5_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-54180-8

  • Online ISBN: 978-3-319-54181-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation