short-paper

Partial Multi-Modal Sparse Coding via Adaptive Similarity Structure Regularization

Authors:

Yueting ZhuangAuthors Info & Claims

MM '16: Proceedings of the 24th ACM international conference on Multimedia

Pages 152 - 156

https://doi.org/10.1145/2964284.2967201

Published: 01 October 2016 Publication History

Abstract

Multi-modal sparse coding has played an important role in many multimedia applications, where data are usually with multiple modalities. Recently, various multi-modal sparse coding approaches have been proposed to learn sparse codes of multi-modal data, which assume that data appear in all modalities, or at least there is one modality containing all data. However, in real applications, it is often the case that some modalities of the data may suffer from missing information and thus result in partial multi-modality data. In this paper, we propose to solve the partial multi-modal sparse coding problem via multi-modal similarity structure regularization. Specifically, we propose a partial multi-modal sparse coding framework termed Adaptive Partial Multi-Modal Similarity Structure Regularization for Sparse Coding (AdaPM²SC), which preserves the similarity structure within the same modality and between different modalities. Experimental results conducted on two real-world datasets demonstrate that AdaPM²SC significantly outperforms the state-of-the-art methods under partial multi-modality scenario.

References

[1]

S. Boyd and L. Vandenberghe. Convex optimization. Cambridge university press, 2004.

[2]

T. Cao, V. Jojic, S. Modla, D. Powell, K. Czymmek, and M. Niethammer. Robust multimodal dictionary learning. In Medical Image Computing and Computer-Assisted Intervention--MICCAI 2013, pages 259--266. Springer, 2013.

[3]

Y. Han, F. Wu, D. Tao, J. Shao, Y. Zhuang, and J. Jiang. Sparse unsupervised dimensionality reduction for multiple view data. Circuits and Systems for Video Technology, IEEE Transactions on, 22(10):1485--1496, 2012.

Digital Library

[4]

X.-Y. Jing, R.-M. Hu, F. Wu, X.-L. Chen, Q. Liu, and Y.-F. Yao. Uncorrelated multi-view discrimination dictionary learning for recognition. In AAAI, 2014.

Digital Library

[5]

H. Lee, A. Battle, R. Raina, and A. Y. Ng. Efficient sparse coding algorithms. In Advances in neural information processing systems, pages 801--808, 2006.

Digital Library

[6]

F. Nie, X. Wang, and H. Huang. Clustering and projected clustering with adaptive neighbors. In SIGKDD, pages 977--986. ACM, 2014.

Digital Library

[7]

S. Shekhar, V. M. Patel, N. M. Nasrabadi, and R. Chellappa. Joint sparse representation for robust multimodal biometrics recognition. TPAMI, 36(1):113--126, 2014.

Digital Library

[8]

Y. Shi, Y. Gao, Y. Yang, Y. Zhang, and D. Wang. Multimodal sparse representation-based classification for lung needle biopsy images. Biomedical Engineering, IEEE Transactions on, 60(10):2675--2685, 2013.

[9]

Q. Wang, L. Si, and B. Shen. Learning to hash on partial multi-modal data. In IJCAI, pages 3904--3910. AAAI Press, 2015.

Digital Library

[10]

S. Wang, L. Zhang, Y. Liang, and Q. Pan. Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In CVPR, pages 2216--2223. IEEE, 2012.

Digital Library

[11]

J. Yu, Y. Rui, and D. Tao. Click prediction for web image reranking using multimodal sparse coding. IEEE Transactions on Image Processing, 23(5):2019--2032, 2014.

[12]

J. Zheng and Z. Jiang. Learning view-invariant sparse representations for cross-view action recognition. In Proceedings of the IEEE International Conference on Computer Vision, pages 3176--3183, 2013.

Digital Library

[13]

M. Zheng, J. Bu, C. Chen, C. Wang, L. Zhang, G. Qiu, and D. Cai. Graph regularized sparse coding for image representation. TIP, 20(5):1327--1336, 2011.

Digital Library

[14]

S. Zheng, B. Xie, K. Huang, and D. Tao. Multi-view pedestrian recognition using shared dictionary learning with group sparsity. In Neural Information Processing, pages 629--638. Springer, 2011.

[15]

S.-Y. L. Y. J. Zhi and H. Zhou. Partial multi-view clustering. In AAAI. AAAI Press, 2014.

[16]

Y. T. Zhuang, Y. F. Wang, F. Wu, Y. Zhang, and W. M. Lu. Supervised coupled dictionary learning with group structures for multi-modal retrieval. In AAAI, 2013.

Digital Library

Cited By

Li YLiu SWang XJing P(2023)Self-supervised deep partial adversarial network for micro-video multimodal classificationInformation Sciences10.1016/j.ins.2022.11.111630(356-369)Online publication date: Jun-2023
https://doi.org/10.1016/j.ins.2022.11.111
Tong WChu XJiang WLi Z(2023)A multimodal dual-fusion entity extraction model for large and complex devicesComputer Communications10.1016/j.comcom.2023.07.026210(1-9)Online publication date: Oct-2023
https://doi.org/10.1016/j.comcom.2023.07.026
Adam KAl-Maadeed SAkbari Y(2022)Hierarchical Fusion Using Subsets of Multi-Features for Historical Arabic Manuscript DatingJournal of Imaging10.3390/jimaging80300608:3(60)Online publication date: 1-Mar-2022
https://doi.org/10.3390/jimaging8030060
Show More Cited By

Index Terms

Partial Multi-Modal Sparse Coding via Adaptive Similarity Structure Regularization
1. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Unsupervised learning and clustering

Recommendations

Noise-robust semi-supervised learning via fast sparse coding

This paper presents a novel noise-robust graph-based semi-supervised learning algorithm to deal with the challenging problem of semi-supervised learning with noisy initial labels. Inspired by the successful use of sparse coding for noise reduction, we ...
Multi-Hypergraph Consistent Sparse Coding
Survey Paper, Regular Papers and Special Issue: Social Media Processing

Sparse representation has been a powerful technique for modeling high-dimensional data. As an unsupervised technique to extract sparse representations, sparse coding encodes the original data into a new sparse code space and simultaneously learns a ...
Structure-Constrained Low-Rank and Partial Sparse Representation with Sample Selection for image classification

In this paper, we propose a novel Structure-Constrained Low-Rank and Partial Sparse Representation algorithm for image classification. First, a Structure-Constrained Low-Rank Dictionary Learning (SCLRDL) algorithm is proposed, which imposes both ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '16: Proceedings of the 24th ACM international conference on Multimedia

October 2016

1542 pages

ISBN:9781450336031

DOI:10.1145/2964284

General Chairs:
Alan Hanjalic
Delft University of Technology
,
Cees Snoek
Qualcomm Research Netherlands / University of Amsterdam
,
Marcel Worring
University of Amsterdam
,
Moderator:
Dick Bulterman
CWI / VU University Amsterdam
,
Program Chairs:
Benoit Huet
EURECOM
,
Aisling Kelliher
Virginia Tech
,
Yiannis Kompatsiaris
CERTH-ITI
,
Jin Li
Microsoft

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

MM '16

Sponsor:

SIGMM

MM '16: ACM Multimedia Conference

October 15 - 19, 2016

Amsterdam, The Netherlands

Acceptance Rates

MM '16 Paper Acceptance Rate 52 of 237 submissions, 22%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
388
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)1

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li YLiu SWang XJing P(2023)Self-supervised deep partial adversarial network for micro-video multimodal classificationInformation Sciences10.1016/j.ins.2022.11.111630(356-369)Online publication date: Jun-2023
https://doi.org/10.1016/j.ins.2022.11.111
Tong WChu XJiang WLi Z(2023)A multimodal dual-fusion entity extraction model for large and complex devicesComputer Communications10.1016/j.comcom.2023.07.026210(1-9)Online publication date: Oct-2023
https://doi.org/10.1016/j.comcom.2023.07.026
Adam KAl-Maadeed SAkbari Y(2022)Hierarchical Fusion Using Subsets of Multi-Features for Historical Arabic Manuscript DatingJournal of Imaging10.3390/jimaging80300608:3(60)Online publication date: 1-Mar-2022
https://doi.org/10.3390/jimaging8030060
Tie YLi XZhang TJin CZhao XTie J(2022)Deep learning based audio and video cross-modal recommendation2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC53654.2022.9945521(2366-2371)Online publication date: 9-Oct-2022
https://doi.org/10.1109/SMC53654.2022.9945521
Akbari YElharrouss OAl-Maadeed S(2022)Feature fusion based on joint sparse representations and wavelets for multiview classificationPattern Analysis and Applications10.1007/s10044-022-01110-226:2(645-653)Online publication date: 22-Sep-2022
https://doi.org/10.1007/s10044-022-01110-2
Shan JHou CTao HZhuge WYi D(2019)Full Representation Data Embedding via Nonoverlapping Historical FeaturesIEEE Transactions on Cybernetics10.1109/TCYB.2018.283767549:8(3006-3019)Online publication date: Aug-2019
https://doi.org/10.1109/TCYB.2018.2837675
Xu NGuo YZheng XWang QLuo XBoll SMu Lee KLuo JZhu WByun HWen Chen CLienhart RMei T(2018)Partial Multi-view Subspace ClusteringProceedings of the 26th ACM international conference on Multimedia10.1145/3240508.3240679(1794-1801)Online publication date: 15-Oct-2018
https://dl.acm.org/doi/10.1145/3240508.3240679
Xu DZhao ZXiao JWu FZhang HHe XZhuang YLiu QLienhart RWang HChen SBoll SChen PFriedland GLi JYan S(2017)Video Question Answering via Gradually Refined Attention over Appearance and MotionProceedings of the 25th ACM international conference on Multimedia10.1145/3123266.3123427(1645-1653)Online publication date: 23-Oct-2017
https://dl.acm.org/doi/10.1145/3123266.3123427
Zhao ZLin JJiang XCai DHe XZhuang YLiu QLienhart RWang HChen SBoll SChen PFriedland GLi JYan S(2017)Video Question Answering via Hierarchical Dual-Level Attention Network LearningProceedings of the 25th ACM international conference on Multimedia10.1145/3123266.3123364(1050-1058)Online publication date: 23-Oct-2017
https://dl.acm.org/doi/10.1145/3123266.3123364

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten