research-article

A RELIEF-based modality weighting approach for multimodal information retrieval

Authors:

Masaru KitsuregawaAuthors Info & Claims

ICMR '12: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval

Article No.: 54, Pages 1 - 8

https://doi.org/10.1145/2324796.2324858

Published: 05 June 2012 Publication History

Abstract

Despite the extensive number of studies for multimodal information fusion, the issue of determining the optimal modalities has not been adequately addressed yet. In this study, a RELIEF-based multimodal feature selection approach (RELIEF-RDR) is proposed. The original RELIEF algorithm is extended for weaknesses in three major issues; multi-labeled data, noise and class-specific feature selection. To overcome these weaknesses, discrimination based weighting mechanism of RELIEF is supported with two additional concepts; representation and reliability capabilities of features, without an increase in computational complexity. These capabilities of features are exploited by using the statistics on dissimilarities of training instances. The experiments conducted on TRECVID 2007 dataset validated the superiority of RELIEF-RDR over RELIEF.

References

[1]

P. K. Atrey, M. S. Kankanhalli, and J. B. Oommen. Goal-oriented optimal subset selection of correlated multimedia streams. ACM Trans. Multimedia Comput. Commun. Appl., 3, February 2007.

Digital Library

[2]

B. Mathieu, S. Essid, T. Fillon, J. Prado, and G. Richard. Yaafe, an easy to use and efficient audio feature extraction software, 2010. Proceedings of the 11th ISMIR conference, Utrecht, Netherlands.

[3]

E. Bruno and S. Marchand-Maillet. Multimodal preference aggregation for multimedia information retrieval. Journal of Multimedia, 4(5):321--329, 2009.

[4]

T. G. Dietterich. Machine-learning research: Four current directions. The AI Magazine, 18(4):97--136, 1998.

[5]

F. J. Ferri, P. Pudil, M. Hatef, and J. Kittler. Comparative study of techniques for large-scale feature selection, 1994. Pattern Recognition in Practice IV, Multiple Paradigms, Comparative Studies and Hybrid Systems.

[6]

I. Guyon and A. Elisseeff. An introduction to variable and feature selection. J. Mach. Learn. Res., 3:1157--1182, Mar. 2003.

Digital Library

[7]

M. A. Hall. Correlation-based Feature Subset Selection for Machine Learning. PhD thesis, Dept. of Computer Science, Univ. of Waikato, New Zealand, Apr. 1999.

[8]

E. B. Hunt, P. J. Stone, and J. Marin. Experiments in induction/Earl B. Hunt, Janet Marin, Philip J. Stone. Academic Press, New York:, 1966.

[9]

A. Jain, K. Nandakumar, and A. Ross. Score normalization in multimodal biometric systems. Pattern Recognition, 38(12):2270--2285, 2005.

Digital Library

[10]

A. K. Jain, R. P. Duin, and J. Mao. Statistical pattern recognition: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22:4--37, 2000.

Digital Library

[11]

A. Jakulin and I. Bratko. Analyzing attribute dependencies. In N. Lavrac, D. Gamberger, H. Blockeel, and L. Todorovski, editors, PKDD, volume 2838 of LNCS, pages 229--240. Springer, 2003.

[12]

M. Kankanhalli, J. Wang, and R. Jain. Experiential sampling on multiple data streams. Multimedia, IEEE Transactions on, 8(5):947--955, 2006.

Digital Library

[13]

K. Kira and L. A. Rendell. A practical approach to feature selection. In Proc. of the 9th Int. Workshop on Machine Learning, ML '92, pages 249--256, San Francisco, CA, USA, 1992. Morgan Kaufmann Publishers Inc.

Digital Library

[14]

J. Kittler. Feature Set Search Algorithms. Pattern Recognition and Signal Processing, pages 41--60, 1978.

[15]

J. Kludas, E. Bruno, and S. Marchand-Maillet. Information fusion in multimedia information retrieval. In Proc. of 5th Int. Workshop on Adaptive Multimedia Retrieval (AMR), Paris, France, July 5--6 2007.

[16]

J. Kludas, E. Bruno, and S. Marchand-Maillet. Can feature information interaction help for information fusion in multimedia problems? Multimedia Tools Appl., 42:57--71, March 2009.

Digital Library

[17]

I. Kononenko. Estimating attributes: analysis and extensions of relief. In Proc. of European Conf. on Machine Learning, pages 171--182, Secaucus, NJ, USA, 1994. Springer-Verlag New York, Inc.

Digital Library

[18]

H. Liu, H. Motoda, and L. Yu. A selective sampling approach to active feature selection. Artif. Intell., 159:49--74, November 2004.

Digital Library

[19]

J. Martínez. Mpeg-7 overview (version 10). Requirements ISO/IEC JTC1/SC29/WG11 N6828, International Organisation For Standardisation, Oct 2003.

[20]

P. Atrey, M. Hossain, A. E. Saddik, M. Kankanhalli. Multimodal fusion for multimedia analysis: a survey. Multimedia Systems, 16:345--379, 2010.

Digital Library

[21]

MPEG. Mpeg-7 reference software experimentation model, 2003. http://standards.iso.org/ittf/PubliclyAvailableStandards/c035364_ISO_IEC_15938-6(E)_Reference_Software.zip.

[22]

P. Over, G. Awad, W. Kraaij, and A. F. Smeaton. Trecvid 2007-overview. In TRECVID'07, 2007.

[23]

B. B. Pineda-Bautista, J. A. Carrasco-Ochoa, and J. F. M. Trinidad. General framework for class-specific feature selection. Expert Syst. Appl., 38(8):10018--10024, 2011.

Digital Library

[24]

N. Poh and J. Kittler. Multimodal Information Fusion: Theory and Applications for Human-Computer Interaction, chapter 8, pages 153--169. Academic Press, 2010.

[25]

J. R. Quinlan. Induction of decision trees. Mach. Learn., 1:81--106, March 1986.

[26]

M. Robnik-Sikonja and I. Kononenko. An adaptation of relief for attribute estimation in regression. In D. H. Fisher, editor, ICML, pages 296--304. Morgan Kaufmann, 1997.

Digital Library

[27]

M. Robnik-Šikonja and I. Kononenko. Theoretical and empirical analysis of relieff and rrelieff. Mach. Learn., 53:23--69, October 2003.

Digital Library

[28]

Y. Saeys, I. n. Inza, and P. Larrañaga. A review of feature selection techniques in bioinformatics. Bioinformatics, 23:2507--2517, September 2007.

Digital Library

[29]

G. Salton, A. Wong, and C. S. Yang. A vector space model for automatic indexing. Commun. ACM, 18:613--620, November 1975.

Digital Library

[30]

M. R. Sikonja. Speeding up relief algorithm with k-d trees. In Proceedings of Electrotehnical and Computer Science Conference (ERK'98), pages 137--140, 1998.

[31]

L. Snidaro, R. Niu, G. Foresti, and P. Varshney. Quality-based fusion of multiple video sensors for video surveillance. SMC-B: Cybernetics, IEEE Trans. on, 37(4):1044--1051, 2007.

Digital Library

[32]

C. G. M. Snoek and M. Worring. Multimodal video indexing: A review of the state-of-the-art. Multimedia Tools and Applications, 25(1):5--35, 2005.

Digital Library

[33]

Y. Sun. Iterative relief for feature weighting: Algorithms, theories, and applications. IEEE Trans. Pattern Anal. Mach. Intell., 29(6):1035--1051, 2007.

Digital Library

[34]

L. Wang, N. Zhou, and F. Chu. A general wrapper approach to selection of class-dependent features. IEEE Transactions on Neural Networks, 19(7):1267--1278, 2008.

Digital Library

[35]

Y. Wu, E. Y. Chang, K. C.-C. Chang, and J. R. Smith. Optimal multimodal fusion for multimedia data analysis. In Proc. of the 12th ACM Multimedia, pages 572--579, New York, NY, USA, 2004. ACM.

Digital Library

[36]

T. Yilmaz, A. Yazici, and Y. Yildirim. Exploiting class-specific features in multi-feature dissimilarity space for efficient querying of images. In FQAS, pages 149--161, 2011.

Digital Library

Cited By

Yilmaz TYazici AKitsuregawa M(2014)RELIEF-MMMultimedia Systems10.1007/s00530-014-0360-620:4(389-413)Online publication date: 1-Jul-2014
https://dl.acm.org/doi/10.1007/s00530-014-0360-6
Gulen EYilmaz TYazici A(2012)Multimodal Information Fusion for Semantic Video AnalysisInternational Journal of Multimedia Data Engineering and Management10.4018/jmdem.20121001033:4(52-74)Online publication date: 1-Oct-2012
https://doi.org/10.4018/jmdem.2012100103

Index Terms

A RELIEF-based modality weighting approach for multimodal information retrieval
1. Computing methodologies
  1. Machine learning
2. Information systems
  1. Information retrieval
    1. Document representation

Recommendations

RELIEF-MM: effective modality weighting for multimedia information retrieval

Fusing multimodal information in multimedia data usually improves the retrieval performance. One of the major issues in multimodal fusion is how to determine the best modalities. To combine the modalities more effectively, we propose a RELIEF-based ...
Iterative RELIEF for Feature Weighting: Algorithms, Theories, and Applications

RELIEF is considered one of the most successful algorithms for assessing the quality of features. In this paper, we propose a set of new feature weighting algorithms that perform significantly better than RELIEF, without introducing a large increase in ...
RELIEF-C: Efficient Feature Selection for Clustering over Noisy Data
ICTAI '11: Proceedings of the 2011 IEEE 23rd International Conference on Tools with Artificial Intelligence

RELIEF is a very effective and extremely popular feature selection algorithm developed for the first time in 1992 by Kira and Rendell. Since then it has been modified and expanded in various ways to make it more efficient. But the original RELIEF and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '12: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval

June 2012

489 pages

ISBN:9781450313292

DOI:10.1145/2324796

Conference Chairs:
Horace H. S. Ip
City University of Hong Kong
,
Yong Rui
Microsoft, China

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 June 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Türkiye Bilimsel ve Teknolojik Arastirma Kurumu

Conference

ICMR '12

Sponsor:

SIGMM

ICMR '12: International Conference on Multimedia Retrieval

June 5 - 8, 2012

Hong Kong, China

Acceptance Rates

ICMR '12 Paper Acceptance Rate 50 of 145 submissions, 34%;

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
103
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yilmaz TYazici AKitsuregawa M(2014)RELIEF-MMMultimedia Systems10.1007/s00530-014-0360-620:4(389-413)Online publication date: 1-Jul-2014
https://dl.acm.org/doi/10.1007/s00530-014-0360-6
Gulen EYilmaz TYazici A(2012)Multimodal Information Fusion for Semantic Video AnalysisInternational Journal of Multimedia Data Engineering and Management10.4018/jmdem.20121001033:4(52-74)Online publication date: 1-Oct-2012
https://doi.org/10.4018/jmdem.2012100103

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten