skip to main content
10.1145/1101149.1101342acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
Article

Putting active learning into multimedia applications: dynamic definition and refinement of concept classifiers

Published: 06 November 2005 Publication History

Abstract

The authors developed an extensible system for video exploitation that puts the user in control to better accommodate novel situations and source material. Visually dense displays of thumbnail imagery in storyboard views are used for shot-based video exploration and retrieval. The user can identify a need for a class of audiovisual detection, adeptly and fluently supply training material for that class, and iteratively evaluate and improve the resulting automatic classification produced via multiple modality active learning and SVM. By iteratively reviewing the output of the classifier and updating the positive and negative training samples with less effort than typical for relevance feedback systems, the user can play an active role in directing the classification process while still needing to truth only a very small percentage of the multimedia data set. Examples are given illustrating the iterative creation of a classifier for a concept of interest to be included in subsequent investigations, and for a concept typically deemed irrelevant to be weeded out in follow-up queries. Filtering and browsing tools making use of existing and iteratively added concepts put the user further in control of the multimedia browsing and retrieval process.

References

[1]
Ahlberg, C. and Shneiderman, B. Visual Information Seeking: Tight Coupling of Dynamic Query Filters with Starfield Displays. In Proc. CHI '94, ACM Press, 1994, 313--317.
[2]
Burges, C.J.C. A Tutorial on Support Vector Machines for Pattern Recognition. Data Mining and Knowledge Discovery, 2, 2 (1998), 121--167.
[3]
Chang, E.Y., Tong, S., and Goh, K.-S. Support Vector Machine Concept-Dependent Active Learning for Image Retrieval. IEEE Transactions on Multimedia (anticipated 2005), http://mmdb2.ece.ucsb.edu/~echang/mm000540.pdf.
[4]
Chang, S.-F., moderator. Multimedia Access and Retrieval: The State of the Art and Future Directions. In Proc. ACM Multimedia '99 (Orlando FL, Nov. 1999), ACM Press, 443--445.
[5]
Christel, M. and Conescu, R. Addressing the Challenge of Visual Information Access from Digital Image and Video Libraries. In Proc JCDL '05, ACM Press, 2005, 69--78.
[6]
Forsyth, D., and Ponce, J. Computer Vision: A Modern Approach. Prentice Hall, Englewood Cliffs, NJ, 2002.
[7]
Freund, Y., and Schapire, R.E. A Decision-Theoretic Generalization of On-line Learning and an Application to Boosting. Journal of Computer and System Sciences, 55, 1, 1997, 119--139.
[8]
Gosselin, P.H., and Cord, M. RETIN AL: An active learning strategy for image category retrieval. In Proc. IEEE Conf. Image Processing (Singapore, October 2004), 2219--2222.
[9]
Hastie, T., Tibshirani, R., and Friedman, J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, 2001.
[10]
Hauptmann, A.G., and Christel, M.G. Successful Approaches in the TREC Video Retrieval Evaluations. Proc. ACM Multimedia '04, ACM Press (2004), 668--675.
[11]
Lee, H. and Smeaton, A.F. Designing the User Interface for the Fischlar Digital Video Library, J. Digital Info. 2(4), http://jodi.ecs.soton.ac.uk/Articles/v02/i04/Lee/, May 2002.
[12]
McCallum, A., and Nigam, K. Employing EM in pool-based active learning for text classification. In Proc. Int'l Conf. on Machine Learning. Morgan Kaufmann, 1998, 350--358.
[13]
Naphade, M., and Smith, J.R. Active Learning for Simultaneous Annotation of Multiple Binary Concepts. In Proc. IEEE Intl. Conf. on Multimedia and Expo (ICME) (Taipei, Taiwan, June, 2004), 77--80.
[14]
Naphade, M.R., and Smith, J.R. On the Detection of Semantic Concepts at TRECVID. Proc. ACM Multimedia '04, ACM Press (2004), 660--667.
[15]
Nguyen, H.T., and Smeulders, A. Active Learning Using Pre-clustering. In Proc. Int'l Conf. on Machine Learning (Banff, Canada, July 2004). ACM Press, 2004.
[16]
Rowe, L.A. and Jain, R., ACM SIGMM Retreat Report on Future Directions in Multimedia Research, http://www.sigmm.org/Events/reports/retreat03/sigmm-retreat03-final.pdf, March, 2004.
[17]
Schneiderman, H., and Kanade, T. Probabilistic Modeling of Local Appearance and Spatial Relationships of Object Recognition. In Conf. Computer Vision and Pattern Recognition (CVPR '98) (Santa Barbara, CA, June, 1998). IEEE Computer Society, 1998, 45--51.
[18]
Tong, S., and Chang, E. Support Vector Machine Active Learning for Image Retrieval. In Proc. ACM Multimedia 2001 (Ottawa, Canada, October, 2001). ACM Press, 2001, 107--118.
[19]
Trant, J. Image Retrieval Benchmark Database Service: A Needs Assessment and Preliminary Develoment Plan. Council on Library and Information Resources and the Coalition for Networked Information, Archives & Museum Informatics, http://www.clir.org/pubs/reports/ trant04/tranttext.pdf, January 2004.
[20]
Wang, L., Chan, K.L., and Zhang, Z. Bootstrapping SVM Active Learning by Incorporating Unlabelled Images for Image Retrieval. In Conf. Computer Vision and Pattern Recognition (CVPR '03) (Madison, WI, June, 1998). IEEE Computer Society, 2003, 629--634.

Cited By

View all
  • (2022)Automated Metadata in Multimedia Information Systems: Creation, Refinement, Use in Surrogates, and EvaluationundefinedOnline publication date: 10-Mar-2022
  • (2016)Visualization-Based Active Learning for Video AnnotationIEEE Transactions on Multimedia10.1109/TMM.2016.261422718:11(2196-2205)Online publication date: 1-Nov-2016
  • (2013)Active Bucket Categorization for High Recall Video RetrievalIEEE Transactions on Multimedia10.1109/TMM.2013.223789415:4(898-907)Online publication date: 1-Jun-2013
  • Show More Cited By

Index Terms

  1. Putting active learning into multimedia applications: dynamic definition and refinement of concept classifiers

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        MULTIMEDIA '05: Proceedings of the 13th annual ACM international conference on Multimedia
        November 2005
        1110 pages
        ISBN:1595930442
        DOI:10.1145/1101149
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Sponsors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 06 November 2005

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. active learning
        2. extensible concept classification
        3. video retrieval

        Qualifiers

        • Article

        Conference

        MM05

        Acceptance Rates

        MULTIMEDIA '05 Paper Acceptance Rate 49 of 312 submissions, 16%;
        Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)19
        • Downloads (Last 6 weeks)4
        Reflects downloads up to 27 Jan 2025

        Other Metrics

        Citations

        Cited By

        View all
        • (2022)Automated Metadata in Multimedia Information Systems: Creation, Refinement, Use in Surrogates, and EvaluationundefinedOnline publication date: 10-Mar-2022
        • (2016)Visualization-Based Active Learning for Video AnnotationIEEE Transactions on Multimedia10.1109/TMM.2016.261422718:11(2196-2205)Online publication date: 1-Nov-2016
        • (2013)Active Bucket Categorization for High Recall Video RetrievalIEEE Transactions on Multimedia10.1109/TMM.2013.223789415:4(898-907)Online publication date: 1-Jun-2013
        • (2013)Coaching the Exploration and Exploitation in Active Learning for Interactive Video RetrievalIEEE Transactions on Image Processing10.1109/TIP.2012.222290222:3(955-968)Online publication date: 1-Mar-2013
        • (2013)A kernel-based framework for image collection explorationJournal of Visual Languages and Computing10.1016/j.jvlc.2012.10.00824:1(53-67)Online publication date: 1-Feb-2013
        • (2013)A novel framework for concept detection on large scale video database and feature poolArtificial Intelligence Review10.1007/s10462-011-9287-x40:4(391-403)Online publication date: 1-Dec-2013
        • (2012)Efficient targeted search using a focus and context video browserACM Transactions on Multimedia Computing, Communications, and Applications10.1145/2379790.23797938:4(1-19)Online publication date: 30-Nov-2012
        • (2012)Interactive Video Indexing With Statistical Active LearningIEEE Transactions on Multimedia10.1109/TMM.2011.217478214:1(17-27)Online publication date: 1-Feb-2012
        • (2012)Inference of Co-occurring Classes: Multi-class and Multi-label ClassificationComputational Intelligence Paradigms in Advanced Pattern Classification10.1007/978-3-642-24049-2_9(171-197)Online publication date: 12-Jan-2012
        • (2011)Coached active learning for interactive video searchProceedings of the 19th ACM international conference on Multimedia10.1145/2072298.2072356(443-452)Online publication date: 28-Nov-2011
        • Show More Cited By

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media