skip to main content
10.1145/1924559.1924596acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicvgipConference Proceedingsconference-collections
research-article

Learning an interactive segmentation system

Published: 12 December 2010 Publication History

Abstract

Many successful applications of computer vision to image or video manipulation are interactive by nature. However, parameters of such systems are often trained neglecting the user. Traditionally, interactive systems have been treated in the same manner as their fully automatic counterparts. Their performance is evaluated by computing the accuracy of their solutions under some fixed set of user interactions. This paper proposes a new evaluation and learning method which brings the user in the loop. It is based on the use of an active robot user -- a simulated model of a human user. We show how this approach can be used to evaluate and learn parameters of state-of-the-art interactive segmentation systems. We also show how simulated user models can be integrated into the popular max-margin method for parameter learning and propose an algorithm to solve the resulting optimisation problem.

References

[1]
amazon.com. Amazon mechanical turk. https://www.mturk.com, 2010.
[2]
X. Bai and G. Sapiro. A geodesic framework for fast interactive image and video segmentation and matting. In ICCV, 2007.
[3]
D. Batra, A. Kowdle, D. Parikh, J. Luo, and T. Chen. iCoseg: Interactive co-segmentation with intelligent scribble guidance. In CVPR, 2010.
[4]
M. Bilgic and L. Getoor. Reflect and correct: A misclassification prediction approach to active inference. ACM TKDD, 3(4), 2009.
[5]
A. Blake, P. Kohli, and C. Rother. Markov Random Fields for Vision and Image Processing. MIT Press, 2011.
[6]
A. Blake, C. Rother, M. Brown, P. Perez, and P. Tor. Interactive image segmentation using an adaptive GMMRF model. In ECCV, 2004.
[7]
Y. Boykov and M. Jolly. Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images. In ICCV, 2001.
[8]
O. Duchenne, J.-Y. Audibert, R. Keriven, J. Ponce, and F. Ségonne. Segmentation by transduction. In CVPR, 2008.
[9]
M. Everingham, L. V. Gool, C. K. I. Williams, J. Winn, and A. Zisserman. http://www.pascal-network.org/challenges/VOC, 2009.
[10]
J. A. Fails and J. Dan R. Olsen. Interactive machine learning. In International Conference on Intelligent User Interfaces, 2003.
[11]
T. Finley and T. Joachims. Training structural SVMs when exact inference is intractable. In ICML, 2008.
[12]
L. Grady. Random walks for image segmentation. PAMI, 28:1--17, 2006.
[13]
V. Gulshan, C. Rother, A. Criminisi, A. Blake, and A. Zisserman. Geodesic star convexity for interactive image segmentation. In CVPR, 2010.
[14]
P. Kohli, L. Ladicky, and P. Torr. Robust higher order potentials for enforcing label consistency. In CVPR, 2008.
[15]
P. Kohli and P. Torr. Efficiently solving dynamic MRFs using graph cuts. In ICCV, 2005.
[16]
Y. Li, J. Sun, C.-K. Tang, and H.-Y. Shum. Lazy snapping. SIGGRAPH, 23, 2004.
[17]
J. Liu, J. Sun, and H.-Y. Shum. Paint selection. In SIGGRAPH, 2009.
[18]
H. Nickisch, P. Kohli, and C. Rother. Learning an interactive segmentation system. Technical report, http://arxiv.org/abs/0912.2492, 2009.
[19]
S. Nowozin and C. H. Lampert. Global connectivity potentials for random field models. In CVPR, 2009.
[20]
C. Rother, V. Kolmogorov, and A. Blake. Grabcut - interactive foreground extraction using iterated graph cuts. SIGGRAPH, 23(3):309--314, 2004.
[21]
B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman. Labelme: a database and web-based tool for image annotation. IJCV, 77:157--173, 2008.
[22]
D. Singaraju, L. Grady, and R. Vidal. P-brush: Continuous valued MRFs with normed pairwise distributions for image segmentation. In CVPR, 2009.
[23]
A. Sorokin and D. Forsyth. Utility data annotation with amazon mechanical turk. In Internet Vision Workshop at CVPR, 2008.
[24]
M. Szummer, P. Kohli, and D. Hoiem. Learning CRFs using graph cuts. In ECCV, 2008.
[25]
B. Taskar, V. Chatalbashev, and D. Koller. Learning associative markov networks. In ICML, 2004.
[26]
I. Tsochantaridis, T. Hofmann, T. Joachims, and Y. Altun. Support vector learning for interdependent and structured output spaces. In ICML, 2004.
[27]
S. Vicente, V. Kolmogorov, and C. Rother. Graph cut based image segmentation with connectivity priors. In CVPR, 2008.
[28]
L. von Ahn and L. Dabbish. Labeling images with a computer game. In SIGCHI, pages 319--326, 2004.
[29]
L. Wasserman. All of Statistics. Springer, 2004.

Cited By

View all
  • (2024)Deep Interactive Segmentation of Medical Images: A Systematic Review and TaxonomyIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.345262946:12(10998-11018)Online publication date: Dec-2024
  • (2020)Interactive Training And Architecture For Deep Object Selection2020 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME46284.2020.9102942(1-6)Online publication date: Jul-2020
  • (2019)Click CarvingInternational Journal of Computer Vision10.1007/s11263-019-01184-2127:9(1321-1344)Online publication date: 20-Sep-2019
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICVGIP '10: Proceedings of the Seventh Indian Conference on Computer Vision, Graphics and Image Processing
December 2010
533 pages
ISBN:9781450300605
DOI:10.1145/1924559
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 December 2010

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. SVMstruct
  2. interactive learning
  3. interactive segmentation

Qualifiers

  • Research-article

Conference

ICVGIP '10

Acceptance Rates

Overall Acceptance Rate 95 of 286 submissions, 33%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)1
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Deep Interactive Segmentation of Medical Images: A Systematic Review and TaxonomyIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.345262946:12(10998-11018)Online publication date: Dec-2024
  • (2020)Interactive Training And Architecture For Deep Object Selection2020 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME46284.2020.9102942(1-6)Online publication date: Jul-2020
  • (2019)Click CarvingInternational Journal of Computer Vision10.1007/s11263-019-01184-2127:9(1321-1344)Online publication date: 20-Sep-2019
  • (2018)Iterative Interaction Training for Segmentation Editing NetworksMachine Learning in Medical Imaging10.1007/978-3-030-00919-9_42(363-370)Online publication date: 15-Sep-2018
  • (2017)UI-netProceedings of the Eurographics Workshop on Visual Computing for Biology and Medicine10.2312/vcbm.20171248(143-147)Online publication date: 7-Sep-2017
  • (2013)Evaluation of Interactive Segmentation Algorithms Using Densely Sampled Correct InteractionsImage Analysis and Processing – ICIAP 201310.1007/978-3-642-41181-6_20(191-200)Online publication date: 2013
  • (2012)Seeded watershed cut uncertainty estimators for guided interactive segmentation2012 IEEE Conference on Computer Vision and Pattern Recognition10.1109/CVPR.2012.6247747(765-772)Online publication date: Jun-2012
  • (2012)Transforming cluster-based segmentation for use in OpenVL by mainstream developersProceedings of the 11th international conference on Computer Vision - Volume Part I10.1007/978-3-642-37410-4_22(254-265)Online publication date: 5-Nov-2012
  • (2012)An Interactive Colour Video Segmentation: A Granular Computing ApproachElectrical Engineering and Intelligent Systems10.1007/978-1-4614-2317-1_11(135-146)Online publication date: 2-May-2012
  • (2011)A statistical approach to interactive image segmentation2011 International Conference on Multimedia Technology10.1109/ICMT.2011.6002016(5260-5263)Online publication date: Jul-2011
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media