User-Driven Sampling Strategies in Image Exploitation
Abstract
Visual analytics and interactive machine learning both try to leverage the complementary strengths of humans and machines to solve complex data exploitation tasks. These fields overlap most significantly when training is involved: the visualization or machine learning tool improves over time by exploiting observations of the human-computer interaction. This paper focuses on one aspect of the human-computer interaction that we call user-driven sampling strategies. Unlike relevance feedback and active learning sampling strategies, where the computer selects which data to label at each iteration, we investigate situations where the user selects which data is to be labeled at each iteration. User-driven sampling strategies can emerge in many visual analytics applications but they have not been fully developed in machine learning. We discovered that in user-driven sampling strategies suggest new theoretical and practical research questions for both visualization science and machine learning. In this paper we identify and quantify the potential benefits of these strategies in a practical image analysis application. We find user-driven sampling strategies can sometimes provide significant performance gains by steering tools towards local minima that have lower error than tools trained with all of the data. Furthermore, in preliminary experiments we find these performance gains are particularly pronouncedmore »
- Authors:
-
- Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
- Publication Date:
- Research Org.:
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1234822
- Report Number(s):
- LA-UR-14-28017
Journal ID: ISSN 0277-786X
- Grant/Contract Number:
- AC52-06NA25396
- Resource Type:
- Journal Article: Accepted Manuscript
- Journal Name:
- Proceedings of SPIE - The International Society for Optical Engineering
- Additional Journal Information:
- Journal Volume: 9017; Journal ID: ISSN 0277-786X
- Publisher:
- SPIE
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; computing systems; feedback; human-computer interaction; image analysis; machine learning; visual analytics; visualization
Citation Formats
Harvey, Neal R., and Porter, Reid B. User-Driven Sampling Strategies in Image Exploitation. United States: N. p., 2013.
Web. doi:10.1117/12.2038581.
Harvey, Neal R., & Porter, Reid B. User-Driven Sampling Strategies in Image Exploitation. United States. https://doi.org/10.1117/12.2038581
Harvey, Neal R., and Porter, Reid B. 2013.
"User-Driven Sampling Strategies in Image Exploitation". United States. https://doi.org/10.1117/12.2038581. https://www.osti.gov/servlets/purl/1234822.
@article{osti_1234822,
title = {User-Driven Sampling Strategies in Image Exploitation},
author = {Harvey, Neal R. and Porter, Reid B.},
abstractNote = {Visual analytics and interactive machine learning both try to leverage the complementary strengths of humans and machines to solve complex data exploitation tasks. These fields overlap most significantly when training is involved: the visualization or machine learning tool improves over time by exploiting observations of the human-computer interaction. This paper focuses on one aspect of the human-computer interaction that we call user-driven sampling strategies. Unlike relevance feedback and active learning sampling strategies, where the computer selects which data to label at each iteration, we investigate situations where the user selects which data is to be labeled at each iteration. User-driven sampling strategies can emerge in many visual analytics applications but they have not been fully developed in machine learning. We discovered that in user-driven sampling strategies suggest new theoretical and practical research questions for both visualization science and machine learning. In this paper we identify and quantify the potential benefits of these strategies in a practical image analysis application. We find user-driven sampling strategies can sometimes provide significant performance gains by steering tools towards local minima that have lower error than tools trained with all of the data. Furthermore, in preliminary experiments we find these performance gains are particularly pronounced when the user is experienced with the tool and application domain.},
doi = {10.1117/12.2038581},
url = {https://www.osti.gov/biblio/1234822},
journal = {Proceedings of SPIE - The International Society for Optical Engineering},
issn = {0277-786X},
number = ,
volume = 9017,
place = {United States},
year = {Mon Dec 23 00:00:00 EST 2013},
month = {Mon Dec 23 00:00:00 EST 2013}
}
Works referenced in this record:
Interactive Machine Learning in Data Exploitation
journal, September 2013
- Porter, Reid; Theiler, James; Hush, Don
- Computing in Science & Engineering, Vol. 15, Issue 5
SIGKDD Special Interest Group on Knowledge Discovery in Data
journal, December 2005
- Basilico, Justin; Shapira, Bracha; Oldridge, Even
- ACM SIGKDD Explorations Newsletter, Vol. 7, Issue 2
Semantic Interaction for Sensemaking: Inferring Analytical Reasoning for Model Steering
journal, December 2012
- Endert, A.; Fiaux, P.; North, C.
- IEEE Transactions on Visualization and Computer Graphics, Vol. 18, Issue 12
Relevance feedback: a power tool for interactive content-based image retrieval
journal, January 1998
- Rui, Yong; Huang, T. S.; Ortega, M.
- IEEE Transactions on Circuits and Systems for Video Technology, Vol. 8, Issue 5, p. 644-655
Proceedings of the 25th international conference on Machine learning - ICML '08
conference, July 2008
- Cohen, William
- 25th Annual International Conference on Machine Learning
Proceedings of the 8th international conference on Intelligent user interfaces
conference, January 2003
- Leake, David
- Eighth International Conference on Intelligent User Interfaces
Works referencing / citing this record:
A Review of User Interface Design for Interactive Machine Learning
text, January 2018
- Dudley, John; Kristensson, Per Ola
- Apollo - University of Cambridge Repository
Power to the Oracle? Design Principles for Interactive Labeling Systems in Machine Learning
journal, January 2020
- Nadj, Mario; Knaeble, Merlin; Li, Maximilian Xiling
- KI - Künstliche Intelligenz, Vol. 34, Issue 2
A Review of User Interface Design for Interactive Machine Learning
journal, July 2018
- Dudley, John J.; Kristensson, Per Ola
- ACM Transactions on Interactive Intelligent Systems, Vol. 8, Issue 2