Verbal assistance to visual search in complex visualizations
Pages 126 - 133
Abstract
We present an exploratory experiment of the possible contribution of multimodal messages to facilitating visual search in complex scenes (e.g., visualizations of very large data sets). The main objective is to evaluate the actual contribution of verbal information to improving the efficiency (i.e., accuracy and speed) of the detection and selection (using the mouse) of visual targets. The evaluation focuses on the performances and subjective satisfaction of future users in three situations where the main free variable is the target's preliminary presentation: (i) display of the isolated target, (ii) verbal designation + information on its location in the scene, and (iii) multimodal (visual + oral) presentation. Multimodal target presentations improved subjects' performances significantly; they also received the highest subjective ratings.
References
[1]
1. Bernsen, N.-O. Foundations of multimodal representations, a taxonomy of representational modalities. Interacting with computers, 6:347-371, 1994.]]
[2]
2. Burhans, D.T., Chopra, R., Srihari, R.K. Domain specific understanding of spatial expressions. In Proc. IJCAI'95 Workshop on the Representation and Processing of Spatial Expressions, Montréal, August 1995, pp. 33-40, 1995.]]
[3]
3. Card, S.K., Mackinlay, J. The structure of the information visualization design space. In Proc. IEEE InfoVis'97, Phoenix, AZ, October 1997, pp. 92-99.]]
[4]
4. Coutaz, J., Nigay, L., Salber, D., Blandford, A., May, J, Young, R. Four easy pieces for assessing the usability of multimodal interaction: the CARE properties. In Proc. IFIP INTERACT'95, Lillehammer, Norway, June 1995, pp. 115-120.]]
[5]
5. Cribbin, T., Chen, C. Exploring cognitive issues in visual information retrieval. In Proc. IFIP INTERACT'01, Tokyo, July 2001, pp. 166-173.]]
[6]
6. Cribbin, T., Chen, C. A study of navigation strategies in spatial semantic visualizations. In Proc. HCI International 2001, New Orleans, August 2001, pp. 948-952.]]
[7]
7. Dollinger, S., Hoyer, W. Age and skill differences in the processing demands of visual inspection. Applied Cognitive Psychology, 10:225-239, 1996.]]
[8]
8. Drury, C.G. Inspection performance. In G. Salvendy (Ed.), Handbook of Industrial Engineering, 2nd Edition, John Wiley & Sons, Inc., chapter 88, pp. 2283-2314, 1992.]]
[9]
9. Engelkamp, J. Modality and modularity of the mind. In Actes du 5ème Colloque de l'ARC, Nancy, France, mars 1992, pp. 321-343.]]
[10]
10. Findlay, J., Gilchrist, I. Eye guidance and visual search. In {22}, chapter13, pp. 295-312, 1998.]]
[11]
11. Frank, A. Formal models for cognition-taxonomy of spatial location description and frames of reference. In C. Freksa, C. Habel, K.F. Wender (Eds.), Spatial Cognition -- An Interdisciplinary Approach to Representing and Processing Spatial Knowledge, Berlin: Springer Verlag, vol. 1, pp. 293-312, 1998.]]
[12]
12. Gramopadhye, A.K., Madhani, K. Visual search and visual lobe size. In Proc. 4th Int. Workshop on Visual Form, Capri, May 2001, LNCS-2059, Berlin: Springer Verlag, pp. 525-531.]]
[13]
13. Harisson, B.L., Vicente, K.J. An experimental evaluation of transparent menu usage. In Proc. ACM CHI'96, Vancouver, April 1996, New York: ACM Press, pp. 391-398.]]
[14]
14. Henderson, J.M., Hollingworth, A. Eye movements during scene viewing: an overview. In {22}, chapter 12, pp.269-293, 1998.]]
[15]
15. Itti, L., Koch, C. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Research, 40(10-12):1489-1506, 2000.]]
[16]
16. Mackay, W., Fayard, A., Frobert, L., Médini, L. Reinventing the familiar: exploring an augmented reality design space for air traffic control. In Proc. ACM CHI'98 Los Angeles, April 1998, New York: ACM Press, pp. 558-565.]]
[17]
17. Pirolli, P., Card, S.K., Van Der Wedge, M.M. The effect of information scent on searching information visualizations of large tree structures. In Proc. AVI'2000, Palermo, May 2000, pp. 161-172.]]
[18]
18. Plesniak, W., Ravikanth, P. Coincident display using haptics and holographic video. In Proc. ACM CHI'98,Los Angeles, April 1998, New York: ACM Press, pp. 304-311.]]
[19]
19. Rasmussen, J. Information processing and human-machine interaction: an approach to cognitive engineering.Amster-dam: North-Holland, 1986.]]
[20]
20. Shipman, F., Marshall, C., Moran, T. Finding and using implicit structure in human-organised spatial layouts of information.In Proc. CHI'95, Denver (CO), May 1995, New York: ACM Press, pp. 346-353.]]
[21]
21. Shneiderman, B. The eyes have it: A task by data type taxonomy for information visualizations. In Proc. IEEE Workshop on Visual Languages, Boulder (CO), September 1996, IEEE Computer Society Press, pp. 336-343.]]
[22]
22. Underwood, G., Ed. Eye guidance in reading and scene perception. Oxford (UK): Elsevier Science, 1998.]]
[23]
23. van Diepen, M.J., Wampers, M., d'Ydewall, G. Functional division of the visual field: moving masks and moving windows. In {22}, chapter 15, pp. 337-355, 1998.]]
[24]
24. Composer une photographie. Disponible à l'adresse http://100iso.free.fr/lessons/composer.htm-Lecture%20de%20l'image]]
Index Terms
- Verbal assistance to visual search in complex visualizations
Recommendations
Oral messages improve visual search
AVI '06: Proceedings of the working conference on Advanced visual interfacesInput multimodality combining speech and hand gestures has motivated numerous usability studies. Contrastingly, issues relating to the design and ergonomic evaluation of multimodal output messages combining speech with visual modalities have not yet ...
Changes in verbal and nonverbal conversational behavior in long-term interaction
ICMI '12: Proceedings of the 14th ACM international conference on Multimodal interactionWe present an empirical investigation of conversational behavior in dyadic interaction spanning multiple conversations, in the context of a developing interpersonal relationship between a health counselor and her clients. Using a longitudinal video ...
Dynamic tactile guidance for visual search tasks
UIST '12: Proceedings of the 25th annual ACM symposium on User interface software and technologyVisual search in large real-world scenes is both time consuming and frustrating, because the search becomes serial when items are visually similar. Tactile guidance techniques can facilitate search by allowing visual attention to focus on a subregion of ...
Comments
Information & Contributors
Information
Published In
November 2003
313 pages
ISBN:1581138032
DOI:10.1145/1063669
- Conference Chair:
- Khaldoun Zreik,
- Editor:
- Thomas Baudel,
- Program Chairs:
- Christophe Kolski,
- Eric Brangier
Copyright © 2003 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
- AFIHM: Ass. Francophone d'Interaction Homme-Machine
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 25 November 2003
Check for updates
Author Tags
Qualifiers
- Article
Acceptance Rates
Overall Acceptance Rate 103 of 199 submissions, 52%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 171Total Downloads
- Downloads (Last 12 months)2
- Downloads (Last 6 weeks)0
Reflects downloads up to 01 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in