skip to main content
research-article

combinFormation: Mixed-initiative composition of image and text surrogates promotes information discovery

Published: 23 December 2008 Publication History

Abstract

combinFormation is a mixed-initiative creativity support tool for searching, browsing, organizing, and integrating information. Images and text are connected to represent surrogates (enhanced bookmarks), optimizing the use of human cognitive facilities. Composition, an alternative to lists and spatial hypertext, is used to represent a collection of surrogates as a connected whole, using principles from art and design. This facilitates the creative process of information discovery, in which humans develop new ideas while finding and collecting information. To provoke the user to think about the large space of potentially relevant information resources, a generative agent proactively engages in collecting information resources, forming image and text surrogates, and composing them visually. The agent develops the collection and its visual representation over time, enabling the user to see ideas and relationships. To keep the human in control, we develop interactive mechanisms for authoring the composition and directing the agent. In a field study in an interdisciplinary course on The Design Process, over a hundred students alternated using combinFormation and Google+Word to collect prior work on information discovery invention assignments. The students that used combinFormation's mixed-initiative composition of image and text surrogates performed better.

References

[1]
Amar, R. and Stasko, J. 2004. A knowledge task-based framework for design and evaluation of information visualizations. In Proceedings of the 8th conference on Information Visualization (InfoViz), 143--150.
[2]
Baddeley, A. D. 1992. Is working memory working?, Quart. J. Exp. Psych. 44A, 1--31.
[3]
Balabanovic, M. 1998. Learning to Surf: Multiagent systems for adaptive Web page recommendation. Ph.D. dissertation. Stanford University Department of Computer Science, Stanford, CA.
[4]
Baldonado, M. Q. W. and Winograd, T. 1997. SenseMaker: An information-exploration interface supporting the contextual evolution of a user's interests. In Proceedings of the ACM CHI. 11--18.
[5]
Belkin, N. 1982. ASK for Information Retrieval: Part I. Background and Theory. J. Documentat. 38, 2, 299--304.
[6]
Bier, E. and Perer, A. 2005. Icon Abacus and Ghost Icons. In Proceedings of the 5th ACM/IEEE Joint Conference on Digital Libraries (JCDL). 404--404.
[7]
Breazeal, C. 2002. Designing Sociable Robots. MIT Press, Cambridge, MA.
[8]
Burke, M. 1999. Organization of Multimedia Resources: Principles and Practice of Information Retrieval. Gower Publishing Company, Hampshire, U.K.
[9]
Cutrell, E., Dumais, S. T., and Teevan J. 2006. Searching to eliminate personal information management. Commun. ACM 49, 1, 58--64.
[10]
Cutting, D. R., Karger, D. R., Pederson, J. O., and Tukey, J. W. 1992. Scatter/gather: A cluster-based approach to browsing large document collections. In Proceedings of ACM IR. 318--329
[11]
Diligenti, M., Coetzee, F., Lawrence, S., Giles, C. L., and Gori, M. 2000. Focused crawling using context graphs. In Proceedings of the 26th International Conference on Very Large Databases (VLDB). 527--534.
[12]
Ding, W., Marchionini, G., and Soergel, D. 1999. Multimodal surrogates for video browsing. In Proceedings of the 4th ACM Conference on Digital Libraries (DL). 85--93.
[13]
Dumais, S. T., Cutrell, E., Cadiz, J. J., Jancke, G., Sarin, R., and Robbins, D. C. 2003. Stuff I've Seen: A system for personal information retrieval and re-use. In Proceedings of the 26th International ACM SIGIR Conference on Research and Development in Information Retrieval. 72--79.
[14]
Eisenstein, S. M. 1975. The Film Sense. Harcourt Brace Jovanovich, New York, NY.
[15]
Finke, R., Ward, T., and Smith, S. M. 1992. Creative Cognition. MIT Press, Cambridge, MA.
[16]
Furnas, G. W. 1986. Generalized fisheye views. In Proceedings of the ACM Computer Human Interaction. 16--23.
[17]
Furnas, G. W. 2006. A fish-eye follow-up: Further reflections on focus + context. In Proceedings of the ACM Computer Human Interaction. 999--1008.
[18]
Glenberg, A. M. 2002. The Indexical hypothesis: Meaning from language, world, and image. In Working with Words and Images. Ablex, Westport, CT.
[19]
Glenberg, A. M. and Langston, W. E. 1992. Comprehension of illustrated text: Pictures help to build mental models. J. Mem. Lang. 31, 2, 129--151.
[20]
Hearst, M. A. and Pederson, J. O. 1996. In Proceedings of ACM IR. 76--84.
[21]
Hill, W. C., Hollan, J. D., Wroblewski, D., and McCandless, T. 1992. Edit wear and read wear. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 3--9.
[22]
Horvitz, E. Uncertainty, action, and interaction. In pursuit of mixed-initiative computing. In Intelligent Systems. IEEE Computer Society Press, Los Alamitos, CA.
[23]
Huynh, D., Mazzocchi, S., and Karger, D. 2005. Piggy Bank: Experience the semantic Web inside your Web browser. In Proceedings of the 2005 International Semantic Web Conference. 413--430.
[24]
Interface Ecology Lab. 2007. combinFormation. http://ecologylab.cs.tamu.edu/combin Formation/.
[25]
Jones, W., Dumais, S., and Bruce, H. 2002. Once found, what then?: A study of ‘keeping’ behaviors in personal use of Web information. In Proceedings of the ASIST Annual Meeting. Vol. 39. 391--402.
[26]
Jones, W., Phuwanartnurak, A. J., Gill, R., and Bruce, H. 2005. Don't take my folders away!: Organizing personal information to get things done. In CHI'05 Extended Abstracts on Human Factors in Computing Systems. ACM Press, New York, NY, 1505--1508.
[27]
Kelly D. and Teevan, J. 2003. Implicit feedback for inferring user preference: a bibliography. ACM SIGIR For. 37, 2, 18--28.
[28]
Kerne, A. CollageMachine, a Model of Interface Ecology, Ph.D. dissertation. New York University, New York, NY.
[29]
Kerne, A. and Khandelwal, M. 2003. Publishing evolving metadocuments on the Web. In Proceedings of the 14th ACM Conference on Hypertext and Hypermedia. 104--105.
[30]
Kerne, A., Koh, E., Dworaczyk, B., Mistrot, J. M., Choi, H., Smith, S. M., Graeber, R., Caruso, D., Webb, A., Hill, R., and Albea, J. 2006. combinFormation: A mixed-initiative system for representing collections as compositions of image and text surrogates. In Proceedings of the 6th ACM/IEEE Joint Conference on Digital Libraries (JCDL). 11--20.
[31]
Kerne, A., Koh, E., Sundaram, V., and Mistrot, J. M. 2005. Generative semantic clustering in spatial hypertext. In Proceedings of the 2005 ACM Symposium on Document Engineering. 84--93.
[32]
Kerne, A., Mistrot, J. M., Khandelwal, M., Sundaram, V., and Koh, E. 2004a. Using composition to re-present personal collections of hypersigns. In Proceedings of Computational Semiotics in Games and New Media (CoSIGN).
[33]
Kerne, A. and Smith, S. 2004. The information discovery framework. In Proceedings of the 2004 Conference on Designing Interactive Systems (DIS). 357--360.
[34]
Kerne, A., Smith, S. M., Mistrot, J. M., Sundaram, V., and Khandelwal, M. 2004b. Mapping interest and design to facilitate creative process during mixed-initiative information composition. In Proceedings of Interaction: Systems, Practice and Theory, Creativity & Cognition Symposium. 1--25.
[35]
Kerne, A. and Sundaram, V. 2003. A recombinant information space. In Proceedings of the Computational Semiotics in Games and New Media (CoSIGN). 48--57.
[36]
Kerne, A., Toups, Z., Dworaczyk, B., and Khandelwal, M. 2008. Componentized XML-Java data binding framework connects software engineering with efficiency. Submitted to OOPSLA.
[37]
Khandelwal, M., Kerne, A., and Mistrot, J. M. 2004. Manipulating history in generative hypermedia. In Proceedings of the Fifteenth ACM Conference on Hypertext and Hypermedia. 139--140.
[38]
Koh, E. and Kerne, A. 2006. I keep collecting: College students build and utilize collections in spite of breakdowns. In Proceedings of European Conf on Digital Libraries (ECDL). 303--314.
[39]
Lippard, L., Ed. 1971. Dadas on Art. Prentice-Hall, Englewood Cliffs, NJ.
[40]
Maier, N. R. F. 1931. Reasoning in humans ü: The solution of a problem and its appearance in consciousness. J. Compar. Physiol. Psych. 12, 181--194.
[41]
Marchionini, G. 1995. Information seeking in electronic environments. Cambridge Series on Human-Computer Interaction. vol. 9. Cambridge University Press, Cambridge, U.K.
[42]
Marshall, C. C. and Bly, S. 2005. Saving and using encountered information: Implications for electronic periodicals. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (SIGCHI). 111--120.
[43]
Marshall, C. C. and Jones, W. 2006. Keeping encountered information. Commun. ACM 49, 1, 66--67.
[44]
Marshall, C. C., and Shipman, F. M. 1994. VIKI: Spatial hypertext supporting emergent structure. In Proceedings of the 1994 ACM European Conference on Hypermedia Technology (ECHT). 13--23.
[45]
Mayer, R. E. and Moreno, R. 2002. Aids to computer-based multimedia learning. Learn. Instruct. 12, 107--119.
[46]
McNee, S. M., Lam, S., Konstan, J. A., and Riedl, J. 2003. Interfaces for eliciting new user preferences in recommender systems. In Proceedings of User Modeling. 178--188.
[47]
Miller, G. A. 2006. WordNet. Princeton University Cognitive Science Laboratory, Princeton, NJ. http://wordnet.princeton.edu/.
[48]
Morrison, J. B., Pirolli, P., and Card, S. K. 2001. A taxonomic analysis of what World Wide Web activities significantly impact people's decisions and actions. In CHI' 01 Extended Abstracts on Human Factors in Computing Systems. ACM Press, New York, NY, 163--164.
[49]
Nardi, B. and Barreau, D. K. 1997. “Finding and reminding” revisited: Appropriate metaphors for file organization at the desktop. SIGCHI Bull. 29, 1.
[50]
Oxford English Dictionary. 1992. 2nd ed., on compact disk. Oxford University Press, Oxford, U.K.
[51]
Pirolli, P. and Card, S. K. 1999. Information foraging. Psychol. Rev. 106, 4, 643--675.
[52]
Porter, M. F. 1980. An algorithm for suffix stripping. Program, 14, 3, 130--137.
[53]
Rocchio, J. 1971. Relevance feedback in information retrieval. In The Smart Retrieval System-Experiments in Automatic Document Processing. Prentice-Hall, Englewood Cliffs, NJ, 313--323.
[54]
Salton, G. and McGill, M. J. 1983. Introduction to Modern Information Retrieval. McGraw-Hill, New York, NY.
[55]
Schraefel, M. C., Zhu, Y., Modjeska, D., Wigdor, D., and Zhao, S. 2002. Hunter gatherer: Interaction support for the creation and management of within-Web-page collections. In Proceedings of the Eleventh International Conference on World Wide Web (WWW). 172--181.
[56]
Shipman, F., Hsieh, H., Moore, J. M., and Zacchi, A. 2004. Supporting personal collections across digital libraries in spatial hypertext. In Proceedings of 6th ACM/IEEE Joint Conference on Digital Libraries (JCDL). 358--367.
[57]
Shneiderman, B. and Maes, P. 1997. Direct manipulation vs. interface agents. ACM Interact. 4, 6.
[58]
Simon, H. A. 1971. Designing organizations for an information-rich world. In Computer, Communications, and the Public Interest. Johns Hopkins Press, Baltimore, MD, 37--72.
[59]
Smith, S. M. 1994. Getting into and out of mental ruts: A theory of fixation, incubation, and insight. In R. J. Sternberg and J. Davidson, Eds. The Nature of Insight, MIT Press, Cambridge, MA, 121--149.
[60]
Shah, J. J., Smith, S. M., Vargas-Hernandez, N., Gerkens, R., and Wulan, M. 2003. Empirical studies of design ideation: Alignment of design experiments with laboratory experiments. In Proceedings of DETC 2003: ASME International Conference on Design Theory and Methodology.
[61]
Spies, W., Ed. 2006. Max Ernst: Life and Work: An Autobiographical Collage. Thames & Hudson, London, U.K.
[62]
Spooky, Dj That Subliminal Kid. 2004. Rhythm Science. Mediawork /MIT Press, Cambridge, MA.
[63]
Sun Microsystems. 2006. Java. http://java.sun.com.
[64]
Tufte, E. 1990. Envisioning Information. Graphics Press, Cheshire, CT.
[65]
White, R. W., Kules, B., Drucker, S. M., and Schraefel, M. C. 2006. Supporting exploratory search. Commun. ACM 49, 4, 36--39.
[66]
Wildemuth, B. M., Marchionini, G., Yang, M., Geisler, G., Wilkens, T., Hughes, A., and Gruss, R. 2003. How fast is too fast?: Evaluating fast forward surrogates for digital video. In Proceedings of the 3rd ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL). 221--230.
[67]
Woodruff, A., Faulring, A., Rosenholtz, R., Morrison, J., and Pirolli, P. 2001. Using thumbnails to search the Web. In Proceedings of ACM CHI. 198--205.
[68]
Woodruff, A., Rosenholtz, R., Morrison, J., Faulring, A., and Pirolli, P. 2002. A comparison of the use of text summaries, plain thumbnails, and enhanced thumbnails for Web search tasks. J. Amer. Soc. Inform. Sci. Tech. 53, 2, 172--185.
[69]
W3C. 2000. Document Object Model (DOM) Level 2 Core Specification. http://www.w3.org/TR/2000/REC-DOM-Level-2-Core-20001113/.
[70]
Yamamoto, Y., Nakakoji, K., Nishinaka, Y., Asada, M., and Matsuda, R. 2005. What is the space for?: The role of space in authoring hypertext representations. In Proceedings of ACM Hypertext. 117--125.

Cited By

View all
  • (2024)Designing digital tools for creative thinking: a case study from elite sports coachingExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3637146(1-11)Online publication date: 11-May-2024
  • (2024)What Counts as ‘Creative’ Work? Articulating Four Epistemic Positions in Creativity-Oriented HCI ResearchProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642854(1-15)Online publication date: 11-May-2024
  • (2023)SearchIdea: An Idea Generation Tool to Support Creativity in Academic SearchProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578294(161-171)Online publication date: 19-Mar-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Information Systems
ACM Transactions on Information Systems  Volume 27, Issue 1
December 2008
208 pages
ISSN:1046-8188
EISSN:1558-2868
DOI:10.1145/1416950
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 December 2008
Accepted: 01 March 2008
Revised: 01 January 2008
Received: 01 January 2007
Published in TOIS Volume 27, Issue 1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Creativity support tools
  2. clustering
  3. collections
  4. creative cognition
  5. exploratory search
  6. field study
  7. focused crawler
  8. information discovery
  9. mixed-initiative systems
  10. relevance feedback
  11. semantics
  12. software agents

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)43
  • Downloads (Last 6 weeks)5
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Designing digital tools for creative thinking: a case study from elite sports coachingExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3637146(1-11)Online publication date: 11-May-2024
  • (2024)What Counts as ‘Creative’ Work? Articulating Four Epistemic Positions in Creativity-Oriented HCI ResearchProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642854(1-15)Online publication date: 11-May-2024
  • (2023)SearchIdea: An Idea Generation Tool to Support Creativity in Academic SearchProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578294(161-171)Online publication date: 19-Mar-2023
  • (2023)Designing Creative AI Partners with COFI: A Framework for Modeling Interaction in Human-AI Co-Creative SystemsACM Transactions on Computer-Human Interaction10.1145/351902630:5(1-28)Online publication date: 23-Sep-2023
  • (2022)Understanding Creative Thinking Processes in Searching for New IdeasProceedings of the 2022 Conference on Human Information Interaction and Retrieval10.1145/3498366.3505783(321-326)Online publication date: 14-Mar-2022
  • (2022)Designing new digital tools to augment human creative thinking at work: An application in elite sports coachingExpert Systems10.1111/exsy.1319440:3Online publication date: 30-Nov-2022
  • (2021)Using computational tools to support journalists’ creativityJournalism10.1177/1464884921101058223:9(1881-1899)Online publication date: 21-Apr-2021
  • (2021)Serendipity in human information behavior: a systematic reviewJournal of Documentation10.1108/JD-02-2021-0029ahead-of-print:ahead-of-printOnline publication date: 19-Jul-2021
  • (2020)Evaluating Creativity Support Tools in HCI ResearchProceedings of the 2020 ACM Designing Interactive Systems Conference10.1145/3357236.3395474(457-476)Online publication date: 3-Jul-2020
  • (2020)Improving Big Data Technologies with Visual Faceted Search2020 8th International Conference on Information Technology and Multimedia (ICIMU)10.1109/ICIMU49871.2020.9243380(44-49)Online publication date: 24-Aug-2020
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media