skip to main content
10.1145/2984511.2984538acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article
Public Access

Supporting Mobile Sensemaking Through Intentionally Uncertain Highlighting

Published: 16 October 2016 Publication History

Abstract

Patients researching medical diagnoses, scientist exploring new fields of literature, and students learning about new domains are all faced with the challenge of capturing information they find for later use. However, saving information is challenging on mobile devices, where the small screen and font sizes combined with the inaccuracy of finger based touch screens makes it time consuming and stressful for people to select and save text for future use. Furthermore, beyond the challenge of simply selecting a region of bounded text on a mobile device, in many learning and data exploration tasks the boundaries of what text may be relevant and useful later are themselves uncertain for the user. In contrast to previous approaches which focused on speeding up the selection process by making the identification of hard boundaries faster, we introduce the idea of intentionally supporting uncertain input in the context of saving information during complex reading and information exploration. We embody this idea in a system that uses force touch and fuzzy bounding boxes along with posthoc expandable context to support identifying and saving information in an intentionally uncertain way on mobile devices. In a two part user study we find that this approach reduced selection time and was preferred by participants over the default system text selection method.

Supplementary Material

MP4 File (p61-chang.mp4)

References

[1]
Adler, A., Gujar, A., Harrison, B. L., O'Hara, K., and Sellen, A. A diary study of work-related reading: design implications for digital reading devices. In Proceedings of the SIGCHI conference on Human factors in computing systems, ACM Press/Addison-Wesley Publishing Co. (1998), 241--248.
[2]
Bier, E. A., Ishak, E. W., and Chi, E. Entity quick click: rapid text copying based on automatic entity extraction. In CHI'06 Extended Abstracts on Human Factors in Computing Systems, ACM (2006), 562--567.
[3]
Capra, R., Marchionini, G., Velasco-Martin, J., and Muller, K. Tools-at-hand and learning in multi-session, collaborative search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '10, ACM (New York, NY, USA, 2010), 951--960.
[4]
Chapuis, O., and Roussel, N. Copy-and-paste between overlapping windows. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2007), 201--210.
[5]
Chen, C., Perrault, S. T., Zhao, S., and Ooi, W. T. Bezelcopy: an efficient cross-application copy-paste technique for touchscreen smartphones. In Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces, ACM (2014), 185--192.
[6]
Faure, G., Chapuis, O., and Roussel, N. Power tools for copying and moving: useful stuff for your desktop. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2009), 1675--1678.
[7]
Han, J., and Lee, G. Push-push: A drag-like operation overlapped with a page transition operation on touch interfaces. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, ACM (2015), 313--322.
[8]
Hart, S. G., and Staveland, L. E. Development of nasa-tlx (task load index): Results of empirical and theoretical research. Advances in psychology 52 (1988), 139--183.
[9]
Hinckley, K., Bi, X., Pahud, M., and Buxton, B. Informal information gathering techniques for active reading. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2012), 1893--1896.
[10]
Ives, Z., Knoblock, C., Minton, S., Jacob, M., Talukdar, P., Tuchinda, R., Ambite, J. L., Muslea, M., and Gazen, C. Interactive data integration through smart copy & paste. arXiv preprint arXiv:0909.1769 (2009).
[11]
Kirsh, D. The intelligent use of space. Artificial intelligence 73, 1 (1995), 31--68.
[12]
Kittur, A., Peters, A. M., Diriye, A., Telang, T., and Bove, M. R. Costs and benefits of structured information foraging. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2013), 2989--2998.
[13]
Lank, E., and Saund, E. Sloppy selection: Providing an accurate interpretation of imprecise selection gestures. Computers & Graphics 29, 4 (2005), 490--500.
[14]
Lee, S., and Lee, G. Design of interaction techniques for clickable touch screens. In Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: The Future of Design, OzCHI '14, ACM (New York, NY, USA, 2014), 568--577.
[15]
Marchionini, G. Exploratory search: from finding to understanding. Communications of the ACM 49, 4 (2006), 41--46.
[16]
Marshall, C. C., and Bly, S. Saving and using encountered information: implications for electronic periodicals. In Proceedings of the Sigchi conference on human factors in computing systems, ACM (2005), 111--120.
[17]
Morris, M. R., Brush, A. B., and Meyers, B. R. Reading revisited: Evaluating the usability of digital display surfaces for active reading tasks. In Horizontal Interactive Human-Computer Systems, 2007. TABLETOP'07. Second Annual IEEE International Workshop on, IEEE (2007), 79--86.
[18]
Peterson, S. E. The cognitive functions of underlining as a study technique. Literacy Research and Instruction 31, 2 (1991), 49--56.
[19]
Pirolli, P., and Card, S. Information foraging. Psychological review 106, 4 (1999), 643.
[20]
Pohl, H., and Murray-Smith, R. Focused and casual interactions: allowing users to vary their level of engagement. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2013), 2223--2232.
[21]
Roth, V., and Turner, T. Bezel swipe: conflict-free scrolling and multiple selection on mobile touch screen devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2009), 1523--1526.
[22]
Schilit, B. N., Golovchinsky, G., and Price, M. N. Beyond paper: supporting active reading with free form digital ink annotations. In Proceedings of the SIGCHI conference on Human factors in computing systems, ACM Press/Addison-Wesley Publishing Co. (1998), 249--256.
[23]
Schwarz, J., Mankoff, J., and Hudson, S. E. An architecture for generating interactive feedback in probabilistic user interfaces. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM (2015), 2545--2554.
[24]
Sinha, R. A cognitive analysis of tagging.
[25]
Stylos, J., Myers, B. A., and Faulring, A. Citrine: providing intelligent copy-and-paste. In Proceedings of the 17th annual ACM symposium on User interface software and technology, ACM (2004), 185--188.
[26]
Tashman, C. S., and Edwards, W. K. Active reading and its discontents: the situations, problems and ideas of readers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2011), 2927--2936.
[27]
Tashman, C. S., and Edwards, W. K. Liquidtext: a flexible, multitouch environment to support active reading. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2011), 3285--3294.
[28]
Teevan, J., Jones, W., and Bederson, B. B. Personal information management. Communications of the ACM 49, 1 (2006), 40--43.
[29]
Zhao, S., Chevalier, F., Ooi, W. T., Lee, C. Y., and Agarwal, A. Autocompaste: auto-completing text as an alternative to copy-paste. In Proceedings of the International Working Conference on Advanced Visual Interfaces, ACM (2012), 365--372.

Cited By

View all
  • (2024)Patterns of Hypertext-Augmented SensemakingProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676338(1-17)Online publication date: 13-Oct-2024
  • (2024)"We Need Structured Output": Towards User-centered Constraints on Large Language Model OutputExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650756(1-9)Online publication date: 11-May-2024
  • (2024)Constrained Highlighting in a Document Reader can Improve Reading ComprehensionProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642314(1-10)Online publication date: 11-May-2024
  • Show More Cited By

Index Terms

  1. Supporting Mobile Sensemaking Through Intentionally Uncertain Highlighting

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    UIST '16: Proceedings of the 29th Annual Symposium on User Interface Software and Technology
    October 2016
    908 pages
    ISBN:9781450341899
    DOI:10.1145/2984511
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 16 October 2016

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. annotation
    2. capture
    3. copy-paste
    4. highlighting.
    5. information
    6. saving
    7. sensemaking

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    UIST '16

    Acceptance Rates

    UIST '16 Paper Acceptance Rate 79 of 384 submissions, 21%;
    Overall Acceptance Rate 561 of 2,567 submissions, 22%

    Upcoming Conference

    UIST '25
    The 38th Annual ACM Symposium on User Interface Software and Technology
    September 28 - October 1, 2025
    Busan , Republic of Korea

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)82
    • Downloads (Last 6 weeks)21
    Reflects downloads up to 24 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Patterns of Hypertext-Augmented SensemakingProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676338(1-17)Online publication date: 13-Oct-2024
    • (2024)"We Need Structured Output": Towards User-centered Constraints on Large Language Model OutputExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650756(1-9)Online publication date: 11-May-2024
    • (2024)Constrained Highlighting in a Document Reader can Improve Reading ComprehensionProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642314(1-10)Online publication date: 11-May-2024
    • (2024)Selenite: Scaffolding Online Sensemaking with Comprehensive Overviews Elicited from Large Language ModelsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642149(1-26)Online publication date: 11-May-2024
    • (2023)1D-Touch: NLP-Assisted Coarse Text Selection via a Semi-Direct GestureProceedings of the ACM on Human-Computer Interaction10.1145/36264837:ISS(463-482)Online publication date: 1-Nov-2023
    • (2023)CiteSee: Augmenting Citations in Scientific Papers with Persistent and Personalized Historical ContextProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580847(1-15)Online publication date: 19-Apr-2023
    • (2023)Text Pin: Improving text selection with mode-augmented handles on touchscreen mobile devicesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103028175(103028)Online publication date: Jul-2023
    • (2022)Fuse: In-Situ Sensemaking Support in the BrowserProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545693(1-15)Online publication date: 29-Oct-2022
    • (2022)Wigglite: Low-cost Information Collection and TriageProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545661(1-16)Online publication date: 29-Oct-2022
    • (2022)Crystalline: Lowering the Cost for Developers to Collect and Organize Information for Decision MakingProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501968(1-16)Online publication date: 29-Apr-2022
    • Show More Cited By

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media