skip to main content
10.1145/1851600.1851650acmotherconferencesArticle/Chapter ViewAbstractPublication PagesmobilehciConference Proceedingsconference-collections
research-article

Data logging plus e-diary: towards an online evaluation approach of mobile service field trial

Published: 07 September 2010 Publication History

Abstract

Many field study methods such as usability test in fields, contextual inquiry, and ethnographic interview can be applied to evaluate user experience of concepts in trials; however, most of such traditional field study methods suffer from weaknesses like resource demanding, time consuming, lack of measurement for longitudinal usage or lack of channels to collect user feedbacks in real time. In this paper, we propose an approach to combine data logging with e-diary for user experience evaluation in field trials. An evaluation case on a mobile service concept is also presented to show this approach.

References

[1]
Kjeldskov, J. and Graham, C. A review of Mobile HCI Research Methods. In Proceedings of Mobile HCI 2003, Springer-Verlag, LNCS 2795, 2003, 317--335.
[2]
Kantner, L., Sova, D. H., and Rosenbaum, S. (2003). Alternative methods for field usability research. In Proceedings of the 21st Annual international Conference on Documentation (San Francisco, CA, USA, October 12 - 15, 2003). SIGDOC '03. 68--72.
[3]
Robert Kraut, William Scherlis, Tridas Mukhopadhyay, Jane Manning, Sara Kiesler, HomeNet: a field trial of residential Internet services, In Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.284--291, April 13-18, 1996, Vancouver, British Columbia, Canada.
[4]
Jensen, K., L. Larsen, L., B (2007). Evaluating the usefulness of mobile services based on captured usage data from longitudinal field trials. In Proceedings of 4th international conference on mobile technology, application and systems (mobility 2007): 675--682.
[5]
Leysia Palen, Marilyn Salzman, Voice-mail diary studies for naturalistic data capture under mobile conditions, Proceedings of the 2002 ACM conference on Computer supported cooperative work, November 16-20, 2002, New Orleans, Louisiana, USA.
[6]
Geven A., Strassl P., Ferro B. et al, Experiencing real-world interaction: results from a NFC user experience field trial, In Proceedings of Mobile HCI2007 (Singapore, September 09-12, 2007). ACM Press, New York, NY, p.234--237.
[7]
GUZDIAL, M., SANTOS, P., BADRE, A., HUDSON,S., AND GRAY, M. 1994. Analyzing and visualizing log files: A computational science of usability. GVU Center TR GIT-GVU-94-8, Georgia Institute of Technology. Available at http://www.cc.gatech.edu/gvu/reports/1994/abstracts/94-08.html.
[8]
Melody Y. Ivory, Marti A Hearst, The state of the art in automating usability evaluation of user interfaces, ACM Computing Surveys (CSUR), v.33 n.4, p.470--516, December 2001.
[9]
Andreas Lecerof, Fabio Paternò, Automatic Support for Usability Evaluation, IEEE Transactions on Software Engineering, v.24 n.10, p.863--888, October 1998.
[10]
David M. Hilbert, David F. Redmiles, Extracting usability information from user interface events, ACM Computing Surveys (CSUR), v.32 n.4, p.384--421, Dec. 2000.
[11]
Burton, M. C. 2001. The value of web log data in use-based design and testing. J. Comput. Med. Commun. 6, 3. Also available at: www.ascusc.org/jcmc/vol6/issue3/burton.html.
[12]
Bolger, N., Davis, A., and Rafaeli, E., Diary methods: capturing life as it is lived. Annual Review of Psychology, 2003. 54(1): p. 579--616.
[13]
Mary Czerwinski, Eric Horvitz, Susan Wilhite, A diary study of task switching and interruptions, Proceedings of the SIGCHI conference on Human factors in computing systems, p.175--182, April 24-29, 2004, Vienna, Austria.
[14]
Timothy Sohn, Kevin A. Li, William G. Griswold, James D. Hollan, A diary study of mobile information needs, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy.
[15]
Kasper Løvborg Jensen, RECON: capturing mobile and ubiquitous interaction in real contexts, Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services, September 15-18, 2009, Bonn, Germany

Cited By

View all
  • (2017)How many ways to use CiteSpace? A study of user interactive events over 14 monthsJournal of the Association for Information Science and Technology10.1002/asi.2377068:5(1234-1256)Online publication date: 27-Jan-2017
  • (2016)Quantified UXProceedings of the 9th Nordic Conference on Human-Computer Interaction10.1145/2971485.2971501(1-10)Online publication date: 23-Oct-2016
  • (2016)From Publics to CommunitiesComputer Supported Cooperative Work10.1007/s10606-016-9252-y25:2-3(193-225)Online publication date: 1-Jun-2016
  • Show More Cited By

Index Terms

  1. Data logging plus e-diary: towards an online evaluation approach of mobile service field trial

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    MobileHCI '10: Proceedings of the 12th international conference on Human computer interaction with mobile devices and services
    September 2010
    552 pages
    ISBN:9781605588353
    DOI:10.1145/1851600
    • General Chairs:
    • Marco de Sá,
    • Luís Carriço,
    • Program Chair:
    • Nuno Correia
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    In-Cooperation

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 07 September 2010

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. data log
    2. diary
    3. evaluation
    4. field trial

    Qualifiers

    • Research-article

    Conference

    MobileHCI '10

    Acceptance Rates

    MobileHCI '10 Paper Acceptance Rate 46 of 225 submissions, 20%;
    Overall Acceptance Rate 202 of 906 submissions, 22%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)6
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2017)How many ways to use CiteSpace? A study of user interactive events over 14 monthsJournal of the Association for Information Science and Technology10.1002/asi.2377068:5(1234-1256)Online publication date: 27-Jan-2017
    • (2016)Quantified UXProceedings of the 9th Nordic Conference on Human-Computer Interaction10.1145/2971485.2971501(1-10)Online publication date: 23-Oct-2016
    • (2016)From Publics to CommunitiesComputer Supported Cooperative Work10.1007/s10606-016-9252-y25:2-3(193-225)Online publication date: 1-Jun-2016
    • (2016)Work or leisure? Designing a user-centered approach for researching activity "in the wild"Personal and Ubiquitous Computing10.1007/s00779-016-0935-720:4(487-515)Online publication date: 1-Aug-2016
    • (2015)FRAMES – A Framework for Adaptable Mobile Event-Contingent Self-report StudiesEnd-User Development10.1007/978-3-319-18425-8_10(141-155)Online publication date: 30-Apr-2015
    • (2014)A Mixed-Method Approach for In-Depth Contextual User ResearchDesign, User Experience, and Usability. Theories, Methods, and Tools for Designing the User Experience10.1007/978-3-319-07668-3_9(86-95)Online publication date: 2014
    • (2014)Mobile Human–Computer InteractionQuality of Experience10.1007/978-3-319-02681-7_23(339-349)Online publication date: 5-Mar-2014
    • (2013)Capturing prolonged interactive experiencesProceedings of the 6th International Conference on Designing Pleasurable Products and Interfaces10.1145/2513506.2513516(86-95)Online publication date: 3-Sep-2013

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media