skip to main content
10.1145/2047196.2047198acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article

Platemate: crowdsourcing nutritional analysis from food photographs

Published: 16 October 2011 Publication History

Abstract

We introduce PlateMate, a system that allows users to take photos of their meals and receive estimates of food intake and composition. Accurate awareness of this information can help people monitor their progress towards dieting goals, but current methods for food logging via self-reporting, expert observation, or algorithmic analysis are time-consuming, expensive, or inaccurate. PlateMate crowdsources nutritional analysis from photographs using Amazon Mechanical Turk, automatically coordinating untrained workers to estimate a meal's calories, fat, carbohydrates, and protein. We present the Management framework for crowdsourcing complex tasks, which supports PlateMate's nutrition analysis workflow. Results of our evaluations show that PlateMate is nearly as accurate as a trained dietitian and easier to use for most users than traditional self-reporting.

References

[1]
Amft, O., Stager, M., Lukowicz, P., and Troster, G. Analysis of chewing sounds for dietary monitoring. In Ubicomp. Springer, 2005, volume 3660 of Lecture Notes in Computer Science, 56--72.
[2]
Ann Yon, B., Johnson, R., Harvey-Berino, J., and Gold, B. The use of a personal digital assistant for dietary self-monitoring does not improve the validity of self-reports of energy intake. Journal of the American Dietetic Association, 106, (2006), 1256--1259.
[3]
Bernstein, M., Little, G., Miller, R., Hartmann, B., Ackerman, M., Karger, D., Crowell, D., and Panovich, K. Soylent: a word processor with a crowd inside. In Proc. UIST'10. ACM, 2010, 313--322.
[4]
Champagne, C., Bray, G., Kurtz, A., Monteiro, J., Tucker, E., Volaufova, J., and Delany, J. Energy intake and energy expenditure:: A controlled study comparing dietitians and non-dietitians. Journal of the American Dietetic Association, 102, 1. (2002), 1428--1432.
[5]
Dinkins, J. Beliefs and attitudes of americans towards their diet. US Department of Agriculture Center for Nutrition Policy and Promotion (2000).
[6]
Goris, A., Westerterp-Plantenga, M., and Westerterp, K. Undereating and underrecording of habitual food intake in obese men selective underreporting of fat intake. The American journal of clinical nutrition, 71, (2000), 130.
[7]
Kitamura, K., de Silva, C., Yamasaki, T., and Aizawa, K. Image processing based approach to food balance analysis for personal food logging. In IEEE International Conference on Multimedia and Exp. (ICME). IEEE, 2010, 625--630.
[8]
Kittur, A., Smus, B., and Kraut, R. CrowdForge: Crowdsourcing Complex Work. Technical Report Carnegie Mellon University-HCII-11--100, Carnegie Mellon University (2011).
[9]
Little, G., Chilton, L., Goldman, M., and Miller, R. Turkit: human computation algorithms on mechanical turk. In Proc. UIST'10. ACM, 2010, 57--66.
[10]
Little, G., Chilton, L. B., Goldman, M., and Miller, R. C. Exploring iterative and parallel human computation processes. In Proc. ACM SIGKDD Workshop on Human Computation. ACM, New York, NY, USA, 2010, HCOMP '10, 68--76.
[11]
Locke, E. and Latham, G. Building a practically useful theory of goal setting and task motivation. American Psychologist, 57, 9 (2002), 705--717.
[12]
Mankoff, J., Hsieh, G., Hung, H. C., Lee, S., and Nitao, E. Using low-cost sensing to support nutritional awareness. In Proc. UbiComp '02. Springer-Verlag, London, UK, 2002, 371--376.
[13]
Mann, T., Tomiyama, A., Westling, E., Lew, A., Samuels, B., and Chatman, J. Medicare's search for effective obesity treatments: Diets are not the answer. American Psychologist, 62, 3 (2007), 220.
[14]
Marcus, A., Wu, E., Karger, D., Madden, S., and Miller, R. Crowdsourced databases: Query processing with people. In CIDR. 2011.
[15]
Martin, C., Anton, S., York-Crowe, E., Heilbronn, L., VanSkiver, C., Redman, L., Greenway, F., Ravussin, E., and Williamson, D. Empirical evaluation of the ability to learn a calorie counting system and estimate portion size and food intake. British Journal of Nutrition, 98, 02 (2007). 439--444.
[16]
Martin, C., Han, H., Coulon, S., Allen, H., Champagne, C., and Anton, S. A novel method to remotely measure food intake of free-living individuals in real time: the remote food photography method. British Journal of Nutrition, 101, 03 (2009). 446--456.
[17]
Pikholz, C., Swinburn, B., and Metcalf, P. Under-reporting of energy intake in the 1997 national nutrition survey. The New Zealand Medical Journal, 117, 1202 (2004).
[18]
Reisner, R. and Thompson, D. The diet industry: A big fat lie. BuisinessWeek (2008).
[19]
Schoeller, D., Bandini, L., and Dietz, W. Inaccuracies in self-reported intake identified by comparison with the doubly labelled water method. Canadian journal of physiology and pharmacology, 68. 7 (1990), 941.
[20]
Surowiecki, J. The wisdom of crowds: why the many are smarter than the few and how collective wisdom shapes business, economies, societies, and nations. Doubleday, 2004.
[21]
Yon, B., Johnson, R., Harvey-Berino, J., Gold, B., and Howard, A. Personal digital assistants are comparable to traditional diaries for dietary self-monitoring during a weight loss program. Journal of behavioral medicine, 30, 2 (2007). 165--175.

Cited By

View all
  • (2025)A partitioning Monte Carlo approach for consensus tasks in crowdsourcingExpert Systems with Applications10.1016/j.eswa.2024.125559262(125559)Online publication date: Mar-2025
  • (2024)OSINT Research Studios: A Flexible Crowdsourcing Framework to Scale Up Open Source Intelligence InvestigationsProceedings of the ACM on Human-Computer Interaction10.1145/36373828:CSCW1(1-38)Online publication date: 26-Apr-2024
  • (2024)Belief Miner: A Methodology for Discovering Causal Beliefs and Causal Illusions from General PopulationsProceedings of the ACM on Human-Computer Interaction10.1145/36372988:CSCW1(1-37)Online publication date: 26-Apr-2024
  • Show More Cited By

Index Terms

  1. Platemate: crowdsourcing nutritional analysis from food photographs

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    UIST '11: Proceedings of the 24th annual ACM symposium on User interface software and technology
    October 2011
    654 pages
    ISBN:9781450307161
    DOI:10.1145/2047196
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 16 October 2011

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. amazon mechanical turk
    2. crowdsourcing
    3. human computation
    4. nutrition
    5. remote food photography

    Qualifiers

    • Research-article

    Conference

    UIST '11

    Acceptance Rates

    UIST '11 Paper Acceptance Rate 67 of 262 submissions, 26%;
    Overall Acceptance Rate 561 of 2,567 submissions, 22%

    Upcoming Conference

    UIST '25
    The 38th Annual ACM Symposium on User Interface Software and Technology
    September 28 - October 1, 2025
    Busan , Republic of Korea

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)47
    • Downloads (Last 6 weeks)5
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)A partitioning Monte Carlo approach for consensus tasks in crowdsourcingExpert Systems with Applications10.1016/j.eswa.2024.125559262(125559)Online publication date: Mar-2025
    • (2024)OSINT Research Studios: A Flexible Crowdsourcing Framework to Scale Up Open Source Intelligence InvestigationsProceedings of the ACM on Human-Computer Interaction10.1145/36373828:CSCW1(1-38)Online publication date: 26-Apr-2024
    • (2024)Belief Miner: A Methodology for Discovering Causal Beliefs and Causal Illusions from General PopulationsProceedings of the ACM on Human-Computer Interaction10.1145/36372988:CSCW1(1-37)Online publication date: 26-Apr-2024
    • (2024)GustosonicSense: Towards understanding the design of playful gustosonic eating experiencesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642182(1-12)Online publication date: 11-May-2024
    • (2024)LabelAId: Just-in-time AI Interventions for Improving Human Labeling Quality and Domain Knowledge in Crowdsourcing SystemsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642089(1-21)Online publication date: 11-May-2024
    • (2024)The Development of CNN-Based Food Scanning Model for Calorie Analysis: A Tool for Promoting Healthier Dietary Choices2024 International Conference on Advancements in Smart, Secure and Intelligent Computing (ASSIC)10.1109/ASSIC60049.2024.10507809(1-6)Online publication date: 27-Jan-2024
    • (2024)Feature-Based Text Search Engine Mitigating Data Diversity Problem Using Pre-Trained Large Language Model for Fast Deployment ServicesIEEE Access10.1109/ACCESS.2024.337347012(48145-48157)Online publication date: 2024
    • (2024)Global–local feature learning for fine-grained food classification based on Swin TransformerEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.108248133(108248)Online publication date: Jul-2024
    • (2024)Intelligent Decision Support in Personal Health: Personalized Health Coaching in Type 2 DiabetesHuman Computer Interaction in Healthcare10.1007/978-3-031-69947-4_17(413-438)Online publication date: 14-Nov-2024
    • (2023)Exploring Opportunities for Multimodality and Multiple Devices in Food JournalingProceedings of the ACM on Human-Computer Interaction10.1145/36042567:MHCI(1-27)Online publication date: 13-Sep-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media