Lee Stearns
Jon E. Froehlich
ABSTRACT
Color identification tools do not identify visual patterns or allow users to quickly inspect multiple locations, which are both important for identifying clothing. We are exploring the use of a finger-based camera that allows users to query clothing colors and patterns by touch. Previously, we demonstrated the feasibility of this approach using a small, highly-controlled dataset and combining two image classification techniques commonly used for object recognition. Here, to improve scalability and robustness, we collect a dataset of fabric images from online sources and apply transfer learning to train an end-to-end deep neural network to recognize visual patterns. This new approach achieves 92% accuracy in a general case and 97% when tuned for images from a finger-mounted camera.
- 3D Carousel Using TweenMax.js & jQuery. Available at: https://codepen.io/johnblazek/pen/nceywGoogle Scholar
- Anna Dickinson, Roos Eisma, and Peter Gregor. 2011. The barriers that older novices encounter to computer use. Universal Access in the Information Society. 10, 3, 261--266. Google ScholarDigital Library
- Arthur Theil Cabreira and Faustina Hwang. 2015. An analysis of mid-air gestures used across three platforms. In Proceedings of the 2015 British HCI Conference (British HCI '15), 257--258. Google ScholarDigital Library
- Arthur Theil Cabreira and Faustina Hwang. 2016. How Do Novice Older Users Evaluate and Perform Mid-Air Gesture Interaction for the First Time?. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction (NordiCHI '16), 122--128. Google ScholarDigital Library
- Caroline J. Ketcham and George E. Stelmach. 2001. Age-Related Declines in Motor Control. In Birren, J. E. & Schaie, K.W. (Eds.), Handbook of the Psychology of Aging. Academic Press.Google Scholar
- Christian Stößel. 2009. Familiarity as a factor in designing finger gestures for elderly users. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '09), Article 78. Google ScholarDigital Library
- Darren Guinness, G. Michael Poor, and Alvin Jude. 2014. Gestures with speech for hand-impaired persons. In Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibility (ASSETS '14). ACM, New York, NY, USA, 259--260. Google ScholarDigital Library
- Donald A. Norman and Jakob Nielsen. 2010. Gestural interfaces: a step backward in usability. interactions 17, 5 (September 2010), 46--49. Google ScholarDigital Library
- Gözel Shakeri, John H. Williamson, and Stephen Brewster. 2017. Novel Multimodal Feedback Techniques for In-Car Mid-Air Gesture Interaction. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '17). ACM, New York, NY, USA, 84--93. Google ScholarDigital Library
- Jan Bobeth, Susanne Schmehl, Ernst Kruijff, Stephanie Deutsch, and Manfred Tscheligi. 2012. Evaluating performance and acceptance of older adults using freehand gestures for TV menu control. In Proceedings of the 10th European Conference on Interactive TV and Video (EuroITV '12). ACM, New York, NY, USA, 35--44. Google ScholarDigital Library
- Kathrin Gerling, Ian Livingston, Lennart Nacke, Regan Mandryk. 2012. Full-body motion-based game interaction for older adults. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12), 1873--1882. Google ScholarDigital Library
- Leah Findlater, Jon E. Froehlich, Kays Fattal, Jacob O. Wobbrock, and Tanya Dastyar. 2013. Age-related differences in performance with touchscreens compared to traditional mouse input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 343--346. Google ScholarDigital Library
- Robert Walter, Gilles Bailly, Nina Valkanova, and Jörg Müller. 2014. Cuenesics: using mid-air gestures to select items on interactive public displays. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services (MobileHCI '14). ACM, New York, NY, USA, 299--308. Google ScholarDigital Library
- Sara J. Czaja and Chin Chin Lee. 2007. Information technology and older adults. In J. A. Jacko & A. Sears (Eds.), The Human Computer-Interaction Handbook (2nd ed., 777--792. Lawrence Erlbaum Associates, New York.Google Scholar
- Swipe Gesture -- Leap Motion Documentation. Available at: https://developerarchive.leapmotion.com/documentation/v2/python/api/ Leap.SwipeGesture.html?proglang=pythonGoogle Scholar
- Wei Lu, Zheng Tong, Jinghui Chu. 2016. Dynamic Hand Gesture Recognition With Leap Motion Controller. IEEE Signal Processing Letters. 23, 9: 1188--1192.Google ScholarCross Ref
- Ying Chih Wang, Richard W. Bohannon, Jay Kapellusch, Arun Garg, Richard C. Gershon. 2015. Dexterity as measured with the 9-Hole Peg Test (9- HPT) across the age span. Journal of Hand Therapy. 28, 1, 53--60.Google ScholarCross Ref
Index Terms
- Applying Transfer Learning to Recognize Clothing Patterns Using a Finger-Mounted Camera
Recommendations
Recognizing Clothing Colors and Visual Textures Using a Finger-Mounted Camera: An Initial Investigation
ASSETS '17: Proceedings of the 19th International ACM SIGACCESS Conference on Computers and AccessibilityWe investigate clothing color and visual texture recognition using images from a finger-mounted camera to support people with visual impairments. Our approach mitigates issues with distance and lighting that can impact the accuracy of existing color and ...
Invisiboard: maximizing display and input space with a full screen text entry method for smartwatches
MobileHCI '16: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and ServicesThe small displays of smartwatches make text entry difficult and time consuming. While text entry rates can be increased, this continues to occur at the expense of available screen display space. Soft keyboards can easily use half the display space of ...
Object tracking across non-overlapping views by learning inter-camera transfer models
In this paper, we introduce a novel algorithm to solve the problem of object tracking across multiple non-overlapping cameras by learning inter-camera transfer models. The transfer models are divided into two parts according to different kinds of cues, ...
Comments