Skip to main content
SpringerLink
Log in

Asynchronous reflections: theory and practice in the design of multimedia mirror systems

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

In this paper, we present a theoretical framing of the functions of a mirror by breaking the synchrony between the state of a reference object and its reflection. This framing provides a new conceptualization of the uses of reflections for various applications. We describe the fundamental technical components of such systems and illustrate the technical challenges in two different forms of electronic mirror systems for apparel shopping. The first example, the Responsive Mirror, is an intelligent video capture and access system for clothes shopping in physical stores that provides personalized asynchronous reflections of clothing items through an implicitly controlled human–computer interface. The Responsive Mirror employs computer vision and machine learning techniques to interpret the visual cues of the shopper’s behavior from cameras to then display two different reflections of the shopper on digital displays: (1) the shopper in previously worn clothing with matching pose and orientation and (2) other people in similar and dissimilar shirts with matching pose and orientation. The second example system is a Countertop Responsive Mirror that differs from the first in that the images do not respond to the real-time movement of the shopper but to frames in a recorded video so that the motion of the shopper in the different recordings are matched non-sequentially. These instantiations of the mirror systems in fitting room and jewelry shopping scenarios are described, focusing on the system architecture and the intelligent computer vision components. The effectiveness of Responsive Mirror is demonstrated by the user study. The paper contributes a conceptualization of reflection and examples of systems illustrating new applications in multimedia systems that break traditional reflective synchronies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. Project Natal shopping scenario (feature video minute 2:30–2:45). http://www.xbox.com/en-US/live/projectnatal/ (last accessed 4 January 2010). Virtual Mirror: http://www.virtualmirror.net/ (last accessed 4 Jan 2010).

  2. A demonstration of the system is shown at BoingBoing Gadgets at: http://gadgets.boingboing.net/2009/06/30/how-parcs-responsive.html (last access 8 January 2010).

References

  1. Altman, I.: The environment and social behavior: privacy, personal space, territory and crowding. Brooks/Cole Publishing Company, Monterey (1975)

    Google Scholar 

  2. Anguelov, D., Lee K., Gokturk, S.B., Sumengen, B.: Contextual identity recognition in personal photo albums. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–7 (2007)

  3. Begole, B., Matsumoto, T., Zhang, W., Liu, J.: Responsive mirror: fitting information for fitting rooms. In: Proceedings of Workshop on Ambient Persuasion at Computer/Human Interaction (CHI) Conference (2008)

  4. Begole, B., Matsumoto, T., Zhang, W., Yee, N., Liu, J., Chu, M.: Designed to fit: challenges of interaction design for clothes fitting room technologies. In: HCI International, LNCS, vol. 5613, pp. 448–457. Springer, Berlin (2009)

  5. Brown, J.: Prada gets personal. BusinessWeek. McGraw-Hill, USA (2002)

  6. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern. Anal. Mach. Intell. 8(6), 679–698 (1986)

    Article  Google Scholar 

  7. Christopher, J.C.B.: A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Disc. 2, 121–167 (1998)

    Article  Google Scholar 

  8. Chu, M., Dalal, B., Walendowski, A., Begole, B.: Countertop responsive mirror: supporting physical retail shopping for sellers, buyers and companions. Submitted to Computer/Human Interaction (CHI) Conference (2010)

  9. Davis, F.: Fashion, Culture and Identity. University Of Chicago Press, USA (1994)

    Google Scholar 

  10. Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley, New York (2001). ISBN: 0-471-05669-3

    MATH  Google Scholar 

  11. Fujinami, K., Kawsar, F., Nakajima, T.: AwareMirror: a personalized display using a mirror. In: Proceedings of Pervasive, pp. 315–332 (2005)

  12. Gallagher, A.C., Chen, T.: Using context to recognize people in consumer images. IPSJ Trans. Comput. Vis. Appl. 1, 115–126 (2009)

    Article  Google Scholar 

  13. Haritaoglu, I., Flickner, M.: Attentive billboards. In: Proceedings of International Conference on Image Analysis and Processing (ICIAP), pp. 162–167 (2001)

  14. Harris, C., Stephens, M.: A combined corner and edge detector. In: Proceedings of the 4th Alvey Vision Conference, pp. 147–151 (1988)

  15. Iwabuchi, E., Nakagawa, M., Siio, I.: Smart makeup mirror: computer-augmented mirror to aid makeup application. HCI International in LNCS, vol. 5613, pp. 495–503. Springer, Berlin (2009)

  16. Kienzle, W., Bakir, G., Franz, M., Scholkopf, B.: Face detection – efficient and rank deficient. In: Proceedings of Advances in Neural Information Processing Systems (NIPS), pp. 673–680 (2005)

  17. Lee, D., Park, J., Lee, M., Hahn, M.: Personalized magic mirror: interactive mirror based on user behavior. In: Proceedings of 5th International Conference on Smart Homes and Health Telematics (ICOST), Nara, Japan, pp. 162–169 (2007)

  18. Lepetit, V., Fua, P.: Monocular model-based 3D tracking of rigid objects: a survey. Foundations and Trends®. Comput. Graphics Vis. 1(1) (2005)

  19. Like.com: http://www.like.com/. Last accessed 5 January 2010

  20. Lindsay, G.: Prada’s High-Tech Misstep. Business 2.0, 1 (2004) http://money.cnn.com/magazines/business2/business2_archive/2004/03/01/363574/index.htm

  21. Nanda, S.: Virtual Mirrors. Reuters (2007). http://www.reuters.com/news/video/videoStory?videoId=5219

  22. RockYou: http://www.rockyou.com/. Last accessed 5 January 2010

  23. Roussel, N., Evans, H., Hansen, H.: Proximity as an interface for video communication. Multimedia IEEE 11(3), 12–16 (2004)

    Article  Google Scholar 

  24. Rozin, D.: Wooden mirror. IEEE Spectr. 38, 3 (2001)

    Google Scholar 

  25. Sivic, J., Zitnick, C.L., Szeliski, R.: Finding people in repeated shot of the same scene. In: Proceedings of 16th British Machine Vision Conference (BMVC), pp. 909–918 (2006)

  26. Song, Y., Leung, T.: Context-aided human recognition – clustering. In: Proceedings of 9th European Conference on Computer Vision (ECCV) 3954, pp. 382–395 (2006)

  27. Tran, Q., Mynatt, E.: Cook’s collage: two exploratory designs. In: Proceedings of Computer/Human Interaction (CHI) Conference (2002)

  28. Truong, K., Abowd, G., Brotherton, J.: Who, what, when, where, how: design issues of capture & access applications. In: Proceedings of International Conference on Ubiquitous Computing (Ubicomp), pp. 209–224. ACM Press, New York (2001)

  29. Zhang, L., Chen, L., Li, M., Zhang, H.: Automated annotation of human faces in family albums. In: Proceedings of ACM Multimedia (MM), pp. 355–358 (2003)

  30. Zhang, W., Matsumoto, T., Liu, J., Chu, M., Begole, B.: An intelligent fitting room using multi-camera perception. In: Proceedings of International Conference on Intelligent User Interfaces (IUI), pp. 60–69 (2008)

  31. Zhang, W., Begole, B., Chu, M., Liu, J., Yee, N.: Real-time clothes comparison based on multi-view vision. In: Proceedings of ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC), 7–11 Sep (2008)

  32. Zhao, M., Teo, Y.W., Liu, S., Chua, T., Jain, R.: Automatic person annotation of family photo album. In: Proceedings of ACM International Conference on Image and Video Retrieval (CIVR), pp. 163–172 (2006)

Download references

Author information

Authors and Affiliations

  1. Like.com, 777 Mariner’s Island Blvd., Suite 510, San Mateo, CA, 94404, USA

    Wei Zhang

  2. Palo Alto Research Center, Inc. (PARC), 3333 Coyote Hill Road, Palo Alto, CA, 94304, USA

    Bo Begole & Maurice Chu

Authors
  1. Wei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo Begole
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maurice Chu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Zhang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhang, W., Begole, B. & Chu, M. Asynchronous reflections: theory and practice in the design of multimedia mirror systems. Multimedia Systems 16, 293–307 (2010). https://doi.org/10.1007/s00530-010-0192-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-010-0192-y

Keywords

Search

Navigation