Abstract
In this paper, we introduce an intelligent augmented reality shopping system that supports image search and 3D virtual shop generation. By capturing an image of the product that the user is looking at, and using image search, we can recommend similar products to the user. Based on the spatial understanding techniques, we developed a 3D virtual shop interface that would be automatically generated in the user’s physical room, where products are arranged around the room at appropriate locations. With the virtual shop display, users would be able to interact with the products’ life-size virtual models by full-hand manipulation. Our system provides more natural interactions such as grabbing and touching. We also carried out an evaluation study for using our system, the participants’ feedback shows that our system provides a better shopping experience with its intuitive search process, immersive shop display, and natural interactions.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Luo, P., Yan, S., Liu, Z., Shen, Z., Yang, S., He, Q.: From online behaviors to offline retailing. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 175–184. ACM, August 2016
Close, A.G., Kukar-Kinney, M.: Beyond buying: motivations behind consumers’ online shopping cart use. J. Bus. Res. 63(9–10), 986–992 (2010). Lu, Y., Smith, S. (2007)
Zhao, Y., Guo, L., Wang, X., Pan, Z.: A 3D virtual shopping mall that has the intelligent virtual purchasing guider and cooperative purchasing functionalities. In 8th International Conference on Computer Supported Cooperative Work in Design, vol. 2, pp. 381–385. IEEE, May 2004
Dou, H., Li, Z., Cai, M., Cheng, K., Masuko, S., Tanaka, J.: Show something: intelligent shopping assistant supporting quick scene understanding and immersive preview. In: Yamamoto, S., Mori, H. (eds.) HCII 2019. LNCS, vol. 11570, pp. 205–218. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22649-7_17
Tanikawa, T., Arai, T., Masuda, T.: Development of micro manipulation system with two-finger micro hand. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 1996, vol. 2, pp. 850–855. IEEE, November 1996
Liu, J., Au, O.K.C., Fu, H., Tai, C.L.: Two-finger gestures for 6DOF manipulation of 3D objects. Comput. Graph. Forum 31(7), 2047–2055 (2012)
Weichert, F., Bachmann, D., Rudak, B., Fisseler, D.: Analysis of the accuracy and robustness of the leap motion controller. Sensors 13(5), 6380–6393 (2013)
Jailungka, P., Charoenseang, S.: Intuitive 3D model prototyping with leap motion and microsoft HoloLens. In: Kurosu, M. (ed.) HCI 2018. LNCS, vol. 10903, pp. 269–284. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91250-9_21
Microsoft HoloLens | Mixed Reality Technology for Business (2019). https://www.microsoft.com/en-us/hololens. Accessed 14 Dec 2019
Leap Motion (2019). https://www.leapmotion.com/. Accessed 14 Dec 2019
Lu, Y., Smith, S.: Augmented reality e-commerce assistant system: trying while shopping. In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4551, pp. 643–652. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-73107-8_72
Levin, A.M., Levin, I.P., Weller, J.A.: A multi-attribute analysis of preferences for online and offline shopping: differences across products, consumers, and shopping stages. J. Electron. Commer. Res. 6(4), 281–290 (2005)
Getting started with MRTK version 2 - Mixed Reality (2020). https://github.com/Microsoft/MixedRealityToolkit-Unity/releases. Accessed 26 Jan 2020
Garon, M., Boulet, P.O., Doironz, J.P., Beaulieu, L., Lalonde, J.F.: Real-time high resolution 3D data on the HoloLens. In: 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), pp. 189–191. IEEE (2016)
Wozniak, P., Vauderwange, O., Mandal, A., Javahiraly, N., Curticapean, D.: Possible applications of the LEAP motion controller for more interactive simulated experiments in augmented or virtual reality. In: Optics Education and Outreach IV, vol. 9946, p. 99460P. International Society for Optics and Photonics, September 2016
Ababsa, F., He, J., Chardonnet, J.R.: Free hand-based 3D interaction in optical see-through augmented reality using leap motion, October 2018
Zhu, W., Owen, C.B., Li, H., Lee, J.H.: Personalized in-store e-commerce with the promopad: an augmented reality shopping assistant. Electron. J. e-Commer. Tools Appl. 1(3), 1–19 (2004)
Dou, H.: Spatial recognition based mixed-reality shop system providing responsive layout. Unpublished master’s thesis, Graduate School of Information, Production and System, Waseda University, Japan, September 2019
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, Z. et al. (2020). Augmented Reality Shopping System Through Image Search and Virtual Shop Generation. In: Yamamoto, S., Mori, H. (eds) Human Interface and the Management of Information. Designing Information. HCII 2020. Lecture Notes in Computer Science(), vol 12184. Springer, Cham. https://doi.org/10.1007/978-3-030-50020-7_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-50020-7_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-50019-1
Online ISBN: 978-3-030-50020-7
eBook Packages: Computer ScienceComputer Science (R0)