Skip to main content
SpringerLink
Log in

RetroFlex: enabling intuitive human–robot collaboration with flexible retroreflective tags

  • Regular Paper
  • Published:
CCF Transactions on Pervasive Computing and Interaction Aims and scope Submit manuscript

Abstract

Efficient and seamless human–robot collaboration relies on carefully-designed user interfaces to relieve the users from heavy mental stress and burden. Current marker-based intuitive interfaces suffer from reduced reliability in a noisy environment, non-scalable wireless connections, and fixed form factors with limited data capacity, which limit the efficiency of real-world applications where frequent and seamless target switching is required. We propose RetroFlex, a flexible retroreflective communication system that enables fluent and intuitive human–robot collaboration. RetroFlex integrates robots with a flexible tag (FlexTag) and leverages visible light backscatter communication to exploit the intrinsic user spatial context to retain the intuitiveness in the interaction process. The users can interact with multiple target robots in a point-and-control manner, which significantly lowers the workload and improves the overall experience. Our evaluation reveals that RetroFlex is able to support room-scale tasks with commodity smartphones with a bit rate of 60 bps at a distance up to 2.5 m and a view angle up to 70\(^{\circ }\) while being robust to different environmental noises. A usability study with 12 users and two real-life tasks demonstrates that our system offers a fluent and satisfying multi-target control experience.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Ahmad, F., Jae Jeon, Y., Jamil, M.: Graphene-based polymer dispersed liquid crystals display—an overview. Mol. Cryst. Liq. Cryst. 669(1), 46–60 (2018)

    Article  Google Scholar 

  • Almeida, L., Menezes, P., Dias, J.: Improving robot teleoperation experience via immersive interfaces. In: 2017 4th Experiment@ International Conference (exp. At’17), pp. 87–92. IEEE (2017)

  • Baygin, M., Yaman, O., Baygin, N., Karakose, M.: A blockchain-based approach to smart cargo transportation using UHF RFID. Expert Syst. Appl. 188, 116030 (2022)

    Article  Google Scholar 

  • Bernardini, F., Buffi, A., Fontanelli, D., Macii, D., Magnago, V., Marracci, M., Motroni, A., Nepa, P., Tellini, B.: Robot-based indoor positioning of UHF-RFID tags: the SAR method with multiple trajectories. IEEE Trans. Instrum. Meas. 70, 1–15 (2020)

    Article  Google Scholar 

  • Bonarini, A.: Communication in human–robot interaction. Curr. Robot. Rep. 1(4), 279–285 (2020)

    Article  Google Scholar 

  • Bonilla, I., Mendoza, M., Gonzalez-Galvan, E.J., Chavez-Olivares, C., Loredo-Flores, A., Reyes, F.: Path-tracking maneuvers with industrial robot manipulators using uncalibrated vision and impedance control. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 42(6), 1716–1729 (2012)

  • Brantner, G., Khatib, O.: Controlling ocean one: human–robot collaboration for deep-sea manipulation. J. Field Robot. 38(1), 28–51 (2021)

    Article  Google Scholar 

  • Chong, J.W.S., Ong, S., Nee, A.Y., Youcef-Youmi, K.: Robot programming using augmented reality: an interactive method for planning collision-free paths. Robot. Comput. Integrated Manuf. 25(3), 689–701 (2009)

    Article  Google Scholar 

  • Doane, J., Golemme, A., West, J.L., Whitehead, J., Jr., Wu, B.-G.: Polymer dispersed liquid crystals for display application. Mol. Cryst. Liq. Cryst. 165(1), 511–532 (1988)

    Google Scholar 

  • Douglas, D.H., Peucker, T.K.: Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartogr. Int. J. Geogr. Inf. Geovisualization 10(2), 112–122 (1973)

  • Du, G., Zhang, P.: A markerless human–robot interface using particle filter and Kalman filter for dual robots. IEEE Trans. Ind. Electron. 62(4), 2257–2264 (2014)

    Article  Google Scholar 

  • Enan, S.S., Fulton, M., Sattar, J.: Robotic detection of a human-comprehensible gestural language for underwater multi-human–robot collaboration. arXiv preprint arXiv:2207.05331 (2022)

  • Granqvist, C.G., Avendaño, E., Azens, A.: Electrochromic coatings and devices: survey of some recent advances. Thin Solid Films 442(1–2), 201–211 (2003)

    Article  Google Scholar 

  • Grasse, L., Boutros, S.J., Tata, M.S.: Speech interaction to control a hands-free delivery robot for high-risk health care scenarios. Front. Robot. AI 8, 612750 (2021)

  • Gummadi, R., Wetherall, D., Greenstein, B., Seshan, S.: Understanding and mitigating the impact of rf interference on 802.11 networks. ACM SIGCOMM Comput. Commun. Rev. 37(4), 385–396 (2007)

  • Hancock, P.A., Billings, D.R., Schaefer, K.E., Chen, J.Y., De Visser, E.J., Parasuraman, R.: A meta-analysis of factors affecting trust in human–robot interaction. Hum. Factors 53(5), 517–527 (2011)

    Article  Google Scholar 

  • Hedayati, H., Walker, M., Szafir, D.: Improving collocated robot teleoperation with augmented reality. In: Proceedings of the 2018 ACM/IEEE International Conference on Human–Robot Interaction, pp. 78–86 (2018)

  • Hu, C., Ma, D., Hassan, M., Hu, W.: Nlc: natural light communication using switchable glass. In: IEEE INFOCOM 2020-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pp. 201–206. IEEE (2020)

  • Hu, Z., Marshall, C., Bicker, R., Taylor, P.: Automatic surface roughing with 3d machine vision and cooperative robot control. Robot. Auton. Syst. 55(7), 552–560 (2007)

    Article  Google Scholar 

  • Islam, M.J., Ho, M., Sattar, J.: Dynamic reconfiguration of mission parameters in underwater human–robot collaboration. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 6212–6219. IEEE (2018)

  • Islam, M.J., Ho, M., Sattar, J.: Understanding human motion and gestures for underwater human–robot collaboration. J. Field Robot. 36(5), 851–873 (2019)

    Article  Google Scholar 

  • Keyes, B., Micire, M., Drury, J.L., Yanco, H.A., Chugo, D.: Improving human–robot interaction through interface evolution. In: Human–Robot Interaction, pp. 183–202. InTech (2010)

  • Lai, K.-Y., Chen, W.-T., Wu, Y.-H., Chen, Y.-F., Tsai, J.-c.: 3D-printed and pdlc-tuned corner cube retroreflector for sunlight communication. In: 2019 International Conference on Optical MEMS and Nanophotonics (OMN), pp. 164–165. IEEE (2019)

  • Lampert, C.M.: Smart switchable glazing for solar energy and daylight control. Sol. Energy Mater. Sol. Cells 52(3–4), 207–221 (1998)

    Article  Google Scholar 

  • Li, J., Liu, A., Shen, G., Li, L., Sun, C., Zhao, F.: Retro-VLC: enabling battery-free duplex visible light communication for mobile and iot applications. In: Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications, pp. 21–26 (2015)

  • Magnago, V., Palopoli, L., Buffi, A., Tellini, B., Motroni, A., Nepa, P., Macii, D., Fontanelli, D.: Ranging-free uhf-rfid robot positioning through phase measurements of passive tags. IEEE Trans. Instrum. Meas. 69(5), 2408–2418 (2019)

    Article  Google Scholar 

  • Mateo, C., Brunete, A., Gambao, E., Hernando, M.: Hammer: an android based application for end-user industrial robot programming. In: 2014 IEEE/ASME 10th International Conference on Mechatronic and Embedded Systems and Applications (MESA), pp. 1–6. IEEE (2014)

  • Nathan, A., Ahnood, A., Cole, M.T., Lee, S., Suzuki, Y., Hiralal, P., Bonaccorso, F., Hasan, T., Garcia-Gancedo, L., Dyadyusha, A., et al.: Flexible electronics: the next ubiquitous platform. In: Proceedings of the IEEE 100 (Special Centennial Issue), pp. 1486–1517 (2012)

  • Punnoose, R.J., Tseng, R.S., Stancil, D.D.: Experimental results for interference between Bluetooth and IEEE 802.11 b DSSS systems. In: IEEE 54th Vehicular Technology Conference. VTC Fall 2001. Proceedings (Cat. No. 01CH37211), vol. 1, pp. 67–71. IEEE (2001)

  • Qin, K., Chen, C., Pu, X., Tang, Q., He, W., Liu, Y., Zeng, Q., Liu, G., Guo, H., Hu, C.: Magnetic array assisted triboelectric nanogenerator sensor for real-time gesture interaction. Nano-micro Lett. 13(1), 1–9 (2021)

    Article  Google Scholar 

  • Quintero, C.P., Li, S., Pan, M.K., Chan, W.P., Van der Loos, H.M., Croft, E.: Robot programming through augmented trajectories in augmented reality. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1838–1844. IEEE (2018)

  • Ramzan, A., Rehman, S., Perwaiz, A.: RFID technology: beyond cash-based methods in vending machine. In: 2017 2nd International Conference on Control and Robotics Engineering (ICCRE), pp. 189–193. IEEE (2017)

  • Shao, S., Khreishah, A., Elgala, H.: Pixelated VLC-backscattering for self-charging indoor IoT devices. IEEE Photonics Technol. Lett. 29(2), 177–180 (2016)

    Article  Google Scholar 

  • Solvang, B., Sziebig, G., Korondi, P.: Robot programming in machining operations. In: Robot Manipulators, pp. 479–496. I-Tech, Vienna, Austria (2008)

    Google Scholar 

  • Stadler, S., Kain, K., Giuliani, M., Mirnig, N., Stollnberger, G., Tscheligi, M.: Augmented reality for industrial robot programmers: workload analysis for task-based, augmented reality-supported robot control. In: 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 179–184. IEEE (2016)

  • Van den Bergh, M., Carton, D., De Nijs, R., Mitsou, N., Landsiedel, C., Kuehnlenz, K., Wollherr, D., Van Gool, L., Buss, M.: Real-time 3d hand gesture interaction with a robot for understanding directions from humans. In: 2011 Ro-Man, pp. 357–362. IEEE (2011)

  • Vergaz, R., Pena, J., Barrios, D., Pérez, I., Torres, J.: Electrooptical behaviour and control of a suspended particle device. Opto-Electron. Rev. 15(3), 154–158 (2007)

    Article  Google Scholar 

  • Villani, V., Sabattini, L., Riggio, G., Secchi, C., Minelli, M., Fantuzzi, C.: A natural infrastructure-less human–robot interaction system. IEEE Robot. Autom. Lett. 2(3), 1640–1647 (2017)

    Article  Google Scholar 

  • Walker, M., Hedayati, H., Lee, J., Szafir, D.: Communicating robot motion intent with augmented reality. In: Proceedings of the 2018 ACM/IEEE International Conference on Human–Robot Interaction, pp. 316–324 (2018)

  • Wang, P., Feng, L., Chen, G., Xu, C., Wu, Y., Xu, K., Shen, G., Du, K., Huang, G., Liu, X.: Renovating road signs for infrastructure-to-vehicle networking: a visible light backscatter communication and networking approach. In: Proceedings of the 26th Annual International Conference on Mobile Computing and Networking, pp. 1–13 (2020)

  • Wu, Y., Wang, P., Xu, K., Feng, L., Xu, C.: Turboboosting visible light backscatter communication. In: Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication, pp. 186–197 (2020)

  • Xu, X., Shen, Y., Yang, J., Xu, C., Shen, G., Chen, G., Ni, Y.: Passivevlc: enabling practical visible light backscatter communication for battery-free IOT applications. In: Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking, pp. 180–192 (2017)

  • Yang, J.: Apparatus and method for suppressing interference caused by coexistence of WiMAX and WiFi. Google Patents. US Patent 8,553,622 (2013)

  • Yang, H.-D., Park, A.-Y., Lee, S.-W.: Gesture spotting and recognition for human–robot interaction. IEEE Trans. Rob. 23(2), 256–270 (2007)

    Article  Google Scholar 

  • Zhang, H., Chen, H., Xi, N., Zhang, G., He, J.: On-line path generation for robotic deburring of cast aluminum wheels. In: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2400–2405. IEEE (2006)

Download references

Funding

This study was funded by the National Natural Science Foundation of China (Grant No. 62022005, 62272010, 62061146001, and 62172008), ICT Grant CARCHB202017, and the Jiangxi Provincial Key R&D Program (Grant No. 20212BBE53004).

Author information

Authors and Affiliations

  1. School of Computer Science, Peking University, No. 5 Yiheyuan Road, Haidian District, 100871, Beijing, China

    Wei Li, Zhe Ou & Chenren Xu

  2. Paul G. Allen School of Computer Science and Engineering, University of Washington, 1410 NE Campus Parkway, Seattle, WA, 98195, USA

    Tuochao Chen

  3. School of Mathematics and Computer Science, Nanchang University, No. 999 Xuefu Road, Nanchang, 330031, Jiangxi, China

    Xin Wen & Zichen Xu

Authors
  1. Wei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tuochao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhe Ou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xin Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zichen Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chenren Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chenren Xu.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article. On behalf of all authors, the corresponding author states that there is no conflict of interest.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (mp4 4294 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, W., Chen, T., Ou, Z. et al. RetroFlex: enabling intuitive human–robot collaboration with flexible retroreflective tags. CCF Trans. Pervasive Comp. Interact. 4, 437–451 (2022). https://doi.org/10.1007/s42486-022-00120-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42486-022-00120-7

Keywords

Search

Navigation