Skip to main content
Log in

FrameTalk: Human and Picture Frame Interaction through the IoT Technology

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

The concept of frame plays a significant role in art design. However, traditional picture frames do not offer human interaction. To create a new dimension of graphic artwork demonstration, we utilize the Internet of Things (IoT) technology to develop human and digital frame interaction. Specifically, this paper uses IoTtalk, an application-layer IoT device management platform, to quickly establish connections and meaningful interactions between IoT devices. The interaction between human and frame is accomplished by implementing an output device of IoTtalk called FrameTalk, which allows the input devices to control the dynamic graphics on a digital frame. The FrameTalk device can be exhibited in specific frame hardware or the screen of any computing device. We illustrate the functional block diagram of the FrameTalk device and its default applications. Then we demonstrate two FrameTalk application examples. The first example is CalligraphyTalk, where an audience can interact with the frame to write Chinese poems. The second example is PortraitGuess that allows an audience to play the historical figures portrait painting guess game by the frame. We measure the delays between the smartphone and the FrameTalk device, and develop both analytic and simulation models to investigate the user experience of the FrameTalk applications. Our study indicates that good user experience can be achieved in FrameTalk.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Nixplay (2018) https://www.nixplay.com. Accessed 7 May 2018

  2. Meural (2018) https://meural.com. Accessed 7 May 2018

  3. Yiiisu (2018) http://yiiisu.com. Accessed 7 May 2018

  4. FRAMED (2018) https://frm.fm. Accessed 7 May 2018

  5. Lin YB, Lin YW, Chih CY, Li TY, Tai CC, Wang YC, Lin FJ, Kuo HC, Huang CC, Hsu SC (2015) EasyConnect: A Management System for IoT Devices and Its Applications for Interactive Design and Art. IEEE Internet Things J 2(6):551–561

    Article  Google Scholar 

  6. Lin YB, Lin YW, Huang CM, Chih CY, Lin P (2017) IoTtalk: A Management Platform for Reconfigurable Sensor Devices. IEEE Internet Things J 4(5):1552–1562

    Article  Google Scholar 

  7. Lin YW, Lin YB, Hsiao CY, Wang YY (2017) IoTtalk-RC: Sensors as Universal Remote Control for Aftermarket Home Appliances. IEEE Internet Things J 4(4):1104–1112

    Article  Google Scholar 

  8. Processing (2018) https://processing.org. Accessed 7 May 2018

  9. Hsiao CY, Chen LK, Lin YB (2017) Interacting with Paintings using IoTtalk. In: IoT’17 Proceedings of the Seventh International Conference on the Internet of Things. doi: https://doi.org/10.1145/3131542.3140269

  10. Hsiao CY, Huang CC, Lin YB, Lin YW (2018) Flower Sermon: An Interactive Visual Design using IoTtalk. Mobile Networks and Applications Journal. https://doi.org/10.1007/s11036-018-1086-z

    Article  Google Scholar 

  11. Human Benchmark (2018) https://www.humanbenchmark.com/tests/reactiontime/statistics. Accessed 7 May 2018

  12. Asia Digital Art Award FUKUOKA (2018) https://adaa.jp/en/winners/winners2017. Accessed 7 May 2018

  13. PortraitGuess demo video (2018) https://www.youtube.com/watch?v=j7V7Xa0hf30&feature=youtu.be. Accessed 9 Oct 2018

  14. CalligraphyTalk demo video (2018) https://www.youtube.com/watch?time_continue=2&v=1KGgCiFVwPK. Accessed 9 Oct 2018

  15. Lai WS, Lin YB, Hsiao CY, Chen LK, Wu CF, Lin SM (2018) FrameTalk: Human and Picture Frame Interaction through the IoT Technology. In: 4th EAI International Conference on IoT as a Service, Xi’an

Download references

Acknowledgments

This work was supported in part by the Ministry of Science and Technology (MOST) under Grant 106-2221-E-009-049- MY2, Grant 107-2622-8-009-020 and Grant 106-2911-I-009- 508, in part by Academia Sinica AS-105-TP-A07, in part by the Center for Open Intelligent Connectivity from the Featured Areas Research Center Program within the framework of the Higher Education Sprout Project by the Ministry of Education (MOE) in Taiwan.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yi-Bing Lin.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lai, WS., Lin, YB., Hsiao, CY. et al. FrameTalk: Human and Picture Frame Interaction through the IoT Technology. Mobile Netw Appl 24, 1475–1485 (2019). https://doi.org/10.1007/s11036-019-01269-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-019-01269-9

Keywords

Navigation