Skip to main content
Springer Nature Link
Log in

An ontology-based approach to integrate TV and IoT middlewares

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Internet of Things (IoT) is the interconnection of physical devices, known of smart objects, that share data and collaborate among them to support users. There are IoT usage scenarios in several domains. In-home domain, the use of smart objects may support monitoring and automation of the environment. In a home, the TV is one of the most used devices. It may also collaborate with the smart objects to enhance the interactive experience of viewers on watching. To support such a meeting, we propose an (i) conceptual model, IoTTV-Ont, that allows modeling entire homes, with smart objects, people’ profile, and viewers’ interaction events with TV applications; (ii) a software architecture that integrates and interoperates digital TV and IoT middleware. This architecture allows the TV applications: to be aware of the physical environment context, to change aspects of the physical environment, to identify the viewers, and to receive multimodal interactions performed by viewers. We demonstrate our work through use cases. For this, we implement a prototype of our software architecture and modeled home using IoTTV-Ont. Then we developed usage scenarios that enhance TV viewers’ experience by supporting features such as content adaptation, multi-sensorial immersive experience, and multimodal user interaction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. http://itu.int/rec/T-REC-H.761

  2. http://hbbtv.org

  3. http://arduino.cc

  4. http://intel.com/content/www/us/en/ark/products/83137/intel-galileo-gen-2-board

  5. http://w3.org/OWL

  6. Dictionary of people-related terms that used in structured data (e.g., RDF, JSON-LD, Linked Data) [6].

  7. http://jena.apache.org

  8. http://github.com/sbrunk/jena-android

  9. http://espertech.com/esper

  10. http://github.com/mobile-event-processing/Asper

  11. http://github.com/makleystonlsdi/SDPEU

  12. http://mqtt.org

  13. http://github.com/technocreatives/moquette

  14. http://eclipse.org/paho

  15. http://github.com/geekscape/mqtt_lua

  16. http://github.com/makleystonlsdi/iDTVModules

  17. http://arduino.cc

  18. http://lsdi.ufma.br/~iottv

  19. http://vmware.com/products/workstation-player.html

  20. http://gingancl.org.br/pt-br/ferramentas

References

  1. Abreu R, Mattos D, dos Santos JAF, Muchaluat-Saade DC (2019) Semi-automatic synchronization of sensory effects in mulsemedia authoring tools. In: Proceedings of the 25th Brazillian symposium on multimedia and the Web - WebMedia 19, ACM Press. https://doi.org/10.1145/3323503.3360302

  2. Al-Fuqaha A, Guizani M, Mohammadi M, Aledhari M, Ayyash M (2015) Internet of things: a survey on enabling technologies, protocols, and applications. IEEE Communications Surveys & Tutorials 17(4):2347–2376. https://doi.org/10.1109/COMST.2015.2444095

    Article  Google Scholar 

  3. Atzori L, Iera A, Morabito G (2010) The internet of things: a survey. Computer Networks 54(15):2787–2805. https://doi.org/10.1016/j.comnet.2010.05.010

    Article  MATH  Google Scholar 

  4. Biljana L, Kire V (2017) A review of internet of things for smart home: Challenges and solutions. J Clean Prod 140:1454–1464. https://doi.org/10.1016/j.jclepro.2016.10.006

    Article  Google Scholar 

  5. Bonino D, Corno F (2008) DogOnt - Ontology Modeling for Intelligent Domotic Environments. Springer, Berlin, pp 790–803. https://doi.org/10.1007/978-3-540-88564-1_51

    Google Scholar 

  6. Brickley D, Miller L. (2007) Foaf vocabulary specification 0.91. Tech. rep., FOAF

  7. Cesar P, Chorianopoulos K (2009) The evolution of tv systems, content, and users toward interactivity. Foundations and Trends® in Human–Computer Interaction 2(4):279–373. https://doi.org/10.1561/1100000008

    Article  Google Scholar 

  8. Chen H, Finin T, Joshi A (2005) The SOUPA Ontology for Pervasive Computing. Birkhäuser Basel, Basel, pp 233–258. https://doi.org/10.1007/3-7643-7361-X_10

    Google Scholar 

  9. Christophe J, Ahmed M, Yacine B (2013) An ambient assisted living framework with automatic self-diagnosis, vol 5

  10. Etzion O, Niblett P (2011) Event Processing in Action, 1st edn. Manning Publications Co., Greenwich, CT USA

  11. Eugster P, Felber P, Guerraoui R, Kermarrec A (2003) The many faces of publish/subscribe. ACM Comput Surv 35(2):114–131. https://doi.org/10.1145/857076.857078

    Article  Google Scholar 

  12. Gomes B, Muniz L, Silva FJ, Ríos L. E. T., Endler M (2017) A comprehensive and scalable middleware for ambient assisted living based on cloud computing and internet of things. Concurrency and Computation: Practice and Experience 29(11):e4043. https://doi.org/10.1002/cpe.4043

    Article  Google Scholar 

  13. Görgü L, Kroon B, Campbell A, O’Hare G (2013) Enabling a Mobile, Dynamic and Heterogeneous Discovery Service in a Sensor Web by Using AndroSIXTH. Springer International Publishing, Cham, pp 287–292. https://doi.org/10.1007/978-3-319-03647-2_26

    Google Scholar 

  14. Gruber T (1993) A translation approach to portable ontology specifications. Knowl Acquis 5(2):199–220. https://doi.org/10.1006/knac.1993.1008

    Article  Google Scholar 

  15. Guarino N (1998) Formal Ontology in Information systems. In: Proceedings of the 1st international conference June 6-8, 1998, Trento, Italy. 1st edn. IOS Press, Amsterdam

  16. Gubbi J, Buyya R, Marusic S, Palaniswami M (2013) Internet of things (iot): A vision, architectural elements, and future directions. Future Gener Comput Syst 29 (7):1645–1660. https://doi.org/10.1016/j.future.2013.01.010

    Article  Google Scholar 

  17. Guedes Á, Cunha M, Fuks H, Colcher S, Barbosa S (2016) Using ncl to synchronize media objects, sensors and actuators. In: XXII Simpósio Brasileiro de Sistemas Multim’idia e Web (Vol. 2): Workshops e Sessão de Pôsteres., Webmedia, Porto Alegre: Sociedade Brasileira de Computação, pp 184–189

  18. Hunkeler U, Truong H, Stanford-Clark A (2008) Mqtt-s; a publish/subscribe protocol for wireless sensor networks. In: 2008. COMSWARE 2008. 3rd international conference on Communication systems software and middleware and workshops, pp 791–798

  19. ISO (2016) ISO/IEC 23005-3:2016: Information technology – Media context and control – Part 3: Sensory information

  20. ITU-T (2014) Nested context language (ncl) and ginga-ncl. Recommendation H.761, International Telecommunication Union, Geneva

  21. Kapri A, Adalgeirsson Y, Kator K, Chung J, Holtzman H Television (tv) as an internet of things (iot) participant (2017). US Patent App. 15/087,813

  22. Lima E, Rabêlo R (2015) An architectural model for communication between the idtv and mobile devices. In: 2015 international conference on Computing, networking and communications (ICNC), IEEE, pp 1102–1105

  23. Lojka M, Pleva M, Kiktová E, Juhár J, Čižmár A (2015) Efficient acoustic detector of gunshots and glass breaking. Multimedia Tools and Applications 75 (17):10441–10469. https://doi.org/10.1007/s11042-015-2903-z

    Article  Google Scholar 

  24. Luckham D (2008) The power of events: an introduction to complex event processing in distributed enterprise systems. In: Bassiliades N, Governatori G, Paschke A (eds) Rule representation, interchange and reasoning on the web. Springer, Berlin, pp 3–3

  25. Miorandi D, Sicari S, Pellegrini FD, Chlamtac I (2012) Internet of things: Vision, applications and research challenges. Ad Hoc Netw 10 (7):1497–1516. https://doi.org/10.1016/j.adhoc.2012.02.016

    Article  Google Scholar 

  26. Noy N, McGuinness D (2001) Ontology development 101: a guide to creating your first ontology. Tech. rep., Stanford Knowledge Systems Laboratory

  27. Perera C, Jayaraman P, Zaslavsky A, Christen P, Georgakopoulos D (2014) Mosden: an internet of things middleware for resource constrained mobile devices. In: 2014 47Th hawaii international conference on system sciences, pp 1053–1062. https://doi.org/10.1109/HICSS.2014.137

  28. Perera C, Zaslavsky A, Christen P, Georgakopoulos D (2014) Context aware computing for the internet of things: a survey. IEEE Communications Surveys Tutorials 16(1):414–454. https://doi.org/10.1109/SURV.2013.042313.00197

    Article  Google Scholar 

  29. Punt M, Bjelica M, Zdravkovic V, Teslic N (2015) An integrated environment and development framework for social gaming using mobile devices, digital tv and internet. Multimedia Tools and Applications 74(18):8137–8169. https://doi.org/10.1007/s11042-014-2045-8

    Article  Google Scholar 

  30. Rosa R, Lucena V (2017) Contextualizing and capturing individual user interactions in shared itv environments. Multimedia Tools and Applications 76 (6):8573–8595. https://doi.org/10.1007/s11042-016-3489-9

    Article  Google Scholar 

  31. Saleme E, Santos C (2015) PlaySEM: A platform for rendering MulSeMedia compatible with MPEG-v. In: Proceedings of the 21st Brazilian symposium on multimedia and the Web, WebMedia ’15. ACM, pp 145–148. https://doi.org/10.1145/2820426.2820450

  32. Silva V, Maia O, Rodrigues M, Lucena V (2016) Universal system for integrating commercial medical devices with standardized digital tv system. In: 2016 IEEE International conference on consumer electronics (ICCE), pp 301–304. https://doi.org/10.1109/ICCE.2016.7430622

  33. Sommaruga L, Formilli T, Rizzo N (2011) Domoml: an integrating devices framework for ambient intelligence solutions. In: Proceedings of the 6th international workshop on enhanced web service technologies, WEWST ’11. ACM, New York, pp 9–15, https://doi.org/10.1145/2031325.2031327

  34. Sulema Y (2016) Mulsemedia vs. multimedia: State of the art and future trends. In: 2016 International conference on systems, signals and image processing (IWSSIP), pp 1–5. https://doi.org/10.1109/IWSSIP.2016.7502696

  35. Talavera L, Endler M, Vasconcelos I, Vasconcelos R, Cunha M, Silva F (2015) The mobile hub concept: Enabling applications for the internet of mobile things. In: 2015 IEEE International conference on pervasive computing and communication workshops (percom workshops), pp 123–128. https://doi.org/10.1109/PERCOMW.2015.7134005

  36. Wehbi A, Cherif A, Tadj C (2012) Modeling ontology for multimodal interaction in ubiquitous computing systems. In: Proceedings of the 2012 ACM conference on ubiquitous computing, UbiComp ’12. ACM, New York, pp 842–849. https://doi.org/10.1145/2370216.2370408

  37. Zhu Q, Wang R, Chen Q, Liu Y, Qin W (2010) Iot gateway: Bridgingwireless sensor networks into internet of things. In: 2010 IEEE/IFIP 8th international conference on Embedded and ubiquitous computing (EUC). IEEE, Hong Kong, pp 347–352. https://doi.org/10.1109/EUC.2010.58

Download references

Author information

Authors and Affiliations

  1. Federal Institute of Tocantins, Colinas of Tocantins, TO, Brazil

    Danne Makleyston G. Pereira

  2. Universidade Federal do Maranhão, São Luís, MA, Brazil

    Francisco José da S. e Silva, Carlos de Salles S. Neto, Davi Viana dos Santos & Luciano Reis Coutinho

  3. Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro, RJ, Brazil

    Álan L. V. Guedes

Authors
  1. Danne Makleyston G. Pereira
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Francisco José da S. e Silva
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Carlos de Salles S. Neto
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Davi Viana dos Santos
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Luciano Reis Coutinho
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Álan L. V. Guedes
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Danne Makleyston G. Pereira.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pereira, D.M.G., da S. e Silva, F.J., de Salles S. Neto, C. et al. An ontology-based approach to integrate TV and IoT middlewares. Multimed Tools Appl 80, 1813–1837 (2021). https://doi.org/10.1007/s11042-020-09645-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09645-4

Keywords