skip to main content
research-article

Mobile haptic e-book system to support 3D immersive reading in ubiquitous environments

Published: 19 August 2013 Publication History

Abstract

In order to leverage the use of various modalities such as audio-visual materials in instilling effective learning behavior we present an intuitive approach of annotation based hapto-audio-visual interaction with the traditional digital learning materials such as e-books. By integrating the home entertainment system in the user's reading experience combined with haptic interfaces we want to examine whether such augmentation of modalities influence the user's learning patterns. The proposed Haptic E--Book (HE-Book) system leverages the haptic jacket, haptic arm band as well as haptic sofa interfaces to receive haptic emotive signals wirelessly in the form of patterned vibrations of the actuators and expresses the learning material by incorporating image, video, 3D environment based augmented display in order to pave ways for intimate reading experience in the popular mobile e-book platform.

References

[1]
Alam, K. M., Hardy, S., Akther, A., and El Saddik, A. 2011a. SMS text based affective haptic application. In Proceedings of the Virtual Reality International Conference (VRIC'11).
[2]
Alam, K. M., Rahman, A. S. M. M., and El Saddik, A. 2011b. HE-book: A prototype haptic interface for immersive e-book reading experience. In Proceedings of the World Haptics Conference (WHC'11). IEEE. 367--371. http://dx.doi.org/10.1109/WHC.2011.5945514.
[3]
Alam, K. M., Rahman, M. A., El Saddik, A., and Gueaieb, W. 2011c. Adding emotional tag to augment context-awareness in social network services. In Proceedings of the Instrumentation and Measurement Technology Conference (I2MTC'11). IEEE, 1--6. http://dx.doi.org/10.1109/IMTC.2011.5944225.
[4]
Arafsha, F., Alam, K. M., and El Saddik A. 2012. EmoJacket: Consumer centric wearable affective jacket to enhance emotional immersion. In Proceedings of the International Conference on Innovations in Information Technology. 350--355. http://dx.doi.org/10.1109/INNOVATIONS.2012.6207766.
[5]
Barghout, A., Cha, J., El Saddik, A., Kammerl, J., and Steinbach, E. 2009. Spatial resolution of vibrotactile perception on the human forearm when exploiting funneling illusion. In Proceedings of the IEEE International Workshop on HAVE. 19--23.
[6]
Billinghurst, M., Kato, H., and Poupyrev, I. 2001. The MagicBook: Moving seamlessly between reality and virtuality. IEEE Comput. Graph. Appl. 21, 3, 6--8. http://dx.doi.org/10.1109/38.920621.
[7]
Brewster, S. A., Chohan, F., and Brown L. M. 2007. Tactile feedback for mobile interactions. In Proceedings of the Conference on Human Factors in Computing Systems. ACM Press, 159--162.
[8]
Brunken, R., Plass, J. L., and Leutner, D. 2003. Direct measurement of cognitive load in multimedia learning. Educ. Psych. 38, 1, 53--61. http://www.informaworld.com/10.1207/S15326985EP3801n 7.
[9]
Card, S. K., Hong, L., Mackinlay, J. D., Chi, E. H., and Chi, H. 2004. 3Book: A scalable 3D virtual book. In Extended Abstracts of the Conference on Human Factors and Computing Systems, ACM Press, 1095--1098.
[10]
Carney, R. and Levin, J. 2002. Pictorial illustrations still improve students' learning from text. Educ. Psych. Rev. 14, 5--26. http://dx.doi.org/10.1023/A:1013176309260 10.1023/A:1013176309260.
[11]
Carver, R. P. 1992. Reading rate: theory, research, and practical implications. J. Read. 36, 2, 84--95. http://www.jstor.org/stable/40016440.
[12]
Chen, N., Guimbretiere, F., Dixon, M., Lewis, C., and Agrawala, M. 2008. Navigation techniques for dual-display e-book readers. In Proceeding of the SIGCHI conference on Human factors in computing systems (CHI'08). ACM, 1779--1788.
[13]
Dale, E. 1969. Audio-Visual Methods in Teaching. Rinehart and Winston.
[14]
Grasset, R., Dunser, A., and Billinghurst, M. 2008. The design of a mixed-reality book: Is it still a real book? In Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR'08). 99--102. http://dx.doi.org/10.1109/ISMAR.2008.4637333.
[15]
Gupta, S. and Jaynes, C. 2006. The universal media book: tracking and augmenting moving surfaces with projected information. In Proceedings of the 5th IEEE and ACM International Symposium on Mixed and Augmented Reality. IEEE, 177--180.
[16]
Haans, A. and Ijsselsteijn, W. 2006. Mediated social touch: a review of current research and future directions. Virtual Real. 9, 2, 149--159.
[17]
Harrison, B. L. 2000. E-books and the future of reading. IEEE Comput. Graph. Appl. 20, 32--39.
[18]
Hoggan, E., Brewster, S. A., and Johnston, J. 2008. Investigating the effectiveness of tactile feedback for mobile touchscreens. In Proceedings of the 26th Annual SIGCHI Conference on Human Factors in Computing Systems (CHI'08). ACM, New York, 1573--1582.
[19]
Hong, L., Chi, E. H., and Card, S. K. 2005. Annotating 3D electronic books. In Extended Abstracts of the Conference on Human Factors in Computing Systems, ACM, 463--1466.
[20]
Kameyama, S. and Ishibashi, Y. 2006. Perception of synchronization errors in haptic and visual communications. Proc. SPIE 6391, 639108. http://dx.doi.org/
[21]
Lemmens, P., Crompvoets, F., Brokken, D., van den Eerenbeemd, J., and de Vries, G.-J. 2009. A body-conforming tactile jacket to enrich movie viewing. In Proceedings of the 3rd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. 7--12. http://dx.doi.org/10.1109/WHC.2009.4810832.
[22]
Likert, R. 1932. A technique for the measurement of attitudes. Arch. Psych. 140, 1--55.
[23]
Mangent, A. 2008. Hypertext fiction reading: Haptics and immersion. Res. Read. 31, 4, 404--419. http://dx.doi.org/10.1111/j.1467-9817.2008.00380.x.
[24]
Park, A., Lee, K. J., and Casalegno, F. 2010. The three dimensions of book evolution in ubiquitous computing age: Digitalization, augmentation, and hypermediation. In Proceedings of the IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing. 374--378. http://dx.doi.org/10.1109/SUTC.2010.50.
[25]
Park, J. and Woo, W. 2009. Multi-layer based authoring tool for Digilog book. In Proceedings of the International Conference on Entertainment Computing (ICEC'09). S. Natkin and J. Dupire, Eds., Lecture Notes in Computer Science, vol. 5709, Springer, 234--239. http://dx.doi.org/10.1007/978-3-642-04052-8n 26 10.1007/978-3-642-04052-826.
[26]
Park, T., Hwang, J., and Hwang, W. 2009. Facilitating the design of vibration for handheld devices. In Human- Computer Interaction. Novel Interaction Methods and Techniques, J. Jacko, Ed., Lecture Notes in Computer Science, vol. 5611, Springer, 496--502.
[27]
Rahal, L., Cha, J., and El Saddik, A. 2009. Continuous tactile perception for vibrotactile displays. In Proceedings of the IEEE International Workshop on Robotic and Sensors Environments (ROSE'09). 86--91. http://dx.doi.org/10.1109/ROSE.2009.5355986.
[28]
Rahman, A. S. M. M., Alam, K. M., and El Saddik, A. 2011. Augmented he-book: A multimedia-based extension to support immersive reading experience. In Proceedings of the 2nd International Conference on Autonomous and Intelligent Systems (AIS'11). Springer, 321--330. http://dl.acm.org/citation.cfm?id=2026956.2026992.
[29]
Rahman, A. S. M. M., Hossain, S. K. A., and El Saddik, A. 2010. Bridging the gap between virtual and real world by bringing an interpersonal haptic communication system in second life. In Proceedings of the IEEE International Symposium on Multimedia.
[30]
Schilit, B. N., Price, M. N., Golovchinsky, G., Tanaka, K., and Marshall, C. C. 1999. The reading appliance revolution. Computer 32, 1, 65--73. http://dx.doi.org/10.1109/2.738306.
[31]
Taketa, N., Hayashi, K., Kato, H., and Noshida, S. 2007. Virtual pop-up book based on augmented reality. In Human Interface and the Management of Information. Interacting in Information Environments, M. Smith and G. Salvendy, Eds., Lecture Notes in Computer Science, vol. 4558, Springer, 475--484. http://dx.doi.org/10.1007/978-3-540-73354-6n52 10.1007/978-3-540-73354-652.
[32]
Toutanova, K. and Manning, C. D. 2000. Enriching the knowledge sources used in a maximum entropy part-of-speech tagger. In Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora. 63--70.
[33]
Welch, G., State, A., Ilie, A., Low, K.-L., Lastra, A., Cairns, B., Towles, H., Fuchs, H., Yang, R., Becker, S., Russo, D., Funaro, J., and van Dam, A. 2005. Immersive electronic books for surgical training. IEEE Multimedia 12, 3, 22--35. http://dx.doi.org/10.1109/MMUL.2005.48.

Cited By

View all
  • (2024)Immersive Multimedia Service Caching in Edge Cloud with Renewable EnergyACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364381820:6(1-23)Online publication date: 8-Mar-2024
  • (2024)Evolution of Mobile Computing: From Text-Based to Visual-Based Interactions2024 International Conference on Optimization Computing and Wireless Communication (ICOCWC)10.1109/ICOCWC60930.2024.10470729(1-6)Online publication date: 29-Jan-2024
  • (2023) Neues Lesen . Unikalität und Multisensorik als Strategien der Literaturvermittlung Journal of Literary Theory10.1515/jlt-2023-200717:1(143-166)Online publication date: 20-Jul-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications
ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 9, Issue 4
August 2013
168 pages
ISSN:1551-6857
EISSN:1551-6865
DOI:10.1145/2501643
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 August 2013
Accepted: 01 February 2013
Revised: 01 August 2012
Received: 01 February 2012
Published in TOMM Volume 9, Issue 4

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Haptic modality
  2. edutainment system
  3. mobile learning
  4. multimodal interface

Qualifiers

  • Research-article
  • Research
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)54
  • Downloads (Last 6 weeks)5
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Immersive Multimedia Service Caching in Edge Cloud with Renewable EnergyACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364381820:6(1-23)Online publication date: 8-Mar-2024
  • (2024)Evolution of Mobile Computing: From Text-Based to Visual-Based Interactions2024 International Conference on Optimization Computing and Wireless Communication (ICOCWC)10.1109/ICOCWC60930.2024.10470729(1-6)Online publication date: 29-Jan-2024
  • (2023) Neues Lesen . Unikalität und Multisensorik als Strategien der Literaturvermittlung Journal of Literary Theory10.1515/jlt-2023-200717:1(143-166)Online publication date: 20-Jul-2023
  • (2022)Guidelines for evaluating wearables’ quality of experience in a mulsemedia contextMultimedia Tools and Applications10.1007/s11042-022-12766-781:30(43283-43314)Online publication date: 1-Dec-2022
  • (2021)Engaging Children’s Reading with Reflective Augmented RealityLibri10.1515/libri-2019-014771:1(51-63)Online publication date: 12-Feb-2021
  • (2021)Exploring Food based Interactive, Multi-Sensory, and Tangible Storytelling ExperiencesProceedings of the 2021 ACM Designing Interactive Systems Conference10.1145/3461778.3462006(651-665)Online publication date: 28-Jun-2021
  • (2021)Using Multisensory Content to Impact the Quality of Experience of Reading Digital BooksACM Transactions on Multimedia Computing, Communications, and Applications10.1145/345867617:4(1-18)Online publication date: 12-Nov-2021
  • (2020)Automatic XML creation for Multisensorial Books2020 XV Conferencia Latinoamericana de Tecnologias de Aprendizaje (LACLO)10.1109/LACLO50806.2020.9381139(1-6)Online publication date: 19-Oct-2020
  • (2019)Preliminary Study of Haptic Media for Future Digital TextbooksProceedings of the 13th International Conference on Ubiquitous Information Management and Communication (IMCOM) 201910.1007/978-3-030-19063-7_41(529-538)Online publication date: 23-May-2019
  • (2019)Improving Multisensory User Experience Through Olfactory StimuliEmotional Engineering, Vol.710.1007/978-3-030-02209-9_13(201-231)Online publication date: 3-Feb-2019
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media