research-article

ExtEdge: Haptic Augmentation of Visual Experiences of a Smartphone by Electro-Tactile Sensation Through the Edges

Authors:

Takayuki Kameoka,

Hiroyuki KajimotoAuthors Info & Claims

SUI '23: Proceedings of the 2023 ACM Symposium on Spatial User Interaction

Article No.: 7, Pages 1 - 8

https://doi.org/10.1145/3607822.3614543

Published: 13 October 2023 Publication History

Abstract

We present ExtEdge, a design and method for augmenting the visual experiences of a smartphone using electro-tactile stimulation. We aim to explore new interactions through tactile augmentation of visual information, such as virtual characters, at the edges of the screen. ExtEdge presents distributed tactile sensations through electrical stimulations from electrode arrays mounted on both edges of the smartphone. This distributed tactile stimulus fits the image at the edges of the screen and provides a more realistic presentation. We first investigated two-point discrimination thresholds in the contact areas of the grasp—middle finger, ring finger, thumb, and thenar eminence—as a performance evaluation of the distributed tactile presentation and found that our design was adequate. We then conducted a user study with some application scenarios and confirmed that ExtEdge enriches visual information.

Supplemental Material

MP4 File

Appendix

Download
120.83 MB

MP4 File

Video

Download
120.83 MB

References

[1]

Jason Alexander, Mark T. Marshall, and Sriram Subramanian. 2011. Adding Haptic Feedback to Mobile Tv. In CHI ’11 Extended Abstracts on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI EA ’11). Association for Computing Machinery, New York, NY, USA, 1975–1980. https://doi.org/10.1145/1979742.1979899

Digital Library

[2]

Cagatay Basdogan, Frederic Giraud, Vincent Levesque, and Seungmoon Choi. 2020. A Review of Surface Haptics: Enabling Tactile Effects on Touch Surfaces. IEEE Transactions on Haptics 13, 3 (2020), 450–470. https://doi.org/10.1109/TOH.2020.2990712

[3]

Olivier Bau, Ivan Poupyrev, Ali Israr, and Chris Harrison. 2010. TeslaTouch: Electrovibration for Touch Surfaces. In Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology (New York, New York, USA) (UIST ’10). Association for Computing Machinery, New York, NY, USA, 283–292. https://doi.org/10.1145/1866029.1866074

Digital Library

[4]

D.J. Beebe, C.M. Hymel, K.A. Kaczmarek, and M.E. Tyler. 1995. A polyimide-on-silicon electrostatic fingertip tactile display. In Proceedings of 17th International Conference of the Engineering in Medicine and Biology Society, Vol. 2. 1545–1546 vol.2. https://doi.org/10.1109/IEMBS.1995.579819

[5]

Melisande Biet, Frederic Giraud, and Betty Lemaire-Semail. 2007. Squeeze film effect for the design of an ultrasonic tactile plate. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control 54, 12 (2007), 2678–2688. https://doi.org/10.1109/TUFFC.2007.596

[6]

Stephen Brewster, Faraz Chohan, and Lorna Brown. 2007. Tactile Feedback for Mobile Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’07). Association for Computing Machinery, New York, NY, USA, 159–162. https://doi.org/10.1145/1240624.1240649

Digital Library

[7]

Masaaki Fukumoto and Toshiaki Sugimura. 2001. Active Click: Tactile Feedback for Touch Panels. In CHI ’01 Extended Abstracts on Human Factors in Computing Systems (Seattle, Washington) (CHI EA ’01). Association for Computing Machinery, New York, NY, USA, 121–122. https://doi.org/10.1145/634067.634141

Digital Library

[8]

Shogo Fukushima and Hiroyuki Kajimoto. 2011. Palm Touch Panel: Providing Touch Sensation through the Device. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (Kobe, Japan) (ITS ’11). Association for Computing Machinery, New York, NY, USA, 79–82. https://doi.org/10.1145/2076354.2076370

Digital Library

[9]

Daniel Groeger, Martin Feick, Anusha Withana, and Jürgen Steimle. 2019. Tactlets: Adding Tactile Feedback to 3D Objects Using Custom Printed Controls. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 923–936. https://doi.org/10.1145/3332165.3347937

Digital Library

[10]

Fabian Hemmert, Gesche Joost, André Knörig, and Reto Wettach. 2008. Dynamic Knobs: Shape Change as a Means of Interaction on a Mobile Phone. In CHI ’08 Extended Abstracts on Human Factors in Computing Systems (Florence, Italy) (CHI EA ’08). Association for Computing Machinery, New York, NY, USA, 2309–2314. https://doi.org/10.1145/1358628.1358675

Digital Library

[11]

Da-Yuan Huang, Ruizhen Guo, Jun Gong, Jingxian Wang, John Graham, De-Nian Yang, and Xing-Dong Yang. 2017. RetroShape: Leveraging Rear-Surface Shape Displays for 2.5D Interaction on Smartwatches. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (Québec City, QC, Canada) (UIST ’17). Association for Computing Machinery, New York, NY, USA, 539–551. https://doi.org/10.1145/3126594.3126610

Digital Library

[12]

Charles Hudin, José Lozada, and Vincent Hayward. 2015. Localized Tactile Feedback on a Transparent Surface through Time-Reversal Wave Focusing. IEEE Transactions on Haptics 8, 2 (2015), 188–198. https://doi.org/10.1109/TOH.2015.2411267

Digital Library

[13]

H. Ishizuka, K. Suzuki, K. Terao, H. Takao, and F. Shimokawa. 2017. Development of high resolution electrostatic tactile display. In 2017 International Conference on Electronics Packaging (ICEP). 484–486. https://doi.org/10.23919/ICEP.2017.7939427

[14]

Sungjune Jang, Lawrence H. Kim, Kesler Tanner, Hiroshi Ishii, and Sean Follmer. 2016. Haptic Edge Display for Mobile Tactile Interaction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 3706–3716. https://doi.org/10.1145/2858036.2858264

Digital Library

[15]

R S Johansson and A B Vallbo. 1979. Tactile sensibility in the human hand: relative and absolute densities of four types of mechanoreceptive units in glabrous skin.The Journal of Physiology 286, 1 (1979), 283–300. https://doi.org/10.1113/jphysiol.1979.sp012619

[16]

Brett R. Jones, Hrvoje Benko, Eyal Ofek, and Andrew D. Wilson. 2013. IllumiRoom: Peripheral Projected Illusions for Interactive Experiences. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 869–878. https://doi.org/10.1145/2470654.2466112

Digital Library

[17]

Hiroyuki Kajimoto. 2012. Electrotactile Display with Real-Time Impedance Feedback Using Pulse Width Modulation. IEEE Transactions on Haptics 5, 2 (2012), 184–188. https://doi.org/10.1109/TOH.2011.39

Digital Library

[18]

Hiroyuki Kajimoto. 2012. Skeletouch: Transparent Electro-Tactile Display for Mobile Surfaces. In SIGGRAPH Asia 2012 Emerging Technologies (Singapore, Singapore) (SA ’12). Association for Computing Machinery, New York, NY, USA, 1–3. https://doi.org/10.1145/2407707.2407728

Digital Library

[19]

Hiroyuki Kajimoto. 2021. Electro-Tactile Display Kit for Fingertip. In 2021 IEEE World Haptics Conference (WHC). 587–587. https://doi.org/10.1109/WHC49131.2021.9517192

[20]

H. Kajimoto, N. Kawakami, S. Tachi, and M. Inami. 2004. SmartTouch: electric skin to touch the untouchable. IEEE Computer Graphics and Applications 24, 1 (2004), 36–43. https://doi.org/10.1109/MCG.2004.1255807

Digital Library

[21]

Sugarragchaa Khurelbaatar, Yuriko Nakai, Ryuta Okazaki, Vibol Yem, and Hiroyuki Kajimoto. 2016. Tactile Presentation to the Back of a Smartphone with Simultaneous Screen Operation. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 3717–3721. https://doi.org/10.1145/2858036.2858099

Digital Library

[22]

Naoki Kimura and Jun Rekimoto. 2018. ExtVision: Augmentation of Visual Experiences with Generation of Context Images for a Peripheral Vision Using Deep Neural Network. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/3173574.3174001

Digital Library

[23]

Dong-Soo Kwon and Seung-Chan Kim. 2008. Haptic interfaces for mobile devices: a survey of the state of the art. Recent Patents on Computer Science 1, 2 (2008), 84–92.

[24]

Jukka Linjama, Ville Mäkinen, and Petro Suvanto. 2009. Interface apparatus for touch input and tactile output communication. Finland Patent PCT/FI2009/050 416, Nov. 26, 2009.

[25]

Jens Maiero, David Eibich, Ernst Kruijff, André Hinkenjann, Wolfgang Stuerzlinger, Hrvoje Benko, and Gheorghita Ghinea. 2019. Back-of-Device Force Feedback Improves Touchscreen Interaction for Mobile Devices. IEEE Transactions on Haptics 12, 4 (2019), 483–496. https://doi.org/10.1109/TOH.2019.2911519

Digital Library

[26]

Lucie Pantera and Charles Hudin. 2020. Multitouch Vibrotactile Feedback on a Tactile Screen by the Inverse Filter Technique: Vibration Amplitude and Spatial Resolution. IEEE Transactions on Haptics 13, 3 (2020), 493–503. https://doi.org/10.1109/TOH.2020.2981307

[27]

Ivan Poupyrev and Shigeaki Maruyama. 2003. Tactile Interfaces for Small Touch Screens. In Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology (Vancouver, Canada) (UIST ’03). Association for Computing Machinery, New York, NY, USA, 217–220. https://doi.org/10.1145/964696.964721

Digital Library

[28]

Ivan Poupyrev, Shigeaki Maruyama, and Jun Rekimoto. 2002. Ambient Touch: Designing Tactile Interfaces for Handheld Devices. In Proceedings of the 15th Annual ACM Symposium on User Interface Software and Technology (Paris, France) (UIST ’02). Association for Computing Machinery, New York, NY, USA, 51–60. https://doi.org/10.1145/571985.571993

Digital Library

[29]

Moshe Solomonow, John Lyman, and Amos Freedy. 1978. Electrotactile two-point discrimination as a function of frequency, body site, laterality, and stimulation codes. Annals of Biomedical Engineering 6 (1978), 117–125. https://doi.org/10.1007/BF02584538

[30]

Joseph C. Stevens and Kenneth K. Choo. 1996. Spatial Acuity of the Body Surface over the Life Span. Somatosensory & Motor Research 13, 2 (1996), 153–166. https://doi.org/10.3109/08990229609051403

[31]

Evan Strasnick, Jackie Yang, Kesler Tanner, Alex Olwal, and Sean Follmer. 2017. ShiftIO: Reconfigurable Tactile Elements for Dynamic Affordances and Mobile Interaction. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 5075–5086. https://doi.org/10.1145/3025453.3025988

Digital Library

[32]

T. Watanabe and S. Fukui. 1995. A method for controlling tactile sensation of surface roughness using ultrasonic vibration. In Proceedings of 1995 IEEE International Conference on Robotics and Automation, Vol. 1. 1134–1139 vol.1. https://doi.org/10.1109/ROBOT.1995.525433

[33]

E.H. Weber. 1978. The Sense of Touch. New York: Academic Press (1978).

[34]

Laura Winfield, John Glassmire, J. Edward Colgate, and Michael Peshkin. 2007. T-PaD: Tactile Pattern Display through Variable Friction Reduction. In Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC’07). 421–426. https://doi.org/10.1109/WHC.2007.105

Digital Library

[35]

Koji Yatani and Khai Nhut Truong. 2009. SemFeel: A User Interface with Semantic Tactile Feedback for Mobile Touch-Screen Devices. In Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology (Victoria, BC, Canada) (UIST ’09). Association for Computing Machinery, New York, NY, USA, 111–120. https://doi.org/10.1145/1622176.1622198

Digital Library

[36]

Zhuoming Zhang, Jessalyn Alvina, Françoise Détienne, and Eric Lecolinet. 2022. Pulling, Pressing, and Sensing with In-Flat: Transparent Touch Overlay for Smartphones. In Proceedings of the 2022 International Conference on Advanced Visual Interfaces (Frascati, Rome, Italy) (AVI 2022). Association for Computing Machinery, New York, NY, USA, Article 12, 9 pages. https://doi.org/10.1145/3531073.3531111

Digital Library

Cited By

Perera PMarasinghe HTakami TKajimoto HWithana AKostakos VKay JHoang T(2024)Integrating Force Sensing with Electro-Tactile Feedback in 3D Printed Haptic InterfacesProceedings of the 2024 ACM International Symposium on Wearable Computers10.1145/3675095.3676612(48-54)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675095.3676612

Index Terms

ExtEdge: Haptic Augmentation of Visual Experiences of a Smartphone by Electro-Tactile Sensation Through the Edges
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
    2. Interaction devices

Recommendations

LivEdge: Haptic Live Stream Interaction on a Smartphone by Electro-Tactile Sensation Through the Edges
SIGGRAPH '23: ACM SIGGRAPH 2023 Emerging Technologies

We present LivEdge, a novel method for live stream interaction on smartphones utilizing electro-tactile sensation through the edges. Conventional interactions between users and a streamer on a smartphone are restricted to the streamer’s response through ...
Haptic Edge Display for Mobile Tactile Interaction
CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

Current mobile devices do not leverage the rich haptic channel of information that our hands can sense, and instead focus primarily on touch based graphical interfaces. Our goal is to enrich the user experience of these devices through bi-directional ...
Wearable haptic display to present virtual mass sensation
SIGGRAPH '07: ACM SIGGRAPH 2007 sketches

In recent times, there have been a number of computer user interface devices that have some haptic feedback functions, such as the DUALSHOCK controller [Sony Computer Entertainment, Inc. 1997] or the Wii Remote [Nintendo Co., Ltd. 2006]. However, their ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SUI '23: Proceedings of the 2023 ACM Symposium on Spatial User Interaction

October 2023

505 pages

ISBN:9798400702815

DOI:10.1145/3607822

Editors:
Tony Huang
UTS, Sydney, Australia
,
Misha Sra
UC Santa Barbara, USA
,
Ferran Argelaguet
INRIA, France
,
Pedro Lopes
University of Chicago, USA
,
Mayra Barrera Machuca
Dalhousie University, Canada

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Japan Society for the Promotion of Science
KDDI Foundation

Conference

SUI '23

Sponsor:

SUI '23: ACM Symposium on Spatial User Interaction

October 13 - 15, 2023

NSW, Sydney, Australia

Acceptance Rates

Overall Acceptance Rate 86 of 279 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
249
Total Downloads

Downloads (Last 12 months)136
Downloads (Last 6 weeks)17

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Perera PMarasinghe HTakami TKajimoto HWithana AKostakos VKay JHoang T(2024)Integrating Force Sensing with Electro-Tactile Feedback in 3D Printed Haptic InterfacesProceedings of the 2024 ACM International Symposium on Wearable Computers10.1145/3675095.3676612(48-54)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675095.3676612

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten