skip to main content
10.1145/3126594.3126654acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article
Public Access

iSoft: A Customizable Soft Sensor with Real-time Continuous Contact and Stretching Sensing

Published: 20 October 2017 Publication History

Abstract

We present iSoft, a single volume soft sensor capable of sensing real-time continuous contact and unidirectional stretching. We propose a low-cost and an easy way to fabricate such piezoresistive elastomer-based soft sensors for instant interactions. We employ an electrical impedance tomography (EIT) technique to estimate changes of resistance distribution on the sensor caused by fingertip contact. To compensate for the rebound elasticity of the elastomer and achieve real-time continuous contact sensing, we apply a dynamic baseline update for EIT. The baseline updates are triggered by fingertip contact and movement detections. Further, we support unidirectional stretching sensing using a model-based approach which works separately with continuous contact sensing. We also provide a software toolkit for users to design and deploy personalized interfaces with customized sensors. Through a series of experiments and evaluations, we validate the performance of contact and stretching sensing. Through example applications, we show the variety of examples enabled by iSoft.

Supplementary Material

suppl.mov (uistf4355-file3.mp4)
Supplemental video

References

[1]
Andy Adler and Robert Guardo. 1996. Electrical impedance tomography: regularized imaging and contrast detection. IEEE Transactions on Medical Imaging 15, 2 (Apr 1996), 170--179.
[2]
Alamusi, Ning Hu, Hisao Fukunaga, Satoshi Atobe, Yaolu Liu, and Jinhua Li. 2011. Piezoresistive Strain Sensors Made from Carbon Nanotubes Based Polymer Nanocomposites. Sensors 11, 11 (2011), 10691--10723. http://www.mdpi.com/1424--8220/11/11/10691
[3]
Hassan Alirezaei, Akihiko Nagakubo, and Yasuo Kuniyoshi. 2007. A highly stretchable tactile distribution sensor for smooth surfaced humanoids. In 2007 7th IEEE-RAS International Conference on Humanoid Robots. 167--173.
[4]
Chin-yu Chien, Rong-Hao Liang, Long-Fei Lin, Liwei Chan, and Bing-Yu Chen. 2015. FlexiBend: Enabling Interactivity of Multi-Part, Deformable Fabrications Using Single Shape-Sensing Strip. In Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology (UIST '15). ACM, NY, NY, USA, 659--663.
[5]
Alex Chortos, Jia Liu, and Zhenan Bao. 2016. Pursuing prosthetic electronic skin. Nature Materials 15, 9 (2016), 937--950.
[6]
Artem Dementyev, Hsin-Liu (Cindy) Kao, and Joseph A. Paradiso. 2015. SensorTape: Modular and Programmable 3D-Aware Dense Sensor Network on a Tape. In Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology (UIST '15). ACM, NY, NY, USA, 649--658.
[7]
Artem Dementyev and Joseph A. Paradiso. 2014. WristFlex: Low-power Gesture Input with Wrist-worn Pressure Sensors. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, NY, NY, USA, 161--166.
[8]
Paul H. Dietz, Benjamin Eidelson, Jonathan Westhues, and Steven Bathiche. 2009. A Practical Pressure Sensitive Computer Keyboard. In Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology (UIST '09). ACM, NY, NY, USA, 55--58.
[9]
Peter M. Edic, Gary J. Saulnier, Jonathan C. Newell, and David Isaacson. 1995. A real-time electrical impedance tomograph. IEEE Transactions on Biomedical Engineering 42, 9 (Sept 1995), 849--859.
[10]
Marcello Ferro, Giovanni Pioggia, Alessandro Tognetti, Nicola Carbonaro, and Danilo De Rossi. 2009. A Sensing Seat for Human Authentication. IEEE Transactions on Information Forensics and Security 4, 3 (Sept 2009), 451--459.
[11]
Sean Follmer, Daniel Leithinger, Alex Olwal, Nadia Cheng, and Hiroshi Ishii. 2012. Jamming User Interfaces: Programmable Particle Stiffness and Sensing for Malleable and Shape-changing Devices. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST '12). ACM, NY, NY, USA, 519--528.
[12]
Nan-Wei Gong, Jürgen Steimle, Simon Olberding, Steve Hodges, Nicholas Edward Gillian, Yoshihiro Kawahara, and Joseph A. Paradiso. 2014. PrintSense: A Versatile Sensing Technique to Support Multimodal Flexible Surface Interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, NY, NY, USA, 1407--1410.
[13]
LJ Gray and E Lutz. 1990. On the treatment of corners in the boundary element method. J. Comput. Appl. Math. 32, 3 (1990), 369--386.
[14]
Jaehyun Han, Jiseong Gu, and Geehyuk Lee. 2014. Trampoline: A Double-sided Elastic Touch Device for Creating Reliefs. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, NY, NY, USA, 383--388.
[15]
PJRC Inc. 2017. Product Detail. (2017). Retrieved April 1, 2017 from http://www.pjrc.com/teensy/index.html.
[16]
Inpil Kang, Mark J Schulz, Jay H Kim, Vesselin Shanov, and Donglu Shi. 2006. A carbon nanotube strain sensor for structural health monitoring. Smart Materials and Structures 15, 3 (2006), 737.
[17]
Thorsten Karrer, Moritz Wittenhagen, Leonhard Lichtschlag, Florian Heller, and Jan Borchers. 2011. Pinstripe: Eyes-free Continuous Input on Interactive Clothing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, NY, NY, USA, 1313--1322.
[18]
Pasi Kauppinen, Jari Hyttinen, and Jaakko Malmivuo. 2006. Sensitivity distribution visualizations of impedance tomography measurement strategies. International Journal of Bioelectromagnetism 8, 1 (2006), 1--9.
[19]
Yoshihiro Kawahara, Steve Hodges, Benjamin S. Cook, Cheng Zhang, and Gregory D. Abowd. 2013. Instant Inkjet Circuits: Lab-based Inkjet Printing to Support Rapid Prototyping of UbiComp Devices. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '13). ACM, NY, NY, USA, 363--372.
[20]
Tsz-Ho Kwok, Charlie CL Wang, Dongping Deng, Yunbo Zhang, and Yong Chen. 2015. Four-dimensional printing for freeform surfaces: Design optimization of origami and kirigami structures. Journal of Mechanical Design 137, 11 (2015), 111413.
[21]
David Ledo, Fraser Anderson, Ryan Schmidt, Lora Oehlberg, Saul Greenberg, and Tovi Grossman. 2017. Pineal: Bringing Passive Objects to Life with Embedded Mobile Devices. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, NY, NY, USA, 2583--2593.
[22]
Joanne Leong, Patrick Parzer, Florian Perteneder, Teo Babic, Christian Rendl, Anita Vogl, Hubert Egger, Alex Olwal, and Michael Haller. 2016. proCover: Sensory Augmentation of Prosthetic Limbs Using Smart Textile Covers. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, NY, NY, USA, 335--346.
[23]
Jhe-Wei Lin, Chiuan Wang, Yi Yao Huang, Kuan-Ting Chou, Hsuan-Yu Chen, Wei-Luan Tseng, and Mike Y. Chen. 2015. BackHand: Sensing Hand Gestures via Back of the Hand. In Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology (UIST '15). ACM, NY, NY, USA, 557--564.
[24]
Darren J. Lipomi, Michael Vosgueritchian, Benjamin C-K. Tee, Sondra L. Hellstrom, Jennifer A. Lee, Courtney H. Fox, and Zhenan Bao. 2011. Skin-like pressure and strain sensors based on transparent elastic fllms of carbon nanotubes. Nature nanotechnology 6, 12 (2011), 788--792.
[25]
F. Lorussi, Enzo Pasquale Scilingo, M. Tesconi, A. Tognetti, and D. De Rossi. 2005. Strain sensing fabric for hand posture and gesture monitoring. IEEE Transactions on Information Technology in Biomedicine 9, 3 (Sept 2005), 372--381.
[26]
Sachi Mizobuchi, Shinya Terasaki, Turo Keski-Jaskari, Jari Nousiainen, Matti Ryynanen, and Miika Silfverberg. 2005. Making an Impression: Force-controlled Pen Input for Handheld Devices. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05). ACM, NY, NY, USA, 1661--1664.
[27]
Vinh P. Nguyen, Sang Ho Yoon, Ansh Verma, and Karthik Ramani. 2014. BendID: Flexible Interface for Localized Deformation Recognition. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '14). ACM, NY, NY, USA, 553--557.
[28]
Simon Olberding, Nan-Wei Gong, John Tiab, Joseph A. Paradiso, and Jürgen Steimle. 2013. A Cuttable Multi-touch Sensor. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). ACM, NY, NY, USA, 245--254.
[29]
Steve Park, Michael Vosguerichian, and Zhenan Bao. 2013. A review of fabrication and applications of carbon nanotube fllm-based flexible electronics. Nanoscale 5, 5 (2013), 1727--1752.
[30]
Yong-Lae Park, Bor-Rong Chen, and Robert J. Wood. 2012. Design and Fabrication of Soft Artificial Skin Using Embedded Microchannels and Liquid Conductors. IEEE Sensors Journal 12, 8 (2012), 2711--2718.
[31]
Pedro Piacenza, Yuchen Xiao, Steve Park, Ioannis Kymissis, and Matei Ciocarlie. 2016. Contact localization through spatially overlapping piezoresistive signals. In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 195--201.
[32]
Ivan Poupyrev, Nan-Wei Gong, Shiho Fukuhara, Mustafa Emre Karagozler, Carsten Schwesig, and Karen E. Robinson. 2016. Project Jacquard: Interactive Digital Textiles at Scale. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 4216--4227.
[33]
Ganna Pugach, Alexandre Pitti, and Philippe Gaussier. 2015. Neural learning of the topographic tactile sensory information of an artificial skin through a self-organizing map. Advanced Robotics 29, 21 (2015), 1393--1409.
[34]
Raf Ramakers, Fraser Anderson, Tovi Grossman, and George Fitzmaurice. 2016. RetroFab: A Design Tool for Retrofitting Physical Interfaces Using Actuators, Sensors and 3D Printing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 409--419.
[35]
Gonzalo Ramos, Matthew Boulos, and Ravin Balakrishnan. 2004. Pressure Widgets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04). ACM, NY, NY, USA, 487--494.
[36]
Christian Rendl, David Kim, Patrick Parzer, Sean Fanello, Martin Zirkl, Gregor Scheipl, Michael Haller, and Shahram Izadi. 2016. FlexCase: Enhancing Mobile Interaction with a Flexible Sensing and Display Cover. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 5138--5150.
[37]
Peter Roberts, Dana D. Damian, Wanliang Shan, Tong Lu, and Carmel Majidi. 2013. Soft-matter capacitive sensor for measuring shear and pressure deformation. In 2013 IEEE International Conference on Robotics and Automation. 3529--3534.
[38]
Ilya Rosenberg and Ken Perlin. 2009. The UnMousePad: An Interpolating Multi-touch Force-sensing Input Pad. In ACM SIGGRAPH 2009 Papers (SIGGRAPH '09). ACM, NY, NY, USA, Article 65, 9 pages.
[39]
Valkyrie Savage, Sean Follmer, Jingyi Li, and Björn Hartmann. 2015. Makers' Marks: Physical Markup for Designing and Fabricating Functional Objects. In Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology (UIST '15). ACM, NY, NY, USA, 103--108.
[40]
Martin Schmitz, Mohammadreza Khalilbeigi, Matthias Balwierz, Roman Lissermann, Max Mühlhäuser, and Jürgen Steimle. 2015. Capricate: A Fabrication Pipeline to Design and 3D Print Capacitive Touch Sensors for Interactive Objects. In Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology (UIST '15). ACM, NY, NY, USA, 253--258.
[41]
David Silvera-Tawil, David Rye, Manuchehr Soleimani, and Mari Velonaki. 2015. Electrical Impedance Tomography for Artificial Sensitive Robotic Skin: A Review. IEEE Sensors Journal 15, 4 (April 2015), 2001--2016.
[42]
David Silvera-Tawil, David Rye, and Mari Velonaki. 2012. Interpretation of the modality of touch on an artificial arm covered with an EIT-based sensitive skin. The International Journal of Robotics Research 31, 13 (2012), 1627--1641.
[43]
Ronit Slyper, Ivan Poupyrev, and Jessica Hodgins. 2011. Sensing Through Structure: Designing Soft Silicone Sensors. In Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '11). ACM, NY, NY, USA, 213--220.
[44]
Yuta Sugiura, Masahiko Inami, and Takeo Igarashi. 2012. A Thin Stretchable Interface for Tangential Force Measurement. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST '12). ACM, NY, NY, USA, 529--536.
[45]
Raw Material Suppliers. 2017. Print-On Silicone Ink. (2017). Retrieved April 1, 2017 from http: //rawmaterialsuppliers.com/print-on-silicone-rubber.
[46]
Tyler N Tallman, Sila Gungor, K W Wang, and Charles E Bakis. 2015. Damage detection via electrical impedance tomography in glass fiber/epoxy laminates with carbon black filler. Structural Health Monitoring 14, 1 (2015), 100--109.
[47]
Karen Vanderloock, Vero Vanden Abeele, Johan A.K. Suykens, and Luc Geurts. 2013. The Skweezee System: Enabling the Design and the Programming of Squeeze Interactions. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). ACM, NY, NY, USA, 521--530.
[48]
M Vauhkonen, W R B Lionheart, L M Heikkinen, P J Vauhkonen, and J P Kaipio. 2001. A MATLAB package for the EIDORS project to reconstruct two-dimensional EIT images. Physiological Measurement 22, 1 (2001), 107. http://stacks.iop.org/0967--3334/22/i=1/a=314
[49]
Wacker. 2017. ELASTOSIL LR 3162 A/B. (2017). Retrieved April 1, 2017 from http://www.wacker.com.
[50]
Martin Weigel, Tong Lu, Gilles Bailly, Antti Oulasvirta, Carmel Majidi, and Jürgen Steimle. 2015. iSkin: Flexible, Stretchable and Visually Customizable On-Body Touch Sensors for Mobile Computing. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 2991--3000.
[51]
Michael Wessely, Theophanis Tsandilas, and Wendy E. Mackay. 2016. Stretchis: Fabricating Highly Stretchable User Interfaces. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, NY, NY, USA, 697--704.
[52]
Takeo Yamada, Yuhei Hayamizu, Yuki Yamamoto, Yoshiki Yomogida, Ali Izadi-Najafabadi, Don N. Futaba, and Kenji Hata. 2011. A stretchable carbon nanotube strain sensor for human-motion detection. Nature Nano 6, 5 (2011), 296--301.
[53]
K. Yamaguchi, J. J. C. Busfield, and A. G. Thomas. 2003. Electrical and mechanical behavior of filled elastomers. I. The effect of strain. Journal of Polymer Science Part B: Polymer Physics 41, 17 (2003), 2079--2089.
[54]
Lining Yao, Ryuma Niiyama, Jifei Ou, Sean Follmer, Clark Della Silva, and Hiroshi Ishii. 2013. PneUI: Pneumatically Actuated Soft Composite Materials for Shape Changing Interfaces. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). ACM, NY, NY, USA, 13--22.
[55]
Sang Ho Yoon, Ke Huo, Vinh P. Nguyen, and Karthik Ramani. 2015. TIMMi: Finger-worn Textile Input Device with Multimodal Sensing in Mobile Interaction. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '15). ACM, NY, NY, USA, 269--272.
[56]
Sang Ho Yoon, Yunbo Zhang, Ke Huo, and Karthik Ramani. 2016. TRing: Instant and Customizable Interactions with Objects Using an Embedded Magnet and a Finger-Worn Device. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, NY, NY, USA, 169--181.
[57]
Yang Zhang, Gierad Laput, and Chris Harrison. 2017. Electrick: Low-Cost Touch Sensing Using Electric Field Tomography. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, NY, NY, USA, 1--14.
[58]
Yang Zhang, Robert Xiao, and Chris Harrison. 2016. Advancing Hand Gesture Recognition with High Resolution Electrical Impedance Tomography. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, NY, NY, USA, 843--850.

Cited By

View all
  • (2024)Multi-Layer Electrical Impedance Tomography Based Soft Tactile Skins2024 IEEE 7th International Conference on Soft Robotics (RoboSoft)10.1109/RoboSoft60065.2024.10521981(1101-1106)Online publication date: 14-Apr-2024
  • (2024)The design and fabrication of a wearable lattice-patterned 3D sensing skinSensors and Actuators A: Physical10.1016/j.sna.2024.115143369(115143)Online publication date: Apr-2024
  • (2023)LattiSense: A 3D-Printable Resistive Deformation Sensor with Lattice StructuresProceedings of the 8th ACM Symposium on Computational Fabrication10.1145/3623263.3623361(1-14)Online publication date: 8-Oct-2023
  • Show More Cited By

Index Terms

  1. iSoft: A Customizable Soft Sensor with Real-time Continuous Contact and Stretching Sensing

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    UIST '17: Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology
    October 2017
    870 pages
    ISBN:9781450349819
    DOI:10.1145/3126594
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 20 October 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. customization
    2. input device
    3. sensing technique
    4. soft sensor
    5. wearables

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    UIST '17

    Acceptance Rates

    UIST '17 Paper Acceptance Rate 73 of 324 submissions, 23%;
    Overall Acceptance Rate 561 of 2,567 submissions, 22%

    Upcoming Conference

    UIST '25
    The 38th Annual ACM Symposium on User Interface Software and Technology
    September 28 - October 1, 2025
    Busan , Republic of Korea

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)250
    • Downloads (Last 6 weeks)41
    Reflects downloads up to 16 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Multi-Layer Electrical Impedance Tomography Based Soft Tactile Skins2024 IEEE 7th International Conference on Soft Robotics (RoboSoft)10.1109/RoboSoft60065.2024.10521981(1101-1106)Online publication date: 14-Apr-2024
    • (2024)The design and fabrication of a wearable lattice-patterned 3D sensing skinSensors and Actuators A: Physical10.1016/j.sna.2024.115143369(115143)Online publication date: Apr-2024
    • (2023)LattiSense: A 3D-Printable Resistive Deformation Sensor with Lattice StructuresProceedings of the 8th ACM Symposium on Computational Fabrication10.1145/3623263.3623361(1-14)Online publication date: 8-Oct-2023
    • (2023)A Neural Network-based Low-cost Soft Sensor for Touch Recognition and Deformation CaptureProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3595963(889-903)Online publication date: 10-Jul-2023
    • (2022)Using Novel Multi-Frequency Analysis Methods to Retrieve Material and Temperature Information in Tactile Sensing AreasSensors10.3390/s2222887622:22(8876)Online publication date: 17-Nov-2022
    • (2022)A Bionic Testbed for Cardiac Ablation ToolsInternational Journal of Molecular Sciences10.3390/ijms23221444423:22(14444)Online publication date: 21-Nov-2022
    • (2022)StretchARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35503056:3(1-26)Online publication date: 7-Sep-2022
    • (2022)SensurfacesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35346166:2(1-19)Online publication date: 7-Jul-2022
    • (2022)Prototyping Soft Devices with Interactive BioplasticsProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545623(1-16)Online publication date: 29-Oct-2022
    • (2022)Squeezy-Feely: Investigating Lateral Thumb-Index Pinching as an Input ModalityProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501981(1-15)Online publication date: 29-Apr-2022
    • Show More Cited By

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media