skip to main content
10.1145/3098279.3098545acmconferencesArticle/Chapter ViewAbstractPublication PagesmobilehciConference Proceedingsconference-collections
research-article

CapSoles: who is walking on what kind of floor?

Published: 04 September 2017 Publication History

Abstract

Foot interfaces, such as pressure-sensitive insoles, still yield unused potential such as for implicit interaction. In this paper, we introduce CapSoles, enabling smart insoles to implicitly identify who is walking on what kind of floor. Our insole prototype relies on capacitive sensing and is able to sense plantar pressure distribution underneath the foot, plus a capacitive ground coupling effect. By using machine-learning algorithms, we evaluated the identification of 13 users, while walking, with a confidence of ∼95% after a recognition delay of ∼1s. Once the user's gait is known, again we can discover irregularities in gait plus a varying ground coupling. While both effects in combination are usually unique for several ground surfaces, we demonstrate to distinguish six kinds of floors, which are sand, lawn, paving stone, carpet, linoleum, and tartan with an average accuracy of ∼82%. Moreover, we demonstrate the unique effects of wet and electrostatically charged surfaces.

References

[1]
Z. O. Abu-Faraj, G. F. Harris, J. H. Abler, and J. J. Wertsch. 1997. A Holter-type, microprocessor-based, rehabilitation instrument for acquisition and storage of plantar pressure data. In Journal of rehabilitation research and development (JRRD 1997), 34. 187--194.
[2]
J. Alexander, T. Han, W. Judd, P. Irani, and S. Subramanian. 2012. Putting your best foot forward: investigating real-world mappings for foot-based gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2012). ACM, 1229--1238.
[3]
P. J. Aranzulla, D. S. Muckle, and J. L. Cunningham. 1998. A portable monitoring system for measuring weight-bearing during tibial fracture healing. In Medical engineering & gphysics (MEP 1998), 20(7). 543--548.
[4]
T. Augsten, K. Kaefer, R. Meusel, C. Fetzer, D. Kanitz, T. Stoff, ... and P. Baudisch. 2010. Multitoe: high-precision interaction with back-projected floors based on high-resolution multi-touch input. In Proceedings of the 23nd annual ACM symposium on User interface software and technology (UIST 2010). ACM, 209--218.
[5]
S. Barrera, P. Romanos, S. Saito, H. Takahashi, and M. Nakajima. 2003. Real time detection interface for walking on CAVE. In Proceedings of Computer Graphics International (CGI 2003). IEEE, 105--110.
[6]
S. J. M. Bamberg, A. Y. Benbasat, S. M. Scarborough, D. E. Krebs, and J. A. Paradiso. 2008. Gait analysis using a shoe-integrated wireless sensor system. In IEEE Transactions on Information Technology in Biomedicine (ITBM 2008), 12(4). IEEE, 413--423.
[7]
M. Bächlin, J. Schumm, D. Roggen, and G. Töster. 2009. Quantifying gait similarity: User authentication and real-world challenge. In Advances in Biometrics (ICB 2009). Springer, 1040--1049.
[8]
C. Boeck, D. Barth, J. Schadow, J. Maute, J. Stock, and M. Lutz. 2014. Arbeitsschuh. Patent No. DE 102014212535 A1.
[9]
A. Bränzel, C. Holz, D. Hoffmann, D. Schmidt, M. Knaust, P. Lühne, ... and P. Baudisch. 2013. GravitySpace: tracking users and their poses in a smart room using a pressure-sensing floor. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2013). ACM, 725--734.
[10]
M. C. Carrozza, A. Persichetti, C. Laschi, F. Vecchi, R. Lazzarini, V. Tamburrelli, ... and P. Dario. 2005. A novel wearable interface for robotic hand prostheses. In Proceedings of 9th International Conference on Rehabilitation Robotics (ICORR 2005). IEEE, 109--112.
[11]
R. Caruana and D. Freitag. 1994. Greedy Attribute Selection. In Proceedings of the Eighth International Conference on Machine Learning (ICML 1994). 28--36.
[12]
J. Cheng, O. Amft, G. Bahle, and P. Lukowicz. 2013. Designing Sensitive Wearable Capacitive Sensors for Activity Recognition. In IEEE Sensors Journal, 13(10). IEEE, 3935--3947.
[13]
I. Choi and C. Ricci. 1997. Foot-mounted gesture detection and its application in virtual environments. In International Conference on Computational Cybernetics and Simulation (SMC 1997). IEEE, 4248--4253.
[14]
A. Crossan, S. Brewster, and A. Ng. 2010. Foot tapping for mobile interaction. In Proceedings of the 24th BCS Interaction Specialist Group Conference (BCS 2010). 418--422.
[15]
P. Das, R. N. Mitra, and N. Suppakitjarak. 2016. Smart shoes with cueing system and remote monitoring of Parkinson's patients: SCARP. In Proceedings of Foundations and Frontiers in Computer, Communication and Electrical Engineering (C2E2 2016). CRC Press, 269--274.
[16]
A. De Luca, M. Harbach, E. von Zezschwitz, M. E. Maurer, B. Slawik, H. Hussmann, and M. Smith. 2014. Now you see me, now you don't: protecting smartphone authentication from shoulder surfers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2014). ACM, 2937--2946.
[17]
C. Dibenedetto, M. A. Oleson, C. Roth, and M. C. Thompson. 2007. Intelligent footwear systems. U.S. Patent No. 7,188,439.
[18]
F-Scan. URL: http://www.tekscan.com/medical/system-fscan1.html
[19]
FootLogger. URL: http://footlogger.com
[20]
K. Fukahori, D. Sakamoto, and T. Igarashi. 2015. Exploring Subtle Foot Plantar-based Gestures with Sock-placed Pressure Sensors. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI 2015). ACM, 3019--3028.
[21]
A. Gioftsidou, P. Malliou, G. Pafis, A. Beneka, G. Godolias, and C. N. Maganaris. 2006. The effects of soccer training and timing of balance training on balance ability. In Journal of applied physiology, 96(6). 659--664.
[22]
T. S. Gross and R. P. Bunch. 1988. Measurement of discrete vertical in-shoe stress with piezoelectric transducers. In Journal of Biomedical Engineering, 10(3). 261--265.
[23]
A. Guo, R. Xiao, and C. Harrison. 2015. CapAuth: Identifying and Differentiating User Handprints on Commodity Capacitive Touchscreens. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces (ITS 2015). ACM, 59--62.
[24]
M. Hannula, A. Sakkinen, and A. Kylmanen. 2007. Development of emfi-sensor based pressure sensitive insole for gait analysis. In Medical Measurement and Applications (MEMEA 2007). IEEE, 1--3.
[25]
M. Haescher, D. J. C. Matthies, G. Bieber, and B. Urban. 2015. CapWalk: A Capacitive Recognition of Walking-Based Activities as a Wearable Assistive Technology. In Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRAE 2015). ACM, 35.
[26]
M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten. 2009. The WEKA data mining software: an update. ACM SIGKDD explorations news, 11(1). 10--18.
[27]
M. Harbach, E. von Zezschwitz, A. Fichtner, A. De Luca, and M. Smith. 2014. It's a hard lock life: A field study of smartphone (un) locking behavior and risk perception. In Symposium on usable privacy and security (SOUPS 2014). 213--230.
[28]
DeWayne L. Hays. 2006. Shoe insole and methods for identification. U.S. Patent Application No. 11/526,500.
[29]
E. M. Hennig, P. R. Cavanagh, H. T. Albert, and N. H. Macmillan, N. H. 1982. A piezoelectric method of measuring the vertical contact stress beneath the human foot. In Journal of Biomedical Engineering, 4(3). 213--222.
[30]
C. Holz, S. Buthpitiya, and M. Knaust. 2015. Bodyprint: Biometric User Identification on Mobile Devices Using the Capacitive Touchscreen to Scan Body Parts. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI 2015). ACM, 3011--3014.
[31]
Insole X. URL: http://sennostore.com/products
[32]
P. Iso-Ketola, T. Karinsalo, M. Myry, A. Halme, T. Salpavaara, J. Lekkala, and J. Vanhala. 2009. Development of a lower extremity rehabilitation aid utilizing an insole-integrated load sensor matrix and a sole-embedded measurement node. In Proceedings of the International Symposium on Wearable Computers (ISWC 2009). IEEE, 107--114.
[33]
S. Jalaliniya, J. Smith, M. Sousa, L. Büthe, and T. Pederson. 2013. Touch-less interaction with medical images using hand & foot gestures. In UbiComp 2013 Adjunct. ACM, 1265--1274.
[34]
H. S. Jeong, J. S. Yoo, M. H. Choi, S. J. Yoon, J. Y. Kim, and T. Y. Kim. 2009. Footwear for detecting and warning an electric leakage capable of preventing the electric shock accident. Patent No. KR20090036732A.
[35]
M. Kljajic and J. Krajnik. 1987. The use of ground reaction measuring shoes in gait evaluation. In Clinical Physics and Physiological Measurement, 8(2). 133.
[36]
A. Konttila, J. Kaartinen, J. Plomp, R. Toivonen, and L. Hopsu. 2011. Wireless weight measurement shoes. In 4th international conference on Biomedical Engineering and Informatics (BMEI 2011). IEEE, 1386--1390.
[37]
J. Leong, P. Parzer, F. Perteneder, T. Babic, C. Rendl, A. Vogl, ... and M. Haller. 2016. proCover: Sensory Augmentation of Prosthetic Limbs Using Smart Textile Covers. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST 2016). ACM, 335--346.
[38]
J. Little and J. Boyd. 1998. Recognizing people by their gait: the shape of motion. In Videre: Journal of Computer Vision Research, 1(2). 1--32.
[39]
Z. Lu, M. S. Lal Khan, and S. Ur Réhman. 2013. Hand and foot gesture interaction for handheld devices. In Proceedings of the 21st ACM international conference on Multimedia (MM 2013). ACM,621--624.
[40]
A. Martínez-Nova, J. C. Cuevas-García, J. Pascual-Huerta, and R. Sánchez-Rodríguez. 2007. BioFoot® in-shoe system: Normal values and assessment of the reliability and repeatability. In The Foot, 17(4). 190--196.
[41]
D. J. C. Matthies, F. Müller, C. Anthes, and D. Kranzlmüller. 2013. ShoeSoleSense: proof of concept for a wearable foot interface for virtual and real environments. In Proceedings of the 19th ACM Symposium on Virtual Reality Software and Technology (VRST 2013). ACM, 93--96.
[42]
D. J. C. Matthies, F. Müller, C. Anthes, and D. Kranzlmüller. 2014. ShoeSoleSense: demonstrating a wearable foot interface for locomotion in virtual environments. In CHI'14 Extended Abstracts on Human Factors in Computing Systems. ACM, 183--184.
[43]
J. Mäntyjärvi, M. Lindholm, E. Vildjiounaite, S. M. Mäkelä, and H. A. Ailisto. 2005. Identifying users of portable devices from gait pattern with accelerometers. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2005), (Vol. 2). IEEE, 973--976.
[44]
A. Meier, D. J. C. Matthies, B. Urban, and R. Wettach. 2015. Exploring vibrotactile feedback on the body and foot for the purpose of pedestrian navigation. In Proceedings of the 2nd international Workshop on Sensor-based Activity Recognition and Interaction (iWOAR 2015). ACM, 11.
[45]
S. Miyazaki and H. Iwakura. 1978. Foot-force measuring device for clinical assessment of pathological gait. In Medical and Biological Engineering and Computing (MBEC 1978), 16(4). 429--436.
[46]
S. Miyazaki, A. Ishida, H. Iwakura, K. Takino, T. Ohkawa, H. Tsubakimoto, H., and N. Hayashi. 1986. Portable limb-load monitor utilizing a thin capacitive transducer. In Journal of Biomedical Engineering, 8(1). 67--71.
[47]
Moticon. URL: http://www.moticon.de
[48]
M. J.-D. Otis and B.-A. J. Menelas. 2014. Method to determine physical properties of ground, foot- worn sensor therefore, and method to advise a user of a risk of falling based thereon. Patent No. WO 2014/032181 A1.
[49]
T. Pakkanen and R. Raisamo. 2004. Appropriateness of foot interaction for non-accurate spatial tasks. In CHI'04 Extended Abstracts on Human Factors in Computing Systems. ACM, 1123--1126.
[50]
J. Paradiso, C. Abler, K. Y. Hsiao, and M. Reynolds. 1997. The magic carpet: physical sensing for immersive environments. In CHI'97 Extended Abstracts on Human Factors in Computing Systems. ACM, 277--278.
[51]
J. Paradiso, E. Hu, and K. Y. Hsiao. 1998. Instrumented footwear for interactive dance. In Proceedings of the XII Colloquium on Musical Informatics (CIM 1998). 24--26.
[52]
Parotec. URL: http://www.paromed.com.au/our-products
[53]
G. Pearson and M. Weiser. 1988. Exploratory evaluation of a planar foot-operated cursor-positioning device. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI 1988). ACM, 13--18.
[54]
Pedar Insoles. URL: http://www.novel.de/novelcontent/pedar
[55]
A. Pu and D. Psaltis. 1999. User identification through sequential input of fingerprints. U.S. Patent No. 5,933,515.
[56]
R. M. Queen, B. B. Haynes, W. M. Hardaker, and W. E. Garrett. 2007. Forefoot loading during 3 athletic tasks. In The American journal of sports medicine, 35(4). 630--636.
[57]
B. Richardson, K. Leydon, M. Fernstrom, and J. A. Paradiso. 2004. Z-Tiles: building blocks for modular, pressure-sensing floorspaces. In CHI'04 Extended Abstracts on Human Factors in Computing Systems. ACM, 1529--1532.
[58]
A. F. Rovers and H. A. Van Essen. 2005. FootIO---Design and Evaluation of a Device to Enable Foot Interaction over a Computer Network. In Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC 2005). IEEE, 521--522.
[59]
A. F. Rovers and H. A. Van Essen. 2006. Guidelines for haptic interpersonal communication applications: an exploration of foot interaction styles. In Virtual Reality (VR 2006), 9(2--3). IEEE, 177--191.
[60]
S. Sanderson and J. H. Erbetta. 2000. Authentication for secure environments based on iris scanning technology. In Visual Biometrics, 8. IET.
[61]
N. Sangsuriyachot and M. Sugimoto. 2012. Novel interaction techniques based on a combination of hand and foot gestures in tabletop environments. In Proceedings of the 10th Asia Pacific Conference on Computer Human Interaction (APCHI 2012). 21--28.
[62]
J. Scott, D. Dearman, K. Yatani, and K. N. Truong. 2010. Sensing foot gestures from the pocket. In Proceedings of the 23nd annual ACM symposium on User interface software and technology (UIST 2010). ACM, 199--208.
[63]
Sensoria Fitness Socks. URL: http://www.sensoriafitness.com
[64]
N. S. Shenck and J.A. Paradiso. 2001. Energy scavenging with shoe-mounted piezoelectrics. IEEE Micro, 21(3). IEEE, 30--42.
[65]
L. Shu, T. Hua, Y. Wang, Q. Li, D. D. Feng, and X. Tao. 2010. In-shoe plantar pressure measurement and analysis system based on fabric pressure sensing array. In IEEE Transactions on Information Technology in Biomedicine (T-ITB 2010), 14(3). IEEE, 767--775.
[66]
B. Slawik. 2014. ShoeSoleSense for Peripheral Interaction. In Workshop on Peripheral Interaction: Shaping the Research and Design Space at CHI 2014.
[67]
J. R. Smith. 1999. Electric Field Imaging. Ph.D. Thesis, MIT - Massachusetts Institute of Technology
[68]
S. Urry. 1999. Plantar pressure-measurement sensors. In Measurement Science and Technology, 10(1), R16.
[69]
R. Velãzquez, O. Bazán, and M. Magaa. 2009. A shoe-integrated tactile display for directional navigation. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2009). IEEE, 1235--1240.
[70]
L. Wang, T. Tan, H. Ning, and W. Hu. 2003. Silhouette analysis-based gait recognition for human identification. In IEEE transactions on pattern analysis and machine intelligence (TPAMI 2003), 25(12). IEEE, 1505--1518.
[71]
R. Weiss and A. De Luca. 2008. PassShapes: utilizing stroke based authentication to increase password memorability. In Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges (NordiCHI 2008). ACM, 383--392.
[72]
J. J. Wertsch, J. G. Webster, and W. J. Tompkins. 1992. A portable insole plantar pressure measurement system. In Journal of Rehab. Research & Development, 29(1). 13--18.
[73]
R. Whalen, J. Quintana, and J. Emery. 1993. A method for continuous monitoring of the ground reaction force during daily activity. In The Physiologist, 36 (1 Suppl.). 139--40.
[74]
M. Wiebe, B. Wurm, and U. Zendler. 2014. A Study on Foot-Operated Input Devices. University of Stuttgart.
[75]
W. Xu, M. C. Huang, N. Amini, J. J. Liu, L. He, and M. Sarrafzadeh. 2012. Smart insole: a wearable system for gait analysis. In Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA 2012). ACM, 18.
[76]
H. Zhu, G. F. Harris, J. J. Wertsch, W. J. Tompkins, and J. G. Webster. 1991. A microprocessor-based data-acquisition system for measuring plantar pressures from ambulatory subjects. In IEEE Transactions on Biomedical Engineering (TBME 1991), 38(7). IEEE, 710--714.

Cited By

View all
  • (2025)Enhancing Gait Analysis and Pathway Classification Through Ground Impedance-Based Shoes: An Innovative ApproachIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2025.353808774(1-12)Online publication date: 2025
  • (2025)ShoeTect2.0: Real-Time Activity Recognition Using MobileNet CNN with Multisensory Smart FootwearSensor-Based Activity Recognition and Artificial Intelligence10.1007/978-3-031-80856-2_17(260-268)Online publication date: 9-Feb-2025
  • (2025)SurfSole: Demonstrating Real-Time Surface Identification via Capacitive Sensing with Neural NetworksSensor-Based Activity Recognition and Artificial Intelligence10.1007/978-3-031-80856-2_16(251-259)Online publication date: 9-Feb-2025
  • Show More Cited By

Index Terms

  1. CapSoles: who is walking on what kind of floor?

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MobileHCI '17: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services
    September 2017
    874 pages
    ISBN:9781450350754
    DOI:10.1145/3098279
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 04 September 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. capacitive sensing
    2. data mining
    3. floor detection
    4. foot interaction
    5. ground surface detection
    6. implicit input
    7. machine learning
    8. shoe interface
    9. smart insole
    10. user identification
    11. wearable computing

    Qualifiers

    • Research-article

    Funding Sources

    • German Federal State of Mecklenburg-Western Pomerania

    Conference

    MobileHCI '17
    Sponsor:

    Acceptance Rates

    MobileHCI '17 Paper Acceptance Rate 45 of 224 submissions, 20%;
    Overall Acceptance Rate 202 of 906 submissions, 22%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)59
    • Downloads (Last 6 weeks)7
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Enhancing Gait Analysis and Pathway Classification Through Ground Impedance-Based Shoes: An Innovative ApproachIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2025.353808774(1-12)Online publication date: 2025
    • (2025)ShoeTect2.0: Real-Time Activity Recognition Using MobileNet CNN with Multisensory Smart FootwearSensor-Based Activity Recognition and Artificial Intelligence10.1007/978-3-031-80856-2_17(260-268)Online publication date: 9-Feb-2025
    • (2025)SurfSole: Demonstrating Real-Time Surface Identification via Capacitive Sensing with Neural NetworksSensor-Based Activity Recognition and Artificial Intelligence10.1007/978-3-031-80856-2_16(251-259)Online publication date: 9-Feb-2025
    • (2024)Body-Area Capacitive or Electric Field Sensing for Human Activity Recognition and Human-Computer InteractionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435558:1(1-49)Online publication date: 6-Mar-2024
    • (2024)TouchEditorProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314547:4(1-29)Online publication date: 12-Jan-2024
    • (2024)Foot Augmentation 101: Design your own Augmented ExperiencesProceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3623509.3634744(1-4)Online publication date: 11-Feb-2024
    • (2024)Shoes With Integrated Electrodes to Measure Floor Impedance and Its Application to Automatic Pathway ClassificationIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.336557373(1-9)Online publication date: 2024
    • (2023)Estimation of Sidewalk Surface Condition with Insole Devices2023 Fourteenth International Conference on Mobile Computing and Ubiquitous Network (ICMU)10.23919/ICMU58504.2023.10412165(1-6)Online publication date: 29-Nov-2023
    • (2023)PneuShoe: A Pneumatic Smart Shoe for Activity Recognition, Terrain Identification, and Weight EstimationProceedings of the 8th international Workshop on Sensor-Based Activity Recognition and Artificial Intelligence10.1145/3615834.3615853(1-5)Online publication date: 21-Sep-2023
    • (2023)RadarFoot: Fine-grain Ground Surface Context Awareness for Smart ShoesProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606738(1-13)Online publication date: 29-Oct-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media