Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Automatic strain sensor design via active learning and data augmentation for soft machines

Nature Machine Intelligence volume 4, pages 84–94 (2022)Cite this article

Subjects

An Author Correction to this article was published on 11 March 2022

This article has been updated

Abstract

Emerging soft machines require high-performance strain sensors to achieve closed-loop feedback control. Machine learning is a versatile tool to uncover complex correlations between fabrication recipes and sensor performance at the device level. Here a three-stage machine learning framework was realized for a high-accuracy prediction model capable of automating the design of strain sensors. First, a support-vector machine classifier was trained by using 351 compositions of various nanomaterials. Second, through 12 active learning loops, 125 strain sensors were stagewise fabricated to enrich the multidimensional dataset. Third, to address the challenge of data scarcity, data augmentation was implemented to synthesize >10,000 virtual data points, followed by genetic algorithm-based selection to optimize the model’s prediction accuracy. Several data-driven design rules for piezoresistive nanocomposites were generalized and validated by in situ microscopic studies. As final demonstrations, model-suggested strain sensors can be integrated into/onto various soft machines to endow them with real-time strain-sensing capabilities.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Three-stage framework for construction of a machine learning-enabled prediction model capable of automatic strain sensor design for soft machines.
Fig. 2: Boundary definition in sensor design space through an SVM.
Fig. 3: Progressive exploration of sensor design space through active learning.
Fig. 4: Model optimization through data augmentation and GA-based selection.
Fig. 5: Statistical analysis of data points collected from active learning.
Fig. 6: Automatic sensor design for soft machines.

Similar content being viewed by others

Data availability

The data that support the plots within this paper and other findings of this study are available from the corresponding authors on reasonable request. Source data are provided with this paper.

Code availability

The Python code to implement the machine learning tasks within this study are available from GitHub (https://github.com/jiali1025/Automatic_Strain_Sensor_Design) or Zenodo (https://doi.org/10.5281/zenodo.5709148)57.

Change history

References

  1. Rus, D. & Tolley, M. T. Design, fabrication and control of soft robots. Nature 521, 467–475 (2015).

    Article  Google Scholar 

  2. Shepherd, R. F. et al. Multigait soft robot. Proc. Natl Acad. Sci. USA 108, 20400–20403 (2011).

    Article  Google Scholar 

  3. Walsh, C. Human-in-the-loop development of soft wearable robots. Nat. Rev. Mater. 3, 78–80 (2018).

    Article  Google Scholar 

  4. Rus, D. & Tolley, M. T. Design, fabrication and control of origami robots. Nat. Rev. Mater. 3, 101–112 (2018).

    Article  Google Scholar 

  5. Kim, Y., Yuk, H., Zhao, R., Chester, S. A. & Zhao, X. Printing ferromagnetic domains for untethered fast-transforming soft materials. Nature 558, 274–279 (2018).

    Article  Google Scholar 

  6. Hu, W., Lum, G. Z., Mastrangeli, M. & Sitti, M. Small-scale soft-bodied robot with multimodal locomotion. Nature 554, 81–85 (2018).

    Article  Google Scholar 

  7. Laschi, C., Mazzolai, B. & Cianchetti, M. Soft robotics: technologies and systems pushing the boundaries of robot abilities. Sci. Rob. 1, eaah3690 (2016).

    Article  Google Scholar 

  8. Kim, Y., Parada, G. A., Liu, S. & Zhao, X. Ferromagnetic soft continuum robots. Sci. Rob. 4, eaax7329 (2019).

    Article  Google Scholar 

  9. Wang, M. et al. Gesture recognition using a bioinspired learning architecture that integrates visual data with somatosensory data from stretchable sensors. Nat. Electron. 3, 563–570 (2020).

    Article  Google Scholar 

  10. Zhou, Z. et al. Sign-to-speech translation using machine-learning-assisted stretchable sensor arrays. Nat. Electron. 3, 571–578 (2020).

    Article  Google Scholar 

  11. Thuruthel, T. G., Shih, B., Laschi, C. & Tolley, M. T. Soft robot perception using embedded soft sensors and recurrent neural networks. Sci. Robot. 4, eaav1488 (2019).

    Article  Google Scholar 

  12. Sinatra, N. R. et al. Ultragentle manipulation of delicate structures using a soft robotic gripper. Sci. Robot. 4, eaax5425 (2019).

    Article  Google Scholar 

  13. Sundaram, S. et al. Learning the signatures of the human grasp using a scalable tactile glove. Nature 569, 698–702 (2019).

    Article  Google Scholar 

  14. Zhang, J. et al. Robotic artificial muscles: current progress and future perspectives. IEEE Trans. Robot. 35, 761–781 (2019).

    Article  Google Scholar 

  15. Mirvakili, S. M. & Hunter, I. W. Artificial muscles: mechanisms, applications, and challenges. Adv. Mater. 30, 1704407 (2018).

    Article  Google Scholar 

  16. Zhao, H., O’Brien, K., Li, S. & Shepherd, R. F. Optoelectronically innervated soft prosthetic hand via stretchable optical waveguides. Sci. Robot. 1, eaai7529 (2016).

    Article  Google Scholar 

  17. Cianchetti, M., Laschi, C., Menciassi, A. & Dario, P. Biomedical applications of soft robotics. Nat. Rev. Mater. 3, 143–153 (2018).

    Article  Google Scholar 

  18. Amjadi, M., Kyung, K.-U., Park, I. & Sitti, M. Stretchable, skin-mountable, and wearable strain sensors and their potential applications: A review. Adv. Funct. Mater. 26, 1678–1698 (2016).

    Article  Google Scholar 

  19. Qiu, A. et al. A path beyond metal and silicon:polymer/nanomaterial composites for stretchable strain sensors. Adv. Funct. Mater. 29, 1806306 (2019).

    Article  Google Scholar 

  20. Cai, Y. et al. Stretchable Ti3C2Tx MXene/carbon nanotube composite based strain sensor with ultrahigh sensitivity and tunable sensing range. ACS Nano 12, 56–62 (2018).

    Article  Google Scholar 

  21. Shi, X., Liu, S., Sun, Y., Liang, J. & Chen, Y. Lowering internal friction of 0D–1D–2D ternary nanocomposite-based strain sensor by fullerene to boost the sensing performance. Adv. Funct. Mater. 28, 1800850 (2018).

    Article  Google Scholar 

  22. Wang, Y. et al. Wearable and highly sensitive graphene strain sensors for human motion monitoring. Adv. Funct. Mater. 24, 4666–4670 (2014).

    Article  Google Scholar 

  23. Shi, X. et al. Bioinspired ultrasensitive and stretchable MXene-based strain sensor via nacre-mimetic microscale ‘brick-and-mortar’ architecture. ACS Nano 13, 649–659 (2019).

    Article  Google Scholar 

  24. Jayathilaka, W. A. D. M. et al. Significance of nanomaterials in wearables: A review on wearable actuators and sensors. Adv. Mater. 31, 1805921 (2019).

    Article  Google Scholar 

  25. Araromi, O. A. et al. Ultra-sensitive and resilient compliant strain gauges for soft machines. Nature 587, 219–224 (2020).

    Article  Google Scholar 

  26. Yang, H. et al. Wireless Ti3C2Tx MXene strain sensor with ultrahigh sensitivity and designated working windows for soft exoskeletons. ACS Nano 14, 11860–11875 (2020).

    Article  Google Scholar 

  27. Wang, H., Totaro, M. & Beccai, L. Toward perceptive soft robots: progress and challenges. Adv. Sci. 5, 1800541 (2018).

    Article  Google Scholar 

  28. Mengüç, Y. et al. Wearable soft sensing suit for human gait measurement. Int. J. Rob. Res. 33, 1748–1764 (2014).

    Article  Google Scholar 

  29. Lu, N. & Kim, D.-H. Flexible and stretchable electronics paving the way for soft robotics. Soft Robot. 1, 53–62 (2014).

    Article  Google Scholar 

  30. Schmidt, J., Marques, M. R. G., Botti, S. & Marques, M. A. L. Recent advances and applications of machine learning in solid-state materials science. npj Comput. Mater. 5, 83 (2019).

  31. Xia, B. et al. Improving the actuation speed and multi-cyclic actuation characteristics of silicone/ethanol soft actuators. Actuators 9, 62 (2020).

  32. Butler, K. T., Davies, D. W., Cartwright, H., Isayev, O. & Walsh, A. Machine learning for molecular and materials science. Nature 559, 547–555 (2018).

    Article  Google Scholar 

  33. Zahrt, A. F. et al. Prediction of higher-selectivity catalysts by computer-driven workflow and machine learning. Science 363, eaau5631 (2019).

    Article  Google Scholar 

  34. Toyao, T. et al. Machine learning for catalysis informatics: recent applications and prospects. ACS Catal. 10, 2260–2297 (2020).

    Article  Google Scholar 

  35. Kitchin, J. R. Machine learning in catalysis. Nat. Catal. 1, 230–232 (2018).

    Article  Google Scholar 

  36. Vamathevan, J. et al. Applications of machine learning in drug discovery and development. Nat. Rev. Drug Discov. 18, 463–477 (2019).

    Article  Google Scholar 

  37. Ekins, S. et al. Exploiting machine learning for end-to-end drug discovery and development. Nat. Mater. 18, 435–441 (2019).

    Article  Google Scholar 

  38. Voznyy, O. et al. Machine learning accelerates discovery of optimal colloidal quantum dot synthesis. ACS Nano 13, 11122–11128 (2019).

    Article  Google Scholar 

  39. Durrer, R. et al. Automated tuning of double quantum dots into specific charge states using neural networks. Phys. Rev. Appl. 13, 054019 (2020).

    Article  Google Scholar 

  40. Li, J. et al. AI applications through the whole life cycle of material discovery. Matter 3, 393–432 (2020).

    Article  Google Scholar 

  41. Cole, J. M. A design-to-device pipeline for data-driven materials discovery. Acc. Chem. Res. 53, 599–610 (2020).

    Article  Google Scholar 

  42. Cao, B. et al. How to optimize materials and devices via design of experiments and machine learning: demonstration using organic photovoltaics. ACS Nano 12, 7434–7444 (2018).

    Article  Google Scholar 

  43. Afsarimanesh, N. et al. A review on fabrication, characterization and implementation of wearable strain sensors. Sens. Actuator A 315, 112355 (2020).

    Article  Google Scholar 

  44. Murphey, Y. L., Guo, H. & Feldkamp, L. A. Neural learning from unbalanced data. Appl. Intell. 21, 117–128 (2004).

    Article  MATH  Google Scholar 

  45. Hoffmann, J. et al. Machine learning in a data-limited regime: augmenting experiments with synthetic data uncovers order in crumpled sheets. Sci. Adv. 5, eaau6792 (2019).

    Article  Google Scholar 

  46. Chen, P.-Y. et al. Multiscale graphene topographies programmed by sequential mechanical deformation. Adv. Mater. 28, 3564–3571 (2016).

    Article  Google Scholar 

  47. Noble, W. S. What is a support vector machine? Nat. Biotechnol. 24, 1565–1567 (2006).

    Article  Google Scholar 

  48. Whitley, D. A genetic algorithm tutorial. Stat. Comput. 4, 65–85 (1994).

    Article  Google Scholar 

  49. Schober, P., Boer, C. & Schwarte, L. A. Correlation coefficients: appropriate use and interpretation. Anesth. Analg. 126, 1763–1768 (2018).

    Article  Google Scholar 

  50. Štrumbelj, E. & Kononenko, I. Explaining prediction models and individual predictions with feature contributions. Knowl. Inf. Syst. 41, 647–665 (2014).

    Article  Google Scholar 

  51. Bach, S. et al. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10, e0130140 (2015).

    Article  Google Scholar 

  52. Zhang, S. et al. Predicting the formability of hybrid organic-inorganic perovskites via an interpretable machine learning strategy. J. Phys. Chem. Lett. 12, 7423–7430 (2021).

    Google Scholar 

  53. Low, J. H. et al. Hybrid tele-manipulation system using a sensorized 3-D-printed soft robotic gripper and a soft fabric-based haptic glove. IEEE Robot. Autom. Let. 2, 880–887 (2017).

    Article  Google Scholar 

  54. Truong, T. V., Viswanathan, V. K., Joseph, V. S. & Alvarado, P. V. Y. Design and characterization of a fully autonomous under-actuated soft batoid-like robot. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 5826–5831 (IEEE, 2019).

  55. Alhabeb, M. et al. Guidelines for synthesis and processing of two-dimensional titanium carbide (Ti3C2Tx MXene). Chem. Mater. 29, 7633–7644 (2017).

    Article  Google Scholar 

  56. Shenton, M. J., Lovell-Hoare, M. C. & Stevens, G. C. Adhesion enhancement of polymer surfaces by atmospheric plasma treatment. J. Phys. D 34, 2754–2760 (2001).

    Article  Google Scholar 

  57. Li, J., Lim, K. & Yang, H. Automatic strain sensor design (v.1.0.3). Zenodo https://doi.org/10.5281/zenodo.5709148 (2021).

Download references

Acknowledgements

We thank C.-H. Yeow from the Department of Biomedical Engineering in the National University of Singapore for providing the soft gripper. We acknowledge the financial support provided by the Singapore RIE2020 Advanced Manufacturing and Engineering Programmatic Grant ‘Accelerated Materials Development for Manufacturing’ by the Agency for Science, Technology and Research under grant no. A1898b0043 (to X.W.). We acknowledge the financial support provided by the Start-Up Fund of University of Maryland, College Park (KFS no. 2957431 to P.-Y.C.). Funding for this research was provided by Maryland Industrial Partnerships under grant no. 6808 (KFS no. 4311103 to P.-Y.C.), Maryland Innovation Initiative (MII) Technology Assessment Award (KFS no. 4308302 to P.-Y.C.), and MOST-AFOSR Taiwan Topological and Nanostructured Materials Grant under grant no. FA2386-21-1-4065 (KFS no. 5284212 to P.-Y.C.).

Author information

Author notes

  1. These authors contributed equally: Haitao Yang, Jiali Li.

Authors and Affiliations

  1. Department of Chemical and Biomolecular Engineering, National University of Singapore, Singapore, Singapore

    Haitao Yang, Jiali Li, Kai Zhuo Lim, Chuanji Pan, Qian Wang, Kerui Li, Shuo Li, Meng Ding & Xiaoli Liu

  2. Engineering Product Development, Singapore University of Technology and Design, Singapore, Singapore

    Tien Van Truong & Pablo Valdivia y. Alvarado

  3. Department of Electrical and Electronic Engineering, Southern University of Science and Technology, Shenzhen, China

    Xiao Xiao

  4. Department of Chemical and Biomolecular Engineering, University of Maryland, College Park, MD, USA

    Tianle Chen & Po-Yen Chen

  5. School of Metallurgic Engineering, Anhui University of Technology, Ma’anshan, China

    Qian Xie

  6. Department of Chemical Engineering, Tsinghua University, Beijing, China

    Xiaonan Wang

Authors
  1. Haitao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiali Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kai Zhuo Lim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chuanji Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tien Van Truong
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Kerui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Shuo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Xiao Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Meng Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Tianle Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Xiaoli Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Qian Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Pablo Valdivia y. Alvarado
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Xiaonan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Po-Yen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

P.-Y.C., X.W., J.L. and H.Y. conceived the project ideas and designed the experiments. H.Y. and S.L. carried out the synthesis of MXene nanosheets and characterizations. H.Y., K.Z.L. and J.L. designed the machine learning framework and implemented the machine learning tasks in Python. H.Y., Q.W., J.L. and C.P. collected the experimental data. T.V.T., Q.W. and H.Y. integrated the model-suggested sensors onto soft swimmer robot and performed related tests. X.X. and H.Y. performed the FEA simulations and analyses. P.-Y.C., X.W., J.L. and H.Y. interpreted the results and co-wrote the manuscript. K.L., S.L., M.D., T.C., X.L. and Q.X. involved in the discussion and manuscript revisions. P.-Y.C, P.V.A. and X.W. supervised this project.

Corresponding authors

Correspondence to Xiaonan Wang or Po-Yen Chen.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Machine Intelligence thanks the anonymous reviewers for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Influential fabrication parameters for tuning strain sensor characteristics.

(a) Schematic illustration of influential fabrication parameters for tuning strain sensor characteristics, including composition, thickness, and morphology of strain sensing layer. (b) Resistance–strain profiles of various G0 sensors with the sensing layers at different MXene/SWNT/PVA ratios. All G0 sensors were with the sensing layers at the same thickness of 800 nm. (c) Resistance–strain profiles of various G0 sensors with the sensing layers at different thicknesses. All G0 sensors were with the sensing layers at the same MXene/SWNT/PVA ratio of 90/5/5. (d) Resistance–strain profiles of Gn sensors with different sensing layer morphologies. All Gn sensors were composed of ps-MXene layers either with planar texture (G0), periodic wrinkles (G1–1D) or isotropic crumples (G1–2D). The MXene/SWNT/PVA ratio and thickness of all ps-MXene layers were controlled to be 100/0/0 and 800 nm, respectively.

Source data

Extended Data Fig. 2 In situ electron microscopic studies to validate data-driven design rules.

(a) SEM images of G0 ps-MXene layers with varying thicknesses under various uniaxial strains. Crack-to-width ratios (b) and crack densities (c) in G0 ps-MXene layers with varying thicknesses. All the ps-MXene layers kept the same MXene/SWNT/PVA ratio of 90/10/0. Crack-to-width ratios (d) and crack densities (e) in G0 ps-MXene layers with varying SWNT loadings (wt.%). All the ps-MXene layers kept the same thickness of 800 nm and 0 wt.% PVA loading. FEA simulation of strain distribution and in situ SEM images of G0 (f), G1–1D (g), and G1–2D (h) ps-MXene layers under various uniaxial strains.

Source data

Supplementary information

Supplementary Information

Supplementary Notes 1–33, Figs. 1–32 and Tables 1–18.

Reporting Summary

Supplementary Video 1

A soft swimmer robot with model-suggested strain sensors for underwater exploration mission.

Supplementary Video 2

Real-time monitoring of a soft swimmer robot.

Source data

Source Data Fig. 1

Feasibility grades of detachment situations of 351 ps-MXene layers from PVDF membranes. Possibility heatmap representing the chances of successfully detaching ps-MXene layers from PVDF membranes.

Source Data Fig. 2

\({\bar L}\) and MSE values of trained navigation models by using different acquisition functions, including variance, random selection, distance and A Score. Cumulative number of fabricated Gn sensors for each loop of active learning; \({\bar{L}}\) and MSE values of evolving navigation model at different stages of active learning.

Source Data Fig. 3

Comparison of MRE values of various candidate models. Model-predicted strain labels and actual sensor characteristics of a G0 sensor. Model-predicted strain labels and actual sensor characteristics of a G1–1D sensor. Model-predicted strain labels and actual sensor characteristics of a G1–2D sensor.

Source Data Fig. 4

Classification of εmax extracted from 125 Gn sensors at different morphologies of strain-sensing layers. Spearman’s ρ statistical analyses for G1–2D, G1–1D and G0 sensors between fabrication parameters (for example, SWNT loading, PVA loading, and sensing layer thickness) and εmax. SHAP values of SWNT loading, PVA loading and sensing layer thickness for G1–2D, G1–1D and G0 sensors.

Source Data Fig. 5

GF-strain profiles of two suggested G1–1D sensors. Resistance–strain profile of Sensor 1 for monitoring a robotic task of grasping a candle. Resistance–strain profile of Sensor 1 for monitoring a robotic task of grasping a glue. GF-strain profiles of two suggested G0 sensors. Resistance–strain profile of an embedded G0 sensor (Sensor 2) in the left fin of a soft swimmer robot.

Source Data Extended Data Fig. 1

Resistance–strain profiles of various G0 sensors with the sensing layers at different MXene/SWNT/PVA ratios. Resistance–strain profiles of various G0 sensors with the sensing layers at different thicknesses. Resistance–strain profiles of Gn sensors with different sensing layer morphologies.

Source Data Extended Data Fig. 2

Crack-to-width ratios and crack densities in G0 ps-MXene layers with varying thicknesses and with varying SWNT loadings (wt%).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, H., Li, J., Lim, K.Z. et al. Automatic strain sensor design via active learning and data augmentation for soft machines. Nat Mach Intell 4, 84–94 (2022). https://doi.org/10.1038/s42256-021-00434-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-021-00434-8

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics