Skip to main content
SpringerLink
Log in

Image segmentation using convolutional neural networks in multi-sensor information fusion

  • Neural Networks
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

In recent years, the rapid advancement of artificial intelligence (AI) has revolutionized various industries, with image processing playing an important role in the revolution. AI-powered image analysis has opened up new possibilities in healthcare diagnostics, autonomous navigation, and security surveillance, fuelling demand for sophisticated image-processing solutions. This paper describes an innovative approach to AI image processing that combines multi-sensor data fusion with convolutional neural networks (CNNs) within a fuzzy neural network framework. This integration uses data from various sensors, including cameras, lidar, and radar, to improve the robustness and precision of image analysis and interpretation. The T–S model serves as the foundation for the information fusion strategy. A comprehensive investigation of deep learning algorithms reveals inherent strengths such as robustness and parallelism. However, it also identifies limitations, particularly in image segmentation tasks, characterized by challenges like premature convergence and prolonged computation times. The paper proposes a quantization technique for deep learning algorithms to address these issues and introduces chaotic optimization to expedite convergence rates. It also presents a novel three-dimensional Otsu threshold segmentation method based on CNNs, which overcomes noise susceptibility in traditional two-dimensional approaches. Integrating Gray morphology and this three-dimensional Otsu threshold segmentation method results in the development of a three-dimensional Gray Otsu model. This model is the basis for designing a fitness function for the CNN algorithm, optimizing its efficacy. Experimental validation demonstrates the proposed algorithm’s effectiveness, achieving an impressive 91% accuracy rate while displaying robust noise resistance and versatility. Comparative assessments against other leading AI architectures, including multilayer perceptron, radial basis function network, recurrent neural network, and long short-term memory network, affirm the superior performance achieved by the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from the corresponding author, upon reasonable request.

References

  • Ali M, Yin B, Kunar A, Sheikh AM, Bilal H et al. (2020) Reduction of multiplications in convolutional neural networks. In: 2020 39th Chinese control conference (CCC). IEEE, pp 7406–7411. https://doi.org/10.23919/CCC50068.2020.9188843.

  • Aslam MS, Dai X, Hou J, Li Q, Ullah R, Ni Z, Liu Y (2020) Reliable control design for composite-driven scheme based on delay networked T–S fuzzy system. Int J Robust Nonlinear Control 30(4):1622–1642

    Article  MathSciNet  MATH  Google Scholar 

  • Aslam MS, Qaisar I, Majid A, Shamrooz S (2023) Adaptive event-triggered robust H∞ control for Takagi–Sugeno fuzzy networked Markov jump systems with time-varying delay. Asian J Control 25(1):213–228

    Article  MathSciNet  Google Scholar 

  • Bilal H, Yin B, Aslam MS et al (2023) A practical study of active disturbance rejection control for rotary flexible joint robot manipulator. Soft Comput 27:4987–5001. https://doi.org/10.1007/s00500-023-08026-x

    Article  Google Scholar 

  • Cao B, Zhao J, Gu Y, Fan S, Yang P (2019) Security-aware industrial wireless sensor network deployment optimization. IEEE Trans Ind Inf 16(8):5309–5316

    Article  Google Scholar 

  • Cheng B, Wang M, Zhao S, Zhai Z, Zhu D, Chen J (2017) Situation-aware dynamic service coordination in an IoT environment. IEEE/ACM Trans Netw 25(4):2082–2095

    Article  Google Scholar 

  • Cheng D, Chen L, Lv C, Guo L, Kou Q (2022) Light-guided and cross-fusion U-Net for anti-illumination image super-resolution. IEEE Trans Circuits Syst Video Technol 32(12):8436–8449

    Article  Google Scholar 

  • Cong R, Sheng H, Yang D, Cui Z, Chen R (2023) Exploiting spatial and angular correlations with deep efficient transformers for light field image super-resolution. IEEE Trans Multimed

  • Fan W, Yang L, Bouguila N (2021) Unsupervised grouped axial data modeling via hierarchical Bayesian nonparametric models with Watson distributions. IEEE Trans Pattern Anal Mach Intell 44(12):9654–9668

    Article  Google Scholar 

  • Fu C, Yuan H, Xu H, Zhang H, Shen L (2023) TMSO-Net: texture adaptive multi-scale observation for light field image depth estimation. J vis Commun Image Represent 90:103731

    Article  Google Scholar 

  • Han Y, Wang B, Guan T, Tian D, Yang G, Wei W, Tang H, Chuah JH (2022) Research on road environmental sense method of intelligent vehicle based on tracking check. IEEE Trans Intell Transp Syst 24(1):1261–1275

    Article  Google Scholar 

  • Hazrat B, Yin B, Kumar A, Ali M, Zhang J, Yao J (2023) Jerk-bounded trajectory planning for rotary flexible joint manipulator: an experimental approach. Soft Comput 27(7):4029–4039. https://doi.org/10.1007/s00500-023-07923-5

    Article  Google Scholar 

  • Kumar A, Shaikh AM, Li Y et al (2021) Pruning filters with L1-norm and capped L1-norm for CNN compression. Appl Intell 51:1152–1160. https://doi.org/10.1007/s10489-020-01894-y

    Article  Google Scholar 

  • Li B, Tan Y, Wu AG, Duan GR (2021) A distributionally robust optimization-based method for stochastic model predictive control. IEEE Trans Autom Control 67(11):5762–5776

    Article  MathSciNet  MATH  Google Scholar 

  • Li Y, Qian J, Feng S, Chen Q, Zuo C (2022) Deep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurement. Opto-Electron Adv 5(5):210021–210031

    Article  Google Scholar 

  • Li L, Wang P, Zheng X, Xie Q, Tao X, Velásquez JD (2023) Dual-interactive fusion for code-mixed deep representation learning in tag recommendation. Inf Fusion 99:101862

    Article  Google Scholar 

  • Liu H, Yuan H, Liu Q, Hou J, Zeng H, Kwong S (2021a) A hybrid compression framework for color attributes of static 3D point clouds. IEEE Trans Circuits Syst Video Technol 32(3):1564–1577

    Article  Google Scholar 

  • Liu Q, Yuan H, Hamzaoui R, Su H, Hou J, Yang H (2021b) Reduced reference perceptual quality model with application to rate control for video-based point cloud compression. IEEE Trans Image Process 30:6623–6636

    Article  Google Scholar 

  • Liu AA, Zhai Y, Xu N, Nie W, Li W, Zhang Y (2021c) Region-aware image captioning via interaction learning. IEEE Trans Circuits Syst Video Technol 32(6):3685–3696

    Article  Google Scholar 

  • Liu M, Zhang X, Yang B, Yin Z, Liu S, Yin L, Zheng W (2023) Three-dimensional modeling of heart soft tissue motion. Appl Sci 13(4):2493

    Article  Google Scholar 

  • Lu S, Liu S, Hou P, Yang B, Liu M, Yin L, Zheng W (2023a) Soft tissue feature tracking based on DeepMatching network. CMES Comput Model Eng Sci 136(1):363–379

    Google Scholar 

  • Lu S, Yang B, Xiao Y, Liu S, Liu M, Yin L, Zheng W (2023b) Iterative reconstruction of low-dose CT based on differential sparse. Biomed Signal Process Control 79:104204

    Article  Google Scholar 

  • Lu S, Yang J, Yang B, Yin Z, Liu M, Yin L, Zheng W (2023c) Analysis and design of surgical instrument localization algorithm. CMES Comput Model Eng Sci 137(1):669

    Google Scholar 

  • Ma X, Dong Z, Quan W, Dong Y, Tan Y (2023) Real-time assessment of asphalt pavement moduli and traffic loads using monitoring data from built-in sensors: optimal sensor placement and identification algorithm. Mech Syst Signal Process 187:109930

    Article  Google Scholar 

  • Mao Y, Zhu Y, Tang Z, Chen Z (2022) A novel airspace planning algorithm for cooperative target localization. Electronics 11(18):2950

    Article  Google Scholar 

  • Min H, Fang Y, Wu X, Lei X, Chen S, Teixeira R, Zhu B, Zhao X, Xu Z (2023) A fault diagnosis framework for autonomous vehicles with sensor self-diagnosis. Expert Syst Appl 224:120002

    Article  Google Scholar 

  • Shamrooz M, Li Q, Hou J (2021) Fault detection for asynchronous T–S fuzzy networked Markov jump systems with new event-triggered scheme. IET Control Theory Appl 15(11):1461–1473

    Article  MathSciNet  Google Scholar 

  • Shen Y, Ding N, Zheng HT, Li Y, Yang M (2020) Modeling relation paths for knowledge graph completion. IEEE Trans Knowl Data Eng 33(11):3607–3617

    Article  Google Scholar 

  • Wang Y, Xu N, Liu AA, Li W, Zhang Y (2021) High-order interaction learning for image captioning. IEEE Trans Circuits Syst Video Technol 32(7):4417–4430

    Article  Google Scholar 

  • Wang W, Chen Z, Yuan X (2022a) Simple low-light image enhancement based on Weber-Fechner law in logarithmic space. Signal Process Image Commun 106:116742

    Article  Google Scholar 

  • Wang F, Wang H, Zhou X, Fu R (2022b) A driving fatigue feature detection method based on multifractal theory. IEEE Sens J 22(19):19046–19059

    Article  Google Scholar 

  • Wang L, Zhai Q, Yin B et al (2019) Second-order convolutional network for crowd counting. In: Proceedings of the SPIE 11198, fourth international workshop on pattern recognition, 111980T (31 July 2019). https://doi.org/10.1117/12.2540362

  • Wu Z, Cao J, Wang Y, Wang Y, Zhang L, Wu J (2018) hPSD: a hybrid PU-learning-based spammer detection model for product reviews. IEEE Trans Cybern 50(4):1595–1606

    Article  Google Scholar 

  • Xu H, Sun Z, Cao Y et al (2023) A data-driven approach for intrusion and anomaly detection using automated machine learning for the Internet of Things. Soft Comput. https://doi.org/10.1007/s00500-023-09037-4

    Article  Google Scholar 

  • Yang M, Wang H, Hu K, Yin G, Wei Z (2022) IA-Net: An inception–attention-module-based network for classifying underwater images from others. IEEE J Oceanic Eng 47(3):704–717

    Article  Google Scholar 

  • Yao W, Guo Y, Wu Y, Guo J (2017) Experimental validation of fuzzy PID control of flexible joint system in presence of uncertainties. In: 2017 36th Chinese control conference (CCC), pp 4192–4197. IEEE. https://doi.org/10.23919/ChiCC.2017.8028015

  • Yin B, Khan J, Wang L, Zhang J, Kumar A (2019) Real-time lane detection and tracking for advanced driver assistance systems. In: 2019 Chinese control conference (CCC). IEEE, pp 6772–6777. https://doi.org/10.23919/ChiCC.2019.8866334

  • Zhang J, Peng S, Gao Y, Zhang Z, Hong Q (2023a) APMSA: adversarial perturbation against model stealing attacks. IEEE Trans Inf Forensics Secur 18:1667–1679

    Article  Google Scholar 

  • Zhang C, Xiao P, Zhao ZT, Liu Z, Yu J, Hu XY, Chu HB, Xu JJ, Liu MY, Zou Q, Zhang L (2023b) A wearable localized surface plasmons antenna sensor for communication and sweat sensing. IEEE Sens J 23:11591

    Article  Google Scholar 

  • Zhao K, Jia Z, Jia F, Shao H (2023) Multi-scale integrated deep self-attention network for predicting remaining useful life of aero-engine. Eng Appl Artif Intell 120:105860

    Article  Google Scholar 

  • Zheng Y, Lv X, Qian L, Liu X (2022) An optimal bp neural network track prediction method based on a GA–ACO hybrid algorithm. J Mar Sci Eng 10(10):1399

    Article  Google Scholar 

  • Zhou G, Liu X (2022) Orthorectification model for extra-length linear array imagery. IEEE Trans Geosci Remote Sens 60:1–10

    Google Scholar 

  • Zhou X, Zhang L (2022) SA-FPN: an effective feature pyramid network for crowded human detection. Appl Intell 52(11):12556–12568

    Article  Google Scholar 

  • Zhou L, Ye Y, Tang T, Nan K, Qin Y (2021) Robust matching for SAR and optical images using multi-scale convolutional gradient features. IEEE Geosci Remote Sens Lett 19:1–5

    Google Scholar 

  • Zhou D, Sheng M, Li J, Han Z (2023) Aerospace integrated networks innovation for empowering 6G: a survey and future challenges. IEEE Commun Surv Tutor 25:975

    Article  Google Scholar 

Download references

Funding

No funding was provided for the completion of this study.

Author information

Authors and Affiliations

  1. School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou Henan, 450001, China

    Wenying Zhang, Min Dong & Li Jiang

Authors
  1. Wenying Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Min Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Li Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenying Zhang.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose. The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any study with human participants or animals performed by the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, W., Dong, M. & Jiang, L. Image segmentation using convolutional neural networks in multi-sensor information fusion. Soft Comput 27, 18353–18372 (2023). https://doi.org/10.1007/s00500-023-09271-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-023-09271-w

Keywords

Search

Navigation