Skip to main content

Advertisement

SpringerLink
Log in

Confusion matrix-based modularity induction into pretrained CNN

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Structurally and functionally, the human brain’s visual cortex inspires convolutional neural networks (CNN). The visual cortex consists of different connected cortical regions. When a cortical area receives an input, it extracts meaningful information and forwards it to its neighboring region. CNN imitates the hierarchical structure of the visual cortex by multiple feature extraction layers. In neurosciences, it is believed that the modular structure of the human brain is the source of its cognitive abilities. This work contributes to the problem of domain decomposition, information routing control in the network, and module integration for image classification by proposing a novel framework to induce modularity in a pretrained CNN. We decompose the input domain of the CNN by employing novel Confusion Matrix driven Centroid Based Clustering (CMCBC) to create functional modules comprised of different pathways. CMCBC is an unsupervised clustering technique that utilizes the k-Medoid algorithm. This approach uses a confusion matrix to find similarities between each pair of classes and medoid for every cluster instead of using a distance function. The proposed framework is evaluated on two benchmark datasets, MNIST and CIFAR10, and the results achieved are promising. On the MNIST dataset, we achieved 98.51% accuracy using our proposed Modular CNN compared to the baseline accuracy of 99.39%. But at the same time, we saved 53% multiplications in the network, which significantly reduced the complexity. Similarly, on the CIFAR10 dataset, our model achieves 78.01% accuracy, 6% less than the baseline accuracy (84%). But when we retrain the network to align the weights further, our model outperformed the baseline model accuracy by 2.78% and achieved 86.78% accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability

The data used in this paper is publicly available. The link to the datasets is as follow:

– MNIST (http://yann.lecun.com/exdb/mnist/)

– CIFAR10 (https://www.cs.toronto.edu/~kriz/CIFAR.html)

References

  1. Anderson A, Shaffer K, Yankov A, Corley CD, Hodas NO (2016) Beyond fine tuning: a modular approach to learning on small data. arXiv:1611.01714

  2. Anwar S, Hwang K, Sung W (2017) Structured pruning of deep convolutional neural networks. ACM J Emerg Technol Comput Syst (JETC) 13(3):32

    Google Scholar 

  3. Anwar S, Hwang K, Sung W (2017) Structured pruning of deep convolutional neural networks. ACM J Emerg Technol Comput Syst (JETC) 13(3):1–18

    Article  Google Scholar 

  4. Bastanfard A, Bastanfard O, Takahashi H, Nakajima M (2004) Toward anthropometrics simulation of face rejuvenation and skin cosmetic. Computer Animation and Virtual Worlds 15(3):347–352

    Article  Google Scholar 

  5. Bastanfard A, Takahashi H, Nakajima M (2004) Toward e-appearance of human face and hair by age, expression and rejuvenation. In: 2004 International conference on cyberworlds, pp 306–311

  6. Blakeney C, Li X, Yan Y, Zong Z (2020) Parallel blockwise knowledge distillation for deep neural network compression. IEEE Transactions on Parallel and Distributed Systems 32(7):1765–1776

    Article  Google Scholar 

  7. Braylan A, Hollenbeck M, Meyerson E, Miikkulainen R (2015) Reuse of neural modules for general video game playing. arXiv:1512.01537

  8. Chihaoui M, Elkefi A, Bellil W, Ben Amar C (2016) A survey of 2d face recognition techniques. Computers 5(4):21

    Article  Google Scholar 

  9. Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251–1258

  10. Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251–1258

  11. Ciregan D, Meier U, Schmidhuber J (2012) Multi-column deep neural networks for image classification. In: 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 3642–3649

  12. Cui Z, Henrickson K, Ke R, Wang Y (2019) Traffic graph convolutional recurrent neural network: a deep learning framework for network-scale traffic learning and forecasting. IEEE Transactions on Intelligent Transportation Systems

  13. Dehshibi MM, Bastanfard A (2010) A new algorithm for age recognition from facial images. Signal Process 90(8):2431–2444

    Article  Google Scholar 

  14. Freeman I, Roese-Koerner L, Kummert A (2018) Effnet: an efficient structure for convolutional neural networks. In: 2018 25th IEEE international conference on image processing (ICIP). IEEE, pp 6–10

  15. Fritsch J (1996) Modular neural networks for speech recognition. CARNEGIE-MELLON UNIV PITTSBURGH PA DEPT OF COMPUTER SCIENCE, Tech. Rep.

  16. Gheorghe T, Ivanovici M (2021) Model-based weight quantization for convolutional neural network compression. In: 2021 16th International conference on engineering of modern electric systems (EMES). IEEE, pp 1–4

  17. Ghosh S, Srinivasa SK, Amon P, Hutter A, Kaup A (2019) Deep network pruning for object detection. In: 2019 IEEE international conference on image processing (ICIP). IEEE, pp 3915–3919

  18. Goldberg Y (2016) A primer on neural network models for natural language processing. J Artif Intell Res 57:345–420

    Article  MathSciNet  Google Scholar 

  19. Gradojevic N, Gençay R., Kukolj D (2009) Option pricing with modular neural networks. IEEE Transactions on Neural Networks 20(4):626–637

    Article  Google Scholar 

  20. Han S, Mao H, Dally WJ (2015) Deep compression: compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv:1510.00149

  21. Happel BL, Murre JM (1994) Design and evolution of modular neural network architectures. Neural Netw 7(6-7):985–1004

    Article  Google Scholar 

  22. He Y, Zhang X, Sun J (2017) Channel pruning for accelerating very deep neural networks. In: Proceedings of the IEEE international conference on computer vision, pp 1389–1397

  23. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861

  24. Huizinga J, Clune J, Mouret J-B (2014) Evolving neural networks that are both modular and regular: hyperneat plus the connection cost technique. In: Proceedings of the 2014 annual conference on genetic and evolutionary computation, pp 697–704

  25. Jain S, Hamidi-Rad S, Racapé F (2021) Low rank based end-to-end deep neural network compression. In: 2021 Data compression conference (DCC). IEEE, pp 233–242

  26. Karim F, Majumdar S, Darabi H, Chen S (2017) Lstm fully convolutional networks for time series classification. IEEE Access 6:1662–1669

    Article  Google Scholar 

  27. Lavin A, Gray S (2016) Fast algorithms for convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4013–4021

  28. LeCun Y, Bengio Y, et al. (1995) Convolutional networks for images, speech, and time series. The Handbook of Brain Theory and Neural Networks 3361 (10):1995

    Google Scholar 

  29. Melin P, Mendoza O, Castillo O (2011) Face recognition with an improved interval type-2 fuzzy logic sugeno integral and modular neural networks. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans 41(5):1001–1012

    Article  Google Scholar 

  30. Mikolov T, Kombrink S, Burget L, Černockỳ J, Khudanpur S (2011) Extensions of recurrent neural network language model. In: 2011 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 5528–5531

  31. Modhej N, Bastanfard A, Teshnehlab M, Raiesdana S (2020) Pattern separation network based on the hippocampus activity for handwritten recognition. IEEE Access 8:212 803–212 817

    Article  Google Scholar 

  32. Phan KT, Maul TH, Vu TT, Lai WK (2018) Dropcircuit: a modular regularizer for parallel circuit networks. Neural Process Lett 47(3):841–858

    Article  Google Scholar 

  33. Ronco E, Gawthrop P (1995) Modular neural networks: a state of the art. Rapport Technique CSC-95026, Center of System and Control, University of Glasgow. http://www.mech.gla.ac.uk/control/report.html

  34. Ronen M, Shabtai Y, Guterman H (2002) Hybrid model building methodology using unsupervised fuzzy clustering and supervised neural networks. Biotech Bioeng 77(4):420–429

    Article  Google Scholar 

  35. Srivastava RK, Greff K, Schmidhuber J (2015) Highway networks. arXiv:15050.00387

  36. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. The J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  37. Szegedy C, Ioffe S, Vanhoucke V, Alemi A (2016) Inception-v4, inception-resnet and the impact of residual connections on learning. arXiv:1602.07261

  38. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4 inception-resnet and the impact of residual connections on learning. In: Thirty-first AAAI conference on artificial intelligence

  39. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826

  40. Terekhov AV, Montone G, O’Regan JK (2015) Knowledge transfer in deep block-modular neural networks. In: Conference on biomimetic and biohybrid systems. Springer, pp 268–279

  41. Tseng MM, Wang C (2014) Modular design, pp 895–897. Springer, Berlin

    Google Scholar 

  42. Verbancsics P, Stanley KO (2011) Constraining connectivity to encourage modularity in hyperneat. In: Proceedings of the 13th annual conference on Genetic and evolutionary computation, pp 1483–1490

  43. Waibel A (1989) Modular construction of time-delay neural networks for speech recognition. Neural Comput 1(1):39–46

    Article  Google Scholar 

  44. Wang T, Wu DJ, Coates A, Ng AY (2012) End-to-end text recognition with convolutional neural networks. In: Proceedings of the 21st international conference on pattern recognition (ICPR2012). IEEE, pp 3304–3308

  45. Watanabe C (2019) Interpreting layered neural networks via hierarchical modular representation. In: International conference on neural information processing. Springer, pp 376–388

  46. Wei W, Wong Y, Du Y, Hu Y, Kankanhalli M, Geng W (2019) A multi-stream convolutional neural network for semg-based gesture recognition in muscle-computer interface. Pattern Recogn Lett 119:131–138

    Article  Google Scholar 

  47. Wen W, Wu C, Wang Y, Chen Y, Li H (2016) Learning structured sparsity in deep neural networks. In: Advances in neural information processing systems, pp 2074–2082

  48. Yuan M, Lin Y (2006) Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 68(1):49–67

    Article  MathSciNet  Google Scholar 

  49. Zhang X, Zhou X, Lin M, Sun J (2018) Shufflenet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6848–6856

  50. Zhao Y, Wang D, Wang L, Liu P (2018) A faster algorithm for reducing the computational complexity of convolutional neural networks. Algorithms 11(10):159

    Article  MathSciNet  Google Scholar 

Download references

Funding

No funds or grants were received.

Author information

Authors and Affiliations

  1. Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, District Swabi, Topi, 23640, Pakistan

    Salman Ahmad, Shahab U. Ansari, Usman Haider, Jalees Ur Rahman & Sajid Anwar

  2. National Center of Artificial Intelligence (NCAI), Saudi Data and Artificial Intelligence Authority (SDAIA), Riyadh, Saudi Arabia

    Kamran Javed

Authors
  1. Salman Ahmad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shahab U. Ansari
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Usman Haider
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kamran Javed
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jalees Ur Rahman
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sajid Anwar
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

S.A.: Study design of neural network architecture and its implementation. SU.A.: Supervision, writing article, article review. U.H.: Study design of neural network architecture, writing article. K.J.: Helped in developing neural network architecture, article review. J.R.: Study design, article review. S.A.: Supervision, writing article, article review.

Corresponding author

Correspondence to Usman Haider.

Ethics declarations

Conflict of Interests

The authors declare no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Sajid Anwar deceased

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahmad, S., Ansari, S.U., Haider, U. et al. Confusion matrix-based modularity induction into pretrained CNN. Multimed Tools Appl 81, 23311–23337 (2022). https://doi.org/10.1007/s11042-022-12331-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12331-2

Keywords

Search

Navigation