Skip to main content
SpringerLink
Log in
Book cover

Asian Conference on Intelligent Information and Database Systems

ACIIDS 2021: Intelligent Information and Database Systems pp 743–753Cite as

Towards Layer-Wise Optimization of Contextual Neural Networks with Constant Field of Aggregation

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12672))

Abstract

In this paper contextual neural networks with different numbers of connection groups in different layers of neurons are considered. It is verified if not-uniform patterns of numbers of groups can influence classification properties of contextual neural networks. Simulations are done in dedicated H2O machine learning environment enhanced with Generalized Backpropagation algorithm. Experiments are performed for selected UCI machine learning problems and cancer gene expression microarray data of bone marrow acute lymphatic and myeloid leukemia.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Mendez, K.M., Broadhurst, D., Reinke S.N., The application of artificial neural networks in metabolomics: a historical perspective. Metabolomics 15(11), 142. Springer (2019). https://doi.org/10.1007/s11306-019-1608-0

  2. Tsai, Y.C., et al.: FineNet: a joint convolutional and recurrent neural network model to forecast and recommend anomalous financial items. In: Proceedings of the 13th ACM Conference on Recom-mender Systems RecSys 2019, New York, USA, pp. 536–537. ACM (2019)

    Google Scholar 

  3. Chen, S., Zhang, S., Shang, J., Chen, B., Zheng, N.: Brain-inspired cognitive model with attention for self-driving cars. IEEE Trans. Cogn. Dev. Syst. 11(1), 13–25. IEEE (2019)

    Google Scholar 

  4. Nasser, I.M., Abu-Naser, S.S.: Lung cancer detection using artificial neural network. Int. J. of Eng. Inf. Syst. (IJEAIS) 3(3), 17–23 (2019)

    Google Scholar 

  5. Suleymanova, I., et al.: A deep convolutional neural network approach for astrocyte detection. Sci. Rep. 8(12878), 1–7 (2018)

    Google Scholar 

  6. Wang, Z.H., Horng, G.J., Hsu, T.H., Chen, C.C., Jong, G.J.: A novel facial thermal feature extraction method for non-contact healthcare system. IEEE Access 8, 86545–86553. IEEE (2020)

    Google Scholar 

  7. Qiming, Z., et al.: Artificial neural networks enabled by nanophotonics. Light: Sci. Appl. 8(1), 14. Nature Publishing Group (2019)

    Google Scholar 

  8. Guest, D., Cranmer, K., Whiteson, D.: Deep learning and its application to LHC physics. Annu. Rev. Nucl. Part. Sci. 68, 1–22 (2018)

    Article  Google Scholar 

  9. Liu, L., Zheng, Y., Tang, D., Yuan, Y., Fan, C., Zhou, K.: Automatic skin binding for production characters with deep graph networks. ACM Trans. Graph. (SIGGRAPH) 38(4), Article 114, 12 (2019)

    Google Scholar 

  10. Gao, D., Li, X., Dong, Y., Peers, P., Xu, K., Tong, X.: Deep inverse rendering for high-resolution SVBRDF estimation from an arbitrary number of images. ACM Trans. Graph. (SIGGRAPH) 38(4), article 134, 15 (2019)

    Google Scholar 

  11. Gong, K., et al.: Iterative PET image reconstruction using convolutional neural network representation. IEEE Trans. Med. Imag. 38(3), 675–685. IEEE (2019)

    Google Scholar 

  12. Munkhdalai, L., Park, K.-H., Batbaatar, E., Theera-Umpon, N., Ryu, K.H.: Deep learning-based demand forecasting for Korean postal delivery service. IEEE Access 8, 188135–188145 (2020)

    Google Scholar 

  13. Batbaatar, E., Li, M., Ryu, K.H.: Semantic-emotion neural network for emotion recognition from text. IEEE Access 7, 111866–111878. IEEE (2019)

    Google Scholar 

  14. Higgins, I., et al.: β-VAE: learning basic visual concepts with a constrained variational framework. In: International Conference Learning Representations. ICLR 2017, vol. 2, no. 5, pp. 1–22 (2017)

    Google Scholar 

  15. Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of GANs for improved quality, stability, and variation. In: International Conference on Learning Representations. ICLR 2018, pp. 1–26 (2018)

    Google Scholar 

  16. Huang, X., Tan, H., Lin, G., Tian, Y.: A LSTM-based bidirectional translation model for optimizing rare words and terminologies. In: 2018 IEEE International Conference on Artificial Intelligence and Big Data (ICAIBD), China, pp. 5077–5086. IEEE (2018)

    Google Scholar 

  17. Athiwaratkun, B., Stokes, J.W.: Malware classification with LSTM and GRU language models and a character-level CNN. In: Proceedings 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), USA, 2017, pp. 2482–2486. IEEE (2017)

    Google Scholar 

  18. Amato, F., et al.: Multilayer perceptron: an intelligent model for classification and intrusion detection. In: 31st International Conference on Advanced Information Networking and Applications Workshops (WAINA), Taipei, Taiwan, pp. 686–691. IEEE (2017)

    Google Scholar 

  19. Dozono, H., Niina, G., Araki, S.: Convolutional self organizing map. In: 2016 IEEE International Conference on Computational Science and Computational Intelligence (CSCI), pp. 767–771. IEEE (2016)

    Google Scholar 

  20. Gościewska, K., Frejlichowski, D.: A combination of moment descriptors, fourier transform and matching measures for action recognition based on shape. In: Krzhizhanovskaya, V.V., et al. (eds.) ICCS 2020. LNCS, vol. 12138, pp. 372–386. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50417-5_28

    Chapter  Google Scholar 

  21. Frejlichowski, D.: Low-level greyscale image descriptors applied for intelligent and contextual approaches. In: Nguyen, N.T., Gaol, F.L., Hong, T.-P., Trawinski, B. (eds.) ACIIDS 2019. LNCS (LNAI), vol. 11431. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-14799-0

  22. Huk, M.: Backpropagation generalized delta rule for the selective attention Sigma-if artificial neural network. Int. J. App. Math. Comp. Sci. 22, 449–459 (2012)

    Google Scholar 

  23. Huk, M.: Notes on the generalized backpropagation algorithm for contextual neural networks with conditional aggregation functions. J. Intell. Fuzzy Syst. 32, 1365–1376. IOS Press (2017)

    Google Scholar 

  24. Huk, M.: Stochastic optimization of contextual neural networks with RMSprop. In: Nguyen, N.T., Jearanaitanakij, K., Selamat, A., Trawiński, B., Chittayasothorn, S. (eds.) ACIIDS 2020. LNCS (LNAI), vol. 12034, pp. 343–352. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-42058-1_29

    Chapter  Google Scholar 

  25. Burnell, E.D., Wołk, K., Waliczek, K., Kern, R.: The impact of constant field of attention on properties of contextual neural networks. In: Nguyen, N.T., Trawinski, B., et al. (eds.) 12th Asian Conference on Intelligent Information and Database Systems, ACIIDS 2020. LNAI, vol. 12034, pp. 364–375, Springer (2020). https://doi.org/10.1007/978-3-030-42058-1_31

  26. Huk, M., Non-uniform initialization of inputs groupings in contextual neural networks. In: Nguyen, N., Gaol F., Hong TP., Trawiński B. (eds) Intelligent Information and Database Systems. ACIIDS 2019. LNCS, vol. 11432, pp. 420–428. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-14802-7_36

  27. Huk, M.: Training contextual neural networks with rectifier activation functions: role and adoption of sorting methods. J. Intell. Fuzzy Syst. 37(6), 7493–7502. IOS Press (2019)

    Google Scholar 

  28. Huk, M.: Weights ordering during training of contextual neural networks with generalized error backpropagation: importance and selection of sorting algorithms. In: Nguyen, N.T., Hoang, D.H., Hong, T.-P., Pham, H., Trawiński, B. (eds.) ACIIDS 2018. LNCS (LNAI), vol. 10752, pp. 200–211. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75420-8_19

    Chapter  Google Scholar 

  29. Szczepanik, M., et al.: Multiple classifier error probability for multi-class problems. Eksploatacja i Niezawodnosc - Maintenance and Reliability 51(3), 12–16 (2011). https://doi.org/10.17531/ein

  30. Huk, M.: Measuring computational awareness in contextual neural networks. In: 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Budapest, pp. 002254–002259 (2016). https://doi.org/10.1109/SMC.2016.7844574

  31. Huk, M., Measuring the effectiveness of hidden context usage by machine learning methods under conditions of increased entropy of noise. In: 2017 3rd IEEE International Conference on Cybernetics (CYBCONF), Exeter, pp. 1–6 (2017). https://doi.org/10.1109/CYBConf.2017.7985787

  32. Huk, M., Pietraszko, J.: Contextual neural-network based spectrum prediction for cognitive radio. In: 4th International Conference on Future Generation Communication Technology (FGCT 2015). IEEE Computer Society, London, UK, pp. 1–5 (2015)

    Google Scholar 

  33. Szczepanik, M., Jóźwiak, I.: Data management for fingerprint recognition algorithm based on characteristic points’ groups. In: New Trends in Databases and Information Systems. Foundations of Computing and Decision Sciences, vol. 38, no. 2, pp. 123–130, Springer (2013). https://doi.org/10.1007/978-3-642-32518-2_40

  34. Szczepanik, M., Jóźwiak, I.: Fingerprint recognition based on minutes groups using directing attention algorithms. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012. LNCS (LNAI), vol. 7268, pp. 347–354. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29350-4_42

    Chapter  Google Scholar 

  35. Kwiatkowski, J., et al.: Context-sensitive text mining with fitness leveling genetic algorithm. In: 2015 IEEE 2nd International Conference on Cybernetics (CYBCONF), Gdynia, Poland, 2015, pp. 1–6. Electronic Publication (2015). https://doi.org/10.1109/CYBConf.2015.7175957. ISBN: 978-1-4799-8321-6

  36. Huk, M.: Using context-aware environment for elderly abuse prevention. In: Nguyen, N.T., Trawiński, B., Fujita, H., Hong, T.-P. (eds.) ACIIDS 2016. LNCS (LNAI), vol. 9622, pp. 567–574. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-662-49390-8_55

    Chapter  Google Scholar 

  37. Huk, M.: Context-related data processing with artificial neural networks for higher reliability of telerehabilitation systems. In: 17th International Conference on E-health Networking, Application & Services (HealthCom). IEEE Computer Society, Boston, USA, pp. 217–221 (2015)

    Google Scholar 

  38. Privitera, C.M., Azzariti, M., Stark, L.W.: Locating regions-of-interest for the Mars Rover expedition. Int. J. Remote Sens. 21, 3327–3347. Taylor and Francis (2000)

    Google Scholar 

  39. UCI Machine Learning Repository. https://archive.ics.uci.edu/ml

  40. Golub, T.R., et al.: Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286, 531–537 (1999)

    Article  Google Scholar 

  41. Glosser, C., Piermarocchi, C., Shanker, B.: Analysis of dense quantum dot systems using a self-consistent Maxwell-Bloch framework. In: Proceedings of 2016 IEEE International Symposium on Antennas and Propagation (USNC-URSI), Puerto Rico, pp. 1323–1324. IEEE (2016)

    Google Scholar 

  42. H2O.ai documentation. https://docs.h2o.ai/h2o/latest-stable/h2o-docs/index.html

  43. Rodriguez, J.D., et al.: Sensitivity analysis of k-fold cross validation in prediction error estimation. IEEE Trans. Patt. Anal. Mach. Int. 32(3), 569–575 (2010)

    Google Scholar 

  44. Bouckaert, R.R.: Estimating replicability of classifier learning experiments. In: Proceedings of the 21st International Conference on Machine Learning, Banf, Canada (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Zilina, Zilina, Slovakia

    Miroslava Mikusova

  2. Rubi Consulting, Bratislava, Slovakia

    Antonin Fuchs

  3. HSBC Digital Solutions, Hong Kong, China

    Adrian Karasiński

  4. Indian Institute of Technology Guwahati, Guwahati, Assam, India

    Rashmi Dutta Baruah

  5. Faculty of Computer Science and Management, Wroclaw University of Science and Technology, Wrocław, Poland

    Rafał Palak

  6. Science Applications International Corporation, Cookeville, USA

    Erik Dawid Burnell

  7. ALM Services Technology Group, Wroclaw, Poland

    Krzysztof Wołk

Authors
  1. Miroslava Mikusova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonin Fuchs
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adrian Karasiński
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rashmi Dutta Baruah
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rafał Palak
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Erik Dawid Burnell
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Krzysztof Wołk
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rafał Palak .

Editor information

Editors and Affiliations

  1. Wrocław University of Science and Technology, Wrocław, Poland

    Ngoc Thanh Nguyen

  2. King Mongkut's Institute of Technology Ladkrabang, Bangkok, Thailand

    Suphamit Chittayasothorn

  3. Nanyang Technological University, Singapore, Singapore

    Dusit Niyato

  4. Wrocław University of Science and Technology, Wrocław, Poland

    Bogdan Trawiński

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mikusova, M. et al. (2021). Towards Layer-Wise Optimization of Contextual Neural Networks with Constant Field of Aggregation. In: Nguyen, N.T., Chittayasothorn, S., Niyato, D., Trawiński, B. (eds) Intelligent Information and Database Systems. ACIIDS 2021. Lecture Notes in Computer Science(), vol 12672. Springer, Cham. https://doi.org/10.1007/978-3-030-73280-6_59

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-73280-6_59

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-73279-0

  • Online ISBN: 978-3-030-73280-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation