Skip to main content
Springer Nature Link
Log in

A Quantum 3D Convolutional Neural Network with Application in Video Classification

  • Conference paper
  • First Online:
Advances in Visual Computing (ISVC 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 13017))

Included in the following conference series:

  • 1512 Accesses

Abstract

Quantum computing seeks to exploit the properties of quantum mechanics to perform computations at a fraction of the cost compared to the classical computing methods. Recently, quantum methods for machine learning have attracted the interest of researchers. Those methods aim to exploit, in the context of machine learning, the potential benefits that the quantum computers should be able to offer in the near future. A particularly interesting area of research in this direction, investigates the union of quantum machine learning models with Convolutional Neural Networks. In this paper we develop a quantum counterpart of a 3D Convolutional Neural Network for video classification, dubbed Q3D-CNN. This is the first approach for quantum video classification we are aware of.

Our model is based on previously proposed quantum machine learning models, where manipulation of the input data is performed in such a way that a fully quantum-mechanical neural network layer can be realized and used to form a Quantum Convolutional Neural Network. We augment this approach by introducing quantum-friendly operations during data-loading and appropriately manipulating the quantum network. We demonstrate the applicability of the proposed Q3D-CNN in video classification using videos from a publicly available dataset. We successfully classify the test dataset using two and three classes using the quantum network and its classical counterpart.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Adcock, J.,et al.: Advances in quantum machine learning. arXiv:1512.02900 December 2015

  2. Allcock, J., Hsieh, C.Y., Kerenidis, I., Zhang, S.: Quantum Algorithms for Feedforward Neural Networks. ACM Trans. Quant. Comput. 1(1), 6:1–6:24 (2020). https://doi.org/10.1145/3411466

  3. Allcock, J., Zhang, S.: Quantum machine learning. Nat. Sci. Rev. 6(1), 26–28 (2019). https://doi.org/10.1093/nsr/nwy149

    Article  Google Scholar 

  4. Behrman, E.C., Nash, L.R., Steck, J.E., Chandrashekar, V.G., Skinner, S.R.: Simulations of quantum neural networks. Inf. Sci. 128(3), 257–269 (2000). https://doi.org/10.1016/S0020-0255(00)00056-6

    Article  MathSciNet  MATH  Google Scholar 

  5. Cerezo, M., et al.: Variational quantum algorithms. arXiv:2012.09265 (2020)

  6. Chatzis, S.P., Kosmopoulos, D.: A nonparametric bayesian approach toward stacked convolutional independent component analysis. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), December 2015

    Google Scholar 

  7. Cong, I., Choi, S., Lukin, M.D.: Quantum convolutional neural networks. Nat. Phys. 15(12), 1273–1278 (2019). https://doi.org/10.1038/s41567-019-0648-8

    Article  Google Scholar 

  8. Dang, Y., Jiang, N., Hu, H., Ji, Z., Zhang, W.: Image classification based on quantum k-nearest-neighbor algorithm. Quantum Inf. Process. 17(9), 1–18 (2018). https://doi.org/10.1007/s11128-018-2004-9

    Article  MATH  Google Scholar 

  9. Garg, S., Ramakrishnan, G.: Advances in quantum deep learning: an overview. arXiv:2005.04316 May 2020

  10. Gawron, P., Lewiński, S.: Multi-spectral image classification with quantum neural network. In: IGARSS 2020–2020 IEEE International Geoscience and Remote Sensing Symposium, pp. 3513–3516, September 2020. https://doi.org/10.1109/IGARSS39084.2020.9323065

  11. Henderson, M., Shakya, S., Pradhan, S., Cook, T.: Quanvolutional neural networks: powering image recognition with quantum circuits. Quantum Mach. Intell. 2(1), 1–9 (2020). https://doi.org/10.1007/s42484-020-00012-y

    Article  Google Scholar 

  12. Hernández, H.I.G., Ruiz, R.T., Sun, G.H.: Image classification via quantum machine learning. arXiv:2011.02831 December 2020

  13. Jeswal, S.K., Chakraverty, S.: Recent developments and applications in quantum neural network: a review. Arch. Comput. Methods Eng. 26(4), 793–807 (2018). https://doi.org/10.1007/s11831-018-9269-0

    Article  MathSciNet  Google Scholar 

  14. Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L.: Large-scale video classification with convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2014

    Google Scholar 

  15. Kerenidis, I., Landman, J., Prakash, A.: Quantum algorithms for deep convolutional neural networks. In: International Conference on Learning Representations, September 2019

    Google Scholar 

  16. Kulkarni, V., Kulkarni, M., Pant, A.: Quantum computing methods for supervised learning. arXiv:2006.12025 June 2020

  17. Lockwood, O., Si, M.: Reinforcement learning with quantum variational circuits. arXiv:2008.07524 August 2020

  18. Materzynska, J., Berger, G., Bax, I., Memisevic, R.: The jester dataset: A large-scale video dataset of human gestures. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 2874–2882. IEEE Computer Society (2019)

    Google Scholar 

  19. Nguyen, N.T., Kenyon, G.T.: Image classification using quantum inference on the d-wave 2x. In: 2018 IEEE International Conference on Rebooting Computing (ICRC), pp. 1–7, November 2018. https://doi.org/10.1109/ICRC.2018.8638596

  20. Niu, X.F., Ma, W.P.: A novel quantum neural network based on multi- level activation function. Laser Phys. Lett. 18(2), 025201 (2021). https://doi.org/10.1088/1612-202X/abd23c

    Article  Google Scholar 

  21. Oh, S., Choi, J., Kim, J.: A tutorial on quantum convolutional neural networks (QCNN). arXiv:2009.09423 September 2020

  22. Perdomo-Ortiz, A., Benedetti, M., Realpe-Gómez, J., Biswas, R.: Opportunities and challenges for quantum-assisted machine learning in near-term quantum computers. Quantum Sci. Technol. 3(3), 030502 (2018). https://doi.org/10.1088/2058-9565/aab859

    Article  Google Scholar 

  23. Schuld, M., Sinayskiy, I., Petruccione, F.: Simulating a perceptron on a quantum computer. Phys. Lett. A 379(7), 660–663 (2015). https://doi.org/10.1016/j.physleta.2014.11.061

    Article  MATH  Google Scholar 

  24. Tacchino, F., Barkoutsos, P., Macchiavello, C., Tavernelli, I., Gerace, D., Bajoni, D.: Quantum implementation of an artificial feed-forward neural network. Quantum Sci. Technol. 5(4), 044010 (2020). https://doi.org/10.1088/2058-9565/abb8e4

    Article  Google Scholar 

  25. Tacchino, F., Barkoutsos, P.K., Macchiavello, C., Gerace, D., Tavernelli, I., Bajoni, D.: Variational learning for quantum artificial neural networks. In: 2020 IEEE International Conference on Quantum Computing and Engineering (QCE), pp. 130–136, October 2020. https://doi.org/10.1109/QCE49297.2020.00026

  26. Tran, D., Wang, H., Torresani, L., Feiszli, M.: Video classification with channel-separated convolutional networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019

    Google Scholar 

  27. Ullah, A., Ahmad, J., Muhammad, K., Sajjad, M., Baik, S.W.: Action recognition in video sequences using deep bi-directional LSTM with CNN features. IEEE Access 6, 1155–1166 (2018). https://doi.org/10.1109/ACCESS.2017.2778011

    Article  Google Scholar 

  28. Wan, K.H., Dahlsten, O., Kristjánsson, H., Gardner, R., Kim, M.S.: Quantum generalisation of feedforward neural networks. npj Quantum Inf. 3(1), 1–8 (2017). https://doi.org/10.1038/s41534-017-0032-4

  29. Wu, Z., Wang, X., Jiang, Y.G., Ye, H., Xue, X.: Modeling spatial-temporal clues in a hybrid deep learning framework for video classification. In: Proceedings of the 23rd ACM International Conference on Multimedia, MM 2015, pp. 461–470. Association for Computing Machinery, New York (2015). https://doi.org/10.1145/2733373.2806222

  30. Xie, S., Sun, C., Huang, J., Tu, Z., Murphy, K.: Rethinking spatiotemporal feature learning: speed-accuracy trade-offs in video classification. In: Proceedings of the European Conference on Computer Vision (ECCV), September 2018

    Google Scholar 

  31. Yue-Hei Ng, J., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., Toderici, G.: Beyond short snippets: deep networks for video classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015

    Google Scholar 

  32. Zhou, N.-R., Liu, X.-X., Chen, Y.-L., Du, N.-S.: Quantum k-nearest-neighbor image classification algorithm based on K-L transform. Int. J. Theoret. Phys. 60(3), 1209–1224 (2021). https://doi.org/10.1007/s10773-021-04747-7

    Article  MATH  Google Scholar 

  33. Zhou, R.: Quantum competitive neural network. Int. J. Theoret. Phys. 49(1), 110 (2009). https://doi.org/10.1007/s10773-009-0183-y

    Article  MATH  Google Scholar 

  34. Zhou, R., Ding, Q.: Quantum M-P neural network. Int. J. Theoret. Phys. 46(12), 3209–3215 (2007). https://doi.org/10.1007/s10773-007-9437-8

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgment

This research has been co-financed by the European Regional Development Fund of the European Union and Greek national funds through the Operational Program “Competitiveness, Entrepreneurship and Innovation”, under the call “RESEARCH - CREATE - INNOVATE” (project code:T2EDK-00982).

Author information

Authors and Affiliations

  1. University of Patras, University Campus, 26504, Rion, Achaia, Greece

    Kostas Blekos & Dimitrios Kosmopoulos

Authors
  1. Kostas Blekos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dimitrios Kosmopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kostas Blekos .

Editor information

Editors and Affiliations

  1. University of Nevada, Reno, NV, USA

    George Bebis

  2. University of Texas at Arlington, Arlington, TX, USA

    Vassilis Athitsos

  3. University of South Carolina, Columbia, SC, USA

    Tong Yan

  4. City University of Hong Kong, Kowloon, Hong Kong

    Manfred Lau

  5. School of Engineering and Computing, University of Durham, Durham, Durham, UK

    Frederick Li

  6. Airbnb, New York, NY, USA

    Conglei Shi

  7. Peking University, Beijing, China

    Xiaoru Yuan

  8. Purdue University, West Lafayette, IN, USA

    Christos Mousas

  9. IST, School of Modeling, Simulation, and Training, Orlando, FL, USA

    Gerd Bruder

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Blekos, K., Kosmopoulos, D. (2021). A Quantum 3D Convolutional Neural Network with Application in Video Classification. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2021. Lecture Notes in Computer Science(), vol 13017. Springer, Cham. https://doi.org/10.1007/978-3-030-90439-5_47

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-90439-5_47

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-90438-8

  • Online ISBN: 978-3-030-90439-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics