Skip to main content
SpringerLink
Log in

An efficient and scalable variational quantum circuits approach for deep reinforcement learning

  • Published:
Quantum Information Processing Aims and scope Submit manuscript

Abstract

Nowadays, machine learning techniques are successfully applied to many problems in industrial and academic fields with classical computers. With the introduction of quantum simulators, the idea of using quantum-computing speed to solve these problems has become widespread. Many researchers are experimenting with using quantum circuits in various machine-learning methods to solve different problems. Due to the limited number of qubits, experiments are on simpler problems. In this study, a variational quantum circuit (VQC) was proposed using amplitude encoding to overcome the limited qubit number barrier and use the advantages of quantum computing more efficiently. The proposed amplitude encoding method and VQC were explained. Generalized by exemplifying how they can be applied to different problems. The proposed approach was applied to a navigation problem. The performance of the proposed approach was evaluated with the number of parameters, the number of qubits needed, and the success rate. As a result, the performance of the proposed approach has been verified.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data Availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Lecun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015). https://doi.org/10.1038/nature14539

    Article  ADS  Google Scholar 

  2. Francois-Lavet, V., Henderson, P., Islam, R., Bellemare, M.G., Pineau, J.: An introduction to deep reinforcement learning. Found Trends Mach Learn. 11(3–4), 219–354 (2018). https://doi.org/10.1561/2200000071

    Article  MATH  Google Scholar 

  3. Kwak Y., Yun W. J., Jung S., Kim J.: Quantum neural networks: concepts, applications, and challenges, (2021)

  4. Chen S. Y.-C., Yang C.-H. H., Qi J., Chen P.-Y., Ma X., Goan H.-S.: Variational quantum circuits for deep reinforcement learning. Jun. 2019.

  5. Hernández H. I. G., Ruiz R. T., and Sun G.-H.: Image classification via quantum machine learning. (2020)

  6. Lockwood O., and Si M.: Reinforcement learning with quantum variational circuits. (2020).

  7. Kimura, T., Shiba, K., Chen, C.-C., Sogabe, M., Sakamoto, K., Sogabe, T.: Variational quantum circuit-based reinforcement learning for POMDP and experimental implementation. Math Probl Eng 2021, 1–11 (2021). https://doi.org/10.1155/2021/3511029

    Article  Google Scholar 

  8. Moro, L., Paris, M.G.A., Restelli, M., Prati, E.: Quantum compiling by deep reinforcement learning. Commun Phys 4(1), 178 (2021). https://doi.org/10.1038/s42005-021-00684-3

    Article  Google Scholar 

  9. Kwak Y., Yun W. J., Jung S., Kim J.-K., Kim J.: Introduction to quantum reinforcement learning: theory and pennylane-based implementation. 2021.

  10. Maheshwari, D., Sierra-Sosa, D., Garcia-Zapirain, B.: Variational quantum classifier for binary classification: real vs synthetic dataset. IEEE Access 10, 3705–3715 (2022). https://doi.org/10.1109/ACCESS.2021.3139323

    Article  Google Scholar 

  11. Bar N. F., Yetis H., Karakose M.: An approach based on quantum reinforcement learning for navigation problems. in 2022 International Conference on Data Analytics for Business and Industry, ICDABI 2022, Institute of Electrical and Electronics Engineers Inc., 2022, pp. 593–597. doi: https://doi.org/10.1109/ICDABI56818.2022.10041570.

  12. Bar N. F., Yetis H., Karakose M.: A quantum-classical hybrid classifier using multi-encoding method for images. in 2023 27th International Conference on Information Technology (IT), IEEE, Feb. 2023, pp. 1–4. doi: https://doi.org/10.1109/IT57431.2023.10078617.

  13. TensorFlow, “Tensorflow Quantum.” https://www.tensorflow.org/quantum (accessed Mar. 21, 2022).

  14. Pennylane, “Pennylane.” https://pennylane.ai (accessed Mar. 21, 2023).

  15. IBM Quantum, “IBM quantum.” https://quantum-computing.ibm.com (accessed Mar. 21, 2022).

  16. Amazon Braket, “Amazon Quantum Computing Service.” https://aws.amazon.com/braket/ (accessed Mar. 21, 2022).

  17. Bar N. F., Yetis H., and Karakose M.: Deep reinforcement learning approach with adaptive reward system for robot navigation in dynamic environments. in Interdisciplinary Research in Technology and Management, London: CRC Press, 2021, pp. 349–355. doi: https://doi.org/10.1201/9781003202240-55.

  18. Bar, N.F., Karakose, M.: Multi-robot navigation in unknown environment based on deep reinforcement learning. Fırat Üniversitesi Mühendislik Bilimleri Dergisi (2022). https://doi.org/10.35234/fumbd.1122947

    Article  Google Scholar 

  19. OpenAI, “A toolkit for developing and comparing reinforcement learning algorithms.” https://gym.openai.com/ (accessed Mar. 21, 2022).

  20. Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction. IEEE Trans Neural Netw 9(5), 1054–1054 (1998). https://doi.org/10.1109/TNN.1998.712192

    Article  Google Scholar 

  21. Tokic M.: Adaptive ε-greedy exploration in reinforcement learning based on value differences. 2010, pp 203–210. doi: https://doi.org/10.1007/978-3-642-16111-7_23.

  22. Yetis H., Karakose M.: A new framework for quantum image processing and application of binary template matching. in 2022 26th International Conference on Information Technology (IT), IEEE, 2022, pp. 1–4. doi: https://doi.org/10.1109/IT54280.2022.9743534.

Download references

Acknowledgements

This article work was carried out within the scope of Niyazi Furkan Bar’s master’s thesis named “Automated Generation of Quantum Computing Models Using Deep Learning.” The supervisor of the thesis is Mehmet Karaköse.

Funding

This study was supported by the TUBITAK (The Scientific and Technological Research Council of Turkey) under Grant No: 121E439.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Firat University, Elazig, Turkey

    Niyazi Furkan Bar, Hasan Yetis & Mehmet Karakose

Authors
  1. Niyazi Furkan Bar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hasan Yetis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mehmet Karakose
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors wrote the main manuscript text and reviewed the manuscript.

Corresponding author

Correspondence to Niyazi Furkan Bar.

Ethics declarations

Conflict of interests

The authors declare that they have no competing interests.

Ethical approval and Consent to participate

Not Applicable.

Consent for publication

Not Applicable.

Human and animal ethics

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bar, N.F., Yetis, H. & Karakose, M. An efficient and scalable variational quantum circuits approach for deep reinforcement learning. Quantum Inf Process 22, 300 (2023). https://doi.org/10.1007/s11128-023-04051-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11128-023-04051-9

Keywords

Search

Navigation