Abstract
The growing availability of quantum computers raises questions about their ability to solve concrete problems. Existing benchmark protocols still lack problem diversity and attempt to summarize quantum advantage in a single metric that measures the quality of found solutions. Unfortunately, the solution quality metric is insufficient for measuring quantum algorithm performance and should be presented along with time and instances coverage metrics. This paper aims to establish the TAQOS protocol to perform a Tight Analysis of Quantum Optimization Systems. The combination of metrics considered by this protocol helps to identify problems and instances liable to produce quantum advantage on Noisy-Intermediate Scale Quantum (NISQ) devices for useful applications. The methodology used for the benchmark process is detailed and an illustrative short case study on the Max-Cut problem is provided.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
D-wave systems, d-wave ocean sdk, release 6.0.1 (2022). https://github.com/dwavesystems/dwave-ocean-sdk
Albash, T., Lidar, D.A.: Demonstration of a scaling advantage for a quantum annealer over simulated annealing. Phys. Rev. X 8(3), 031016 (2018)
Barr, R.S., Golden, B.L., Kelly, J.P., Resende, M.G.C., Stewart, W.R.: Designing and reporting on computational experiments with heuristic methods. J. Heuristics 1(1), 9–32 (1995)
Basso, J., et al.: The quantum approximate optimization algorithm at high depth for maxcut on large-girth regular graphs and the sherrington-kirkpatrick model. arXiv preprint arXiv:2110.14206 (2021)
Blume-Kohout, R., Young, K.C.: A volumetric framework for quantum computer benchmarks. Quantum 4, 362 (2020)
Cross, A.W., Bishop, L.S., Sheldon, S., Nation, P.D., Gambetta, J.M.: Validating quantum computers using randomized model circuits. Phys. Rev. A 100, 032328 (2019)
Dunning, I., Gupta, S., Silberholz, J.: What works best when? a systematic evaluation of heuristics for max-cut and QUBO. INFORMS J. Comput. 30(3) (2018)
Farhi, E., Goldstone, J., Gutmann, S.: A quantum approximate optimization algorithm. arXiv preprint arXiv:1411.4028 (2014)
Fellous Asiani, M., et al.: Optimizing resource efficiencies for scalable full-stack quantum computers. arXiv preprint arXiv:2209.05469 (2022)
Gilbert, V.: Taqos (2023). https://github.com/CEA-LIST/Quantum-Benchmark-CEA-LIST/tree/main/TAQOS
Harrigan, M.P., et al.: Quantum approximate optimization of non-planar graph problems on a planar superconducting processor. Nat. Phys. 17(3), 332–336 (2021)
Hen, I., Job, J., Albash, T., Rønnow, T.F., et al.: Probing for quantum speedup in spin-glass problems with planted solutions. Phys. Rev. A 92, 042325 (2015)
Knill, E., et al.: Randomized benchmarking of quantum gates. Phys. Rev. A 77, 012307 (2008)
Li, A., Stein, S., Krishnamoorthy, S., Ang, J.: Qasmbench: A low-level qasm benchmark suite for nisq evaluation and simulation. arXiv preprint arXiv:2005.13018 (2020)
Lilja, D.J.: Measuring computer performance: a practitioner’s guide. Cambridge University Press (2005)
Lubinski, T., Coffrin, C., McGeoch, C., Sathe, et al.: Optimization applications as quantum performance benchmarks. arXiv preprint arXiv:2302.02278 (2023)
Magesan, E., Gambetta, J.M., Emerson, J.: Scalable and robust randomized benchmarking of quantum processes. Phys. Rev. Lett. 106, 180504 (2011)
Martiel, S., Ayral, T., Allouche, C.: Benchmarking quantum coprocessors in an application-centric, hardware-agnostic, and scalable way. IEEE Trans. Quantum Eng. 2, 1–11 (2021)
Oshiyama, H., Ohzeki, M.: Benchmark of quantum-inspired heuristic solvers for quadratic unconstrained binary optimization. Scient. Reports 12(1) (2022)
Preskill, J.: Quantum computing in the nisq era and beyond. Quantum 2, 79 (2018)
Rice, J.R.: The algorithm selection problem. In: Advances in Computers, pp. 65–118. Elsevier (1976)
Smith-Miles, K., Baatar, D., Wreford, B., Lewis, R.: Towards objective measures of algorithm performance across instance space. Comput. Oper. Res. 45, 12–24 (2014)
Xue, C., Chen, Z.Y., Wu, Y.C., Guo, G.P.: Effects of quantum noise on quantum approximate optimization algorithm. Chin. Phys. Lett. 38(3), 030302 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gilbert, V., Louise, S., Sirdey, R. (2023). TAQOS: A Benchmark Protocol for Quantum Optimization Systems. In: Mikyška, J., de Mulatier, C., Paszynski, M., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M. (eds) Computational Science – ICCS 2023. ICCS 2023. Lecture Notes in Computer Science, vol 10477. Springer, Cham. https://doi.org/10.1007/978-3-031-36030-5_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-36030-5_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36029-9
Online ISBN: 978-3-031-36030-5
eBook Packages: Computer ScienceComputer Science (R0)