Skip to main content
SpringerLink
Log in
Book cover

Experimental Methods for the Analysis of Optimization Algorithms pp 161–184Cite as

Algorithm Survival Analysis

  • Chapter
  • First Online:

Abstract

Algorithm selection is typically based on models of algorithm performance,learned during a separate offline training sequence, which can be prohibitively expensive. In recent work, we adopted an online approach, in which models of the runtime distributions of the available algorithms are iteratively updated and used to guide the allocation of computational resources, while solving a sequence of problem instances. The models are estimated using survival analysis techniques, which allow us to reduce computation time, censoring the runtimes of the slower algorithms. Here, we review the statistical aspects of our online selection method, discussing the bias induced in the runtime distributions (RTD) models by the competition of different algorithms on the same problem instances.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Aalen O (1978) Nonparametric inference for a family of counting processes. Annals of Statistics 6:701–726

    Article  MATH  MathSciNet  Google Scholar 

  • Akritas M (1994) Nearest neighbor estimation of a bivariate distribution under random censoring. Annals of Statistics 22:1299–1327

    Article  MATH  MathSciNet  Google Scholar 

  • Auer P, Cesa-Bianchi N, Freund Y, Schapire RE (2003) The nonstochastic multiarmed bandit problem. SIAM Journal on Computing 32(1):48–77

    Article  MathSciNet  Google Scholar 

  • Beran R (1981) Nonparametric regression with randomly censored survival data. Tech. rep., University of California, Berkeley, CA

    Google Scholar 

  • Bishop CM (1995) Neural networks for pattern recognition. Oxford University Press

    Google Scholar 

  • Boddy M, Dean TL (1994) Deliberation scheduling for problem solving in timeconstrained environments. Artificial Intelligence 67(2):245–285

    Article  MATH  Google Scholar 

  • Collet D (2003) Modeling survival data in medical research. Chapman & Hall/CRC, Boca Raton

    Google Scholar 

  • Cox D (1972) Regression models and life-tables. Journal of the Royal Statistics Society, Series B 34:187–220

    MATH  Google Scholar 

  • Cox D, Oakes D (1984) Analysis of survival data. Chapman & Hall, London

    Google Scholar 

  • Finkelstein L, Markovitch S, Rivlin E (2002) Optimal schedules for parallelizing anytime algorithms: The case of independent processes. Tech. rep., CS Department, Technion, Haifa, Israel

    Google Scholar 

  • Finkelstein L, Markovitch S, Rivlin E (2003) Optimal schedules for parallelizing anytime algorithms: The case of shared resources. Journal of Artificial Intelligence Research 19:73–138

    MATH  MathSciNet  Google Scholar 

  • Fitzmaurice G, Davidian M, Verbeke G, Molenberghs G (2008) Longitudinal Data Analysis. Chapman & Hall/CRC

    Google Scholar 

  • Fleming T, Harrington D (1991) Counting processes and survival analysis. Wiley, New York, NY

    MATH  Google Scholar 

  • Frost D, Rish I, Vila L (1997) Summarizing CSP hardness with continuous probability distributions. In: Kuipers B, et al. (eds) Fourteenth National Conference on Artificial Intelligence, pp 327–333

    Google Scholar 

  • Gagliolo M, Schmidhuber J (2006a) Impact of censored sampling on the performance of restart strategies. In: Benhamou F (ed) Principles and Practice of Constraint Programming, Springer, pp 167–181

    Chapter  Google Scholar 

  • Gagliolo M, Schmidhuber J (2006b) Learning dynamic algorithm portfolios. Annals of Mathematics and Artificial Intelligence 47(3–4):295–328

    MATH  MathSciNet  Google Scholar 

  • Gagliolo M, Schmidhuber J (2007) Learning restart strategies. In: Veloso MM (ed) Twentieth International Joint Conference on Artificial Intelligence, vol 1, AAAI, pp 792–797

    Google Scholar 

  • Gagliolo M, Schmidhuber J (2008a) Algorithm selection as a bandit problem with unbounded losses. Tech. Rep. IDSIA - 07 - 08, IDSIA

    Google Scholar 

  • Gagliolo M, Schmidhuber J (2008b) Towards distributed algorithm portfolios. In: Corchado JM, et al. (eds) International Symposium on Distributed Computing and Artificial Intelligence (DCAI 2008), Springer, pp 634–643

    Google Scholar 

  • Gent I, Walsh T (1999) The search for satisfaction. Tech. rep., Dept. of Computer Science, University of Strathclyde

    Google Scholar 

  • Gomes CP, Selman B (2001) Algorithm portfolios. Artificial Intelligence 126(1–2):43–62

    Article  MATH  MathSciNet  Google Scholar 

  • Gomes CP, Selman B, Kautz H (1998) Boosting combinatorial search through randomization. In: Mostow J, et al. (eds) Fifteenth National Conference on Artificial Intelligence, pp 431–437

    Google Scholar 

  • Gomes CP, Selman B, Crato N, Kautz H (2000) Heavy-tailed phenomena in satisfiability and constraint satisfaction problems. Journal of Automated Reasoning 24(1–2):67–100

    Article  MATH  MathSciNet  Google Scholar 

  • Hansen EA, Zilberstein S (2001) Monitoring and control of anytime algorithms: A dynamic programming approach. Artificial Intelligence 126(1–2):139–157

    Article  MATH  MathSciNet  Google Scholar 

  • Hogg T, Williams CP (1994) The hardest constraint problems: a double phase transition. Artificial Intelligence 69(1–2):359–377

    Article  MATH  Google Scholar 

  • Hoos HH (1999) On the run-time behaviour of stochastic local search algorithms for SAT. In: Hendler J, et al. (eds) Sixteenth National Conference on Artificial Intelligence, pp 661–666

    Google Scholar 

  • Hoos HH (2002) A mixture-model for the behaviour of SLS algorithms for SAT In: Hendler JA (ed) Eighteenth National Conference on Artificial Intelligence, pp 661–667

    Google Scholar 

  • Hoos HH, Stützle T (2000) SATLIB: An online resource for research on SAT. In: Gent I, et al. (eds) SAT 2000 — Highlights of Satisfiability Research in the Year 2000, IOS, pp 283–292

    Google Scholar 

  • Hoos HH, Stützle T (2004) Stochastic Local Search : Foundations & Applications. Morgan Kaufmann

    Google Scholar 

  • Huberman BA, Lukose RM, Hogg T (1997) An economic approach to hard computational problems. Science 27:51–53

    Article  Google Scholar 

  • Ibrahim JG, Chen MH, Sinha D (2001) Bayesian Survival Analysis. Springer

    MATH  Google Scholar 

  • Kaplan EL, Meyer P (1958) Nonparametric estimation from incomplete samples. Journal of the American Statistical Association 73:457–481

    Article  Google Scholar 

  • Keilegom IV, Akritas M, Veraverbeke N (2001) Estimation of the conditional distribution in regression with censored data: a comparative study. Computational Statistics and Data Analysis 35:487–500

    Article  MATH  MathSciNet  Google Scholar 

  • Klein JP, Moeschberger ML (2003) Survival Analysis: Techniques for Censored and Truncated Data, 2nd edn. Springer

    MATH  Google Scholar 

  • Leyton-Brown K, Nudelman E, Shoham Y (2002) Learning the empirical hardness of optimization problems: The case of combinatorial auctions. In: Van Hentenryck P (ed) Principles and Practice of Constraint Programming, pp 91–100

    Google Scholar 

  • Li CM, Anbulagan (1997) Heuristics based on unit propagation for satisfiability problems. In: Georgeff MP, et al. (eds) Fifteenth International Joint Conference on Artificial Intelligence, pp 366–371

    Google Scholar 

  • Li CM, Huang W (2005) Diversification and determinism in local search for satisfiability In: Bacchus F, et al. (eds) Theory and Applications of Satisfiability Testing, Springer, pp 158–172

    Chapter  Google Scholar 

  • Li G, Doss H (1995) An approach to nonparametric regression for life history data using local linear fitting. Annals of Statistics 23:787–823

    Article  MATH  MathSciNet  Google Scholar 

  • Liang K, Self S, Bandeen-Roche K, Zeger S (1995) Some recent developments for regression analysis of multivariate failure time data. Lifetime Data Analysis 1:403–415

    Article  MATH  Google Scholar 

  • Luby M, Sinclair A, Zuckerman D (1993) Optimal speedup of las vegas algorithms. Information Processing Letters 47(4):173–180

    Article  MATH  MathSciNet  Google Scholar 

  • Machin D, Cheung Y, Parmar M (2006) Survival Analysis. A Practical Approach. Wiley, UK, second edition

    Book  MATH  Google Scholar 

  • Mackay DC (2002) Information Theory, Inference and Learning Algorithms. Cambridge University Press

    Google Scholar 

  • Mitchell D, Selman B, Levesque H (1992) Hard and easy distributions of SAT problems. In: Swartout W (ed) Tenth National Conference on Artificial Intelligence, pp 459–465

    Google Scholar 

  • Nelson W (1972) Theory and applications of hazard plotting for censored failure data. Technometrics 14:945–965

    Article  Google Scholar 

  • Nelson W (1982) Applied Life Data Analysis. Wiley, New York, NY

    Book  MATH  Google Scholar 

  • Nielsen J, Linton O (1995) Kernel estimation in a nonparametric marker dependent hazard model. Annals of Statistics 23:1735–1748

    Article  MATH  MathSciNet  Google Scholar 

  • Nudelman E, Brown KL, Hoos HH, Devkar A, Shoham Y (2004) Understanding random SAT: Beyond the clauses-to-variables ratio. In:Wallace M(ed) Principles and Practice of Constraint Programming, Springer, pp 438–452

    Google Scholar 

  • Pintilie M (2006) Competing Risks. A practical perspective. Wiley, New York, NY

    Google Scholar 

  • Putter H, Fiocco M, Geskus R (2006) Tutorial in biostatistics: Competing risks and multi-state models. Statistics in Medicine 26:2389–2430

    Article  MathSciNet  Google Scholar 

  • Rice JR (1976) The algorithm selection problem. In: Rubinoff M, Yovits MC (eds) Advances in computers, vol 15, Academic, New York, pp 65–118

    Google Scholar 

  • Spierdijk L (2005) Nonparametric conditional hazard rate estimation: a local linear approach. Tech. Rep. TW Memorandum, University of Twente

    Google Scholar 

  • Tsang E (1993) Foundations of Constraint Satisfaction. Academic, London and San Diego

    Google Scholar 

  • Tsiatis A (1975) A nonidentifiability aspect of the problem of competing risks. Proceedings of the National Academy of Sciences of the USA 72(1):20–22

    Article  MATH  MathSciNet  Google Scholar 

  • Vilalta R, Drissi Y (2002) A perspective view and survey of meta-learning. Artificial Intelligence Review 18(2):77–95

    Article  Google Scholar 

  • Wang JL (2005) Smoothing hazard rate. In: Armitage P, et al. (eds) Encyclopedia of Biostatistics, 2nd Edition, vol 7, Wiley, pp 4986–4997

    Google Scholar 

  • Wichert L, Wilke RA (2005) Application of a simple nonparametric conditional quantile function estimator in unemployment duration analysis. Tech. Rep. ZEW Discussion Paper No. 05-67, Centre for European Economic Research

    Google Scholar 

  • Xu L, Hutter F, Hoos HH, Leyton-Brown K (2008) SATzilla: Portfolio-based algorithm selection for SAT. Journal of Artificial Intelligence Research 32 (2008) 565–606 32:565–606

    MATH  Google Scholar 

Download references

Acknowledgements

The authors would like to thank an anonymous reviewer, and Faustino Gomez, for useful comments on a draft of this chapter. The first author was supported by the Swiss National Science Foundation with a grant for prospective researchers (n. PBTI2–118573).

Author information

Authors and Affiliations

  1. IRIDIA, CoDE, Université Libre de Bruxelles, Brussels, Belgium

    Matteo Gagliolo

  2. Faculty of Informatics, University of Lugano, Lugano, Switzerland

    Matteo Gagliolo

  3. Institut de Statistique, Université Catholique de Louvain, Louvain-la-Neuve, Belgium

    Catherine Legrand

Authors
  1. Matteo Gagliolo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Catherine Legrand
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Matteo Gagliolo or Catherine Legrand .

Editor information

Editors and Affiliations

  1. Institute of Computer Science, Faculty of Computer Science, Cologne University of Applied Sciences, Steinmüllerallee 1, Gummersbach, 51643, Germany

    Thomas Bartz-Beielstein

  2. , Department of Mathematics, University of Southern Denmark, Campusvej 55, Odense, 5230, Denmark

    Marco Chiarandini

  3. CISUC, Department of Informatics Engineering, University of Coimbra, Coimbra, 3030-290, Portugal

    Luís Paquete

  4. Algorithm Engineering, TU Dortmund, Otto-Hahn-Str. 14, Dortmund, 44227, Germany

    Mike Preuss

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Gagliolo, M., Legrand, C. (2010). Algorithm Survival Analysis. In: Bartz-Beielstein, T., Chiarandini, M., Paquete, L., Preuss, M. (eds) Experimental Methods for the Analysis of Optimization Algorithms. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02538-9_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-02538-9_7

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-02537-2

  • Online ISBN: 978-3-642-02538-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation