Skip to main content
SpringerLink
Log in

Automata, Probability, and Recursion

  • Conference paper
Book cover Implementation and Applications of Automata (CIAA 2008)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 5148))

Included in the following conference series:

  • 461 Accesses

Abstract

We discuss work on the modeling and analysis of systems with probabilistic and recursive features. Recursive Markov chains extend ordinary finite state Markov chains with the ability to invoke other Markov chains in a potentially recursive manner. The equivalent model of Probabilistic Pushdown Automata extends ordinary pushdown automata with probabilistic actions. Both of these are natural abstract models for probabilistic programs with procedures, and related systems. They generalize other classical well-studied stochastic models, e.g. Stochastic Context-free Grammars and (Multi-type) Branching Processes, that arise in a variety of areas. More generally, Recursive Markov Decision Processes and Recursive Stochastic Games can be used to model recursive systems that have both probabilistic and nonprobabilistic, controllable actions. In recent years there has been substantial work on the algorithmic analysis of these models, regarding basic questions of termination, reachability, and analysis of the properties of their executions. In this talk we will present some of the basic theory, algorithmic methods, results, and challenges.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abney, S., McAllester, D., Pereira, F.: Relating probabilistic grammars and automata. In: Proc. 37th Ann. Meeting of Ass. for Comp. Linguistics, pp. 542–549. Morgan Kaufmann, San Francisco (1999)

    Chapter  Google Scholar 

  2. Allender, E., Bürgisser, P., Kjeldgaard-Pedersen, J., Miltersen, P.B.: On the complexity of numerical analysis. In: 21st IEEE Computational Complexity Conference (2006)

    Google Scholar 

  3. Alur, R., Yannakakis, M.: Model checking of hierarchical state machines. ACM Trans. Prog. Lang. Sys. 23(3), 273–303 (2001)

    Article  Google Scholar 

  4. Alur, R., Benedikt, M., Etessami, K., Godefroid, P., Reps, T.W., Yannakakis, M.: Analysis of recursive state machines. ACM Trans. Progr. Lang. Sys. 27, 786–818 (2005)

    Article  Google Scholar 

  5. Bini, D., Latouche, G., Meini, B.: Numerical methods for Structured Markov Chains. Oxford University Press, Oxford (2005)

    MATH  Google Scholar 

  6. Bouajjani, A., Esparza, J., Maler, O.: Reachability analysis of pushdown automata: Applications to model checking. In: Mazurkiewicz, A., Winkowski, J. (eds.) CONCUR 1997. LNCS, vol. 1243, pp. 135–150. Springer, Heidelberg (1997)

    Google Scholar 

  7. Brázdil, T., Brozek, V., Forejt, V., Kučera, A.: Reachability in recursive Markov decision processes. In: Baier, C., Hermanns, H. (eds.) CONCUR 2006. LNCS, vol. 4137, pp. 358–374. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  8. Brázdil, T., Kučera, A., Esparza, J.: Analysis and prediction of the long-run behavior of probabilistic sequential programs with recursion. In: Proc. of FOCS 2005, pp. 521–530 (2005)

    Google Scholar 

  9. Brázdil, T., Kučera, A., Stražovský, O.: Decidability of temporal properties of probabilistic pushdown automata. In: Diekert, V., Durand, B. (eds.) STACS 2005. LNCS, vol. 3404. Springer, Heidelberg (2005)

    Google Scholar 

  10. Canny, J.: Some algebraic and geometric computations in PSPACE. In: Proc. of 20th ACM STOC, pp. 460–467 (1988)

    Google Scholar 

  11. Condon, A.: The complexity of stochastic games. Inf. & Comp. 96(2), 203–224 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  12. Courcoubetis, C., Yannakakis, M.: The complexity of probabilistic verification. Journal of the ACM 42(4), 857–907 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  13. Courcoubetis, C., Yannakakis, M.: Markov decision processes and regular events. IEEE Trans. on Automatic Control 43(10), 1399–1418 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  14. de Alfaro, L., Majumdar, R.: Quantitative solution of omega-regular games. J. Comp. Sys. Sc. 68(2), 374–397 (2004)

    Article  MATH  Google Scholar 

  15. Durbin, R., Eddy, S.R., Krogh, A., Mitchison, G.: Biological Sequence Analysis: Probabilistic models of Proteins and Nucleic Acids. Cambridge U. Press (1999)

    Google Scholar 

  16. Esparza, J., Gawlitza, T., Kiefer, S., Seidl, H.: Approximative methods for motonone systems of min-max-polynomial equations. In: Proc. 35th ICALP (2008)

    Google Scholar 

  17. Esparza, J., Hansel, D., Rossmanith, P., Schwoon, S.: Efficient algorithms for model checking pushdown systems. In: Emerson, E.A., Sistla, A.P. (eds.) CAV 2000. LNCS, vol. 1855, pp. 232–247. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  18. Esparza, J., Kiefer, S., Luttenberger, M.: Convergence thresholds of Newton’s method for monotone polynomial equations. In: Proc. STACS (2008)

    Google Scholar 

  19. Esparza, J., Kučera, A., Mayr, R.: Model checking probabilistic pushdown automata. In: Proc. of 19th IEEE LICS 2004 (2004); Full version in Logical Methods in Computer Science 2(1) (2006)

    Google Scholar 

  20. Esparza, J., Kučera, A., Mayr, R.: Quantitative analysis of probabilistic pushdown automata: expectations and variances. In: Proc. of 20th IEEE LICS (2005)

    Google Scholar 

  21. Etessami, K., Wojtczak, D., Yannakakis, M.: Recursive Stochastic Games with Positive Rewards. In: Proc. 35th ICALP (2008)

    Google Scholar 

  22. Etessami, K., Wojtczak, D., Yannakakis, M.: Quasi-birth-death processes, tree-like QBDs, probabilistic 1-counter automata, and pushdown systems (submitted, 2008)

    Google Scholar 

  23. Etessami, K., Yannakakis, M.: Recursive Markov chains, stochastic grammars, and monotone systems of non-linear equations. In: Diekert, V., Durand, B. (eds.) STACS 2005. LNCS, vol. 3404, pp. 340–352. Springer, Heidelberg (2005), http://homepages.inf.ed.ac.uk/kousha/bib_index.html

    Google Scholar 

  24. Etessami, K., Yannakakis, M.: Algorithmic verification of recursive probabilistic state machines. In: Halbwachs, N., Zuck, L.D. (eds.) TACAS 2005. LNCS, vol. 3440, pp. 253–270. Springer, Heidelberg (2005)

    Google Scholar 

  25. Etessami, K., Yannakakis, M.: Recursive Markov Decision Processes and Recursive Stochastic Games. In: Caires, L., Italiano, G.F., Monteiro, L., Palamidessi, C., Yung, M. (eds.) ICALP 2005. LNCS, vol. 3580, pp. 891–903. Springer, Heidelberg (2005)

    Google Scholar 

  26. Etessami, K., Yannakakis, M.: Checking LTL Properties of Recursive Markov Chains. In: Proc. 2nd Intl. Conf. on Quantitative Evaluation of Systems. IEEE, Los Alamitos (2005)

    Google Scholar 

  27. Etessami, K., Yannakakis, M.: Efficient Qualitative Analysis of Classes of Recursive Markov Decision Processes and Simple Stochastic Games. In: Durand, B., Thomas, W. (eds.) STACS 2006. LNCS, vol. 3884, pp. 634–645. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  28. Etessami, K., Yannakakis, M.: Recursive concurrent stochastic games. In: Bugliesi, M., Preneel, B., Sassone, V., Wegener, I. (eds.) ICALP 2006. LNCS, vol. 4052, pp. 324–335. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  29. Etessami, K., Yannakakis, M.: On the complexity of Nash equilibria and other fixed points. In: Proc. of 48th IEEE FOCS (2007)

    Google Scholar 

  30. Etessami, K., Yannakakis, M.: Recursive Markov Processes (in preparation, 2008)

    Google Scholar 

  31. Fagin, R., Karlin, A., Kleinberg, J., Raghavan, P., Rajagopalan, S., Rubinfeld, R., Sudan, M., Tomkins, A.: Random walks with “back buttons” (extended abstract). In: ACM Symp. on Theory of Computing, pp. 484–493 (2000); Full version in Ann. of App. Prob., 11, pp 810–862 (2001)

    Google Scholar 

  32. Filar, J., Vrieze, K.: Competitive Markov Decision Processes. Springer, Heidelberg (1997)

    MATH  Google Scholar 

  33. Garey, M.R., Graham, R.L., Johnson, D.S.: Some NP-complete geometric problems. In: 8th ACM Symp. on Theory of Computing, pp. 10–22 (1976)

    Google Scholar 

  34. Haccou, P., Jagers, P., Vatutin, V.A.: Branching Processes: Variation, Growth, and Extinction of Populations. Cambridge U. Press (2005)

    Google Scholar 

  35. Harris, T.E.: The Theory of Branching Processes. Springer, Heidelberg (1963)

    MATH  Google Scholar 

  36. Jagers, P.: Branching Processes with Biological Applications. Wiley, Chichester (1975)

    MATH  Google Scholar 

  37. Kiefer, S., Luttenberger, M., Esparza, J.: On the convergence of Newton’s method for monotone systems of polynomial equations. In: Proc. 39th Symp. on Theory of Computation (STOC), pp. 217–226 (2007)

    Google Scholar 

  38. Kimmel, M., Axelrod, D.E.: Branching processes in biology. Springer, Heidelberg (2002)

    MATH  Google Scholar 

  39. Kolmogorov, A.N., Sevastyanov, B.A.: The calculation of final probabilities for branching random processes. Dokl. Akad. Nauk SSSR 56, 783–786 (1947) (Russian)

    MATH  Google Scholar 

  40. Kwiatkowska, M.: Model checking for probability and time: from theory to practice. In: 18th IEEE LICS, pp. 351–360 (2003)

    Google Scholar 

  41. Latouche, G., Ramaswami, V.: Introduction to Matrix Analytic Methods in Stochastic Modeling. ASA-SIAM series on statistics and applied probability (1999)

    Google Scholar 

  42. Manning, C., Schütze, H.: Foundations of Statistical Natural Language Processing. MIT Press, Cambridge (1999)

    MATH  Google Scholar 

  43. Nederhof, M.J., Satta, G.: Using Newton’s method to compute the partition function of a PCFG (unpublished manuscript, 2006)

    Google Scholar 

  44. Neyman, A., Sorin, S. (eds.): Stochastic Games and Applications. Kluwer, Dordrecht (2003)

    MATH  Google Scholar 

  45. Neuts, M.F.: Stuctured Stochastic Matrices of M/G/1 Type and their applications. Marcel Dekker, New York (1989)

    Google Scholar 

  46. Paz, A.: Introduction to Probabilistic Automata. Academic Press, London (1971)

    MATH  Google Scholar 

  47. Puterman, M.L.: Markov Decision Processes. Wiley, Chichester (1994)

    MATH  Google Scholar 

  48. Renegar, J.: On the computational complexity and geometry of the first-order theory of the reals, parts I-III. J. Symb. Comp. 13(3), 255–352 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  49. Sakakibara, Y., Brown, M., Hughey, R., Mian, I.S., Sjolander, K., Underwood, R., Haussler, D.: Stochastic context-free grammars for tRNA modeling. Nucleic Acids Research 22(23), 5112–5120 (1994)

    Article  Google Scholar 

  50. Sevastyanov, B.A.: The theory of branching processes. Uspehi Mathemat. Nauk 6, 47–99 (1951) (Russian)

    MathSciNet  Google Scholar 

  51. Shapley, L.S.: Stochastic games. Proc. Nat. Acad. Sci. 39, 1095–1100 (1953)

    Article  MATH  MathSciNet  Google Scholar 

  52. Tiwari, P.: A problem that is easier to solve on the unit-cost algebraic RAM. Journal of Complexity, 393–397 (1992)

    Google Scholar 

  53. van Houdt, B., Blondia, C.: Tree structured QBD Markov chains and tree-like QBD processes. Stochastic Models 19(4), 467–482 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  54. Vardi, M.: Automatic verification of probabilistic concurrent finite-state programs. In: Proc. of 26th IEEE FOCS, pp. 327–338 (1985)

    Google Scholar 

  55. Wojtczak, D., Etessami, K.: Premo: an analyzer for probabilistic recursive models. In: Grumberg, O., Huth, M. (eds.) TACAS 2007. LNCS, vol. 4424. Springer, Heidelberg (2007), http://groups.inf.ed.ac.uk/premo/

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Columbia University,  

    Mihalis Yannakakis

Authors
  1. Mihalis Yannakakis
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Oscar H. Ibarra Bala Ravikumar

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Yannakakis, M. (2008). Automata, Probability, and Recursion. In: Ibarra, O.H., Ravikumar, B. (eds) Implementation and Applications of Automata. CIAA 2008. Lecture Notes in Computer Science, vol 5148. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-70844-5_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-70844-5_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-70843-8

  • Online ISBN: 978-3-540-70844-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation