Abstract
Given a finite set of words w 1, ..., w n independently drawn according to a fixed unknown distribution law P called a stochastic language, a usual goal in Grammatical Inference is to infer an estimate of P in some class of probabilistic models, such as Probabilistic Automata (PA). Here, we study the class \({{\mathcal S}_{\mathbb R}^{rat}(\Sigma)}\) of rational stochastic languages, which consists in stochastic languages that can be generated by Multiplicity Automata (MA) and which strictly includes the class of stochastic languages generated by PA. Rational stochastic languages have minimal normal representation which may be very concise, and whose parameters can be efficiently estimated from stochastic samples. We design an efficient inference algorithm DEES which aims at building a minimal normal representation of the target. Despite the fact that no recursively enumerable class of MA computes exactly \({{\mathcal S}_{\mathbb Q}^{rat}(\Sigma)}\), we show that DEES strongly identifies \({{\mathcal S}_{\mathbb Q}^{rat}(\Sigma)}\) in the limit. We study the intermediary MA output by DEES and show that they compute rational series which converge absolutely and which can be used to provide stochastic languages which closely estimate the target.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Beimel, A., Bergadano, F., Bshouty, N.H., Kushilevitz, E., Varricchio, S.: On the applications of multiplicity automata in learning. In: IEEE Symposium on Foundations of Computer Science, pp. 349–358 (1996)
Beimel, A., Bergadano, F., Bshouty, N.H., Kushilevitz, E., Varricchio, S.: Learning functions represented as multiplicity automata. Journal of the ACM 47(3), 506–530 (2000)
Bergadano, F., Varricchio, S.: Learning behaviors of automata from multiplicity and equivalence queries. In: Italian Conf. on Algorithms and Complexity (1994)
Berstel, J., Reutenauer, C.: Les séries rationnelles et leurs langages. Masson (1984)
Carrasco, R.C., Oncina, J.: Learning stochastic regular grammars by means of a state merging method. In: Carrasco, R.C., Oncina, J. (eds.) ICGI 1994. LNCS, vol. 862, pp. 139–152. Springer, Heidelberg (1994)
Denis, F., Esposito, Y.: Learning classes of probabilistic automata. In: Shawe-Taylor, J., Singer, Y. (eds.) COLT 2004. LNCS (LNAI), vol. 3120, pp. 124–139. Springer, Heidelberg (2004)
Denis, F., Esposito, Y.: Rational stochastic languages. Technical report, LIF - Université de Provence (2006), http://hal.ccsd.cnrs.fr/ccsd-00019728
Gold, E.M.: Language identification in the limit. Inform. Control 10, 447–474 (1967)
Hardy, G.H., Wright, E.M.: An introduction to the theory of numbers. Oxford University Press, Oxford (1979)
Lugosi, G.: Pattern classification and learning theory. In: Principles of Nonparametric Learning, pp. 1–56. Springer, Heidelberg (2002)
Sakarovitch, J.: Éléments de théorie des automates. Éditions Vuibert (2003)
Salomaa, A., Soittola, M.: Automata: Theoretic Aspects of Formal Power Series. Springer, Heidelberg (1978)
Thollard, F., Dupont, P., de la Higuera, C.: Probabilistic DFA inference using Kullback-Leibler divergence and minimality. In: Proc. 17th ICML, pp. 975–982. KAUFM
Vapnik, V.N.: Statistical Learning Theory. John Wiley, Chichester (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Denis, F., Esposito, Y., Habrard, A. (2006). Learning Rational Stochastic Languages. In: Lugosi, G., Simon, H.U. (eds) Learning Theory. COLT 2006. Lecture Notes in Computer Science(), vol 4005. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11776420_22
Download citation
DOI: https://doi.org/10.1007/11776420_22
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-35294-5
Online ISBN: 978-3-540-35296-9
eBook Packages: Computer ScienceComputer Science (R0)