Skip to main content
Springer Nature Link
Log in

An Estimate Method of the Minimum Entropy of Natural Languages

  • Conference paper
  • First Online:
Computational Linguistics and Intelligent Text Processing (CICLing 2003)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2588))

  • 916 Accesses

Abstract

The study of minimum entropy of English has a long history and has made a great progress, but only a few studies on other languages have been reported in literature so far. In this paper, we present a new method to estimate the minimum entropy of character in natural languages, based on two hypotheses of conservation of information quantity. We also verified the hypotheses empirically through experiments with two natural languages, Japanese and Chinese.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. K. Kita, T. Nakamura, M. Nagata, “Voice Language Processing,” Morikita Publishing, Inc., 1996

    Google Scholar 

  2. N. Mori, O. Yamaji, “Estimating the Upper Limit of Information Quantity in Japanese Language,” Information Processing Society Journal, Vol. 38, No. 11, pp. 2192–2199, 1997

    Google Scholar 

  3. K. Asai, “About Entropy in Japanese,” Measuring Japanese Language, pp. 4–7, 1965

    Google Scholar 

  4. C.E. Shannon, “Prediction and Entropy of Printed English,” Bell System Technical Journal, Vol. 30, pp. 50–64, 1951.

    Google Scholar 

  5. P.F. Brown, S.A.D. Pietra, R.L. Mercer, “An Estimate of an Upper Bound for the Entropy of English,” Computational Linguistics, Vol.18, No. 1, pp. 31–20, 1992.

    Google Scholar 

  6. P.F. Brown, V.J.D. Pietra, P.V. deSouza, J.C. Lai, R.L. Mercer, “Class-Based ngram Models of Natural Language,” Computational Linguistics, Vol. 18, No. 4, pp. 467–479, 1992.

    Google Scholar 

  7. T.M. Cover, R.C. King, “A Convergent Gambling Estimate of the Entropy of English,” IEEE Trans. on Information Theory, Vol.-IT-24, No.-4, pp. 413–421, 1978.

    Article  MathSciNet  Google Scholar 

  8. F. Ren, J. Nie, “The Concept of Sensitive Word in Chinese-Survey in a Machine-Readable Dictionary,” Journal of Natural Language Processing, Vol. 6, No. 1, pp. 59–78, 1999.

    Google Scholar 

  9. L. Fan, F. Ren, Y. Miyanaga, K. Tochinai, “Automatic Composition of Chinese Compound Words for Chinese-Japanese Machine Translation,” Transactions of Information Processing Society of Japan, Vol. 33, No. 9, pp. 1103–1113, 1992.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Engineering, The University of Tokushima, 770-8506, Tokushima, Japan

    Fuji Ren

  2. Department of Research, A.G.I. Inc., 106-0043, Tokyo, Japan

    Shunji Mitsuyoshi

  3. Dept. of Electrical and Computer Eng., Florida Int’l University, 33174, Miami Florida

    Kang Yen

  4. Institute of Automation, Chinese Academy of Science, 100080, Beijing, China

    Chengqing Zong

  5. Faculty of Engineering, Hiroshima Kokusai Gakuin University, 739-0321, Hiroshima, Japan

    Hongbing Zhu

Authors
  1. Fuji Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shunji Mitsuyoshi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kang Yen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chengqing Zong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hongbing Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Centro de Investigación en Computación (CIC), Instituto Politécnico Nacional (IPN), Col. Zacatenco, CP 07738, Mexico D.F., Mexico

    Alexander Gelbukh

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Ren, F., Mitsuyoshi, S., Yen, K., Zong, C., Zhu, H. (2003). An Estimate Method of the Minimum Entropy of Natural Languages. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2003. Lecture Notes in Computer Science, vol 2588. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36456-0_39

Download citation

  • DOI: https://doi.org/10.1007/3-540-36456-0_39

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-00532-2

  • Online ISBN: 978-3-540-36456-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation