Multiple-level information source and entropy-reduced transformation models

https://doi.org/10.1016/0031-3203(91)90077-IGet rights and content

Abstract

An approach has been proposed to analyse systematically the changes in entropy which occur in the different stages of a pattern recognizer. It models the entire pattern recognition system as a multiple level information source (MLIS). For a typical recognition system there are four levels in this information source, i.e. IS1 to IS4, and they can be divided into two categories: entropy-reduced and entropy-increased. By examining the internal structures of a pattern recognition system, MLIS allows us to address all the different factors which increase the entropy at the different levels of the entire system. Such factors include (1) IS1: Intrinsic characteristics of the target pattern set, (2) IS2: Distortions due to the transducer and digitization, and (3) IS4: Information loss due to feature extraction process. A theoretical analysis of the entropy distribution in the MLIS indicates that in order to improve the performance of a pattern recognition system, the entropy of the MLIS must be reduced in all the different levels. Specifically, IS1, IS2 and IS4 must be converted. To perform this conversion, a theoretical framework called entropy-reduced transformation (ERT) has been developed. Two examples are presented to illustrate how the proposed MLIS solves a practical engineering problem of pattern recognition and how the ERT is applied to design a pattern recognizer.

References (15)

  • R.J. Clausius

    Die Mechanische Wermetheorie

    (1887)
  • C.E. Shannon

    A mathematical theory of communication

    Bell Syst. Tech. J.

    (1948)
  • S. Guisau

    On the most rational algorithm of recognition

    Kybernetik

    (1968)
  • L.N. Landa

    The characteristic selection problem in recognition systems

    IRE Trans. Inf. Theory

    (1962)
  • J.T. Tou et al.

    Pattern Recognition Principles

  • S. Watanabe

    Pattern Recognition: Human and Mechanical

  • S. Watanabe et al.

    Recent developments of the minimum entropy algorithm

There are more references available in the full text version of this article.

Cited by (1)

View full text