Skip to main content
SpringerLink
Log in

The Entropy Based Approach to Modeling and Evaluating Autonomy and Intelligence of Robotic Systems

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

This review paper presents the Entropy approach to modeling and performance evaluation of Intelligent Machines (IMs), which are modeled as hierarchical, multi-level structures. It provides a chronological summary of developments related to intelligent control, from its origins to current advances. It discusses fundamentals of the concept of Entropy as a measure of uncertainty and as a control function, which may be used to control, evaluate and improve through adaptation and learning performance of engineering systems. It describes a multi-level, hierarchical, architecture that is used to model such systems, and it defines autonomy and machine intelligence for engineering systems, with the aim to set foundations necessary to tackle related challenges. The modeling philosophy for the systems under consideration follows the mathematically proven principle of Increasing Precision with Decreasing Intelligence (IPDI). Entropy is also used in the context of N-Dimensional Information Theory to model the flow of information throughout such systems and contributes to quantitatively evaluate uncertainty, thus, autonomy and intelligence. It is explained how Entropy qualifies as a unique, single, measure to evaluate autonomy, intelligence and precision of task execution. The main contribution of this review paper is that it brings under one forum research findings from the 1970’s and 1980’s, and that it supports the argument that even today, given the unprecedented existing computational power, advances in Artificial Intelligence, Deep Learning and Control Theory, the same foundational framework may be followed to study large-scale, distributed Cyber Physical Systems (CPSs), including distributed intelligence and multi-agent systems, with direct applications to the SmartGrid, transportation systems and multi-robot teams, to mention but a few applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Antsaklis, P.J., Passino, K.M. (eds.): An Introduction to Intelligent and Autonomous Control. Kluwer Academic Publishers, Dordrecht (1993)

  2. Antsaklis, P.J., Passino, K.M., S. J. Wang.: An introduction to autonomous control systems. IEEE Control Syst. Mag. 11(4), 5–13 (1991). Reprinted in Neuro-Control Systems: Theory and Applications, M.M. Gupta and D.H. Rao Eds., Chapter 4, Part 1, pp 81-89 IEEE Press (1994)

    Article  Google Scholar 

  3. Antsaklis, P.J., Kantor, J.C.: Intelligent control for high autonomy process control systems. In: Davis, J.F., Stephanopoulos, G., Venkatasubramanian, V. (eds.) Proceedings, Intelligent Systems in Process Engineering. AIChE Symposium Series, vol. 92, No. 312, pp 37–46 (1996)

  4. Antsaklis, P.J.: Intelligent Control. Encyclopedia of Electrical and Electronics Engineering, vol. 10, pp. 493–503. Wiley (1999)

  5. Antsaklis, P.J., Passino, K.M., Wang, S.J.: Autonomous control systems: Architecture and fundamental issues (invited). In: Proc. of the 1988 American Control Conference, pp. 602-607, Atlanta, Georgia, June, 15–17 (1988)

  6. Antsaklis, P.J., Passino, K.M., Wang, S.J : Towards intelligent autonomous control systems: Architecture and fundamental issues. J. Intell. Robot. Syst. 1, 315–342 (1989)

    Article  Google Scholar 

  7. Antsaklis, P.J., Passino, K.M., Wang, S.J.: An introduction to autonomous control systems. In: Proceedings, 5th IEEE International Symposium on Intelligent Control, pp. 21–26. Philadelphia, Pennsylvania, Sept. 5–7 (1990)

  8. Antsaklis, P.J.: On intelligent control: Report of the IEEE CSS task force on intelligent control, technical report of the ISIS (interdisciplinary studies of intelligent systems) Group, No. ISIS-94-001. University of Notre Dame (1994)

  9. Antsaklis, P.J.: Defining intelligent control, report of the task force on intelligent control, P.J Antsaklis, chair. In: IEEE Control Systems Magazine, pp. 4-5 & 58–66, June. Also Inproceedings of the 1994 International Symposium on Intelligent Control, pp. (i)-(xvii), Columbus, OH, August 16–18, 1994 (1994)

  10. Antsaklis, P.J.: Towards autonomous intelligent control systems, keynote address. Summary in Proceedings, 1991 AI Simulation and Planning in High Autonomy Systems, pp. 2–3, Cocoa Beach, FL, April 1–2 (1991)

  11. Antsaklis, P.J.: On modeling, analysis and design of high autonomy control systems, Proceedings 1992, Workshop on Intelligent Autonomous Control Systems, pp. 128–147/ Leshem, Israel, November 2–3 (1992)

  12. Antsaklis, P.J.: Towards autonomous control systems. Proceedings, Mediterranean Symposium on New Directions in Control Theory and Applications, pp. 274–277. Chania, Crete, Greece, June 21–23 (1993)

  13. Antsaklis, P.J.: Intelligent control for high autonomy in unmanned underwater vehicles. In: Proceedings of the NSF/ISR Workshop on ’Undersea Robotics and Intelligent Control’, pp. 25–32, Lisboa, Portugal, March 2–3 (1995)

  14. Antsaklis, P.J.: Intelligent learning control, guest editor’s introduction. IEEE Control Syst. 15(3), 5–7 (1995). Special Issue on ’Intelligence and Learning’ of the IEEE Control Systems Magazine, Vol.15, No.3, pp. 5-80, June 1995

    Article  Google Scholar 

  15. Antsaklis, P.J.: Setting the stage: Some autonomous thoughts on autonomy, introduction to the panel discussion: Autonomy in engineering systems: What is it and Why is it Important? In: Proceedings of the ISIC/CIRA/ISAS’98, Joint Conference on the Science and Technology of Intelligent Systems, pp. 520–521, National Institute of Standards and Technology (NIST), Gaithersburg, MD, September 14–17 (1998)

  16. Antsaklis, P.J.: Emerging control technologies: What does the future hold?, Keynote address summary. In: Proceedings, IFAC Workshop on Control Applications and Ergonomics in Agriculture, pp. 1–3, Athens, Greece, June 14–17 (1998)

  17. Antsaklis, P.J., Passino, K.M.: Introduction to intelligent control systems with high degree of autonomy. In: Antsaklis, P.J., Passino, K.M. (eds.) Introduction to Intelligent and Autonomous Control. Chapter 1, pp 1–26. Kluwer (1992)

  18. Passino, K.M., Antsaklis, P.J.: Modeling and analysis of artificially intelligent planning systems. In: Antsaklis, P.J., Passino, K.M. (eds.) Introduction to Intelligent and Autonomous Control. Chapter 8, pp 191–214. Kluwer (1992)

  19. Antsaklis, P.J., Lemmon, M.D., Stiver, J.A.: Learning to be autonomous: Intelligent supervisory control, Technical Report of the ISIS (Interdisciplinary Studies of Intelligent Systems) Group, No. ISIS-93-003, University of Notre Dame, April. Also in Intelligent Control: Theory and Practice, Chapter 2 pp. 28–62. Gupta M.M., Sinha N.K., eds., IEEE Press, Piscataway, NJ (1995) (1993)

  20. Antsaklis, P.J.: Hybrid and supervisory control systems in autonomous underwater vehicles. In: First International Workshop on Autonomous Underwater Vehicles for Shallow Waters and Coastal Environments, pp. 135–139. Lafayette, Louisiana, February 17–19 (1998)

  21. Koutsoukos, X.D., Antsaklis, P.J.: Computational issues in intelligent control discrete-event and hybrid systems. In: Sinha, N.K., Gupta, M.M. (eds.) Soft Computing and Intelligent Systems: Theory and Practice. Also, Technical Report isis-99-005, Dept. of Electrical Engr., Univ. of Notre Dame, May 1999, pp 39–69. Academic Press (2000)

  22. Antsaklis, P.J.: Hybrid supervisory control in intelligent autonomous systems. In: Proceedings, 1stIFAC/IEEE Symposium on Systems Structure and Control. Prague, Czech Republic, 29–31 August (2001)

  23. Saridis, G.N.: Toward the realization of intelligent controls. Proc. IEEE 67, 1115–1133 (1979)

    Article  Google Scholar 

  24. Albus, J.S.: Outline for a theory of intelligence. IEEE Trans. Syst. Man Cybern. 21(3), 432–509 (1991)

    Article  MathSciNet  Google Scholar 

  25. Meystel, A.: Autonomous Mobile Robots. World Scientific, River Edge (1991)

    Book  MATH  Google Scholar 

  26. White, D.A., Sofge, D.A. (eds.): Handbook of Intelligent Control Neural, Fuzzy, and Adaptive Approaches. Van Nostrand Reinhold, New York (1992)

  27. Gupta, M.M., Sinha, N.K. (eds.): Intelligent Control: Theory and Practice. IEEE Press, Piscataway (1994)

  28. Michalski, R.S., Carbonell, J.G., Mitchell, T.M.: Machine Learning-An Artificial Intelligence Approach, Palo Alto, Tioga. Also R.S. Michalski, and G. Tecuci (Eds.), Machine Learning - A Multistrategy Approach, San Francisco: Morgan-Kaufmann, l994, Vol IV (1983)

  29. Shavlik, J.W., Dietterich, T.G. (eds.): Readings in Machine Learning. Morgan-Kaufmann, San Francisco (1990)

  30. Fu, K.S.: Learning control systems–review and outlook. IEEE Trans. Autom. Control 15, 210–221 (1970)

    Article  MathSciNet  Google Scholar 

  31. Mendel, J.M., Fu, K.S.: Adaptive, Learning and Pattern Recognition Systems. Academic Press, New York (1970)

    MATH  Google Scholar 

  32. Sklansky, J.: Learning systems for automatic control. IEEE Trans. Autom. Control 11, 6–19 (1966)

    Article  MathSciNet  Google Scholar 

  33. Tsypkin, Y.A.Z.: Adaptation and Learning in Automatic Systems. Academic Press, New York (1971)

    MATH  Google Scholar 

  34. Tsypkin, Y.A.Z.: Self-learning-what is it? IEEE Trans on Automatic Control 13, 608–612 (1968)

    Article  MathSciNet  Google Scholar 

  35. Passino, K.M., Antsaklis, P.J.: Event rates and aggregation in hierarchical discrete event systems. J. Discret. Event Dyn. Syst. Theory Appl. 1(3), 271–287 (1992)

    Article  MATH  Google Scholar 

  36. Antsaklis, P.J., Stiver, J.A., Lemmon, M.D.: Hybrid system modeling and autonomous control systems. In: Grossman, R.L., Nerode, A., Ravn, A.P., Rischel, H. (eds.) Hybrid Systems. LNCS 736, 366–392. Springer, New York (1993)

  37. Stiver, J.A., Antsaklis, P.J., Lemmon, M.D.: A Logical DES Approach to the Design of Hybrid Control Systems, Mathematical and Computer Modeling, Special issue on Discrete Event Systems 23(11/12) (1996)

  38. Grossman, R.L., Nerode, A., Ravn, A.P., Rischel, H. (eds.): Hybrid Systems. Springer, New York (1993). LNCS 736

  39. Antsaklis, P., Kohn, W., Nerode, A., Sastry, S. (eds.): Hybrid Systems II. Springer, New York (1995). LNCS 999

  40. Alur, R., Henzinger, T., Sontag, E. (eds.): Hybrid Systems III. Springer, New York (1996). LNCS 1066

  41. Antsaklis, P., Kohn, W., Nerode, A., Sastry, S. (eds.): Hybrid Systems IV. Springer, New York (1997). LNCS

  42. Valavanis, K.P. (ed.): Applications of Intelligent Control to Engineering Systems, International Series on Intelligent Systems, Control and Automation: Science and Engineering, vol. 39. Springer, Berlin (2009)

  43. Valavanis, K.P., Saridis, G.N.: Intelligent Robotic Theory Systems: Design and Applications. Kluwer Academic Publishers (1992)

  44. Valavanis, K.P., Saridis, G.N.: Information theoretic modeling of robotic and automation systems. In: Leondes, C.T. (ed.) Control and Dynamic Systems; Manufacturing and Automation Systems: Techniques and Technologies, Part 4 of 5, vol. 48, pp 387–416. Academic Press (1991)

  45. Valavanis, K.P., Saridis, G.N.: Knowledge based architectural models for intelligent robotic systems. In: Saridis, G.N. (ed.) Advances in Automation and Robotics, vol. 2, pp 1–19. JAI Press (1991)

  46. Valavanis, K.P.: Theory and design of intelligent robotic systems. In: Tzafestas, S.G. (ed.) Intelligent Robotic Systems, pp 45–83. Marcel Dekker (1991)

  47. Ramaswamy, S., Valavanis, K.P.: Hierarchical time extended petri nets (HEPN’s) based error identification and recovery for multilevel systems. IEEE Trans. Syst. Man Cybern. 26(1), 164–175 (1996)

    Article  Google Scholar 

  48. Gracanin, D., Srinivasan, P., Valavanis, K.P.: Parameterized Petri nets and their application to planning and coordination in intelligent systems. IEEE Trans. Syst. Man Cybern. 24(10), 1483–1497 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  49. Valavanis, K.P., Stellakis, H.: A general organizer model for robotic assemblies and intelligent robotic systems. IEEE Trans. Syst. Man Cybern. 21(2), 302–317 (1991)

    Article  Google Scholar 

  50. Valavanis, K.P., Saridis, G.N.: Probabilistic modeling of intelligent robotic systems. IEEE Trans. Robot. Autom. 7(1), 164–171 (1991)

    Article  Google Scholar 

  51. Stellakis, H., Valavanis, K.P.: Fuzzy logic based formulation of the organizer of an intelligent robotic system. J. Intell. Robot. Syst. 4(1), 1–24 (1991)

    Article  Google Scholar 

  52. Valavanis, K.P., Carelo, S.J.: An efficient planning technique for robotic assemblies and intelligent robotic systems. J. Intell. Robot. Syst. 3(4), 321–347 (1990). Winter

    Article  Google Scholar 

  53. Valavanis, K.P., Yuan, P.H.: Hardware and software for hierarchical intelligent robotic systems. J. Intell. Robot. Syst. 1(4), 343–373 (1989)

    Article  Google Scholar 

  54. Valavanis, K.P., Saridis, G.N.: Information theoretic modeling of intelligent robotic systems. IEEE Trans. Syst. Man Cybern. 18(6), 852–872 (1988)

    Article  MathSciNet  Google Scholar 

  55. Saridis, G.N., Valavanis, K.P.: Analytical design of intelligent machines. Automatica 24(2), 123–133 (1988)

    Article  MATH  Google Scholar 

  56. Albus, J.S., McCain, H.G., Lumia, R.: NASA/NBS Standard Reference Model for Telerobot Control Systems Architecture (NASREM), NIST Technical Report 1235. Gaithersburg, MD (1989)

  57. Astrom, K.J.: Expert control. Automatica (1986)

  58. Athans, M.: Command and control theory: A challenge to control science. IEEE Transactions on Automatic Control (1987)

  59. Conant, R.C.: Laws of information which govern systems. IEEE Trans. SMC 6, 4 (1976)

    MathSciNet  MATH  Google Scholar 

  60. Findeisen, B., et al.: Control and Coordination in Hierarchical Systems. Wiley (1980)

  61. Mesarovic, et al.: Theory of Hierarchical Multilevel Systems, Mathematics in Science and Engineering, vol. 68. Academic Press (1970)

  62. Meystel, A.: Intelligent control in robotics. Journal of Robotic Systems (1988)

  63. Meystel, A.: Intelligent motion control in anthropomorphic machines. In: Andriole, S. (ed.) Applied AI. Pentrocellis Books, Princeton (1985)

  64. Acar, L., Ozguner, U.: Design of knowledge-rich hierarchical controllers for large functional systems. IEEE Transactions on SMC

  65. Acar, L., Ozguner, U.: Design of hierarchically distributed expert controllers for large-scale systems. In: Proceedings 4th IEEE ISIC. Albany (1989)

  66. Lin, F.: Robust and adaptive supervisory control of discrete event systems. IEEE Trans. Autom. Control 38 (12), 1848–1852 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  67. Angluin, D.: Queries and concept learning. Mach. Learn. 2(4), 319–342 (1988)

    MathSciNet  Google Scholar 

  68. Valavanis, K.P., Vachtsevanos, G.J.: (Editors and authors of multiple Chapters), Handbook of Unmanned Aerial Vehicles (UAVs), Five Volumes, ISBN 978-90-481-9706-4, Springer published in August of 2014 (2014)

  69. The Role of Autonomy in DOD Systems, DOD, Defense Science Board, Task Force Report July 2012

  70. Brooks, R.: A robust layered control system for a mobile robot. IEEE J. Robot. Autom. 2(1), 14–23 (1986)

    Article  Google Scholar 

  71. Vachtsevanos, G., Lewis, F., Roemer, M., Hess, A., Wu, B.: Intelligent Fault Diagnosis and Prognosis for Engineering Systems. Wiley (2006)

  72. Schrage, D., Vachtsevanos, G.: Software-enabled control for intelligent UAVs. In: Proceedings International Conference on Control Applications. Hawaii (1999)

  73. Vachtsevanos, G.: Hierarchical control. In: Ruspini, E., Bonissone, P., Pedrycz, W. (eds.) Handbook of Fuzzy Computation. Institute of Physics Publishing (1998)

  74. Fiksel, J.: Designing resilient, sustainable systems. Environ Sci Technol 37, 23 (2003)

    Article  Google Scholar 

  75. Rieger, C.G., Gertman, D.I., McQueen, M.A.: Resilient Control Systems: Next Generation Design Research. Idaho National Laboratory INL/CON-08-15208 (2009)

  76. Saridis, G.N.: Self-Organizing Control of Stochastic Systems. Marcel Dekker, New York (1977)

    MATH  Google Scholar 

  77. Saridis, G.N., Stephanou, H.E.: A hierarchically intelligent control for a bionic arm. Proceedings, IEEE CDC (1975)

  78. Saridis, G.N., Stephanou, H.E.: A hierarchical approach to the control of a prosthetic arm. IEEE Trans. SMC 7, 6 (1977)

    Google Scholar 

  79. Sheridan, T.B., Ferrell, W.R.: Man-Machine Systems. MIT Press (1974)

  80. Antsaklis, P.J.: The Quest for Autonomy Revisited. Technical Report of the ISIS Group at the University of Notre Dame (2011)

  81. Saxena, A., Wu, B., Vachtsevanos, G.: Integrated diagnosis and prognosis architecture for fleet vehicles using dynamic case based reasoning. In: Proceedings, IEEE Autotestcon 2005, pp. 96–102. (Best Paper Award) (2005)

  82. Khawaja, T, Vachtsevanos, G., Wu B: Reasoning about uncertainty in prognosis: A confidence prediction neural network approach. NAFIPS (2005)

  83. Saha, B., Vachtsevanos, G.: A novel model-based reasoning approach to system-level diagnostics of a helicopter intermediate gearbox. In: Proceedings, 60th Meeting of the Society for Machinery Failure Prevention Technology, pp. 281–290 (2006)

  84. Saha, B., Vachtsevanos, G.: A model-based reasoning approach to system fault diagnosis. WSEAS Trans. Syst. 5(8), 1997–2004 (2006)

    Google Scholar 

  85. Lindsay, R.B., Margenau, H.: Foundations of Physics. Dover (1957)

  86. Shannon, C., Weaver, W.: The Mathematical Theory of Communication. Illinois Books (1963)

  87. Shannon, C.E.: Prediction and entropy of printed English. Bell Syst. Tech. J. 30, 50–64 (1951)

    Article  MATH  Google Scholar 

  88. Lindsay, R.B., Margenau, H.: Foundations of Physics. Dover (1957)

  89. Lima, P.U., Saridis, G.N.: Design of Intelligent Control Systems based on Hierarchical Stochastic Automata. World Scientific (1996)

  90. Saridis, G.N.: Entropy in Control Engineering. World Scientific (2001)

  91. Saridis, G.N.: Stochastic Processes, Estimation and Control – The Entropy Approach. Wiley (1995)

  92. Kapur, J.N., Kesavan, H.K.: Entropy Optimization Principles with Applications. Academic Press (1992)

  93. Kondepudi, D., Prigogine, I.: Modern Thermodynamics. Wiley (1998)

  94. Hilden-Minton, J.A.: Multilevel Diagnostics for Mixed and Hierarchical Linear Models, PhD Thesis, UCLA (1995)

  95. Shannon Entropy, Chapter 6, downloaded from the internet

  96. Stepanic, J. Jr, Stefancic, H., et al.: Approach to a quantitative description of social systems based on thermodynamic formalism. Entropy 2, 98–105 (2000)

    Article  Google Scholar 

  97. Anderson, B.D.O., Yu, C., Fidan, B., Hendrickx, J.M.: Rigid graph control architectures for autonomous formations. IEEE Control Systems Magazine (2008)

  98. Boettcher, K.L.: An information theoretic model of the decision maker, LIDS-TH-1096, MSc Thesis, MIT (1981)

  99. Broekstra, G.: On the representation of identification of structure systems. Int. J. Gene. Syst. 9, 1271–1293 (1978)

    Article  MATH  Google Scholar 

  100. Drenick, R.F.: Organization and control. In: Ho, Miller (eds.) Directions in Large Scale Systems. Plenum, New York (1976)

  101. Galbraith, J.R.: Organization Design Addison. Wesley (1977)

  102. Graham, J.H., Saridis, G.N.: Linguistic decision structures for hierarchical systems. IEEE Trans. SMC SMC-12(3), 325–333 (1982)

    MATH  Google Scholar 

  103. Hall, S.A.: Information theoretic models for storage and memory, LIDS-TH-1232, MIT (1982)

  104. Jaynes, E.T.: Information theory and statistical mechanics. Physical Review 106, 4 (1957)

    Article  MathSciNet  MATH  Google Scholar 

  105. Jaynes, E.T.: On the rationale of maximum entropy methods. Proc. IEEE 70, 9 (1982)

    Article  Google Scholar 

  106. Kalata, P., Priemer, R.: On minimal error entropy stochastic approximation. Int. J. Syst. 5, 9 (1974)

    MathSciNet  MATH  Google Scholar 

  107. Kalata, P., Priemer, R: When should smoothing occur? In: Proceedings of IEEE (1974)

  108. Kalata, P., Priemer, R.: On system identification with and without uncertainty. J. Cybern. 8, 31–50 (1978)

    Article  MATH  Google Scholar 

  109. Kalata, P., Priemer, R.: Linear prediction, filtering, and smoothing: An information theoretic approach. Inform. Sci. 17, 1–14 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  110. Koomen, C.J.: The entropy of design: A study on the meaning of creativity. IEEE Trans. SMC SMC-15 (1), 16–30 (1985)

    MathSciNet  Google Scholar 

  111. Papoulis, A.: Probability, Random Variables and Stochastic Processes, 3rd edn. McGraw Hill (1991)

  112. Sanderson, A.C.: Parts entropy methods for robotic assembly system design. In: Proceedings of IEEE ICRA (1984)

  113. Saridis, G.N., McInroy, J.E.: Reliability analysis in intelligent machines. Technical Report, CIRSSE, No. 39 (1989)

  114. Stabile, D.A.: The design of information structures: Basic allocation structures for organizations, LIDS-Th-1008. MSc Thesis, MIT (1981)

Download references

Author information

Authors and Affiliations

  1. John Evans Professor and Director, Research and Innovation, Daniels Felix Ritchie School of Engineering and Computer Science, University of Denver, Denver, CO, 80210, USA

    Kimon P. Valavanis

Authors
  1. Kimon P. Valavanis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kimon P. Valavanis.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Valavanis, K.P. The Entropy Based Approach to Modeling and Evaluating Autonomy and Intelligence of Robotic Systems. J Intell Robot Syst 91, 7–22 (2018). https://doi.org/10.1007/s10846-018-0905-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-018-0905-6

Keywords

Search

Navigation