Skip to main content
SpringerLink
Log in

MeasApplInt - a novel intelligence metric for choosing the computing systems able to solve real-life problems with a high intelligence

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Intelligent agent-based systems are applied for many real-life difficult problem-solving tasks in domains like transport and healthcare. In the case of many classes of real-life difficult problems, it is important to make an efficient selection of the computing systems that are able to solve the problems very intelligently. The selection of the appropriate computing systems should be based on an intelligence metric that is able to measure the systems intelligence for real-life problem solving. In this paper, we propose a novel universal metric called MeasApplInt able to measure and compare the real-life problem solving machine intelligence of cooperative multiagent systems (CMASs). Based on their measured intelligence levels, two studied CMASs can be classified to the same or to different classes of intelligence. MeasApplInt is compared with a recent state-of-the-art metric called MetrIntPair. The comparison was based on the same principle of difficult problem-solving intelligence and the same pairwise/matched problem-solving intelligence evaluations. Our analysis shows that the main advantage of MeasApplInt versus the compared metric, is its robustness. For evaluation purposes, we performed an illustrative case study considering two CMASs composed of simple reactive agents providing problem-solving intelligence at the systems’ level. The two CMASs have been designed for solving an NP-hard problem with many applications in the standard, modified and generalized formulation. The conclusion of the case study, using the MeasApplInt metric, is that the studied CMASs have the same real-life problems solving intelligence level. An additional experimental evaluation of the proposed metric is attached as an Appendix.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Anthon A, Jannett TC (2007) Measuring machine intelligence of an agent-based distributed sensor network system. In: Elleithy K (ed) Advances and innovations in systems, computing sciences and software engineering. Springer, pp 531–535

  2. Arif M, Illahi M, Karim A, Shamshirband S, Alam KA, Farid S, Iqbal S, Buang Z, Balas VE (2015) An architecture of agent-based multi-layer interactive e-learning and e-testing platform. Qual Quant 49 (6):2435–2458

    Article  Google Scholar 

  3. Arik S, Iantovics LB, Szilagyi SM (2017) OutIntSys - a novel method for the detection of the most intelligent cooperative multiagent systems. In: Liu D et al (eds) 24th International conference on neural information processing, Guangzhou, China, November 14-18. Neural Information Processing, LNCS, 10637:31–40

  4. Barnett V, Lewis T (1994) Outliers in statistical data, 3rd edn. Wiley, New York

    MATH  Google Scholar 

  5. Beni G, Wang J (1993) Swarm intelligence in cellular robotic systems. In: Dario P, Sandini G, Aebischer P (eds) Robots and biological systems: towards a new bionics? NATO ASI Series (Series F: Computer and Systems Sciences), vol 102. Springer, Berlin, pp 703–712

  6. Bejar II, Whalen SJ (2003) Methods and systems for presentation and evaluation of constructed responses assessed by human evaluators, US Patent 6,526,258

  7. Besold T, Hernandez-Orallo J, Schmid U (2015) Can machine intelligence be measured in the same way as human intelligence? Kunstl Intell 29(3):291–297

    Article  Google Scholar 

  8. Boctor FF, Laporte G, Renaud J (2003) Heuristics for the traveling purchaser problem. Comput Oper Res 30:491–504

    Article  MATH  Google Scholar 

  9. Bonett DG, Wright TA (2000) Sample size requirements for estimating Pearson, Kendall and Spearman correlations. Psychometrika 65:23–28

    Article  MATH  Google Scholar 

  10. Box FJ (1987) Guinness, gosset, fisher, and small samples. Stat Sci 2(1):45–52

    Article  MathSciNet  MATH  Google Scholar 

  11. Brady SG, Fisher BL, Schultz TR, Ward PS (2014) The rise of army ants and their relatives: diversification of specialized predatory doryline ants. BMC Evol Biol 14:2–14

    Article  Google Scholar 

  12. Bullnheimer B, Hartl RF, Strauss C (1999) A new rank based version of the ant system. A computational study. CEJOR 7(1):25–38

    MathSciNet  MATH  Google Scholar 

  13. Chakraborty UK, Konar D, Roy S, Choudhury S (2016) Intelligent fuzzy spelling evaluator for e-Learning systems. Educ Inf Technol 21(1):171–184

    Article  Google Scholar 

  14. Chakravarti IM, Laha RG, Roy J (1967) Handbook of methods of applied statistics, vol I. Wiley, New York, pp 392–394

    Google Scholar 

  15. Chliaoutakis A, Chalkiadakis G (2016) Agent-based modeling of ancient societies and their organization structure. Auton Agent Multi-Agent Syst 30(6):1072–1116

    Article  Google Scholar 

  16. Coelho CGC, Abreu CG, Ramos RM, Mendes AHD, Teodoro G, Ralha CG (2016) MASE-BDI: Agent-based simulator for environmental land change with efficient and parallel auto-tuning. Appl Intell 45(3):904–922

    Article  Google Scholar 

  17. Chmait N, Dowe DL, Green DG, Li YF, Insa-Cabrera J (2015) Measuring universal intelligence in agent-based systems using the anytime intelligence test. Technical Report, Monash University, Report Num, 2015/279

  18. Chouhan SS, Niyogi R (2017) MAPJA: multi-agent planning with joint actions. Appl Intell 47(4):1044–1058

    Article  Google Scholar 

  19. Colom R, Karama S, Jung RE, Haier RJ (2010) Human intelligence and brain networks. Dialogues Clin Neurosci 12(4):489–501

    Google Scholar 

  20. Colorni A, Dorigo M, Maniezzo V (1991) Distributed optimization by ant colonies. In: Actes de la première conférence européenne sur la vie artificielle. Paris, France, Elsevier Publishing, 134–142

  21. Conley W (1988) Travelling salesman problem solved with simulation techniques. Int J Syst Sci 19(10):2115–2122

    Article  MathSciNet  MATH  Google Scholar 

  22. Conley W (1989) Two truck travelling salesman simulation. Int J Syst Sci 20(12):2495–2514

    Article  MathSciNet  MATH  Google Scholar 

  23. Conley W (1990) Multi-stage Monte Carlo optimization applied to a large travelling salesman problem. Int J Syst Sci 21(3):547–566

    Article  MATH  Google Scholar 

  24. Conover WJ (1973) On methods of handling ties in the wilcoxon signed-rank test. J Am Stat Assoc 68 (344):985–988

    Article  MathSciNet  MATH  Google Scholar 

  25. Cordon O, Herrera F, de Viana IF, Moreno L (2000) A new ACO model integrating evolutionary computation concepts: The Best-Worst ant system. In: Proceedings of ANTS’2000. From ant colonies to artificial ants: second international workshop on ant algorithms, Brussels, Belgium, September 7–9, 22–29

  26. Cordon O, de Viana IF, Herrera F (2002) Analysis of the best-worst ant system and its variants on the QAP. In: Dorigo M, Di Caro G, Sampels M (eds) Ant algorithms, vol 2463. Springer, LNCS, Berlin, Heidelberg, pp 228–234

  27. Crisan GC, Pintea CM, Palade V (2017) Emergency management using geographic information systems: application to the first Romanian traveling salesman problem instance. Knowl Inf Syst 50(1):265–285

    Article  Google Scholar 

  28. Dantzig GB, Ramser JH (1959) The truck dispatching problem. Manag Sci 6:80–91

    Article  MathSciNet  MATH  Google Scholar 

  29. Dantzig G, Fulkerson D, Johnson S (1954) Solution of a large scale traveling salesman problem. Oper Res 2:393–410

    MathSciNet  MATH  Google Scholar 

  30. David HA, Gunnink JL (1997) The paired t test under artificial pairing. Am Stat 51(1):9–12

    MathSciNet  Google Scholar 

  31. Dorigo M (1992) Optimization, learning and natural algorithms. PhD thesis, Politecnico di Milano, Italy

  32. Dorigo M, Gambardella LM (1997) Ant colony system: a cooperative learning approach to the traveling salesman problem. IEEE Trans Evol Comput 1(1):53–66

    Article  Google Scholar 

  33. Dowe DL, Hernández-Orallo J (2014) How universal can an intelligence test be? Adapt Behavior Animals Animats Softw Agents Robots Adapt Syst Arch 22(1):51–69

  34. Everitt B (1998) The cambridge dictionary of statistics Cambridge. Cambridge University Press, New York

    MATH  Google Scholar 

  35. Fay MP, Proschan MA (2010) Wilcoxon–mann–whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules. Stat Surveys 4:1–39

    Article  MathSciNet  MATH  Google Scholar 

  36. Ferrucci D, Levas A, Bagchi S, Gondek D, Mueller ET (2013) Watson: beyond jeopardy! Artif Intell 199–200:93–105

    Article  Google Scholar 

  37. Franklin D, Abrao A (2000) Measuring software agent’s intelligence. In: Proceedings of international conference: advances in infrastructure for electronical business science and education on the internet. L’Aquila, Italy

  38. Galton F (1886) Regression towards mediocrity in hereditary stature. J Anthropol Inst G B Irel 15:246–263

    Google Scholar 

  39. Grotschel M, Padberg MW (1978) On the symmetric travelling salesman problem: theory and computation. In: Henn R, Korte B, Oettli W (eds) Optimization and operations research. Lecture notes in economics and mathematical systems. vol 157, Springer, Berlin, pp 105–115

  40. Hernandez-Orallo J, Dowe DL (2010) Measuring universal intelligence: towards an anytime intelligence test. Artif Intell 174(8):1508–1539

    Article  MathSciNet  Google Scholar 

  41. Hernández-Orallo J, Dowe DL, Hernández-Lloreda MV (2014) Universal psychometrics: measuring cognitive abilities in the machine kingdom. Cogn Syst Res 27:50–74

    Article  Google Scholar 

  42. Hibbard B (2011) Measuring agent intelligence via hierarchies of environments. Artificial General Intelligence, Lecture Notes in Computer Science 6830:303–308

    Article  Google Scholar 

  43. Hsieh FS (2017) A hybrid and scalable multi-agent approach for patient scheduling based on Petri net models. Appl Intell 7(4):1068–1086

    Article  MathSciNet  Google Scholar 

  44. Iantovics LB, Emmert-Streib F, Arik S (2017) Metrintmeas a novel metric for measuring the intelligence of a swarm of cooperating agents. Cogn Syst Res 45:17–29

    Article  Google Scholar 

  45. Iantovics LB, Rotar C, Niazi AN (2018) Metrintpair-a novel accurate metric for the comparison of two cooperative multiagent systems intelligence based on paired intelligence measurements. Int J Intell Syst 33(3):463–486

    Article  Google Scholar 

  46. Iantovics LB, Zamfirescu CB (2013) ERMS: an evolutionary reorganizing multiagent system, innovative computing. Inf Control 9(3):1171–1188

    Google Scholar 

  47. Iqbal S, Altaf W, Aslam M, Mahmood W, Khan MUG (2016) Application of intelligent agents in health-care: review. Artif Intell Rev 46(1):83–112

    Article  Google Scholar 

  48. Johnson BR, Borowiec ML, Chiu JC, Lee EK, Atallah J, Ward PS (2013) Phylogenomics resolves evolutionary relationships among ants, bees, and wasps. Curr Biol 23(20):1–5

    Article  Google Scholar 

  49. Jussila J, Vuori V, Okkonen J, Helander N (2017) Reliability and perceived value of sentiment analysis for twitter data. In: Kavoura A, Sakas D, Tomaras P (eds) Strategic innovative marketing. Springer proceedings in business and economics. Springer, Cham, pp 43–48

  50. Kafali O, Yolum P (2016) PISAGOR: a proactive software agent for monitoring interactions. Knowl Inf Syst 47(1):215–239

    Article  Google Scholar 

  51. Kwon H, Pack DJ (2012) A robust mobile target localization method for cooperative unmanned aerial vehicles using sensor fusion quality. J Intell Robot Syst 65(1):479–493

    Article  Google Scholar 

  52. Leeuwen JV (ed) (1998) Handbook of theoretical computer science, vol A. Algorithms and complexity. Elsevier, Amsterdam

  53. Lowry R Concepts & applications of inferential statistics. http://vassarstats.net/textbook

  54. Mann PS (1995) Introductory statistics, 2nd edn. Wiley, New York

    MATH  Google Scholar 

  55. Mann HB, Whitney DR (1947) On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat 18(1):50–60

    Article  MathSciNet  MATH  Google Scholar 

  56. Marusteri M, Bacarea V (2010) Comparing groups for statistical differences: how to choose the right statistical test? Biochemia Medica 20(1):15–32

    Article  Google Scholar 

  57. Merkle D, Middendorf M (2005) On solving permutation scheduling problems with ant colony optimization. Int J Syst Sci 36(5):255–266

    Article  MathSciNet  MATH  Google Scholar 

  58. Munteanu C, Rosa A (2004) Gray-scale image enhancement as an automatic process driven by evolution. IEEE Trans Syst Man Cybern B Cybern 34(2):1292–1298

    Article  Google Scholar 

  59. Myers JL, Well AD (2003) Research design and statistical analysis, 2nd edn. Lawrence Erlbaum, Mahwah, p 508

    Book  Google Scholar 

  60. Neisser U, Boodoo G, Bouchard TJ, Boykin AW, Brody N, Ceci SJ, Halpern DF, Loehlin JC, Perloff R, Sternberg RJ, Urbina S (1996) Intelligence: knowns and unknowns. Am Psychol 51(2):77–101

    Article  Google Scholar 

  61. Newborn M (1997) Kasparov vs deep blue: computer chess comes of age. Springer, New York

    Book  Google Scholar 

  62. Niazi M, Hussain A (2011) Agent-based computing from multi-agent systems to agent-based models: a visual survey. Scientometrics 89(2):479–499

    Article  Google Scholar 

  63. Nick TG (2007) Descriptive statistics. Topics in biostatistics. Methods Mol Biol 404:33–52

    Article  Google Scholar 

  64. Ouaarab A, Ahiod B, Yang XS (2014) Discrete cuckoo search algorithm for the travelling salesman problem. Neural Comput Appl 24:1659

    Article  Google Scholar 

  65. Pearson K (1895) Notes on regression and inheritance in the case of two parents. Proc R Soc Lond 58:240–242

    Article  Google Scholar 

  66. Pholdee N, Bureerat S (2016) Hybrid real-code ant colony optimisation for constrained mechanical design. Int J Syst Sci 47(2):474–491

    Article  MATH  Google Scholar 

  67. Prakasam A, Savarimuthu N (2016) Metaheuristic algorithms and probabilistic behaviour: a comprehensive analysis of ant colony optimization and its variants. Artif Intell Rev 45(1):97–130

    Article  Google Scholar 

  68. Pratt JW (1959) Remarks on zeros and ties in the Wilcoxon signed rank procedures. J Am Stat Assoc 54 (287):655–667

    Article  MathSciNet  MATH  Google Scholar 

  69. Pratt JW, Gibbons JD (1981) Concepts of nonparametric theory. Springer, New York

    Book  MATH  Google Scholar 

  70. Rosing MT (1999) 13C-depleted carbon microparticles in >3700-Ma sea-floor sedimentary rocks from West Greenland. Science 283(5402):674–676

    Article  Google Scholar 

  71. Rouse WB, Sandra H (1983) Rouse analysis and classification of human error. IEEE Trans Syst Man Cybern SMC-13(4):539—549

    Article  Google Scholar 

  72. Runkler TA (2005) Ant colony optimization of clustering models. Int J Int Syst 20:1233–1251

    Article  MATH  Google Scholar 

  73. Schreiner K (2000) Measuring IS: toward a US standard. IEEE Intell Syst Their Appl 15(5):19–21

    Article  Google Scholar 

  74. Sanghi P, Dowe DL (2003) A computer program capable of passing I.Q. tests. In: Slezak PP (ed) Proceedings of the joint international conference on cognitive science, 4th ICCS international conference on cognitive science and 7th ASCS Australasian society for cognitive science (ICCS/ASCS 2003). Sydney, NSW, Australia, pp 570–575

  75. Sharkey AJC (2006) Robots, insects and swarm intelligence. Artif Intell Rev 26(4):255–268

    Article  Google Scholar 

  76. Saska M, Vonasek V, Krajnik T, Preucil L (2014) Coordination and navigation of heterogeneous MAV–UGV formations localized by a ‘hawk-eye’-like approach under a model predictive control scheme. Int J Robot Res 33(10):1393–1412

    Article  Google Scholar 

  77. Shapiro SS, Wilk MB (1965) An analysis of variance test for normality. Biometrika 52(3-4):591–611

    Article  MathSciNet  MATH  Google Scholar 

  78. Sharpanskykh A, Haest R (2016) An agent-based model to study compliance with safety regulations at an airline ground service organization. Appl Intell 45(3):881–903

    Article  Google Scholar 

  79. Siegel S (1956) Non-parametric statistics for the behavioral sciences. McGraw-Hill, New York, pp 75–83

    Google Scholar 

  80. Siorpaes K, Simperl E (2010) Human intelligence in the process of semantic content creation. World Wide Web 13(1-2):33–59

    Article  Google Scholar 

  81. Song ZC, Ge YZ, Duan H, Qiu XG (2016) Agent-based simulation systems for emergency management. Int J Autom Comput 13(2):89–98

    Article  Google Scholar 

  82. Stigler SM (1989) Francis galton’s account of the invention of correlation. Stat Sci 4(2):73–79

    Article  MathSciNet  MATH  Google Scholar 

  83. Stutzle T, Hoos HH (1997) The MAX-MIN ant system and local search for the traveling salesman problem. In: Proceedings ICEC97. IEEE Press, Piscataway, pp 309–314

  84. Stützle T, Hoos HH (2000) MAX MIN ant system. Futur Gener Comput Syst 16:889–914

    Article  MATH  Google Scholar 

  85. Tokody D, Mezei IJ, Schuster G (2017) An overview of autonomous intelligent vehicle systems. In: Jármai K, Bolló B (eds) Vehicle and automotive engineering. Lecture notes in mechanical engineering, vol PartF12. Springer, pp 287–307

  86. Turing AM (1950) Computing machinery and intelligence. Mind 59:433–460

    Article  MathSciNet  Google Scholar 

  87. Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bull 1(6):80–83

    Article  Google Scholar 

  88. Winklerova Z (2013) Maturity of the particle swarm as a metric for measuring the collective intelligence of the swarm. Advances in Swarm Intelligence, LNCS 7928:40–54

    Article  Google Scholar 

  89. Won ZB, Do CB, Jeong YK, Han S (2002) Machine intelligence quotient: its measurements and applications. Fuzzy Sets Syst 127(1):3–16

    Article  MathSciNet  MATH  Google Scholar 

  90. Zarandi MHF, Hadavandi E, Turksen IB (2012) A hybrid fuzzy intelligent agent-based system for stock price prediction. Int J Intell Syst 27(11):947–969

    Article  Google Scholar 

  91. Zhang Y, Wang H, Zhang Y, Chen Y (2011) Best-worst ant system. In: Proceedings of the 3rd international conference on advanced computer control (ICACC), pp 392–395

  92. Yager RR (1997) Intelligent agents for World Wide Web advertising decisions. Int J Intell Syst 12(5):379–390

    Article  Google Scholar 

  93. Yang K, Galis A, Guo X, Liu D (2003) rule-driven mobile intelligent agents for real-time configuration of IP networks, knowledge-based intelligent information and engineering systems. Lect Notes Comput Sci 2773:921–928

    Article  Google Scholar 

Download references

Acknowledgments

This work has been funded by the CHIST-ERA programme supported by the Future and Emerging Technologies (FET) programme of the European Union through the ERA-NET funding scheme under the grant agreements, title SOON - Social Network of Machines.

Author information

Authors and Affiliations

  1. Informatics Department, University of Medicine, Pharmacy, Sciences and Technology of Targu Mures, Gh. Marinescu 38, Mures County, Targu Mures, 540139, Romania

    László Barna Iantovics

  2. Information Technology Department, University of Miskolc, Miskolc, Hungary

    László Kovács

  3. Computer Science Department, 1 Decembrie 1918 University, Alba Iulia, Romania

    Corina Rotar

Authors
  1. László Barna Iantovics
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. László Kovács
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Corina Rotar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to László Barna Iantovics.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(PDF 97.0 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Iantovics, L.B., Kovács, L. & Rotar, C. MeasApplInt - a novel intelligence metric for choosing the computing systems able to solve real-life problems with a high intelligence. Appl Intell 49, 3491–3511 (2019). https://doi.org/10.1007/s10489-019-01440-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01440-5

Keywords

Search

Navigation