Skip to main content

Advertisement

SpringerLink
Log in

Performance Evaluation of Machine Learning Methods in Cultural Modeling

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Cultural modeling (CM) is an emergent and promising research area in social computing. It aims to develop behavioral models of human groups and analyze the impact of culture factors on human group behavior using computational methods. Machine learning methods, in particular classification, play a critical role in such applications. Since various cultural-related data sets possess different characteristics, it is important to gain a computational understanding of performance characteristics of various machine learning methods. In this paper, we investigate the performance of seven representative classification algorithms using a benchmark cultural modeling data set and analyze the experimental results as to group behavior forecasting.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Subrahmanian V S. Computer science: Cultural modeling in real time. Science, 2007, 317(5844): 1509–1510.

    Article  Google Scholar 

  2. Subrahmanian V S, Albanese M, Martinez M V, Nau D, Reforgiato D, Simari G I, Sliva A, Wilkenfeld J, Udrea O. CARA: A cultural-reasoning architecture. IEEE Intelligent Systems, 2007, 22(2): 12–16.

    Article  Google Scholar 

  3. Khuller S, Martinez V, Nau D, Simari G, Sliva A, Subrahmanian V S. Finding most probable worlds of logic programs. In Proc. the First International Conference on Scalable Uncertainty Management, Washington DC, USA, October 10–12, 2007, pp.45–59.

  4. Martinez V, Simari G I, Sliva A, Subrahmanian V S. CONVEX: Context vectors as a paradigm for learning group behaviors based on similarity. IEEE Intelligent Systems, 2007, 23(4): 51–57.

    Article  Google Scholar 

  5. Wang F Y. Is culture computable? IEEE Intelligent Systems, 2009, 24(2): 2–3.

    Article  Google Scholar 

  6. Wang F Y, Carley K M, Zeng D, Mao W. Social computing: From social informatics to social intelligence. IEEE Intelligent Systems, 2007, 22(2): 79–83.

    Article  Google Scholar 

  7. Wang F Y. Toward a paradigm shift in social computing: The ACP approach. IEEE Intelligent Systems, 2007, 22(5): 65–67.

    Article  Google Scholar 

  8. Zeng D, Wang F Y, Carley K M. Social computing. IEEE Intelligent Systems, 2007, 22(5): 20–22.

    Article  MATH  Google Scholar 

  9. Minorities at risk organizational behavior dataset. Minorities at Risk Project, University of Maryland, College Park: Center for International Development and Conflict Management, 2008, http://www.cidcm.umd.edu/mar.

  10. Hand D J, Yu K. Idiot’s Bayes — Not so stupid after all? International Statistical Review, 2001, 69(3): 385–398.

    Article  MATH  Google Scholar 

  11. Vladimir N Vapnik. Support Vector Estimation of Functions. Statistical Learning Theory, Haykin S (ed.), Springer-Verlag, 1998, pp.375–570.

  12. Jain A K, Mao J, Mohiuddin K M. Artificial neural networks: A tutorial. Computer, 1996, 29(3): 31–44.

    Article  Google Scholar 

  13. Christopher M Bishop. Neural Networks for Pattern Recognition. Oxford University Press, 1995.

  14. Kotsiantis S, Zaharakis I, Pintelas P. Machine learning: A review of classification and combining techniques. Artificial Intelligence Review, 2006, 26(3): 159–190.

    Article  Google Scholar 

  15. Breiman L. Random forests. Machine Learning, 2001, 45(1): 5–32.

    Article  MATH  Google Scholar 

  16. Thabtah F. A review of associative classification mining. The Knowledge Engineering Review, 2007, 22(1): 37–65.

    Article  Google Scholar 

  17. Baralis E, Garza P. A lazy approach to pruning classification rules. In Proc. the Second IEEE International Conference on Data Mining, Maebashi, Japan, December 9–12, 2002, pp.35–42.

  18. Minorities at Risk Project. College Park, MD: Center for International Development and Conflict Management, 2005, http://www.cidcm.umd.edu/mar/.

  19. Ling C X, Huang J, Zhang H. AUC: A statistically consistent and more discriminating measure than accuracy. In Proc. the Eighteenth International Joint Conference on Artificial Intelligence, Acapulco, Mexico, August 9–15, 2003, pp. 329–341.

  20. Japkowicz N, Stephen S. The class imbalance problem: A systematic study. Intelligent Data Analysis, 2002, 6(5): 429–450.

    MATH  Google Scholar 

  21. Kohavi R, Wolpert D H. Bias plus variance decomposition for zero-one loss functions. In Proc. the Thirteenth International Conference on Machine Learning, Bari, Italy, July 3–6, 1996, pp.275–283.

  22. Govindarajan M. Text mining technique for data mining application. Proceedings of World Academy of Science, Engineering and Technology, 2007, 26(104): 544–549.

    Google Scholar 

  23. Liu B, Hsu W, Ma Y. Integrating classification and association rule mining. In Proc. the Third International Conference on Knowledge Discovery and Data Mining, New York, USA, August 27–31, 1998, pp.80–86.

  24. Li W, Han J, Pei J. CMAR: Accurate and efficient classification based on multiple class-association rules. In Proc. the First IEEE International Conference on Data Mining, San Jose, USA, November 29–December 2, 2001, pp.369–376.

  25. Yin X, Han J. CPAR: Classification based on predictive association rules. In Proc. the Third SIAM International Conference on Data Mining, San Francisco, USA, May 1–3, 2003, pp.369–376.

  26. Grčar M, Mladenič D, Fortuna B, Grobelnik M. Data sparsity issues in the collaborative filtering framework. In Proc. the Seventh International Workshop on Knowledge Discovery on the Web, Chicago, USA, August 21, 2005, pp.58–76.

  27. Nathalie J. Class imbalances: Are we focusing on the right issue? In Proc. the ICML 2003 Workshop on Learning from Imbalanced Data Sets, Washington DC, USA, August 21, 2003.

  28. Japkowicz N. The class imbalance problem: Significance and strategies. In Proc. the Second International Conference on Artificial Intelligence, Las Vegas, USA, June 26–29, 2000, pp.111–117.

  29. Zhang J. kNN approach to unbalanced data distributions: A case study involving information extraction. In Proc. the ICML2003 Workshop on Learning from Imbalanced Data Sets, Washington DC, USA, August 21, 2003.

  30. Barandela R, Sanchez J S, Garcia V, Rangel E. Strategies for learning in class imbalance problems. Pattern Recognition, 2003, 36(3): 849–851.

    Article  Google Scholar 

  31. Rish I, Hellerstein J, Thathachar J. An analysis of data characteristics that affect naive Bayes performance. Technical Report, IBM T.J. Watson Research Center, 2001.

  32. Domingos P, Pazzani M. On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning, 1997, 29(2/3): 103–130.

    Article  MATH  Google Scholar 

  33. Martinez-Arroyo M, Sucar L E. Learning an optimal naive Bayes classifier. In Proc. the Eighteenth International Conference on Pattern Recognition, Hong Kong, China, August 20–24, 2006, p.958.

  34. Zhang C, Yu P S, Bell D. Domain-driven data mining. IEEE Transactions on Knowledge and Data Engineering, 2009, 21(2): 301.

    Article  Google Scholar 

  35. Cao L, Zhang C. Domain-driven, actionable knowledge discovery. IEEE Intelligent Systems, 2007, 22(4): 78–88.

    Article  Google Scholar 

  36. Nau D, Wilkenfeld J. Computational cultural dynamics. IEEE Intelligent Systems, 2008. 23(4): 18–19.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Key Laboratory of Complex Systems and Intelligence Science, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Xiao-Chen Li, Wen-Ji Mao (Senior Member, CCF, Member, ACM), Daniel Zeng (Senior Member, IEEE), Peng Su & Fei-Yue Wang (Senior Member CCF, Fellow, IEEE)

  2. Department of Management Information Systems, University of Arizona, Tucson, AZ, 85721, U.S.A.

    Daniel Zeng (Senior Member, IEEE)

  3. School of Management Engineering, Shandong Jianzhu University, Jinan, 250013, China

    Peng Su

Authors
  1. Xiao-Chen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wen-Ji Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peng Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fei-Yue Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wen-Ji Mao.

Additional information

This work is supported in part by the National Natural Science Foundation of China under Grant Nos. 60621001, 60875028, 60875049, and 70890084, the Ministry of Science and Technology of China under Grant No. 2006AA010106, and the Chinese Academy of Sciences under Grant Nos. 2F05N01, 2F08N03 and 2F07C01.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Li, XC., Mao, WJ., Zeng, D. et al. Performance Evaluation of Machine Learning Methods in Cultural Modeling. J. Comput. Sci. Technol. 24, 1010–1017 (2009). https://doi.org/10.1007/s11390-009-9290-8

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-009-9290-8

Keywords

Search

Navigation