Skip to main content
SpringerLink
Log in

Evolutionary Computation and Big Data: Key Challenges and Future Directions

  • Conference paper
  • First Online:
Book cover Data Mining and Big Data (DMBD 2016)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9714))

Included in the following conference series:

Abstract

Over the past few years, big data analytics has received increasing attention in all most all scientific research fields. This paper discusses the synergies between big data and evolutionary computation (EC) algorithms, including swarm intelligence and evolutionary algorithms. We will discuss the combination of big data analytics and EC algorithms, such as the application of EC algorithms to solving big data analysis problems and the use of data analysis methods for designing new EC algorithms or improving the performance of EC algorithms. Based on the combination of EC algorithms and data mining techniques, we understand better the insights of data analytics, and design more efficient algorithms to solve real-world big data analytics problems. Also, the weakness and strength of EC algorithms could be analyzed via the data analytics along the optimization process, a crucial entity in EC algorithms. Key challenges and future directions in combining big data and EC algorithms are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abraham, A., Grosan, C., Ramos, V. (eds.): Swarm Intelligence in Data Mining, Studies in Computational Intelligence, vol. 34. Springer, Heidelberg (2006)

    MATH  Google Scholar 

  2. Alexander, F.J., Hoisie, A., Szalay, A.: Big data. Comput. Sci. Eng. 13(6), 10–13 (2011)

    Article  Google Scholar 

  3. Bellman, R.: Adaptive Control Processes: A guided Tour. Princeton University Press, Princeton (1961)

    Book  MATH  Google Scholar 

  4. Bui, L.T., Michalewicz, Z., Parkinson, E., Abello, M.B.: Adaptation in dynamic environments: a case study in mission planning. IEEE Trans. Evol. Comput. 16(2), 190–209 (2012)

    Article  Google Scholar 

  5. Chai, T., Jin, Y., Sendhoff, B.: Evolutionary complex engineering optimization: opportunities and challenges. IEEE Comput. Intell. Mag. 8(3), 12–15 (2013)

    Article  Google Scholar 

  6. Cheng, S., Shi, Y., Qin, Q., Bai, R.: Swarm intelligence in big data analytics. In: Yin, H., Tang, K., Gao, Y., Klawonn, F., Lee, M., Weise, T., Li, B., Yao, X. (eds.) IDEAL 2013. LNCS, vol. 8206, pp. 417–426. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  7. Cheng, S., Zhang, Q., Qin, Q.: Big data analytic with swarm intelligence. Ind. Manag. Data Syst. 116(4) (2016, in press)

    Google Scholar 

  8. Coello, C.A.C., Lamont, G.B., Veldhuizen, D.A.V.: Evolutionary Algorithms for Solving Multi-Objective Problems. Genetic and Evolutionary Computation Series, 2nd edn. Springer, New York (2007)

    MATH  Google Scholar 

  9. Coello, C.A.C., Dehuri, S., Ghosh, S. (eds.): Swarm Intelligence for Multi-objective Problems in Data Mining, Studies in Computational Intelligence, vol. 242. Springer, Heidelberg (2009)

    Google Scholar 

  10. Dhar, V.: Data science and prediction. Commun. ACM 56(12), 64–73 (2013)

    Article  Google Scholar 

  11. Domingos, P.: A few useful things to know about machine learning. Commun. ACM 55(10), 78–87 (2012)

    Article  Google Scholar 

  12. Donoho, D.L.: 50 years of data science. Technical report, Stanford University September 2015

    Google Scholar 

  13. Dorigo, M., Stützle, T.: Ant Colony Optimization. MIT Press, Cambridge (2004)

    MATH  Google Scholar 

  14. Eberhart, R., Shi, Y.: Computational Intelligence: Concepts to Implementations. Morgan Kaufmann Publisher, San Francisco (2007)

    Book  MATH  Google Scholar 

  15. Fayyad, U., Piatetsky-Shapiro, G., Smyth, P.: From data mining to knowledge discovery in databases. AI Mag. 17(3), 37–54 (1996)

    Google Scholar 

  16. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Series in Statistics, 2nd edn. Springer, New York (2009)

    Book  MATH  Google Scholar 

  17. Hauschild, M., Pelikan, M.: An introduction and survey of estimation of distribution algorithms. Swarm Evol. Comput. 1(3), 111–128 (2011)

    Article  Google Scholar 

  18. Hu, J., Fu, M.C., Marcus, S.I.: A model reference adaptive search method for global optimization. Oper. Res. 55(3), 549–568 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  19. Jin, Y., Branke, J.: Evolutionary optimization in uncertain environments - a survey. IEEE Trans. Evol. Comput. 9(3), 303–317 (2005)

    Article  Google Scholar 

  20. Jin, Y., Hammer, B.: Computational intelligence in big data. IEEE Comput. Intell. Mag. 9(3), 12–13 (2014)

    Article  Google Scholar 

  21. Jin, Y., Sendhoff, B.: A systems approach to evolutionary multiobjective structural optimization and beyond. IEEE Comput. Intell. Mag. 4(3), 62–76 (2009)

    Article  Google Scholar 

  22. Kennedy, J., Eberhart, R., Shi, Y.: Swarm Intelligence. Morgan Kaufmann Publisher, San Francisco (2001)

    Google Scholar 

  23. Kim, Y.S.: Multi-objective clustering with data- and human-driven metrics. J. Comput. Inf. Syst. 51(4), 64–73 (2011)

    Google Scholar 

  24. Lee, J.A., Verleysen, M.: Nonlinear Dimensionality Reduction. Information Science and Statistics. Springer, New York (2007)

    Book  MATH  Google Scholar 

  25. Li, L., Tang, K.: History-based topological speciation for multimodal optimization. IEEE Trans. Evol. Comput. 19(1), 136–150 (2015)

    Article  Google Scholar 

  26. Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., Byers, A.H.: Big data: the next frontier for innovation, competition, and productivity. Technical report, McKinsey Global Institute, May 2011

    Google Scholar 

  27. Morrison, R.W., De Jong, K.A.: A test problem generator for non-stationary environments. In: Proceedings of the 1999 Congress on Evolutionary Computation (CEC 1999), vol. 3, pp. 2047–2053, July 1999

    Google Scholar 

  28. Pelikan, M., Goldberg, D.E., Lobo, F.G.: A survey of optimization by building and using probabilistic models. Comput. Optim. Appl. 21(1), 5–20 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  29. Rajaraman, A., Leskovec, J., Ullman, J.D.: Mining of Massive Datasets. Cambridge University Press, Cambridge (2012)

    Google Scholar 

  30. Shi, Y.: Brain storm optimization algorithm. In: Tan, Y., Shi, Y., Chai, Y., Wang, G. (eds.) ICSI 2011, Part I. LNCS, vol. 6728, pp. 303–309. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  31. Shi, Y.: An optimization algorithm based on brainstorming process. Int. J. Swarm Intell. Res. (IJSIR) 2(4), 35–62 (2011)

    Article  Google Scholar 

  32. Yang, P., Tang, K., Lu, X.: Improving estimation of distribution algorithm on multimodal problems by detecting promising areas. IEEE Trans. Cybern. 45(8), 1438–1449 (2015)

    Article  Google Scholar 

  33. Yang, S., Li, C.: A clustering particle swarm optimizer for locating and tracking multiple optima in dynamic environments. IEEE Trans. Evol. Comput. 14(6), 959–974 (2010)

    Article  Google Scholar 

  34. Yang, Z., Tang, K., Yao, X.: Differential evolution for high-dimensional function optimization. In: Proceedings of 2007 IEEE Congress on Evolutionary Computation (CEC 2007), pp. 35231–3530. IEEE (2007)

    Google Scholar 

  35. Yang, Z., Tang, K., Yao, X.: Scalability of generalized adaptive differential evolution for large-scale continuous optimization. Soft. Comput. 15(11), 2141–2155 (2011)

    Article  Google Scholar 

  36. Zhou, Z.H., Chawla, N.V., Jin, Y., Williams, G.J.: Big data opportunities and challenges: discussions from data analytics perspectives. IEEE Comput. Intell. Mag. 9(4), 62–74 (2014)

    Article  Google Scholar 

  37. Zlochin, M., Birattari, M., Meuleau, N., Dorigo, M.: Model-based search for combinatorial optimization: a critical survey. Ann. Oper. Res. 131, 373–395 (2004)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

This work is partially supported by National Natural Science Foundation of China under Grant Numbers 60975080, 61273367, 61571238, and 61302158.

Author information

Authors and Affiliations

  1. School of Computer Science, Shaanxi Normal University, Xi’an, China

    Shi Cheng

  2. School of Computer Science and Technology, Nanjing University of Posts and Telecommunications, Nanjing, China

    Bin Liu

  3. Department of Electrical and Electronic Engineering, Xi’an Jiaotong-Liverpool University, Suzhou, China

    Yuhui Shi

  4. Department of Computing, University of Surrey, Guildford, Surrey, UK

    Yaochu Jin

  5. University of Science and Technology of China, Hefei, China

    Bin Li

Authors
  1. Shi Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuhui Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yaochu Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shi Cheng .

Editor information

Editors and Affiliations

  1. Peking University, Beijing, China

    Ying Tan

  2. Xi'an Jiaotong-Liverpool University, Suzhou, China

    Yuhui Shi

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Cheng, S., Liu, B., Shi, Y., Jin, Y., Li, B. (2016). Evolutionary Computation and Big Data: Key Challenges and Future Directions. In: Tan, Y., Shi, Y. (eds) Data Mining and Big Data. DMBD 2016. Lecture Notes in Computer Science(), vol 9714. Springer, Cham. https://doi.org/10.1007/978-3-319-40973-3_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-40973-3_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-40972-6

  • Online ISBN: 978-3-319-40973-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation