Skip to main content
Springer Nature Link
Log in

Dynamic Multi-Objective Optimization with jMetal and Spark: A Case Study

  • Conference paper
  • First Online:
Machine Learning, Optimization, and Big Data (MOD 2016)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10122))

Included in the following conference series:

Abstract

Technologies for Big Data and Data Science are receiving increasing research interest nowadays. This paper introduces the prototyping architecture of a tool aimed to solve Big Data Optimization problems. Our tool combines the jMetal framework for multi-objective optimization with Apache Spark, a technology that is gaining momentum. In particular, we make use of the streaming facilities of Spark to feed an optimization problem with data from different sources. We demonstrate the use of our tool by solving a dynamic bi-objective instance of the Traveling Salesman Problem (TSP) based on near real-time traffic data from New York City, which is updated several times per minute. Our experiment shows that both jMetal and Spark can be integrated providing a software platform to deal with dynamic multi-optimization problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/jMetal/jMetalSP.

  2. 2.

    At the time of writing this paper, the data can be obtained from this URL: http://207.251.86.229/nyc-links-cams/LinkSpeedQuery.txt.

References

  1. Editorial: Community cleverness required. Nature 455, 1 (2008)

    Google Scholar 

  2. White, T.: Hadoop: The Definitive Guide, 1st edn. O’Reilly Media Inc., Sebastopol (2009)

    Google Scholar 

  3. Zaharia, M., Chowdhury, M., Franklin, M., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. In: Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing, Berkeley, CA, USA, HotCloud 2010, pp. 10. USENIX Association (2010)

    Google Scholar 

  4. Marr, M.: Big Data: Using SMART Big Data Analytics and Metrics to Make Better Decisions and Improve Performance. Wiley, Hoboken (2015)

    Google Scholar 

  5. Nam, T., Pardo, T.: Smart city as urban innovation: focusing on management, policy, and context. In: Proceedings of the 5th International Conference on Theory and Practice of Electronic Governance, ICEGOV 2011, pp. 185–194. ACM (2011)

    Google Scholar 

  6. Garcia-Nieto, J., Olivera, A., Alba, E.: Optimal cycle program of traffic lights with particle swarm optimization. IEEE Trans. Evol. Comput. 17, 823–839 (2013)

    Article  Google Scholar 

  7. NYCDOT: New York City traffic speed detectors data set (2016). http://nyctmc.org

  8. Papadimitriou, C.H.: The Euclidean travelling salesman problem is NP-complete. Theor. Comput. Sci. 4, 237–244 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  9. Gartner Inc.: Gartner’s hype cycle for advanced analytics and data science (2015). https://www.gartner.com/doc/3087721/hype-cycle-advanced-analytics-data

  10. Durillo, J., Nebro, A.: jMetal: a java framework for multi-objective optimization. Adv. Eng. Softw. 42, 760–771 (2011)

    Article  Google Scholar 

  11. Deb, K.: Multi-objective Optimization Using Evolutionary Algorithms. Wiley, New York (2001)

    MATH  Google Scholar 

  12. Coello, C., Lamont, G., van Veldhuizen, D.: Multi-objective Optimization Using Evolutionary Algorithms, 2nd edn. Wiley, New York (2007)

    MATH  Google Scholar 

  13. Farina, M., Deb, K., Amato, P.: Dynamic multiobjective optimization problems: test cases, approximations, and applications. IEEE Trans. Evol. Comput. 8, 425–442 (2004)

    Article  MATH  Google Scholar 

  14. Nebro, A., Durillo, J.J., Vergne, M.: Redesigning the jMetal multi-objective optimization framework. In: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO Companion 2015, pp. 1093–1100. ACM, New York (2015)

    Google Scholar 

  15. Reinelt, G.: TSPLIB - a traveling salesman problem library. INFORMS J. Comput. 3, 376–384 (1991)

    Article  MATH  Google Scholar 

  16. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6, 182–197 (2002)

    Article  Google Scholar 

  17. Google Inc.: Encoded polyline algorithm format (2016). https://developers.google.com/maps/documentation/utilities/polylinealgorithm

  18. Google Inc.: Google maps distance matrix API (2016). https://developers.google.com/maps/documentation/distance-matrix

Download references

Acknowledgments

This work is partially funded by Grants TIN2011-25840 (Ministerio de Ciencia e Innovación) and P11-TIC-7529 and P12-TIC-1519 (Plan Andaluz de Investigación, Desarrollo e Innovación). Cristóbal Barba-González is supported by Grant BES-2015-072209 (Ministerio de Economía y Competitividad).

Author information

Authors and Affiliations

  1. European Organization for Nuclear Research (CERN), Geneva, Switzerland

    José A. Cordero

  2. Khaos Research Group, Ada Byron Research Building, Departamento de Lenguajes y Ciencias de la Computación, University of Málaga, Málaga, Spain

    Antonio J. Nebro, Cristóbal Barba-González, José García-Nieto, Ismael Navas-Delgado & José F. Aldana-Montes

  3. Distributed and Parallel Systems Group, University of Innsbruck, Innsbruck, Austria

    Juan J. Durillo

Authors
  1. José A. Cordero
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonio J. Nebro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cristóbal Barba-González
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Juan J. Durillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. José García-Nieto
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ismael Navas-Delgado
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. José F. Aldana-Montes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Antonio J. Nebro .

Editor information

Editors and Affiliations

  1. Department of Industrial and Systems Engineering, University of Florida, Gainesville, Florida, USA

    Panos M. Pardalos

  2. Semantic Technology Laboratory, National Research Council (CNR), Catania, Italy

    Piero Conca

  3. Dipartimento di Sociologia e Metodi della Ricerca Sociale, Università di Catania, Catania, Italy

    Giovanni Giuffrida

  4. Department of Mathematics and Computer Science, University of Catania, Catania, Italy

    Giuseppe Nicosia

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Cordero, J.A. et al. (2016). Dynamic Multi-Objective Optimization with jMetal and Spark: A Case Study. In: Pardalos, P., Conca, P., Giuffrida, G., Nicosia, G. (eds) Machine Learning, Optimization, and Big Data. MOD 2016. Lecture Notes in Computer Science(), vol 10122. Springer, Cham. https://doi.org/10.1007/978-3-319-51469-7_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-51469-7_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-51468-0

  • Online ISBN: 978-3-319-51469-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics