Skip to main content
SpringerLink
Log in

Scheduling Algorithms for Distributed Cosmic Ray Detection Using Apache Mesos

  • Conference paper
  • First Online:
High Performance Computing (CARLA 2016)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 697))

Included in the following conference series:

Abstract

This article presents two scheduling algorithms applied to the processing of astronomical images to detect cosmic rays on distributed memory high performance computing systems. We extend our previous article that proposed a parallel approach to improve processing times on image analysis using the Image Reduction and Analysis Facility IRAF software and the Docker project over Apache Mesos. By default, Mesos introduces a simple list scheduling algorithm where the first available task is assigned to the first available processor. On this paper we propose two alternatives for reordering the tasks allocation in order to improve the computational efficiency. The main results show that it is possible to reduce the makespan getting a speedup = 4.31 by adjusting how jobs are assigned and using Uniform processors.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Tancredi, G., Cromwell, G., Deustua, S., Gonzalez, G., Nesmachnow, S., Schnyder, G.: Geophysics using Hubble Space Telescope. Hubble Space Telescope Cycle 24 approved proposal (2016)

    Google Scholar 

  2. NOAO: IRAF Project Home Page, July 2016. http://iraf.noao.edu/

  3. Schnyder, G., Nesmachnow, S.: Improving the performance of cosmic ray detection using Apache Mesos. In: International Supercomputing Conference in México (2016)

    Google Scholar 

  4. The Apache Software Foundation: Mesos, July 2016. http://mesos.apache.org/

  5. Mesosphere Inc.: Marathon: a cluster-wide init and control system for services in cgroups or Docker containers, July 2016. https://mesosphere.github.io/marathon/

  6. The Apache Software Foundation: Apache ZooKeeper, July 2016. http://zookeeper.apache.org/

  7. Golpayegani, N., Halem, M.: Cloud computing for satellite data processing on high end compute clusters. In: International Conference on Cloud Computing (2009)

    Google Scholar 

  8. Ali, M., Kumar, J.: Implementation of image processing system using handover technique with map reduce based on big data in the cloud environment. Int. Arab J. Inf. Technol. 13(2), 326–331 (2016)

    Google Scholar 

  9. Adam, T.L., Chandy, K.M., Dickson, J.R.: A comparison of list schedules for parallel processing systems. Commun. ACM 17(12), 685–690 (1974)

    Article  MATH  Google Scholar 

  10. Coffman, E.G., Sethi, R.: Algorithms minimizing mean flow time: schedule-length properties. Acta Informatica 6(1), 1–14 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  11. Graham, R.L.: Bounds on multiprocessing timing anomalies. SIAM J. Appl. Math. 17(2), 416–429 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  12. Kovács, A.: Tighter approximation bounds for LPT scheduling in two special cases. J. Discret. Algorithms 7(3), 327–340 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  13. Oyetunji, E.O.: Some common performance measures in scheduling problems: review article. Res. J. Appl. Sci. Eng. Technol. 1(2), 6–9 (2009)

    Google Scholar 

  14. Wiley, K., Connolly, A., Gardner, J., Krughoff, S., Balazinska, M., Howe, B., Kwon, Y., Bu, Y.: Astronomy in the cloud: using MapReduce for image co-addition. Publ. Astron. Soc. Pac. 123(901), 366–380 (2011)

    Article  Google Scholar 

  15. Singh, N., Browne, L.M., Butler, R.: Parallel astronomical data processing with Python: recipes for multicore machines. Astron. Comput. 2, 1–10 (2013)

    Article  Google Scholar 

  16. Graham, R., Lawler, E., Lenstra, J., Kan, A.: Optimization, approximation in deterministic sequencing, scheduling: a survey. Ann. Discret. Math. 5, 287–326 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  17. Eshaghian, M.: Heterogeneous Computing. Artech House, Norwood (1996)

    Google Scholar 

  18. Horowitz, E., Sahni, S.: Exact and approximate algorithms for scheduling nonidentical processors. J. ACM 23(2), 317–327 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  19. Nesmachnow, S.: Parallel multiobjective evolutionary algorithms for batch scheduling in heterogeneous computing and grid systems. Comput. Optim. Appl. 55(2), 515–544 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  20. Leung, J., Kelly, L., Anderson, J.: Handbook of Scheduling: Algorithms, Models, and Performance Analysis. CRC Press Inc., Boca Raton (2004)

    Google Scholar 

  21. Cirne, W., Brasileiro, F., Sauvé, J., Andrade, N., Paranhos, D., Santos-Neto, E.: Grid computing for bag of tasks applications. In: Proceedings of 3rd IFIP Conference on E-Commerce, E-Business and E-Government (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Universidad de la República, Montevideo, Uruguay

    Germán Schnyder, Sergio Nesmachnow & Gonzalo Tancredi

  2. CICESE Research Center, Ensenada, Baja California, Mexico

    Andrei Tchernykh

Authors
  1. Germán Schnyder
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sergio Nesmachnow
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gonzalo Tancredi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andrei Tchernykh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Germán Schnyder .

Editor information

Editors and Affiliations

  1. Universidad Industrial de Santander, Bucaramanga, Colombia

    Carlos Jaime Barrios Hernández

  2. Centro de Investigación y de Estudios Avanzados, CINVESTAV-IPN, Ciudad de México, Mexico

    Isidoro Gitler

  3. Instituto Nacional de Investigaciones Nucleares, La Marquesa, Estado de México, Mexico

    Jaime Klapp

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Schnyder, G., Nesmachnow, S., Tancredi, G., Tchernykh, A. (2017). Scheduling Algorithms for Distributed Cosmic Ray Detection Using Apache Mesos. In: Barrios Hernández, C., Gitler, I., Klapp, J. (eds) High Performance Computing. CARLA 2016. Communications in Computer and Information Science, vol 697. Springer, Cham. https://doi.org/10.1007/978-3-319-57972-6_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-57972-6_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-57971-9

  • Online ISBN: 978-3-319-57972-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation