Skip to main content
SpringerLink
Log in
Book cover

Proceedings of Fifth International Conference on Soft Computing for Problem Solving pp 345–353Cite as

Improvising and Optimizing Resource Utilization in Big Data Processing

  • Conference paper
  • First Online:

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 436))

Abstract

This paper is to improvising and optimizing the scenario of Big data processing in cloud computing. A homogeneous cluster setup supports static nature of processing which is a huge disadvantage for optimizing the response time towards clients. In order to avail utmost client satisfaction, the host server needs to be upgraded with the latest technology to fulfil all requirements. Big data processing is a common frequent event in today’s Internet and the proposed framework improvises the response time. This will also make sure that the user gets its entire requirement fulfilled in optimal time. In order to avail utmost client satisfaction, the server needs to eliminate homogeneous cluster setup that is encountered usually in parallel data processing. The homogeneous cluster setup is static in nature and dynamic allocation of resources is not possible in this kind of environment. This will improve the overall resource utilization and, consequently, reduce the processing cost.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Wu, S., Li, F., Mehrotra, S., Ooi, B.C.: Query optimization for massively parallel data processing. School of Computing, National University of Singapore, March 2012

    Google Scholar 

  2. Parallel Data Processing. http://server-demo-ec2.cloveretl.com/clover/docs/clustering-parallel-processing.html

  3. Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. In OSDI’04: Proceedings of the 6th Conference on Symposium on Operating Systems Design & Implementation, pp. 10–10. Berkeley, CA, USA, 2004. USENIX Association

    Google Scholar 

  4. Chih Yang, H., Dasdan, A., Hsiao, R.-L., Parker, D.S.: Map-Reduce-Merge: simplified relational data processing on large clusters. In: SIGMOD’07: Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data, pp. 1029–1040. New York, NY, USA, 2007. ACM

    Google Scholar 

  5. Lee, K.H., Lee, Y.J.: Big data processing with Map Reduce: A Survey. Department of Computer Science KAIST, December 2011

    Google Scholar 

  6. Gropp, W., Lusk, E., Skjellum, A.: Using MPI: Portable Parallel Programming with the Message-Passing Interface. MIT Press, Cambridge, MA (1999)

    Google Scholar 

  7. Deelman, E., Singh, G., Su, M.-H., Blythe, J., Gil, Y., Kesselman, C., Mehta, G., Vahi, K., Berriman, G.B., Good, J., Laity, A., Jacob, J.C., Katz, D.S.: Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Sci. Program. 13(3), 219–237 (2005)

    Google Scholar 

  8. Pike, R., Dorward, S., Griesemer, R., Quinlan, S.: Interpreting the data: parallel analysis with Sawzall. Sci. Program. 13(4), 277–298 (2005)

    Google Scholar 

  9. Thain, D., Tannenbaum, T., Livny, M.: Distributed computing in practice: the condor experience. Concurrency Comput.: Pract. Exp. (2004)

    Google Scholar 

  10. Pike, R., Dorward, S., Griesemer, R., Quinlan, S.: Interpreting the data: parallel analysis with Sawzall. Sci. Program. 13(4), 277–298 (2005)

    Google Scholar 

  11. Jiang, D., et al.: Map-join-reduce: towards scalable and efficient data analysis on large clusters. IEEE Trans. Knowl. Data Eng. (2010)

    Google Scholar 

  12. Li, B., et al.: A platform for scalable one-pass analytics using MapReduce. In: Proceedings of the 2011 ACM SIGMOD, 2011

    Google Scholar 

  13. Babu, S.: Towards automatic optimization of map reduce programs. In: Proceedings of the 1st ACM Symposium on Cloud Computing, pp. 137–142 (2010)

    Google Scholar 

  14. Arpaci-Dusseau, A.C., Arpaci-Dusseau, R.H., Culler, D.E., Hellerstein, J.M., Patterson, D.A.: High-performance sorting on networks of workstations. In: Proceedings of the 1997 ACM SIGMOD International Conference on Management of Data, Tucson, Arizona, May 1997

    Google Scholar 

  15. Thusoo, A., Sarma, J.S., Jain, N., Shao, Z., Chakka, P., Anthony, S., Liu, H., Wychoff, P., Murthy, R.: Hive—a warehousing solution over a map-reduce framework. In: VLDB, 2009

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, NIMS University, Jaipur, India

    Praveen Kumar

  2. Shri Karni College, Jaipur, Rajasthan, India

    Vijay Singh Rathore

Authors
  1. Praveen Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vijay Singh Rathore
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Praveen Kumar .

Editor information

Editors and Affiliations

  1. Dept of Applied Sci & Eng, Indian Instit of Tech Roorkee, Roorkee, India

    Millie Pant

  2. Department of Mathematics, Indian Inst of Tech Roorkee, Roorkee, India

    Kusum Deep

  3. Chankyapuri, Rm 327, South Asian Univ, Akbar Bhawan, New Delhi, India

    Jagdish Chand Bansal

  4. Department of Mathematics and Comp Sci, Liverpool Hope University, LIVERPOOL, United Kingdom

    Atulya Nagar

  5. Department of Mathematics, National Inst of Tech Silchar, Silchar, Assam, India

    Kedar Nath Das

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer Science+Business Media Singapore

About this paper

Cite this paper

Praveen Kumar, Rathore, V.S. (2016). Improvising and Optimizing Resource Utilization in Big Data Processing. In: Pant, M., Deep, K., Bansal, J., Nagar, A., Das, K. (eds) Proceedings of Fifth International Conference on Soft Computing for Problem Solving. Advances in Intelligent Systems and Computing, vol 436. Springer, Singapore. https://doi.org/10.1007/978-981-10-0448-3_28

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-0448-3_28

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-0447-6

  • Online ISBN: 978-981-10-0448-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation