Skip to main content
Springer Nature Link
Log in

Lightweight component-based scalability

  • Regular Paper
  • Published:
International Journal on Digital Libraries Aims and scope Submit manuscript

Abstract

Digital libraries and information management systems are increasingly being developed according to component models with well-defined APIs and often with Web-accessible interfaces. In parallel with metadata access and harvesting, Web 2.0 mashups have demonstrated the flexibility of developing systems as independent distributed components. It can be argued that such distributed components also can be an enabler for scalability of service provision in medium to large systems. To test this premise, this article discusses how an existing component framework was modified to include support for scalability. A set of lightweight services and extensions were created to migrate and replicate services as the load changes. Experiments with the prototype system confirm that this system can in fact be quite effective as an enabler of transparent and efficient scalability, without the need to resort to complex middleware or substantial system reengineering. Finally, specific problems areas have been identified as future avenues for exploration at the crucial intersection of digital libraries and high-performance computing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Anderson, D.P., Cobb, J., Korpela, E., Lebofsky M., Werthimer, D.: SETI@home: an experiment in public-resource computing. Commun. ACM 45(11), 56–61. ACM Press, New York (2002)

    Google Scholar 

  2. Andresen, D., Yang, T., Egecioglu, O., Ibarra, O.H., Smith, T.R.: Scalability issues for high performance digital libraries on the world wide web. Technical Report, March 1996, Department of Computer Science, University of California, Santa Barbara (1996)

  3. Bar, M.: openMosix, a Linux Kernel Extension for Single System Image Clustering. In: Proceedings of Linux Kongress: 10th International Linux System Technology Conference, Saarbrücken, Germany, 15–16 October, pp. 94–102 (2003)

  4. Candela, L., Castelli, D., Pagano, P.: A reference architecture for digital library systems. In: DELOS Conference on Digital Libraries. Grand Hotel Continental, Tirrenia, Pisa, Italy, 13–14 February (2007)

  5. Castelli, D., Pagano, P.: A system for building expandable digital libraries. In: Delcambre, L., Henry, G. (ed.) Third ACM/IEEE-CS Joint Conference on Digital Libraries, Houston, USA, 27–31 May, 335–345. IEEE Computer Society, Washington, DC, USA (2003)

  6. Catalyst IT.: Technical Evaluation of selected Open Source Repository Systems. Open Access Repositories in New Zealand. http://eduforge.org/docman/view.php/131/1062/Repository%20Evaluation%20Document.pdf (2006)

  7. Don, K.J., Bainbridge, D., Witten, I.H.: The design of Greenstone 3: an agent based dynamic digital library. Technical report, December 2002. Department of Computer Science, University of Waikato, Hamilton, New Zealand. http://www.greenstone.org/manuals/gs3design.pdf (2002)

  8. Foster, I.: Globus toolkit version 4: software for service-oriented systems. In: Gaudiot, J., Ni, L. (eds.) IFIP International Conference on Network and Parallel Computing, Beijing, China, 30 November–3 December, LNCS 3779, pp. 2–13. Springer, Berlin (2005)

  9. Glasner, J.: Britannica Redux. Wired Magazine, March 1999. http://www.wired.com/techbiz/media/news/1999/11/32278

  10. Haedstrom, M.: Research challenges in digital archiving and long-term preservation. In: Proceedings of NSF Post Digital Library Futures Workshop, Cape Cod, 15–17 June. http://www.sis.pitt.edu/~dlwkshop/paper_hedstrom.html (2003)

  11. About OpenPBS. http://www.openpbs.org/about.html

  12. IBM World Community Grid. http://www.worldcommunitygrid.org/about_us/viewAboutUs.do

  13. Imafouo, A.: A scalability survey in IR and DL. TCDL Bulletin, 2, 2. http://www.ieee-tcdl.org/Bulletin/v2n2/imafouo/imafouo.html

  14. Lagoze, C., Davis, J. R.: Dienst—an architecture for distributed document libraries. Commun. ACM, 38(4), 47. ACM Press, New York (1995)

    Google Scholar 

  15. Lagoze, C., Van de Sompel, H.: The open archives initiative: building a low barrier interoperability framework. In: Borgman, C. (ed.) JCDL 2001, Roanoke, VA., USA, 17–23 June, pp. 54–62. ACM Press, New York, NY (2001)

  16. MPI-2: Extensions to the Message-Passing Interface. MPI-1.2 and MPI-2 standards. http://www.mpi-forum.org/docs/mpi2-report.pdf

  17. Naughton, T., Scott, S.L., Barrett, B., Squyres, J., Lumsdaine. A, Fang, Y., Mashayekhi, V.: Looking Inside the OSCAR Cluster Toolkit. In: Dell Powers Solutions. http://www.csm.ornl.gov/PR/dell_1.html (2002)

  18. Omar, M.: Component-based digital library scalability using clusters. MSc Thesis, University of Cape Town (2007)

  19. Papadopoulos, P.M., Katz, M.J., Bruno, G.: NPACI rocks: tools and techniques for easily deploying manageable Linux clusters. In: 3rd IEEE International Conference on Cluster Computing (CLUSTER’01), Sutton Place Hotel, Newport Beach, California, USA, 8–11 October, 258. IEEE (2001)

  20. Staples, T., Wayland, R., Payette, S.: The Fedora Project: An Open-source Digital Object Repository System. D-Lib Magazine, 9, 4. http://www.dlib.org/dlib/april03/staples/04staples.html April (2003)

  21. Suleman, H., Fox, E.A.: A Framework for Building Open Digital Libraries. D-Lib Magazine, 7, 12. http://www.dlib.org/dlib/december01/suleman/12suleman.html December (2001)

  22. Suleman, H.: Analysis and Evaluation of Service-Oriented Architectures for Digital Libraries. In: Turker, C., Agosti, M., Schek, H. (eds.) Peer-to-Peer, Grid, and Service-Orientation in Digital Library Architectures, 6th Thematic Workshop of the EU Network of Excellence DELOS, Cagliari, Italy, 24–25 June 2004, Revised Selected Papers. Lecture Notes in Computer Science 3664, 130–146. Springer, Berlin. http://pubs.cs.uct.ac.za/archive/00000278/01/delos_2005_paper_eval_full_revised.pdf (2005)

  23. Suleman, H.: Parallelising harvesting. In: Sugimoto, S., Hunter, J., Rauber, A., Morishima, A. (eds.) Proceedings of 9th International Conference on Asian Digital Libraries (ICADL 2006), Tokyo, Japan, 27–30 November, 81–90. Springer, Berlin. http://pubs.cs.uct.ac.za/archive/00000328/01/icadl_2006_oaihpc.pdf (2006)

  24. Suleman, H., Fox, E.A.: Designing protocols in support of digital library componentization. In: Agosti, M., Thanos, C. (eds.) Proceedings of 6th European Conference on Research and Advanced Technology for Digital Libraries (ECDL2002), LNCS 2458, Rome, Italy, 16–18 September, pp. 568–582. Springer, Berlin. http://www.husseinsspace.com/publications/ecdl_2002_paper_odl.pdf (2002)

  25. Sunderam V.S.: PVM: A framework for parallel distributed computing. Concurrency Practice Exp. 2, 315–339 (1990)

    Article  Google Scholar 

  26. Su, A., Choffnes, D.R., Kuzmanovic, A., Bustamante, F.E.: Drafting behind Akamai (travelocity-based detouring). In: Proceedings of the Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM 2006), Pisa, Italy, 11–15 September, pp. 435–446. ACM, New York (2006)

  27. Tannenbaum, T., Wright, D., Miller, K., Livny, M.: Condor—A Distributed Job Scheduler. In: Sterling, T. (ed.) Beowulf Cluster Computing with Linux. The MIT Press, Cambridge, MA, USA (2002)

  28. van de Sompel, H., Bekaert J., Liu, X.: aDORe: a modular and standards-based digital object repository at the los alamos national laboratory. In: Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL’05), 367, ACM, New York (2005)

Download references

Author information

Authors and Affiliations

  1. University of Cape Town, Cape Town, South Africa

    Hussein Suleman, Christopher Parker & Muammar Omar

Authors
  1. Hussein Suleman
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Christopher Parker
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Muammar Omar
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Hussein Suleman.

Additional information

This project was made possible by funding from University of Cape Town and NRF (Grant number: GUN2073203).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Suleman, H., Parker, C. & Omar, M. Lightweight component-based scalability. Int J Digit Libr 9, 115–124 (2008). https://doi.org/10.1007/s00799-008-0047-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00799-008-0047-8

Keywords