Abstract
During the winter semester 1996/1997 Kurt Mehlhorn gave a lecture series on algorithms for very large data sets. Towards the end he covered graph traversal problems like finding shortest paths on sparse graphs in external-memory. Kurt concluded the topic stating that there may not be much hope to solve these basic problems I/O-efficiently. The author, just about to finish his master’s studies those days, took this as a challenge. The following paper reviews some detours, dead-ends, and happy ends of the author’s still ongoing research on external-memory graph traversal.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Abello, J., Buchsbaum, A., Westbrook, J.: A functional approach to external graph algorithms. Algorithmica 32(3), 437–458 (2002)
Aggarwal, A., Vitter, J.S.: The input/output complexity of sorting and related problems. Communications of the ACM 31(9), 1116–1127 (1988)
Ahuja, R.K., Magnanti, T.L., Orlin, J.B.: Network flows: Theory, Algorithms and Applications. Prentice Hall, Englewood Cliffs (1993)
Ajwani, D., Beckmann, A., Jacob, R., Meyer, U., Moruz, G.: On computational models for flash memory devices. In: Proc. 8th Int. Symposium on Experimental Algorithms (SEA). LNCS. Springer, Heidelberg (2009)
Ajwani, D., Malinger, I., Meyer, U., Toledo, S.: Characterizing the performance of flash memory storage devices and its impact on algorithm design. In: McGeoch, C.C. (ed.) WEA 2008. LNCS, vol. 5038, pp. 208–219. Springer, Heidelberg (2008)
Ajwani, D., Dementiev, R., Meyer, U.: A computational study of external-memory BFS algorithms. In: SODA, pp. 601–610 (2006)
Ajwani, D., Meyer, U., Osipov, V.: Improved external memory BFS implementation. In: Proc. Workshop on Algorithm Engineering and Experiments, ALENEX. SIAM, Philadelphia (2007)
Allulli, L., Lichodzijewski, P., Zeh, N.: A faster cache-oblivious shortest-path algorithm for undirected graphs with bounded edge lengths. In: Proceedings of the 18th annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pp. 910–919 (2007)
Arge, L., Meyer, U., Toma, L.: External memory algorithms for diameter and all-pairs shortest-paths on sparse graphs. In: Díaz, J., Karhumäki, J., Lepistö, A., Sannella, D. (eds.) ICALP 2004. LNCS, vol. 3142, pp. 146–157. Springer, Heidelberg (2004)
Arge, L., Meyer, U., Toma, L., Zeh, N.: On external-memory planar depth first search. J. Graph Algorithms Appl. 7(2), 105–129 (2003)
Brodal, G., Fagerberg, R., Meyer, U., Zeh, N.: Cache-oblivious data structures and algorithms for undirected breadth-first search and shortest paths. In: Hagerup, T., Katajainen, J. (eds.) SWAT 2004. LNCS, vol. 3111, pp. 480–492. Springer, Heidelberg (2004)
Chen, M., Chowdhury, R.A., Ramachandran, V., Roche, D.L., Tong, L.: Priority queues and Dijkstra’s algorithm. Technical Report TR-07-54, The University of Texas at Austin, Department of Computer Sciences (October 2007)
Chiang, Y.-J., Goodrich, M.T., Grove, E.F., Tamasia, R., Vengroff, D.E., Vitter, J.S.: External memory graph algorithms. In: Proc. 6th Ann. Symposium on Discrete Algorithms, pp. 139–149. ACM-SIAM, New York (1995)
Chowdhury, R.A., Ramachandran, V.: External-memory exact and approximate all-pairs shortest-paths in undirected graphs. In: Proc. 16th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pp. 735–744 (2005)
Chowdury, R., Ramachandran, V.: External-memory exact and approximate all-pairs shortest-paths in undirected graphs. In: Proc. 16th Ann. Symposium on Discrete Algorithms (SODA), pp. 735–744. ACM-SIAM, New York (2005)
Cormen, T.H., Leiserson, C.E., Rivest, R.L.: Introduction to Algorithms. McGraw-Hill, New York (1990)
Crauser, A., Mehlhorn, K., Meyer, U., Sanders, P.: A parallelization of dijkstra’s shortest path algorithm. In: Brim, L., Gruska, J., Zlatuška, J. (eds.) MFCS 1998. LNCS, vol. 1450, pp. 722–731. Springer, Heidelberg (1998)
de Kunder, M.: World wide web size (2009), http://www.worldwidewebsize.com/
Dementiev, R., Kettner, L., Sanders, P.: STXXL: Standard Template library for XXL data sets. Software: Practice and Experience 38(6), 589–637 (2008)
Dijkstra, E.W.: A note on two problems in connexion with graphs. Numerische Mathematik 1, 269–271 (1959)
Donato, D., Laura, L., Leonardi, S., Meyer, U., Millozzi, S., Sibeyn, J.F.: Algorithms and experiments for the webgraph. J. Graph Algorithms Appl. 10(2), 219–236 (2006)
Edelkamp, S., Meyer, U.: Theory and practice of time-space trade-offs in memory limited search. In: Baader, F., Brewka, G., Eiter, T. (eds.) KI 2001. LNCS, vol. 2174, pp. 169–184. Springer, Heidelberg (2001)
Fredman, M.L., Tarjan, R.E.: Fibonacci heaps and their uses in improved network optimization algorithms. Journal of the ACM 34, 596–615 (1987)
Frigo, M., Leiserson, C.E., Prokop, H., Ramachandran, S.: Cache-oblivious algorithms. In: 40th Annual Symposium on Foundations of Computer Science, pp. 285–297. IEEE Computer Society Press, Los Alamitos (1999)
Goldberg, A.V.: A simple shortest path algorithm with linear average time. In: Meyer auf der Heide, F. (ed.) ESA 2001. LNCS, vol. 2161, pp. 230–241. Springer, Heidelberg (2001)
Goldberg, A.V.: Shortest path algorithms: Engineering aspects. In: Eades, P., Takaoka, T. (eds.) ISAAC 2001. LNCS, vol. 2223, pp. 502–513. Springer, Heidelberg (2001)
Goldberg, A.V.: A practical shortest path algorithm with linear expected time. SIAM J. Comput. 37(5), 1637–1655 (2008)
Golden, B., Magnanti, T.: Transportation planning: Network models and their implementation. Studies in Operations Management, 365–518 (1978)
Hagerup, T.: Simpler computation of single-source shortest paths in linear average time. Theory Comput. Syst. 39(1), 113–120 (2006)
Haverkort, H., Toma, L.: I/O-efficient algorithms on near-planar graphs. In: Correa, J.R., Hevia, A., Kiwi, M. (eds.) LATIN 2006. LNCS, vol. 3887, pp. 580–591. Springer, Heidelberg (2006)
Jájá, J.: An Introduction to Parallel Algorithms. Addison-Wesley, Reading (1992)
Kumar, V., Schwabe, E.J.: Improved algorithms and data structures for solving graph problems in external memory. In: Proc. 8th Symp. on Parallel and Distrib. Processing, pp. 169–177. IEEE, Los Alamitos (1996)
Lipton, R.J., Tarjan, R.E.: A separator theorem for planar graphs. SIAM Journal on Applied Mathematics 36(2), 177–189 (1979)
Madduri, K., Bader, D.A., Berry, J.W., Crobak, J.R.: An experimental study of a parallel shortest path algorithm for solving large-scale graph instances. In: Proc. 9th Workshop on Algorithm Engineering and Experiments (ALENEX). SIAM, Philadelphia (2007)
Magnien, C., Latapy, M., Habib, M.: Fast computation of empirically tight bounds for the diameter of massive graphs (submitted, 2007), http://www-rp.lip6.fr/~latapy/Diameter/
Maheshwari, A., Zeh, N.: I/O-efficient planar separators. SIAM J. Comput. 38(3), 767–801 (2008)
Mehlhorn, K., Meyer, U.: External-memory breadth-first search with sublinear I/O. In: Möhring, R.H., Raman, R. (eds.) ESA 2002. LNCS, vol. 2461, pp. 723–735. Springer, Heidelberg (2002)
Meyer, U.: External memory BFS on undirected graphs with bounded degree. In: Proc. 12th Ann. Symp. on Discrete Algorithms, pp. 87–88. ACM–SIAM, New York (2001)
Meyer, U.: Heaps are better than buckets: Parallel shortest paths on unbalanced graphs. In: Sakellariou, R., Keane, J.A., Gurd, J.R., Freeman, L. (eds.) Euro-Par 2001. LNCS, vol. 2150, pp. 343–351. Springer, Heidelberg (2001)
Meyer, U.: Buckets strike back: Improved parallel shortest paths. In: Proc. 16th Intern. Parallel and Distributed Processing Symposium (IPDPS 2002). IEEE, Los Alamitos (2002)
Meyer, U.: Design and Analysis of Sequential and Parallel Single–Source Shortest–Paths Algorithms. PhD thesis, Universität des Saarlandes (2002)
Meyer, U.: On dynamic Breadth-First Search in external-memory. In: Proc. 25th Annual Symposium on Theoretical Aspects of Computer Science (STACS), pp. 551–560. IBFI Dagstuhl (2008)
Meyer, U.: On trade-offs in external-memory diameter-approximation. In: Gudmundsson, J. (ed.) SWAT 2008. LNCS, vol. 5124, pp. 426–436. Springer, Heidelberg (2008)
Meyer, U., Osipov, V.: Improved external memory BFS implementation. In: Proc. 11th Workshop on Algorithm Engineering and Experiments (ALENEX), pp. 85–96. ACM-SIAM, New York (2009)
Meyer, U., Sanders, P.: Parallel shortest path for arbitrary graphs. In: Bode, A., Ludwig, T., Karl, W.C., Wismüller, R. (eds.) Euro-Par 2000. LNCS, vol. 1900, pp. 461–470. Springer, Heidelberg (2000)
Meyer, U., Zeh, N.: I/O-efficient undirected shortest paths. In: Di Battista, G., Zwick, U. (eds.) ESA 2003. LNCS, vol. 2832, pp. 434–445. Springer, Heidelberg (2003)
Meyer, U., Zeh, N.: I/O-efficient undirected shortest paths with unbounded edge lengths. In: Azar, Y., Erlebach, T. (eds.) ESA 2006. LNCS, vol. 4168, pp. 540–551. Springer, Heidelberg (2006)
Meyer, U.: Average-case complexity of single-source shortest-paths algorithms: lower and upper bounds. J. Algorithms 48(1), 91–134 (2003)
Meyer, U., Sanders, P.: Δ-stepping: a parallelizable shortest path algorithm. J. Algorithms 49(1), 114–152 (2003)
Moore, E.F.: The shortest path through a maze. In: Proc. Intern. Symp. on the Theory of Switching, pp. 285–292. Harvard University Press (1959)
Munagala, K., Ranade, A.: I/O-complexity of graph algorithms. In: Proc. 10th Ann. Symposium on Discrete Algorithms (SODA), pp. 687–694. ACM-SIAM, New York (1999)
Sach, B., Clifford, R.: An empirical study of cache-oblivious priority queues and their application to the shortest path problem (February 2008), http://www.cs.bris.ac.uk/~sach/COSP/
Schwartz, M., Stern, T.E.: Routing techniques used in computer communication networks. IEEE Transactions on Communications, 539–552 (1980)
Sibeyn, J.F., Abello, J., Meyer, U.: Heuristics for semi-external depth first search on directed graphs. In: Proc. Symposium on Parallel Algorithms and Architectures (SPAA), pp. 282–292. ACM, New York (2002)
Smith, J.R.: Parallel algorithms for depth-first searches. I. planar graphs. SIAM Journal on Computing 15(3), 814–830 (1986)
Vitter, J.S.: Algorithms and Data Structures for External Memory. Foundations and Trends in Theoretical Computer Science. Now Publishers (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Meyer, U. (2009). Via Detours to I/O-Efficient Shortest Paths. In: Albers, S., Alt, H., Näher, S. (eds) Efficient Algorithms. Lecture Notes in Computer Science, vol 5760. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03456-5_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-03456-5_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03455-8
Online ISBN: 978-3-642-03456-5
eBook Packages: Computer ScienceComputer Science (R0)