Skip to main content

Numerical Thinking in Algorithm Design and Analysis

  • Chapter
  • First Online:
Computer Science
  • 4201 Accesses

Abstract

To me, numerical analysis is one of the most fascinating fields in computing. It is at the intersection of computer science and mathematics; it concerns subjects that can be either continuous or discrete; it involves algorithm design as well as software implementation; and it has success throughout engineering, business, medicine, social sciences, natural sciences, and digital animation fields. Its community has both theorists and practitioners, who often respect and admire each other’s work – in fact, in this area there are many practitioners who are also great theoreticians. Its objectives to solve larger and larger problems have pushed the envelope of computer science, particularly in the advancement of computer architectures, compiler technologies, programming languages, and software tools. Its collaborative culture and genuine need to share data and information among scientists and engineers have led a physicist to make a connection between the hypertext idea and the Internet protocols to create the world wide web. Many pioneers in computing including John von Neumann, Alan Turing, Claude Shannon, Richard Hamming, James Wilkinson, Velvel Kahan, and Gene Golub contributed to this field, not to mention the foot prints left by many great minds before them, e.g., Newton’s method, Lagrange interpolation, Gaussian elimination, Euler’s method, Jacobi iteration, and Chebyshev polynomials.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • I. Abraham, Y. Bartal, O. Neiman. Nearly Tight Low Stretch Spanning Trees. FOCS 2008, Pages 781-790.

    Google Scholar 

  • N. Alon, R. M. Karp, D. Peleg, and D. West. A graph-theoretic game and its application to the k-server problem. SIAM Journal on Computing, 24(1):78–100, February 1995.

    Article  MATH  MathSciNet  Google Scholar 

  • C. J. Alpert and S.-Z. Yao. Spectral partitioning: the more eigenvectors, the better. In DAC ’95: Proceedings of the 32nd ACM/IEEE conference on Design automation, pages 195–200. ACM, 1995.

    Google Scholar 

  • R. Andersen, C. Borgs, J. T. Chayes, J. E. Hopcroft, K. Jain, and V. S. Mirrokni, and S.-H. Teng. Robust PageRank and locally computable spam detection features. In Fourth International Workshop on Adversarial Information Retrieval on the Web, ACM International Conference Proceeding Series, 2008 69–76.

    Google Scholar 

  • R. Andersen, C. Borgs, J. T. Chayes, J. E. Hopcraft, V. S. Mirrokni, and S.-H. Teng. Local computation of pagerank contributions. In Anthony Bonato and Fan R. K. Chung, editors, WAW, volume 4863 of Lecture Notes in Computer Science, pages 150–165. Springer, 2007.

    Google Scholar 

  • R. Andersen, F. Chung, and K. Lang. Local graph partitioning using pagerank vectors. Proceedings: 47th Annual Symposium on Foundations of Computer Science, pages 475–486, 2006.

    Google Scholar 

  • R. Andersen, Y. Peres. Finding sparse cuts locally using evolving sets. STOC, 2009: 235–244.

    Google Scholar 

  • S. Arora. Polynomial time approximation schemes for euclidean traveling salesman and other geometric problems. J. ACM, 45(5):753–782, 1998.

    Article  MATH  MathSciNet  Google Scholar 

  • S. Arora, E. Hazan, and S. Kale. 0(sqrt (log n)) approximation to Sparsest Cut in õ(n2) time. In IEEE FOCS’ 04, pages 238–247, 2004.

    Google Scholar 

  • S. Arora and S. Kale. A combinatorial, primal-dual approach to semidefinite programs. In ACM STOC ’07, pages 227–236, 2007.

    Google Scholar 

  • S. Arora, S. Rao, and U. Vazirani. Expander flows, geometric embeddings and graph partitioning. In ACM STOC ’04, pages 222–231, 2004.

    Google Scholar 

  • D. Arthur and S. Vassilvitskii. Worst-case and smoothed analysis of the icp algorithm, with an application to the k-means method. In IEEE FOCS ’06, pages 153–164, 2006.

    Google Scholar 

  • D. Arthur and B. Manthey and H. Röglin. k-Means Has Polynomial Smoothed Complexity. In IEEE FOCS, pages 405–414, 2009.

    Google Scholar 

  • Nina Balcan and Avrim Blum and Anupam Gupta Approximate clustering without the approximation. SIAM/ACM SODA 2009: 1068–1077.

    Google Scholar 

  • J. Batson, D. A. Spielman, and N. Srivastava. Twice-Ramanujan sparsifiers. Available at http://arxiv.org/abs/0808.0163, 2008.

  • M. Bebendorf and W. Hackbusch. Existence of H-matrix approximants to the inverse FE-matrix of elliptic operators with L -coefficients. Numerische Mathematik, 95(1):1–28, July 2003.

    Article  MATH  MathSciNet  Google Scholar 

  • A. A. Benczúr and D. R. Karger. Approximating s-t minimum cuts in O(n2) time. In ACM STOC ’96,

    Google Scholar 

  • M. Bern, J.Gilbert, B. Hendrickson, N. Nguyen, and S. Toledo. Support-graph preconditioners. In SIAM. J. Matrix Anal. and Appl. 27(4), 930–951, 2006.

    Google Scholar 

  • Avrim Blum and John Dunagan. Smoothed analysis of the perceptron algorithm for linear programming. In SIAM/ACM SODA ’02, pages 905–914, 2002.

    Google Scholar 

  • E. G. Boman and B. Hendrickson. Support theory for preconditioning. SIAM Journal on Matrix Analysis and Applications, 25(3):694–717, 2003.

    Article  MATH  MathSciNet  Google Scholar 

  • E. G. Boman, B. Hendrickson, S. Vavasis. Solving epplitic finite element systems in nearly-linear time with support preconditioners.

    Google Scholar 

  • W. L. Briggs, V. E. Henson, and S. F. McCormick. A Multigrid Tutorial, 2nd Edition. SIAM, 2001.

    Google Scholar 

  • S. Brin and L. Page, The anatomy of a large-scale hypertextual Web search engine, Proceedings of the seventh international conference on World Wide Web 7, 107–117, 1998.

    Google Scholar 

  • J. Cheeger. A lower bound for smallest eigenvalue of laplacian. In Problems in Analysis, pages 195–199, In R.C. Gunning editor,, Princeton University Press, 1970.

    Google Scholar 

  • X. Chen, X. Deng, and S.-H. Teng. Settling the complexity of computing two-player Nash equilibria. In J. ACM, 56(3): (2009a)

    Google Scholar 

  • X. Chen, D. Dai, Y. Du, and S.-H. Teng. Settling the Complexity of Arrow-Debreu Equilibria in Markets with Additively Separable Utilities. In IEEE FOCS, 273–282, 2009b.

    Google Scholar 

  • Siu-Wing Cheng, Tamal K. Dey, Herbert Edelsbrunner, Michael A. Facello, and Shang-Hua Teng. Sliver exudation. Journal of ACM, 47:883–904, 2000.

    Article  MathSciNet  Google Scholar 

  • P Chew. There is a planar graph almost as good as the complete graph. In SCG ’86: Proceedings of the second annual symposium on Computational geometry, pages 169–177. ACM, 1986.

    Google Scholar 

  • P. Christiano, J. A. Kelner, A. Madry, D. A. Spielman, and S.-H. Teng. Electrical Flows, Laplacian Systems, and Faster Approximation of Maximum Flow in Undirected Graphs http://arxiv.org/abs/1010.2921.

  • T. Cormen. C. Leiserson, R. Rivest, and C. Stein. Introduction to Algorithms, 3rd edition.

    Google Scholar 

  • S. I. Daitch and D. A. Spielman. Support-graph preconditioners for 2-dimensional trusses.

    Google Scholar 

  • S. I. Daitch and D. A. Spielman. Faster approximate lossy generalized flow via interior point algorithms. In ACM STOC’08, pages 451–460, 2008.

    Google Scholar 

  • Valentina Damerow, Friedhelm Meyer auf der Heide, Harald Räcke, Christian Scheideler, and Christian Sohler. Smoothed motion complexity. In Proc. 11th Annual European Symposium on Algorithms (ESA’03), pages 161–171, 2003.

    Google Scholar 

  • I. Daubechies, Ten Lectures on Wavelets, Society for Industrial and Applied Mathematics, 1992

    Google Scholar 

  • P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar Acquiring the Reflectance Field of a Human Face SIGGRAPH Conference Proceedings, 2000

    Google Scholar 

  • S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman. Indexing by latent semantic analysis Journal of the American Society for Information Science, 41 (6), 391407, 1990.

    Google Scholar 

  • James Demmel. Applied Numerical Linear Algebra. SIAM, 1997.

    Google Scholar 

  • Jian Ding, James R. Lee, and Yuval Peres. Cover times, blanket times, and majorizing measures. ACM STOC’ 11, 2011.

    Google Scholar 

  • W. E. Donath and A. J. Hoffman. Algorithms for partitioning of graphs and computer logic based on eigenvectors of connection matrices. IBM Technical Disclosure Bulletin, 15:938–944, 1972.

    Google Scholar 

  • D. L. Donoho, Compressed Sensing, IEEE Transactions on Information Theory, 52(4), 12891306, 2006.

    Google Scholar 

  • S. Dughmi and T. Roughgarden Black-Box Randomized Reductions in Algorithmic Mechanism Design. IEEE FOCS 2010: 775–784

    Google Scholar 

  • Herbert Edelsbrunner, Xiang-Yang Li, Gary Miller, Andreas Stathopoulos, Dafna Talmor, Shang-Hua Teng, Alper Üngör, and Noel Walkington. Smoothing and cleaning up slivers. In STOC ’00, pages 273–277, 2000.

    Google Scholar 

  • M. Elkin, Y. Emek, D. A. Spielman, and S.-H. Teng. Lower-stretch spanning trees. SIAM Journal on Computing, 32(2):608–628, 2008.

    Article  MathSciNet  Google Scholar 

  • Matthias Englert, Heiko Röglin, and Berthold Vöcking. Worst case and probabilistic analysis of the 2-opt algorithm for the tsp: extended abstract. In SIAM/ACM SODA ’07, pages 1295–1304, 2007.

    Google Scholar 

  • S. Even and R. E. Tarjan. Network flow and testing graph connectivity. SIAM Journal on Computing, 4(4):507–518, Dec. 1975.

    Article  MATH  MathSciNet  Google Scholar 

  • M. Fiedler. Algebraic connectibity of graphs. Czechoslovak Mathematical Journal, 23(98):298–305, 1973.

    MathSciNet  Google Scholar 

  • A. M. Frieze, G. L. Miller, and S.-H. Teng. Separator Based Parallel Divide and Conquer in Computational Geometry. ACM SPAA 1992: 420-429

    Google Scholar 

  • J. A. George. Nested dissection of a regular finite element mesh. SIAM J. Numer. Anal., 10:345–363, 1973.

    Article  MATH  MathSciNet  Google Scholar 

  • J. R. Gilbert and R. E. Tarjan. The analysis of a nested dissection algorithm. Numerische Mathematik, 50(4):377–404, February 1987.

    Article  MATH  MathSciNet  Google Scholar 

  • A. V. Goldberg and S. Rao. Beyond the ow decomposition barrier. J. ACM, 45(5):783797, 1998.

    Google Scholar 

  • G. H. Golub and M. Overton. The convergence of inexact Chebychev and Richardson iterative methods for solving linear systems. Numerische Mathematik, 53:571–594, 1988.

    Article  MATH  MathSciNet  Google Scholar 

  • G. H. Golub and C. F. Van Loan. Matrix Computations. second edition, 1989.

    Google Scholar 

  • K. Gremban. Combinatorial Preconditioners for Sparse, Symmetric, Diagonall y Dominant Linear Systems. PhD thesis, Carnegie Mellon University, CMU-CS-96-123, 1996.

    Google Scholar 

  • A. Gulli and A. Signorini. The indexable web is more than 11.5 billion pages. In WWW ’05: Special interest tracks and posters of the 14th international conference on World Wide Web, pages 902–903. ACM, 2005.

    Google Scholar 

  • Li-Sha Huang and Shang-Hua Teng. On the approximation and smoothed complexity of leontief market equilibria. In FAW: Frontiers of Algorithms Workshop, pages 96–107, 2007.

    Google Scholar 

  • A. Joshi. Topics in Optimization and Sparse Linear Systems, Ph.D. thesis, UIUC, 1997.

    Google Scholar 

  • A. T. Kalai and A. Samorodnitsky and S.-H. Teng Learning and Smoothed Analysis. In IEEE FOCS’09, 395–404, 2009.

    Google Scholar 

  • J. A. Kelner and E. Nikolova. On the hardness and smoothed complexity of quasi-concave minimization. In IEEE FOCS’07, pages 552–563, 2007.

    Google Scholar 

  • J. A. Kelner and D A. Spielman. A randomized polynomial-time simplex algorithm for linear programming. In ACM STOC ’06, pages 51–60, 2006.

    Google Scholar 

  • R. Khandekar, S. Rao, and U. Vazirani. Graph partitioning using single commodity flows. In ACM STOC ’06, pages 385–390, 2006.

    Google Scholar 

  • J. Kleinberg. Authoritative sources in a hyperlinked environment. J. ACM, 46 (5), 1999, 604–632.

    Article  MATH  MathSciNet  Google Scholar 

  • A. Kolla, Y. Makarychev, A. Saberi, and S.-H. Teng Subgraph Sparsification. ACM STOC’10 2010.

    Google Scholar 

  • I. Koutis. G. Miller, and R. Peng. Solving SDD linear systems in time Õ(mlognlog(1 / ε)). arXiv:1102.4842.

    Google Scholar 

  • I. Koutis. G. Miller, and D. Tolliver. Combinatorial preconditioners and multilevel solvers for problems in computer vision and image prcessing. International Symp. of Visual Computing, 1067–1078, 2009.

    Google Scholar 

  • C.E. Lemke. Bimatrix equilibrium points and mathematical programming. Management Science, 11:681–689, 1965.

    Article  MathSciNet  Google Scholar 

  • C.E. Lemke and JR. J.T. Howson. Equilibrium points of bimatrix games. J. Soc. Indust. Appl. Math., 12:413–423, 1964.

    Google Scholar 

  • X.-Y. Li and S.-H. Teng. Generate sliver free three dimensional meshes. In ACM-SIAM SODA’01, pages 28–37, 2001.

    Google Scholar 

  • R. J. Lipton, D. J. Rose, and R. E. Tarjan. Generalized nested dissection. SIAM Journal on Numerical Analysis, 16(2):346–358, April 1979.

    Article  MATH  MathSciNet  Google Scholar 

  • T. Leighton and S. Rao. Multicommodity max-flow min-cut theorems and their use in designing approximation algorithms. Journal of the ACM, 46(6):787–832, November 1999.

    Article  MATH  MathSciNet  Google Scholar 

  • L. Lovasz and M. Simonovits. Random walks in a convex body and an improved volume algorithm. RSA: Random Structures & Algorithms, 4:359–412, 1993.

    Article  MATH  MathSciNet  Google Scholar 

  • B. Maggs, G. Miller, O. Parekh, R. Ravi, and S. M. Woo. Finding effective support-tree preconditioners. ACM SPAA 176–185, 2005.

    Google Scholar 

  • A. Madry and J. Kelner Faster generation of random spanning trees. IEEE FOCS’09, 2009.

    Google Scholar 

  • Gary L. Miller, Dafna Talmor, Shang-Hua Teng, and Noel Walkington. A delaunay based numerical method for three dimensions: generation, formulation, and partition. In ACM STOC ’95, pages 683–692, 1995.

    Google Scholar 

  • G. L. Miller, S.-H. Teng, W. P. Thurston, and S. A. Vavasis. Separators for sphere-packings and nearest neighbor graphs. J. ACM, 44(1): 1–29 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  • G. L. Miller, S.-H. Teng, W. P. Thurston, and S. A. Vavasis. Geometric Separators for Finite-Element Meshes. SIAM J. Scientific Computing, 19(2): 364–386 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  • Joseph S. B. Mitchell. Guillotine subdivisions approximate polygonal subdivisions: A simple polynomial-time approximation scheme for geometric tsp, k-mst, and related problems. SIAM J. Comput., 28(4):1298–1309, 1999.

    Google Scholar 

  • Michael Mitzenmacher and Salil P. Vadhan. Why simple hash functions work: exploiting the entropy in a data stream. SIAM-ACM SODA 2008:746–755.

    Google Scholar 

  • J. Nash. Noncooperative games. Annals of Mathematics, 54:289–295, 1951,.

    Article  MathSciNet  Google Scholar 

  • E. Nikolova, J. A. Kelner, M. Brand, and M. Mitzenmacher. Stochastic shortest paths via quasi-convex maximization. In ESA’06: Proceedings of the 14th conference on Annual European Symposium, pages 552–563, 2006.

    Google Scholar 

  • L. Orecchia, L. J. Schulman, U. V. Vazirani, and N. K. Vishnoi. On partitioning graphs via single commodity flows. In ACM STOC ’08, pages 461–470, 2008.

    Google Scholar 

  • J. Rief. Efficient approximate solution of sparse linear systems. Computer and Mathematics with Applications, 36 (9): 37–58, 1998.

    Article  Google Scholar 

  • H. Röglin and S.-H. Teng. Smoothed analysis of multiobjective optimization. IEEE FOCS, 681–690, 2009.

    Google Scholar 

  • H. Röglin and B. Vöcking. Smoothed analysis of integer programming. Math. Program., 110(1):21–56, 2007.

    Article  MathSciNet  Google Scholar 

  • M. Rudelson and R. Vershynin. The littlewood-offord problem and invertibility of random matrices. UC Davis, 2006.

    Google Scholar 

  • Arvind Sankar, Daniel A. Spielman, and Shang-Hua Teng. Smoothed analysis of the condition numbers and growth factors of matrices. SIAM Journal on Matrix Analysis and Applications, to appear, 2005.

    Google Scholar 

  • Guido Schäfer, Luca Becchetti, Stefano Leonardi, Alberto Marchetti-Spaccamela, and Tjark Vredeveld. Average case and smoothed competitive analysis of the multi-level feedback algorithm. In IEEE FOCS ’03, page 462, 2003.

    Google Scholar 

  • A. Sharma, X. Liu, P. Miller, A. Nakano, R. K. Kalia, P. Vashishta, W. Zhao, T. J. Campbell, and A. Haas. Immersive and interactive exploration of billion-atom systems. In VR ’02: Proceedings of the IEEE Virtual Reality Conference 2002, page 217. IEEE Computer Society, 2002.

    Google Scholar 

  • J. R. Shewchuk. Tetrahedral mesh generation by Delaunay refinement. In Proc. 14th Annual ACM Symposium on Computational Geometry, pages 86–95, 1998.

    Google Scholar 

  • G. Shklarski and S. Toledo. Rigidity in finite-element matrices: Sufficient conditions for the rigidity of structures and substructures. SIAM J. Matrix Anal. and Appl. 30(1): 7–40, 2008.

    Article  MATH  MathSciNet  Google Scholar 

  • D. Spielman. Graphs and networks: Random walks and spectral graph drawing. Computer Science, Yale, http://www.cs.yale.edu/homes/spielman/462/lect4-07.pdf, Sept. 18, 2007.

  • D. A. Spielman and N. Srivastava. Graph sparsification by effective resistances. In ACM STOC, pages 563–568, 2008.

    Google Scholar 

  • D. Spielman and S.-H. Teng. Spectral partitioning works: planar graphs and finite element meshes. In IEEE FOCS ’96, pages 96–105, 1996.

    Google Scholar 

  • Daniel A. Spielman and Shang-Hua Teng. Smoothed analysis of algorithms: Why the simplex algorithm usually takes polynomial time. J. ACM, 51(3):385–463, 2004.

    Google Scholar 

  • Daniel A. Spielman and Shang-Hua Teng. Smoothed Analysis: An attempt to explain the behavior of algorithms in practice. CACM, 52(10):77–84, 2009.

    Google Scholar 

  • D. A. Spielman and S.-H. Teng. Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems. ACM STOC, 81–90, 2003.

    Google Scholar 

  • D. A. Spielman and S.-H. Teng. A local clustering algorithm for massive graphs and its application to nearly-linear time graph partitioning. CoRR, abs/0809.3232, 2008. Available at http://arxiv.org/abs/0809.3232.

  • D. A. Spielman and S.-H. Teng. Nearly-linear time algorithms for preconditioning and solving symmetric, diagonally dominant linear systems. CoRR, abs/cs/0607105, 2008. Available at http://www.arxiv.org/abs/cs.NA/0607105.

  • D. A. Spielman and S.-H. Teng. Spectral sparsification of graphs. CoRR, abs/0808.4134, 2008. Available at http://arxiv.org/abs/0808.4134.

  • G. Strang. Linear Algebra and its Application, 2nd. Ed. Academic Press, New York, San Francisco, London, 1980.

    Google Scholar 

  • D. A. Tolliver. Spectral rounding and image segmentation. PhD thesis, Pittsburgh, PA, USA, 2006. Adviser-Miller,, Gary L. and Adviser-Collins,, Robert T.

    Google Scholar 

  • D. A. Tolliver and G. L. Miller. Graph partitioning by spectral rounding: Applications in image segmentation and clustering. In CVPR ’06: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 1053–1060. IEEE Computer Society, 2006.

    Google Scholar 

  • L. N. Trefethen and D. Bau. Numerical Linear Algebra. SIAM, Philadelphia, PA, 1997.

    Book  MATH  Google Scholar 

  • P. Vaidya. Solving linear equations with symmetric diagonally dominant matrices by constructing good preconditioners. An invited at IMA, U. Minnesota, Oct. 1991.

    Google Scholar 

  • R. Vershynin. Beyond hirsch conjecture: Walks on random polytopes and smoothed complexity of the simplex method. In IEEE FOCS ’06, pages 133–142, 2006.

    Google Scholar 

  • Van H. Vu and Terence Tao. The condition number of a randomly perturbed matrix. In ACM STOC ’07, pages 248–255, 2007.

    Google Scholar 

  • J. H. Wilkinson. Error analysis of direct methods of matrix inversion. J. Assoc. Comput. Mach., 8:261–330, 1961.

    MathSciNet  Google Scholar 

  • D. Zhou, J. Huang and B. Schlkopf. Learning from Labeled and Unlabeled Data on a Directed Graph. the 22nd International Conference on Machine Learning, 1041–1048. 2005.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shang-Hua Teng .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Teng, SH. (2011). Numerical Thinking in Algorithm Design and Analysis. In: Blum, E., Aho, A. (eds) Computer Science. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-1168-0_15

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-1168-0_15

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-1167-3

  • Online ISBN: 978-1-4614-1168-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics