Abstract
To me, numerical analysis is one of the most fascinating fields in computing. It is at the intersection of computer science and mathematics; it concerns subjects that can be either continuous or discrete; it involves algorithm design as well as software implementation; and it has success throughout engineering, business, medicine, social sciences, natural sciences, and digital animation fields. Its community has both theorists and practitioners, who often respect and admire each other’s work – in fact, in this area there are many practitioners who are also great theoreticians. Its objectives to solve larger and larger problems have pushed the envelope of computer science, particularly in the advancement of computer architectures, compiler technologies, programming languages, and software tools. Its collaborative culture and genuine need to share data and information among scientists and engineers have led a physicist to make a connection between the hypertext idea and the Internet protocols to create the world wide web. Many pioneers in computing including John von Neumann, Alan Turing, Claude Shannon, Richard Hamming, James Wilkinson, Velvel Kahan, and Gene Golub contributed to this field, not to mention the foot prints left by many great minds before them, e.g., Newton’s method, Lagrange interpolation, Gaussian elimination, Euler’s method, Jacobi iteration, and Chebyshev polynomials.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
I. Abraham, Y. Bartal, O. Neiman. Nearly Tight Low Stretch Spanning Trees. FOCS 2008, Pages 781-790.
N. Alon, R. M. Karp, D. Peleg, and D. West. A graph-theoretic game and its application to the k-server problem. SIAM Journal on Computing, 24(1):78–100, February 1995.
C. J. Alpert and S.-Z. Yao. Spectral partitioning: the more eigenvectors, the better. In DAC ’95: Proceedings of the 32nd ACM/IEEE conference on Design automation, pages 195–200. ACM, 1995.
R. Andersen, C. Borgs, J. T. Chayes, J. E. Hopcroft, K. Jain, and V. S. Mirrokni, and S.-H. Teng. Robust PageRank and locally computable spam detection features. In Fourth International Workshop on Adversarial Information Retrieval on the Web, ACM International Conference Proceeding Series, 2008 69–76.
R. Andersen, C. Borgs, J. T. Chayes, J. E. Hopcraft, V. S. Mirrokni, and S.-H. Teng. Local computation of pagerank contributions. In Anthony Bonato and Fan R. K. Chung, editors, WAW, volume 4863 of Lecture Notes in Computer Science, pages 150–165. Springer, 2007.
R. Andersen, F. Chung, and K. Lang. Local graph partitioning using pagerank vectors. Proceedings: 47th Annual Symposium on Foundations of Computer Science, pages 475–486, 2006.
R. Andersen, Y. Peres. Finding sparse cuts locally using evolving sets. STOC, 2009: 235–244.
S. Arora. Polynomial time approximation schemes for euclidean traveling salesman and other geometric problems. J. ACM, 45(5):753–782, 1998.
S. Arora, E. Hazan, and S. Kale. 0(sqrt (log n)) approximation to Sparsest Cut in õ(n2) time. In IEEE FOCS’ 04, pages 238–247, 2004.
S. Arora and S. Kale. A combinatorial, primal-dual approach to semidefinite programs. In ACM STOC ’07, pages 227–236, 2007.
S. Arora, S. Rao, and U. Vazirani. Expander flows, geometric embeddings and graph partitioning. In ACM STOC ’04, pages 222–231, 2004.
D. Arthur and S. Vassilvitskii. Worst-case and smoothed analysis of the icp algorithm, with an application to the k-means method. In IEEE FOCS ’06, pages 153–164, 2006.
D. Arthur and B. Manthey and H. Röglin. k-Means Has Polynomial Smoothed Complexity. In IEEE FOCS, pages 405–414, 2009.
Nina Balcan and Avrim Blum and Anupam Gupta Approximate clustering without the approximation. SIAM/ACM SODA 2009: 1068–1077.
J. Batson, D. A. Spielman, and N. Srivastava. Twice-Ramanujan sparsifiers. Available at http://arxiv.org/abs/0808.0163, 2008.
M. Bebendorf and W. Hackbusch. Existence of H-matrix approximants to the inverse FE-matrix of elliptic operators with L ∞-coefficients. Numerische Mathematik, 95(1):1–28, July 2003.
A. A. Benczúr and D. R. Karger. Approximating s-t minimum cuts in O(n2) time. In ACM STOC ’96,
M. Bern, J.Gilbert, B. Hendrickson, N. Nguyen, and S. Toledo. Support-graph preconditioners. In SIAM. J. Matrix Anal. and Appl. 27(4), 930–951, 2006.
Avrim Blum and John Dunagan. Smoothed analysis of the perceptron algorithm for linear programming. In SIAM/ACM SODA ’02, pages 905–914, 2002.
E. G. Boman and B. Hendrickson. Support theory for preconditioning. SIAM Journal on Matrix Analysis and Applications, 25(3):694–717, 2003.
E. G. Boman, B. Hendrickson, S. Vavasis. Solving epplitic finite element systems in nearly-linear time with support preconditioners.
W. L. Briggs, V. E. Henson, and S. F. McCormick. A Multigrid Tutorial, 2nd Edition. SIAM, 2001.
S. Brin and L. Page, The anatomy of a large-scale hypertextual Web search engine, Proceedings of the seventh international conference on World Wide Web 7, 107–117, 1998.
J. Cheeger. A lower bound for smallest eigenvalue of laplacian. In Problems in Analysis, pages 195–199, In R.C. Gunning editor,, Princeton University Press, 1970.
X. Chen, X. Deng, and S.-H. Teng. Settling the complexity of computing two-player Nash equilibria. In J. ACM, 56(3): (2009a)
X. Chen, D. Dai, Y. Du, and S.-H. Teng. Settling the Complexity of Arrow-Debreu Equilibria in Markets with Additively Separable Utilities. In IEEE FOCS, 273–282, 2009b.
Siu-Wing Cheng, Tamal K. Dey, Herbert Edelsbrunner, Michael A. Facello, and Shang-Hua Teng. Sliver exudation. Journal of ACM, 47:883–904, 2000.
P Chew. There is a planar graph almost as good as the complete graph. In SCG ’86: Proceedings of the second annual symposium on Computational geometry, pages 169–177. ACM, 1986.
P. Christiano, J. A. Kelner, A. Madry, D. A. Spielman, and S.-H. Teng. Electrical Flows, Laplacian Systems, and Faster Approximation of Maximum Flow in Undirected Graphs http://arxiv.org/abs/1010.2921.
T. Cormen. C. Leiserson, R. Rivest, and C. Stein. Introduction to Algorithms, 3rd edition.
S. I. Daitch and D. A. Spielman. Support-graph preconditioners for 2-dimensional trusses.
S. I. Daitch and D. A. Spielman. Faster approximate lossy generalized flow via interior point algorithms. In ACM STOC’08, pages 451–460, 2008.
Valentina Damerow, Friedhelm Meyer auf der Heide, Harald Räcke, Christian Scheideler, and Christian Sohler. Smoothed motion complexity. In Proc. 11th Annual European Symposium on Algorithms (ESA’03), pages 161–171, 2003.
I. Daubechies, Ten Lectures on Wavelets, Society for Industrial and Applied Mathematics, 1992
P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar Acquiring the Reflectance Field of a Human Face SIGGRAPH Conference Proceedings, 2000
S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman. Indexing by latent semantic analysis Journal of the American Society for Information Science, 41 (6), 391407, 1990.
James Demmel. Applied Numerical Linear Algebra. SIAM, 1997.
Jian Ding, James R. Lee, and Yuval Peres. Cover times, blanket times, and majorizing measures. ACM STOC’ 11, 2011.
W. E. Donath and A. J. Hoffman. Algorithms for partitioning of graphs and computer logic based on eigenvectors of connection matrices. IBM Technical Disclosure Bulletin, 15:938–944, 1972.
D. L. Donoho, Compressed Sensing, IEEE Transactions on Information Theory, 52(4), 12891306, 2006.
S. Dughmi and T. Roughgarden Black-Box Randomized Reductions in Algorithmic Mechanism Design. IEEE FOCS 2010: 775–784
Herbert Edelsbrunner, Xiang-Yang Li, Gary Miller, Andreas Stathopoulos, Dafna Talmor, Shang-Hua Teng, Alper Üngör, and Noel Walkington. Smoothing and cleaning up slivers. In STOC ’00, pages 273–277, 2000.
M. Elkin, Y. Emek, D. A. Spielman, and S.-H. Teng. Lower-stretch spanning trees. SIAM Journal on Computing, 32(2):608–628, 2008.
Matthias Englert, Heiko Röglin, and Berthold Vöcking. Worst case and probabilistic analysis of the 2-opt algorithm for the tsp: extended abstract. In SIAM/ACM SODA ’07, pages 1295–1304, 2007.
S. Even and R. E. Tarjan. Network flow and testing graph connectivity. SIAM Journal on Computing, 4(4):507–518, Dec. 1975.
M. Fiedler. Algebraic connectibity of graphs. Czechoslovak Mathematical Journal, 23(98):298–305, 1973.
A. M. Frieze, G. L. Miller, and S.-H. Teng. Separator Based Parallel Divide and Conquer in Computational Geometry. ACM SPAA 1992: 420-429
J. A. George. Nested dissection of a regular finite element mesh. SIAM J. Numer. Anal., 10:345–363, 1973.
J. R. Gilbert and R. E. Tarjan. The analysis of a nested dissection algorithm. Numerische Mathematik, 50(4):377–404, February 1987.
A. V. Goldberg and S. Rao. Beyond the ow decomposition barrier. J. ACM, 45(5):783797, 1998.
G. H. Golub and M. Overton. The convergence of inexact Chebychev and Richardson iterative methods for solving linear systems. Numerische Mathematik, 53:571–594, 1988.
G. H. Golub and C. F. Van Loan. Matrix Computations. second edition, 1989.
K. Gremban. Combinatorial Preconditioners for Sparse, Symmetric, Diagonall y Dominant Linear Systems. PhD thesis, Carnegie Mellon University, CMU-CS-96-123, 1996.
A. Gulli and A. Signorini. The indexable web is more than 11.5 billion pages. In WWW ’05: Special interest tracks and posters of the 14th international conference on World Wide Web, pages 902–903. ACM, 2005.
Li-Sha Huang and Shang-Hua Teng. On the approximation and smoothed complexity of leontief market equilibria. In FAW: Frontiers of Algorithms Workshop, pages 96–107, 2007.
A. Joshi. Topics in Optimization and Sparse Linear Systems, Ph.D. thesis, UIUC, 1997.
A. T. Kalai and A. Samorodnitsky and S.-H. Teng Learning and Smoothed Analysis. In IEEE FOCS’09, 395–404, 2009.
J. A. Kelner and E. Nikolova. On the hardness and smoothed complexity of quasi-concave minimization. In IEEE FOCS’07, pages 552–563, 2007.
J. A. Kelner and D A. Spielman. A randomized polynomial-time simplex algorithm for linear programming. In ACM STOC ’06, pages 51–60, 2006.
R. Khandekar, S. Rao, and U. Vazirani. Graph partitioning using single commodity flows. In ACM STOC ’06, pages 385–390, 2006.
J. Kleinberg. Authoritative sources in a hyperlinked environment. J. ACM, 46 (5), 1999, 604–632.
A. Kolla, Y. Makarychev, A. Saberi, and S.-H. Teng Subgraph Sparsification. ACM STOC’10 2010.
I. Koutis. G. Miller, and R. Peng. Solving SDD linear systems in time Õ(mlognlog(1 / ε)). arXiv:1102.4842.
I. Koutis. G. Miller, and D. Tolliver. Combinatorial preconditioners and multilevel solvers for problems in computer vision and image prcessing. International Symp. of Visual Computing, 1067–1078, 2009.
C.E. Lemke. Bimatrix equilibrium points and mathematical programming. Management Science, 11:681–689, 1965.
C.E. Lemke and JR. J.T. Howson. Equilibrium points of bimatrix games. J. Soc. Indust. Appl. Math., 12:413–423, 1964.
X.-Y. Li and S.-H. Teng. Generate sliver free three dimensional meshes. In ACM-SIAM SODA’01, pages 28–37, 2001.
R. J. Lipton, D. J. Rose, and R. E. Tarjan. Generalized nested dissection. SIAM Journal on Numerical Analysis, 16(2):346–358, April 1979.
T. Leighton and S. Rao. Multicommodity max-flow min-cut theorems and their use in designing approximation algorithms. Journal of the ACM, 46(6):787–832, November 1999.
L. Lovasz and M. Simonovits. Random walks in a convex body and an improved volume algorithm. RSA: Random Structures & Algorithms, 4:359–412, 1993.
B. Maggs, G. Miller, O. Parekh, R. Ravi, and S. M. Woo. Finding effective support-tree preconditioners. ACM SPAA 176–185, 2005.
A. Madry and J. Kelner Faster generation of random spanning trees. IEEE FOCS’09, 2009.
Gary L. Miller, Dafna Talmor, Shang-Hua Teng, and Noel Walkington. A delaunay based numerical method for three dimensions: generation, formulation, and partition. In ACM STOC ’95, pages 683–692, 1995.
G. L. Miller, S.-H. Teng, W. P. Thurston, and S. A. Vavasis. Separators for sphere-packings and nearest neighbor graphs. J. ACM, 44(1): 1–29 (1997)
G. L. Miller, S.-H. Teng, W. P. Thurston, and S. A. Vavasis. Geometric Separators for Finite-Element Meshes. SIAM J. Scientific Computing, 19(2): 364–386 (1998)
Joseph S. B. Mitchell. Guillotine subdivisions approximate polygonal subdivisions: A simple polynomial-time approximation scheme for geometric tsp, k-mst, and related problems. SIAM J. Comput., 28(4):1298–1309, 1999.
Michael Mitzenmacher and Salil P. Vadhan. Why simple hash functions work: exploiting the entropy in a data stream. SIAM-ACM SODA 2008:746–755.
J. Nash. Noncooperative games. Annals of Mathematics, 54:289–295, 1951,.
E. Nikolova, J. A. Kelner, M. Brand, and M. Mitzenmacher. Stochastic shortest paths via quasi-convex maximization. In ESA’06: Proceedings of the 14th conference on Annual European Symposium, pages 552–563, 2006.
L. Orecchia, L. J. Schulman, U. V. Vazirani, and N. K. Vishnoi. On partitioning graphs via single commodity flows. In ACM STOC ’08, pages 461–470, 2008.
J. Rief. Efficient approximate solution of sparse linear systems. Computer and Mathematics with Applications, 36 (9): 37–58, 1998.
H. Röglin and S.-H. Teng. Smoothed analysis of multiobjective optimization. IEEE FOCS, 681–690, 2009.
H. Röglin and B. Vöcking. Smoothed analysis of integer programming. Math. Program., 110(1):21–56, 2007.
M. Rudelson and R. Vershynin. The littlewood-offord problem and invertibility of random matrices. UC Davis, 2006.
Arvind Sankar, Daniel A. Spielman, and Shang-Hua Teng. Smoothed analysis of the condition numbers and growth factors of matrices. SIAM Journal on Matrix Analysis and Applications, to appear, 2005.
Guido Schäfer, Luca Becchetti, Stefano Leonardi, Alberto Marchetti-Spaccamela, and Tjark Vredeveld. Average case and smoothed competitive analysis of the multi-level feedback algorithm. In IEEE FOCS ’03, page 462, 2003.
A. Sharma, X. Liu, P. Miller, A. Nakano, R. K. Kalia, P. Vashishta, W. Zhao, T. J. Campbell, and A. Haas. Immersive and interactive exploration of billion-atom systems. In VR ’02: Proceedings of the IEEE Virtual Reality Conference 2002, page 217. IEEE Computer Society, 2002.
J. R. Shewchuk. Tetrahedral mesh generation by Delaunay refinement. In Proc. 14th Annual ACM Symposium on Computational Geometry, pages 86–95, 1998.
G. Shklarski and S. Toledo. Rigidity in finite-element matrices: Sufficient conditions for the rigidity of structures and substructures. SIAM J. Matrix Anal. and Appl. 30(1): 7–40, 2008.
D. Spielman. Graphs and networks: Random walks and spectral graph drawing. Computer Science, Yale, http://www.cs.yale.edu/homes/spielman/462/lect4-07.pdf, Sept. 18, 2007.
D. A. Spielman and N. Srivastava. Graph sparsification by effective resistances. In ACM STOC, pages 563–568, 2008.
D. Spielman and S.-H. Teng. Spectral partitioning works: planar graphs and finite element meshes. In IEEE FOCS ’96, pages 96–105, 1996.
Daniel A. Spielman and Shang-Hua Teng. Smoothed analysis of algorithms: Why the simplex algorithm usually takes polynomial time. J. ACM, 51(3):385–463, 2004.
Daniel A. Spielman and Shang-Hua Teng. Smoothed Analysis: An attempt to explain the behavior of algorithms in practice. CACM, 52(10):77–84, 2009.
D. A. Spielman and S.-H. Teng. Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems. ACM STOC, 81–90, 2003.
D. A. Spielman and S.-H. Teng. A local clustering algorithm for massive graphs and its application to nearly-linear time graph partitioning. CoRR, abs/0809.3232, 2008. Available at http://arxiv.org/abs/0809.3232.
D. A. Spielman and S.-H. Teng. Nearly-linear time algorithms for preconditioning and solving symmetric, diagonally dominant linear systems. CoRR, abs/cs/0607105, 2008. Available at http://www.arxiv.org/abs/cs.NA/0607105.
D. A. Spielman and S.-H. Teng. Spectral sparsification of graphs. CoRR, abs/0808.4134, 2008. Available at http://arxiv.org/abs/0808.4134.
G. Strang. Linear Algebra and its Application, 2nd. Ed. Academic Press, New York, San Francisco, London, 1980.
D. A. Tolliver. Spectral rounding and image segmentation. PhD thesis, Pittsburgh, PA, USA, 2006. Adviser-Miller,, Gary L. and Adviser-Collins,, Robert T.
D. A. Tolliver and G. L. Miller. Graph partitioning by spectral rounding: Applications in image segmentation and clustering. In CVPR ’06: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 1053–1060. IEEE Computer Society, 2006.
L. N. Trefethen and D. Bau. Numerical Linear Algebra. SIAM, Philadelphia, PA, 1997.
P. Vaidya. Solving linear equations with symmetric diagonally dominant matrices by constructing good preconditioners. An invited at IMA, U. Minnesota, Oct. 1991.
R. Vershynin. Beyond hirsch conjecture: Walks on random polytopes and smoothed complexity of the simplex method. In IEEE FOCS ’06, pages 133–142, 2006.
Van H. Vu and Terence Tao. The condition number of a randomly perturbed matrix. In ACM STOC ’07, pages 248–255, 2007.
J. H. Wilkinson. Error analysis of direct methods of matrix inversion. J. Assoc. Comput. Mach., 8:261–330, 1961.
D. Zhou, J. Huang and B. Schlkopf. Learning from Labeled and Unlabeled Data on a Directed Graph. the 22nd International Conference on Machine Learning, 1041–1048. 2005.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Teng, SH. (2011). Numerical Thinking in Algorithm Design and Analysis. In: Blum, E., Aho, A. (eds) Computer Science. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-1168-0_15
Download citation
DOI: https://doi.org/10.1007/978-1-4614-1168-0_15
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-1167-3
Online ISBN: 978-1-4614-1168-0
eBook Packages: Computer ScienceComputer Science (R0)