skip to main content
10.1145/1007352.1007438acmconferencesArticle/Chapter ViewAbstractPublication PagesstocConference Proceedingsconference-collections
Article

A decentralized algorithm for spectral analysis

Published: 13 June 2004 Publication History

Abstract

In many large network settings, such as computer networks, social networks, or hyperlinked text documents, much information can be obtained from the network's spectral properties. However, traditional centralized approaches for computing eigenvectors struggle with at least two obstacles: the data may be difficult to obtain (both due to technical reasons and because of privacy concerns), and the sheer size of the networks makes the computation expensive. A decentralized, distributed algorithm addresses both of these obstacles: it utilizes the computational power of all nodes in the network and their ability to communicate, thus speeding up the computation with the network size. And as each node knows its incident edges, the data collection problem is avoided as well.Our main result is a simple decentralized algorithm for computing the top k eigenvectors of a symmetric weighted adjacency matrix, and a proof that it converges essentially in OMIXlog2 n) rounds of communication and computation, where τMIX is the mixing time of a random walk on the network. An additional contribution of our work is a decentralized way of actually detecting convergence, and diagnosing the current error. Our protocol scales well, in that the amount of computation performed at any node in any one round, and the sizes of messages sent, depend polynomially on k, but not on the (typically much larger) number n of nodes.

References

[1]
D. Achlioptas, A. Fiat, A. Karlin, and F. McSherry. Web search via hub synthesis. In Proc. 42nd IEEE Symp. on Foundations of Computer Science, 2001.
[2]
Y. Azar, A. Fiat, A. Karlin, F. McSherry, and J. Saia. Spectral analysis of data. In Proc. 33rd ACM Symp. on Theory of Computing, 2001.
[3]
I. Benjamini and L. Lovász. Global information from local observation. In Proc. 43rd IEEE Symp. on Foundations of Computer Science, 2002.
[4]
S. Boyd, P. Diaconis, and L. Xiao. Fastest mixing markov chain on a graph. Submitted to SIAM Review.
[5]
S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah. Gossip and mixing times of random walks on random graphs. Submitted.
[6]
S. Brin and L. Page. The anatomy of a large-scale hypertextual web search engine. Computer Networks and ISDN Systems, 30:107--17, 1998.
[7]
F. Chung. Spectral Graph Theory. American Mathematical Society, 1997.
[8]
S. Deerwester, S. Dumais, T. Landauer, G. Furnas, and R. Harshman. Indexing by latent semantic analysis. J. of the American Society for Information Sciences, 41:391--407, 1990.
[9]
M. Fiedler. A property of eigenvectors of nonnegative symmetric matrices and its applications to graph theory. Czechoslovak Mathematical Journal, 25:619--633, 1975.
[10]
K. Gallivan, M. Heath, E. Ng, B. Peyton, R. Plemmons, J. Ortega, C. Romine, A. Sameh, and R. Voigt. Parallel Algorithms for Matrix Computations. Society for Industrial and Applied Mathematics, 1990.
[11]
G. Golub and C. V. Loan. Matrix Computations. Johns Hopkins University Press, third edition, 1996.
[12]
M. Granovetter. The strength of weak ties. American Journal of Sociology, 78:1360--1380, 1973.
[13]
R. Kannan, S. Vempala, and A. Vetta. On clusterings: Good, bad and spectral. In Proc. 41st IEEE Symp. on Foundations of Computer Science, 2000.
[14]
D. Kempe, A. Dobra, and J. Gehrke. Computing aggregate information using gossip. In Proc. 44th IEEE Symp. on Foundations of Computer Science, 2003.
[15]
J. Kleinberg. Authoritative sources in a hyperlinked environment. J. of the ACM, 46:604--632, 1999.
[16]
F. Leighton and S. Rao. Multicommodity max-flow min-cut theorems and their use in designing approximation algorithms. J. of the ACM, 46, 1999.
[17]
F. McSherry. Spectral partitioning of random graphs. In Proc. 42nd IEEE Symp. on Foundations of Computer Science, pages 529--537, 2001.
[18]
A. Ng, M. Jordan, and Y. Weiss. On spectral clustering: Analysis and an algorithm. In Proc. 14th Advances in Neural Information Processing Systems, 2002.
[19]
S. Ratnasamy, P. Francis, M. Handley, R. Karp, and S. Shenker. A scalable content-addressable network. In Proc. ACM SIGCOMM Conference, pages 161--172, 2001.
[20]
A. Rowstron and P. Druschel. Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems. In Proc. 18th IFIP/ACM Intl. Conf. on Distributed Systems Platforms (Middleware 2001), pages 329--350, 2001.
[21]
L. Saloff-Coste. Lectures on finite markov chains. In Lecture Notes in Mathematics 1665, pages 301--408. Springer, 1997. École d'été de St. Flour 1996.
[22]
G. Stewart. On the perturbation of LU and cholesky factors. IMA Journal of Numerical Analysis, 1997.
[23]
I. Stoica, R. Morris, D. Karger, F. Kaashoek, and H. Balakrishnan. Chord: A scalable peer-to-peer lookup service for internet applications. In Proc. ACM SIGCOMM Conference, pages 149--160, 2001.
[24]
P. Wedin. Perturbation theory for pseudo-inverses. BIT, 13:217--232, 1973.
[25]
B. Y. Zhao, J. Kubiatowicz, and A. Joseph. Tapestry: An infrastructure for fault-tolerant wide-area location and routing. Technical Report UCB/CSD-01-1141, UC Berkeley, 2001.

Cited By

View all
  • (2024)A Decentralised Asynchronous Optimisation Algorithm with an Application to Phase Retrieval2024 IEEE 13rd Sensor Array and Multichannel Signal Processing Workshop (SAM)10.1109/SAM60225.2024.10636375(1-5)Online publication date: 8-Jul-2024
  • (2022)A Graph Neural Network Based Decentralized Learning SchemeSensors10.3390/s2203103022:3(1030)Online publication date: 28-Jan-2022
  • (2022)Scalable Moment-Based Inference for Latent Dirichlet AllocationMachine Learning and Knowledge Discovery in Databases10.1007/978-3-662-44845-8_19(290-305)Online publication date: 10-Mar-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
STOC '04: Proceedings of the thirty-sixth annual ACM symposium on Theory of computing
June 2004
660 pages
ISBN:1581138520
DOI:10.1145/1007352
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 June 2004

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Markov chain
  2. decentralized algorithm
  3. eigenvectors
  4. large networks
  5. spectral analysis

Qualifiers

  • Article

Conference

STOC04
Sponsor:
STOC04: Symposium of Theory of Computing 2004
June 13 - 16, 2004
IL, Chicago, USA

Acceptance Rates

Overall Acceptance Rate 1,469 of 4,586 submissions, 32%

Upcoming Conference

STOC '25
57th Annual ACM Symposium on Theory of Computing (STOC 2025)
June 23 - 27, 2025
Prague , Czech Republic

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)40
  • Downloads (Last 6 weeks)6
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)A Decentralised Asynchronous Optimisation Algorithm with an Application to Phase Retrieval2024 IEEE 13rd Sensor Array and Multichannel Signal Processing Workshop (SAM)10.1109/SAM60225.2024.10636375(1-5)Online publication date: 8-Jul-2024
  • (2022)A Graph Neural Network Based Decentralized Learning SchemeSensors10.3390/s2203103022:3(1030)Online publication date: 28-Jan-2022
  • (2022)Scalable Moment-Based Inference for Latent Dirichlet AllocationMachine Learning and Knowledge Discovery in Databases10.1007/978-3-662-44845-8_19(290-305)Online publication date: 10-Mar-2022
  • (2020)Find Your PlaceSIAM Journal on Computing10.1137/19M124302649:4(821-864)Online publication date: 1-Jan-2020
  • (2020)Expected Convergence Rate to Consensus in Asymmetric Networks: Analysis and Distributed EstimationIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMC.2017.269302650:3(972-987)Online publication date: Mar-2020
  • (2020)Metric Representations of Networks: A Uniqueness ResultICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP40776.2020.9054190(5345-5349)Online publication date: May-2020
  • (2020)Step-by-step community detection in volume-regular graphsTheoretical Computer Science10.1016/j.tcs.2020.09.036847(49-67)Online publication date: Dec-2020
  • (2019)Spectral Complexity of Directed Graphs and Application to Structural DecompositionComplexity10.1155/2019/96108262019(1-18)Online publication date: 1-Jan-2019
  • (2019)Distributed Graph Clustering and SparsificationACM Transactions on Parallel Computing10.1145/33642086:3(1-23)Online publication date: 2-Nov-2019
  • (2019)Recognition of Stores' Relationship Based on Constrained Spectral ClusteringProceedings of the 2019 3rd International Conference on Innovation in Artificial Intelligence10.1145/3319921.3319946(111-115)Online publication date: 15-Mar-2019
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media