poster

A Unified, Flexible Framework in Network Topology Generation for Distributed Machine Learning

Authors:

Jianhao Liu,

Xiaoyan Li,

Yanhua Liu,

Weibei FanAuthors Info & Claims

APNet '23: Proceedings of the 7th Asia-Pacific Workshop on Networking

Pages 196 - 197

https://doi.org/10.1145/3600061.3603132

Published: 05 September 2023 Publication History

Get Access

Abstract

In this study, we propose a unified framework for designing a class of server-centric network topologies for DML by adopting top-down design method and combinatorial design theory. Simulation results show that this flexible framework is capable of effectively supporting various DML tasks. Our framework can generate compatible topologies that meet various resource constraints and different DML tasks.

References

[1]

[1] baidu-allreduce. https://github.com/baidu-research/baidu-allreduce.

Google Scholar

[2]

[2] Yijia Chang et al. Systematic topology design for large-scale networks: A unified framework. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications. IEEE, 347–356, 2020.

Google Scholar

[3]

[3] Jinkun Geng et al. HiPS: Hierarchical parameter synchronization in large-scale distributed machine learning. In Proceedings of the 2018 Workshop on Network Meets AI & ML. 1–7, 2018.

Google Scholar

[4]

[4] Chuanxiong Guo et al. BCube: a high performance, server-centric network architecture for modular data centers. In Proceedings of the ACM SIGCOMM 2009 conference on Data communication. 63–74, 2009.

Google Scholar

[5]

[5] Douglas Stinson. Combinatorial designs: constructions and analysis. Springer Science & Business Media. 2007.

Google Scholar

[6]

[6] Songtao Wanget al. A scalable, high-performance, and fault-tolerant network architecture for distributed machine learning. IEEE/ACM Transactions on Networking 28, 04 (2020). 1752-–1764, 2020.

Google Scholar

[7]

[7] Shuai Wang et al. Impact of network topology on the performance of DML: Theoretical analysis and practical factors. In IEEE INFOCOM 2019-IEEE conference on computer communications. IEEE, 1729–1737, 2019.

Google Scholar

Recommendations

MALEF: Framework for distributed machine learning and data mining

Growing importance of distributed data mining techniques has recently attracted attention of researchers in multiagent domain. In this paper we present a novel framework MultiAgent Learning Framework (MALEF) designed for both the agent-based distributed ...
A CAD/CAE incorporate software framework using a unified representation architecture

An incorporate CAD/CAE software framework for structural design is proposed.A unified representation architecture is presented.Design and analysis information use the same data structure.CAD model display and CAE post-processor shares the unified mesh ...
A framework for agent-based distributed machine learning and data mining
AAMAS '07: Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems

This paper proposes a framework for agent-based distributed machine learning and data mining based on (i) the exchange of meta-level descriptions of individual learning processes among agents and (ii) online reasoning about learning success and learning ...

Comments

Information & Contributors

Information

Published In

APNet '23: Proceedings of the 7th Asia-Pacific Workshop on Networking

June 2023

229 pages

ISBN:9798400707827

DOI:10.1145/3600061

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 September 2023

Check for updates

Qualifiers

Poster
Research
Refereed limited

Funding Sources

Natural Science Foundation of Fujian Province
National Natural Science Foundation of China

Conference

APNET 2023

APNET 2023: 7th Asia-Pacific Workshop on Networking

June 29 - 30, 2023

Hong Kong, China

Acceptance Rates

Overall Acceptance Rate 50 of 118 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
58
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)3

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Abstract

References

Recommendations

MALEF: Framework for distributed machine learning and data mining

A CAD/CAE incorporate software framework using a unified representation architecture

A framework for agent-based distributed machine learning and data mining

Comments

Information

Published In

Publisher

Publication History

Check for updates

Qualifiers

Funding Sources

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Login options

Full Access

View options

PDF

eReader

HTML Format

Share

Share this Publication link

Share on social media

Affiliations