skip to main content
10.1145/3400286.3418229acmconferencesArticle/Chapter ViewAbstractPublication PagesracsConference Proceedingsconference-collections
research-article

Performance improvement of PCI Express adapter cards by adjusting the location of DMA functions

Published: 25 November 2020 Publication History

Abstract

The PCIe (PCI Express) bus has long played a key role in interconnecting devices inside a system. In addition, advances in PCIe technology have made it possible to connect between servers using the PCIe bus. In this study, we've tried to improve the performance of our PCIe adapter cards for expanding the PCIe bus and connecting servers. Especially, we've looked for ways to make the most of the DMA capabilities offered by the PCIe switch chips mounted on our adapter cards. Our experimental results show that the dual ports method using multiple DMAs in each adapter card simultaneously, improves the performance up to 1.7 times compared to using a single port.

References

[1]
Ajay. V. Bhatt. 2002. Creating a PCI Express interconnect. Technical Report.
[2]
Broadcom®. 2011. Product Brief: PEX8733, PCI Express Gen 3 Switch, 32 Lanes, 18 Ports. (2011). Retrieved May 11, 2020 from https://www.broadcom.com/products/pcie-switches-bridges/pcie-switches/pex8733
[3]
Broadcom®. 2019. PLX Software Development Kits. (2019). Retrieved May 11, 2020 from https://www.broadcom.com/products/pcie-switches-bridges/software-dev-kit
[4]
Broadcom®. 2020. PCI Express Switches. (2020). Retrieved May 11, 2020 from https://www.broadcom.com/products/pcie-switches-bridges/pcie-switches
[5]
Ahmed Bu-Khamsin. 2012. Socket direct protocol over PCI express interconnect: design, implementation and evaluation. Master's thesis. Thesis of M.Sc.
[6]
Kwangho Cha, Junglok Yu, Sangwan Kim, Sichul Kim, and Sungho Kim. 2019. Multipurpose adapter card and integration method therefor. (2019). US Patent No. 10,387,363, Filed December 5th., 2017, Issued Aug. 20th., 2019.
[7]
Bob Hansen. 2013. Extending SAS Connectivity in the Data Center. (2013). Retrieved May 11, 2020 from https://www.snia.org/sites/default/files/files2/files2/SDC2013/presentations/BlockStorage/BobHansen_Extending_SAS_Connectivity_Data_Center-v1.pdf
[8]
Rui Hou, Tao Jiang, Liuhang Zhang, Pengfei Qi, Jianbo Dong, Haibin Wang, Xiongli Gu, and Shujie Zhang. 2013. Cost effective data center servers. In 2013 IEEE 19th International Symposium on High Performance Computer Architecture (HPCA). 179--187. https://doi.org/10.1109/HPCA.2013.6522317
[9]
IDT®. 2020. PCI Express (PCIe) Solutions. (2020). Retrieved May 11, 2020 from https://www.idt.com/us/en/products/interface-connectivity/pci-express-solutions
[10]
Hossein Kavianipour, Steffen Muschter, and Christian Bohm. 2014. High Performance FPGA-Based DMA Interface for PCIe. IEEE Transactions on Nuclear Science 61, 2 (April 2014), 745--749. https://doi.org/10.1109/TNS.2014.2304691
[11]
Sang-Gyum Kim, Yang-Woo Lee, Seung-Ho Lim, and Kwangho Cha. 2020. Switchless Interconnect Network with PCIe Non-Transparent Bridge Interface. Advances in Computer Science and Ubiquitous Computing(Lecture Notes in Electrical Engineering) 536 (2020), 97--102. https://doi.org/10.1007/978-981-13-9341-9_17
[12]
Kyungmo Koo, Junglok Yu, Sangwan Kim, Min Choi, and Kwangho Cha. 2018. Implementation of Multipurpose PCI Express Adapter Cards with On-Board Optical Module. Journal of Information Processing Systems 14, 1 (January 2018), 270--279. https://doi.org/10.3745/JIPS.01.0022
[13]
Seung-Ho Lim, Ki-Woong Park, and Kwangho Cha. 2019. Developing an Open-SHMEM Model Over a Switchless PCIe Non-Transparent Bridge Interface. In 2019 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). 593--602. https://doi.org/10.1109/IPDPSW.2019.00104
[14]
Microsemi®. 2019. PCIe Switches. (2019). Retrieved May 11, 2020 from https://www.microsemi.com/product-directory/ics/3724-pcie-switches
[15]
Rolf Neugebauer, Gianni Antichi, José Fernando Zazo, Yury Audzevich, Sergio López-Buedo, and Andrew W. Moore. 2018. Understanding PCIe Performance for End Host Networking. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication (SIGCOMM '18). Association for Computing Machinery, New York, NY, USA, 327--341. https://doi.org/10.1145/3230543.3230560
[16]
PCI-SIG. 2015. PCI Express® Base Specification Revision 3.1a. (2015). Retrieved May 11, 2020 from https://members.pcisig.com/wg/PCI-SIG/document/8257
[17]
Jack Regula. 2004. White paper: Using Non-transparent Bridging in PCI Express Systems. Technical Report.
[18]
L. Rota, M. Caselle, S. Chilingaryan, A. Kopmann, and M. Weber. 2015. A PCIe DMA Architecture for Multi-Gigabyte Per Second Data Transmission. IEEE Transactions on Nuclear Science 62, 3 (June 2015), 972--976. https://doi.org/10.1109/TNS.2015.2426877
[19]
Cheol Shim, Kwangho Cha, and Min Choi. 2019. Design and implementation of initial OpenSHMEM on PCIe NTB based cloud computing. Cluster Computing 22, Supp 1 (January 2019), 1815--1826. https://doi.org/10.1007/s10586-018-1707-0
[20]
PLX Technology and Avago Technologies. 2012. White paper: A Demonstration of PCI Express Generation 3 over a Fiber Optical Link. Technical Report.
[21]
A Triossi, D Barrientos, M Bellato, D Bortolato, R Isocrate, G Rampazzo, and S Ventura. 2013. A PCI Express optical link based on low-cost transceivers qualified for radiation hardness. Journal of Instrumentation 8, 02 (feb 2013), C02011--C02011. https://doi.org/10.1088/1748-0221/8/02/c02011

Cited By

View all
  • (2024)Architectures and Data Transmission Protocol for SpaceFibre - PCIe Bridges2024 Wave Electronics and its Application in Information and Telecommunication Systems (WECONF)10.1109/WECONF61770.2024.10564622(1-6)Online publication date: 3-Jun-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
RACS '20: Proceedings of the International Conference on Research in Adaptive and Convergent Systems
October 2020
300 pages
ISBN:9781450380256
DOI:10.1145/3400286
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 November 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. DMA
  2. Interconnection network
  3. Non-transparent bridge
  4. On-board optical module
  5. PCI Express

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

RACS '20
Sponsor:

Acceptance Rates

RACS '20 Paper Acceptance Rate 42 of 148 submissions, 28%;
Overall Acceptance Rate 393 of 1,581 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)7
  • Downloads (Last 6 weeks)0
Reflects downloads up to 19 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Architectures and Data Transmission Protocol for SpaceFibre - PCIe Bridges2024 Wave Electronics and its Application in Information and Telecommunication Systems (WECONF)10.1109/WECONF61770.2024.10564622(1-6)Online publication date: 3-Jun-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media