skip to main content
10.1145/3626246.3654690acmconferencesArticle/Chapter ViewAbstractPublication PagesmodConference Proceedingsconference-collections
tutorial

SmartNICs in the Cloud: The Why, What and How of In-network Processing for Data-Intensive Applications

Published: 09 June 2024 Publication History

Abstract

In modern datacenters and clouds, Resource Disaggregation has been adopted as a way of offering scalability and efficient resource utilization for large-scale applications. Provisioning CPU, memory, and storage resources independently for distributed data-intensive applications is a great enabler but it also brings challenges, especially in the form of networking and processing overhead. To reduce this overhead and to make disaggregation-related tasks significantly more efficient, cloud providers are offloading these to the network, i.e., to Smart Network Interface Cards (SmartNICs) and Smart Switches. Beyond this specific use-case, the presence of such programmable hardware in commercial clouds creates future opportunities for offloading application-level operations, e.g., parts of SQL queries or ML pipelines. To map out this exciting space, in this tutorial we take a detailed look at SmartNICs, explaining how they work, giving examples of what they are good for, and highlighting how they can best be utilized to make future data-intensive and distributed systems more efficient.

References

[1]
Audibert, A., Chen, Y., Graur, D., Klimovic, A., Simsa, J., and Thekkath, C. A. A case for disaggregation of ml data processing. arXiv preprint arXiv:2210.14826 (2022).
[2]
Barroso, L. A., Hölzle, U., and Ranganathan, P. The datacenter as a computer: Designing warehouse-scale machines. Springer Nature, 2019.
[3]
Burstein, I. Nvidia data center processing unit (dpu) architecture. In 2021 IEEE Hot Chips 33 Symposium (HCS) (2021), IEEE, pp. 1--20.
[4]
Chung, E., Fowers, J., Ovtcharov, K., Papamichael, M., Caulfield, A., Massengill, T., Liu, M., Lo, D., Alkalay, S., Haselman, M., et al. Serving dnns in real time at datacenter scale with project brainwave. iEEE Micro 38, 2 (2018), 8--20.
[5]
Dastidar, J., Riddoch, D., Moore, J., Pope, S., and Wesselkamper, J. The amd 400-g adaptive smartnic system on chip: A technology preview. IEEE Micro 43, 03 (2023), 40--49.
[6]
Delimitrou, C., and Kozyrakis, C. Quasar: Resource-efficient and qos-aware cluster management. ACM SIGPLAN Notices 49, 4 (2014), 127--144.
[7]
Eran, H., Zeno, L., Tork, M., Malka, G., and Silberstein, M. {NICA}: An infrastructure for inline acceleration of network applications. In 2019 USENIX Annual Technical Conference (USENIX ATC 19) (2019), pp. 345--362.
[8]
Faghih, F., István, Z., and Dinu, F. The next 700 heterogeneous olap systems: A framework to answer what-if design questions. Poster Session of ACM EuroSys'23 Conference.
[9]
Firestone,D., Putnam, A., Mundkur, S., Chiou,D., Dabagh, A., Andrewartha, M., Angepat, H., Bhanu, V., Caulfield, A., Chung, E., et al. Azure accelerated networking:{SmartNICs} in the public cloud. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18) (2018), pp. 51--66.
[10]
Hennessy, J. L., and Patterson, D. A. A new golden age for computer architecture. Communications of the ACM 62, 2 (2019), 48--60.
[11]
István, Z., Kara, K., Sidler, D., et al. Fpga-accelerated analytics: From single nodes to clusters. Foundations and Trends® in Databases 9, 2 (2020), 101--208.
[12]
István, Z., Ponnapalli, S., and Chidambaram, V. Software-defined data protection: Low overhead policy compliance at the storage layer is within reach! Proceedings of the VLDB Endowment 14, 7 (2021), 1167--1174.
[13]
István, Z., Sidler, D., and Alonso, G. Caribou: Intelligent distributed storage. Proceedings of the VLDB Endowment 10, 11 (2017), 1202--1213.
[14]
Kanev, S., Darago, J. P., Hazelwood, K., Ranganathan, P., Moseley, T., Wei, G.-Y., and Brooks, D. Profiling a warehouse-scale computer. In Proceedings of the 42nd Annual International Symposium on Computer Architecture (2015), pp. 158--169.
[15]
Klimovic, A., Kozyrakis, C., Thereska, E., John, B., and Kumar, S. Flash storage disaggregation. In Proceedings of the Eleventh European Conference on Computer Systems (2016), pp. 1--15.
[16]
Lee, J. H., Zhang, H., Lagrange, V., Krishnamoorthy, P., Zhao, X., and Ki, Y. S. Smartssd: Fpga accelerated near-storage data analytics on ssd. IEEE Computer architecture letters 19, 2 (2020), 110--113.
[17]
Pandis, I. The evolution of amazon redshift. Proceedings of the VLDB Endowment 14, 12 (2021), 3162--3174.
[18]
Seemakhupt, K., Stephens, B. E., Khan, S., Liu, S., Wassel, H., Yeganeh, S. H., Snoeren, A. C., Krishnamurthy, A., Culler, D. E., and Levy, H. M. A cloudscale characterization of remote procedure calls. In Proceedings of the 29th Symposium on Operating Systems Principles (2023), pp. 498--514.
[19]
Shahrad, M., Fonseca, R., Goiri, I., Chaudhry, G., Batum, P., Cooke, J., Laureano, E., Tresness, C., Russinovich, M., and Bianchini, R. Serverless in the wild: Characterizing and optimizing the serverless workload at a large cloud provider. In 2020 USENIX annual technical conference (USENIX ATC 20) (2020), pp. 205--218.
[20]
Sidler, D., István, Z., and Alonso, G. Low-latency tcp/ip stack for data center applications. In 2016 26th International Conference on Field Programmable Logic and Applications (FPL) (2016), IEEE, pp. 1--4.
[21]
Thostrup, L., Failing, D., Ziegler, T., and Binnig, C. A dbms-centric evaluation of bluefield dpus on fast networks. In 13th International Workshop on Accelerating Analytics and Data Management Systems Using Modern Processor and Storage Architectures (2022).
[22]
Zilberman, N., Audzevich, Y., Covington, G. A., and Moore, A. W. Netfpga sume: Toward 100 gbps as research commodity. IEEE micro 34, 5 (2014), 32--41.

Cited By

View all
  • (2024)Towards Disaggregated NDP Architectures for Large-scale Graph AnalyticsSC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SCW63240.2024.00202(1622-1629)Online publication date: 17-Nov-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGMOD/PODS '24: Companion of the 2024 International Conference on Management of Data
June 2024
694 pages
ISBN:9798400704222
DOI:10.1145/3626246
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 June 2024

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. FPGAs
  2. database management systems
  3. programmable NICs

Qualifiers

  • Tutorial

Conference

SIGMOD/PODS '24
Sponsor:

Acceptance Rates

Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)266
  • Downloads (Last 6 weeks)38
Reflects downloads up to 01 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Towards Disaggregated NDP Architectures for Large-scale Graph AnalyticsSC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SCW63240.2024.00202(1622-1629)Online publication date: 17-Nov-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media