research-article

An adaptive hash-based multilayer scheduler for L7-filter on a highly threaded hierarchical multi-core server

Authors:

Guangdeng Liao,

Laxmi N. Bhuyan,

Bin LiuAuthors Info & Claims

ANCS '09: Proceedings of the 5th ACM/IEEE Symposium on Architectures for Networking and Communications Systems

Pages 50 - 59

https://doi.org/10.1145/1882486.1882497

Published: 19 October 2009 Publication History

Abstract

Ubiquitous multi-core-based web servers and edge routers are increasingly popular in deploying computationally intensive Deep Packet Inspection (DPI) programs. Previous work has shown the benefits of connection locality-based scheduling on multi-core servers to improve L7-filter performance. However, we show that highly threaded hierarchical multi-core processors, such as the Sun Niagara 2 processor, accumulate imbalanced workload at each resource layer. This workload imbalance potentially offsets the benefits from connection locality. In addition, connection-locality-based load balance fails to work when network traffic is unevenly distributed.

In this paper, we propose an adaptive hash-based multilayer scheduler for a highly threaded hierarchical Sun Niagara 2 server. Our scheduler maintains connection locality and adaptively adjusts the scheduling to balance the real time workload. The original Highest Random Weight (HRW) hash guarantees the connection locality but only balances the workload over the number of different connections. We enhance the original single layer HRW into a hierarchical "hash tree" scheduler to balance the connection workload in accordance with the hierarchical processor architecture. We then optimize our multilayer scheduler to adaptively adjust scheduling decisions based on service time at each level, further improving the system load balance. Our scheduler is shown to increase the system throughput by 59.2% compared to the previously proposed connection locality optimization.

References

[1]

Application Layer Packet Classifier for Linux (L7-filter), http://17-filter.sourceforge.net/.

[2]

B. Brodie, et al., "A Scalable Architecture for High-Throughput Regular-Expression Pattern Matching", ISCA '06.

Digital Library

[3]

Cisco IOS Netflow, http://www.cisco.com/en/US/products/ps6601/products_ios_protocol_group_home.html

[4]

Cisco Systems. Scaling the world wide web. Available from http://cio.cisco.com/warp/public/751/advtg/swww_wp.htm.

[5]

A. C. Dusseau, et al., "Effective Distributed Scheduling of Parallel Workloads", SIGMETRICS 1996.

Digital Library

[6]

A. Fedorova, et al., "Cache-Fair Thread Scheduling for Multicore Processors", OSDI '06.

[7]

D. Guo, et al., "A Scalable Multithreaded L7-filter Design for Multi-core Servers", ANCS 2008.

Digital Library

[8]

N. Hua, et al., "Variable-Stride Multi-Pattern Matching for Scalable Deep Packet Inspection", IEEE INFOCOM '09.

[9]

Huawei MSCG Hierarchical DPI Solution, http://www.huawei.com/products/datacomm/catalog.do?id=1 219

[10]

Raj Jain and Shawn A. Routheir, "Packet trains - measurements and a new model for computer network traffic", IEEE Journal on Selected Areas in Communications, 4(6):986--995, September 1986.

Digital Library

[11]

Juniper M Series Multiservice Edge Routers, http://www.juniper.net/us/en/local/pdf/datasheets/1000042-en.pdf

[12]

Lukas Kencl, Jean-Yves Le Boudec, "Adaptive Load Sharing for Network Processor", IEEE INFOCOM 2002.

[13]

S. Kumar, et al., "Advanced Algorithms for Fast and Scalable Deep Packet Inspection", ANCS 2006.

Digital Library

[14]

S. Kumar, et al., "Algorithms to Accelerate Multiple Regular Expressions Matching for Deep Packet Inspection", SIGCOMM '06.

Digital Library

[15]

libnids, http://libnids.sourceforge.net/

[16]

Harlan McGhan, "Niagara 2 Opens the Floodgates - Niagara 2 Design is the Closest thing Yet to a True Server on a Chip", The Insider's Guide to Microprocessor Hardware, 11/6/06--01.

[17]

MIT DARPA Intrusion Detection Data Sets, http://www.ll.mit.edu/IST/ideval/data/2000/2000_data_index.html.

[18]

A. Mitra, et al., "Compiling PCRE to FPGA for Accelerating SNORT IDS", ANCS 2007.

Digital Library

[19]

P. Piyachon and Y. Luo, "Efficient Memory Utilization on Network Processors for Deep Packet Inspection", ANCS 2006.

Digital Library

[20]

SPECweb 2005 Published Results, http://www.spec.org/web2005/results/

[21]

Receive Side Scaling (RSS), http://www.microsoft.com/whdc/device/network/NDIS_RSS.mspx/.

[22]

K. W. Ross, "Hash Routing For Collections of Shared Web Caches", IEEE Network, Vol. 11, No. 6 November-December 1997.

Digital Library

[23]

Steve Sistare, "The UltraSparc T2 Processor and the Solaris Operating System", Oct 09, 2007, http://blogs.sun.com/sistare/entry/the_ultrasparc_t2_processor_and

[24]

Reid G. Smith, "The contract Net Protocol: High-level Communication and Control in a distributed Problem Solver", ACM Transactions on Computers, pages 1104--1113, December 1980.

Digital Library

[25]

SNORT Network Intrusion Detection System, http://www.snort.org/

[26]

D. Tam, et al., "Thread Clustering: Sharing-Aware Scheduling on SMP-CMP-SMT Multiprocessors", EuroSys '07.

Digital Library

[27]

L. Tan, et al., "A High Throughput String Matching Architecture for Intrusion Detection and Prevention", ISCA '05.

Digital Library

[28]

D. G. Thaler, C. V. Ravishankar, "Using name-based mappings to Increase Hit Rates", IEEE/ACM Transactions on Networking, Vol. 6 No. 1 pp. 1--14, Feburary 1998.

Digital Library

[29]

B. Veal, et al., "Performance Scalability of a Multi-core Web Server", ANCS 2007.

Digital Library

[30]

J. Verdu, et al., "MultiLayer processing - an execution model for parallel stateful packet processing", ANCS 2008.

Digital Library

[31]

F. Yu, et al., Fast and memory-efficient regular expression matching for deep packet inspection, ANCS 2006.

Digital Library

Cited By

Li CSong LZeng X(2021)An Adaptive Throughput-First Packet Scheduling Algorithm for DPDK-Based Packet Processing SystemsFuture Internet10.3390/fi1303007813:3(78)Online publication date: 19-Mar-2021
https://doi.org/10.3390/fi13030078
Qu QMa ZClausen AJorgensen B(2021)A Comprehensive Review of Machine Learning in Multi-objective Optimization2021 IEEE 4th International Conference on Big Data and Artificial Intelligence (BDAI)10.1109/BDAI52447.2021.9515233(7-14)Online publication date: 2-Jul-2021
https://doi.org/10.1109/BDAI52447.2021.9515233
Le YHuang L(2013)Migrating OpenDPI to Multi-core Platforms for Performance EnhancementProceedings of the 2013 Fourth World Congress on Software Engineering10.1109/WCSE.2013.46(251-256)Online publication date: 3-Dec-2013
https://dl.acm.org/doi/10.1109/WCSE.2013.46
Show More Cited By

Index Terms

An adaptive hash-based multilayer scheduler for L7-filter on a highly threaded hierarchical multi-core server
1. Computer systems organization
  1. Architectures
    1. Parallel architectures
      1. Multiple instruction, multiple data

Recommendations

A scalable multithreaded L7-filter design for multi-core servers
ANCS '08: Proceedings of the 4th ACM/IEEE Symposium on Architectures for Networking and Communications Systems

L7-filter is a significant component in Linux's QoS framework that classifies network traffic based on application layer data. It enables subsequent distribution of network resources in respect to the priority of applications. Considerable research has ...
An efficient parallelized L7-filter design for multicore servers

L7-filter is a significant deep packet inspection (DPI) extension to Netfilter in Linux's QoS framework. It classifies network traffic based on information hidden in the packet payload. Although the computationally intensive payload classification can ...
A Light-Weight Scheduler for Single-ISA Asymmetric Multi-core Processor Using Online Profiling
SNPD '11: Proceedings of the 2011 12th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing

Single-ISA asymmetric multi-core processors are thought as next generation emerging multi-core architecture which has better performance per watt over symmetric multi-core processors. However, they also pose unique challenges to operating system, which ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ANCS '09: Proceedings of the 5th ACM/IEEE Symposium on Architectures for Networking and Communications Systems

October 2009

227 pages

ISBN:9781605586304

DOI:10.1145/1882486

General Chairs:
Peter Onufryk
IDT
,
K. K. Ramakrishnan
AT&T Labs - Research
,
Program Chairs:
Patrick Crowley
Washington University in St. Louis
,
John Wroclawski
USC

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication
SIGARCH: ACM Special Interest Group on Computer Architecture
IEEE-CS\TCCA: TC on Computer Arhitecture

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

ANCS '09

Sponsor:

SIGCOMM
SIGARCH
IEEE-CS\TCCA

ANCS '09: Symposium on Architecture for Networking and Communications Systems

October 19 - 20, 2009

New Jersey, Princeton

Acceptance Rates

Overall Acceptance Rate 88 of 314 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
224
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li CSong LZeng X(2021)An Adaptive Throughput-First Packet Scheduling Algorithm for DPDK-Based Packet Processing SystemsFuture Internet10.3390/fi1303007813:3(78)Online publication date: 19-Mar-2021
https://doi.org/10.3390/fi13030078
Qu QMa ZClausen AJorgensen B(2021)A Comprehensive Review of Machine Learning in Multi-objective Optimization2021 IEEE 4th International Conference on Big Data and Artificial Intelligence (BDAI)10.1109/BDAI52447.2021.9515233(7-14)Online publication date: 2-Jul-2021
https://doi.org/10.1109/BDAI52447.2021.9515233
Le YHuang L(2013)Migrating OpenDPI to Multi-core Platforms for Performance EnhancementProceedings of the 2013 Fourth World Congress on Software Engineering10.1109/WCSE.2013.46(251-256)Online publication date: 3-Dec-2013
https://dl.acm.org/doi/10.1109/WCSE.2013.46
Guo DBhuyan L(2011)A QoS aware multicore hash scheduler for network applications2011 Proceedings IEEE INFOCOM10.1109/INFCOM.2011.5934884(1089-1097)Online publication date: Apr-2011
https://doi.org/10.1109/INFCOM.2011.5934884
Kuang JBhuyan LXie HGuo D(2011)E-AHRWProceedings of the 2011 ACM/IEEE Seventh Symposium on Architectures for Networking and Communications Systems10.1109/ANCS.2011.15(45-56)Online publication date: 3-Oct-2011
https://dl.acm.org/doi/10.1109/ANCS.2011.15
Nelms TAhamad MLin BMogul JIyer R(2010)Packet scheduling for deep packet inspection on multi-core architecturesProceedings of the 6th ACM/IEEE Symposium on Architectures for Networking and Communications Systems10.1145/1872007.1872033(1-11)Online publication date: 25-Oct-2010
https://dl.acm.org/doi/10.1145/1872007.1872033
Gao Xia Bin Liuy (2010)Accelerating network applications on X86-64 platformsProceedings of the The IEEE symposium on Computers and Communications10.1109/ISCC.2010.5546496(906-912)Online publication date: 22-Jun-2010
https://dl.acm.org/doi/10.1109/ISCC.2010.5546496

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten