Skip to main content
Log in

An index-split Bloom filter for deep packet inspection

  • Research Papers
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Deep packet inspection (DPI) scans both packet headers and payloads to search for predefined signatures. As link rates and traffic volumes of Internet are constantly growing, DPI is facing the high performance challenge of how to achieve line-speed packet processing with limited embedded memory. The recent trie bitmap content analyzer (TriBiCa) suffers from high update overhead and many false positive memory accesses, while the shared-node fast hash table (SFHT) suffers from high update overhead and large memory requirements. This paper presents an index-split Bloom filter (ISBF) to overcome these issues. Given a set of off-chip items, an index of each item is split apart into several groups of constant bits, and each group of bits uses an array of on-chip parallel counting Bloom filters (CBFs) to represent the overall off-chip items. When an item is queried, several groups of on-chip parallel CBFs constitute an index of an off-chip item candidate for a match. Furthermore, we propose a lazy deletion algorithm and vacant insertion algorithm to reduce the update overhead of ISBF, where an on-chip deletion bitmap is used to update on-chip parallel CBFs, not adjusting other related off-chip items. The ISBF is a time/space-efficient data structure, which not only achieves O(1) average memory accesses of insertion, deletion, and query, but also reduces the memory requirements. Experimental results demonstrate that compared with the TriBiCa and SFHT, the ISBF significantly reduces the off-chip memory accesses and processing time of primitive operations, as well as both the on-chip and off-chip memory sizes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Paxson V, Asanovic K, Dharmapurikar S, et al. Rethinking hardware support for network analysis and intrusion prevention. In: Proceedings of USENIX Workshop on Hot Topics in Security 2006. Vancouver: USENIX Press, 2006

    Google Scholar 

  2. Estan C, Varghese G. New directions in traffic measurement and accounting. In: Proceedings of ACM SIGCOMM 2001. San Diego: ACM Press, 2001

    Google Scholar 

  3. Lakshminarayanan K, Rangarajan A, Venkatachary S. Algorithms for advanced packet classification with ternary CAMs. In: Proceedings of ACM SIGCOMM 2005. Philadelphia: ACM Press, 2005

    Google Scholar 

  4. Bonomi F, Mitzenmacher M, Panigrapy R, et al. Beyond Bloom filters: from approximate membership checks to approximate state machines. In: Proceedings of ACM SIGCOMM 2006. Pisa: ACM Press, 2006

    Google Scholar 

  5. Roesch M. Snort c lightweight intrusion detection for networks. In: Proceedings of LISA 1999. Seattle: USENIX Press, 1999

    Google Scholar 

  6. Paxon V. Bro: A system for detecting network intruders in real-time. Comput Networks, 1999, 31: 2435–2463

    Article  Google Scholar 

  7. Levandoski J, Sommer E, Strait M. Application layer packet classifier for Linux. http://l7-filter.sourceforge.net, 2008

  8. Sen S, Spatscheck O, Wang D. Accurate, scalable in-network identification of P2P traffic using application signatures. In: Proceedings of www 2004. Manhattan: ACM Press, 2004

    Google Scholar 

  9. Karagiannis T, Broido A, Faloutsos M, et al. Transport layer identification of p2p traffic. In: Proceedings of IMC 2004. Taormina: ACM Press, 2004

    Google Scholar 

  10. Clark C R, Schimmel D E. Scalable pattern matching on high-speed networks. In: Proceedings of IEEE FCCM 2004. Napa: IEEE Press, 2004

    Google Scholar 

  11. Sourdis I, Pnevmatikatos D. Pre-decoded CAMs for efficient and high-speed NIDS pattern matching. In: Proceedings of IEEE FCCM 2004. Napa: IEEE Press, 2004

    Google Scholar 

  12. Yu F, Katz R, Lakshman T V. Gigabit rate packet pattern-matching using TCAM. In: Proceedings of IEEE ICNP 2004. Berlin: IEEE Press, 2004

    Google Scholar 

  13. Piyachon P, Luo Y. Efficient memory utilization on network processors for deep packet inspection. In: Proceedings of ACM/IEEE ANCS 2006. San Jose: ACM Press, 2006

    Google Scholar 

  14. Lu H, Zheng K, Liu B, et al. A memory-efficient parallel string matching architecture for high-speed intrusion detection. IEEE J Select Areas Commun, 2006, 34: 1793–1804

    Article  Google Scholar 

  15. Artan N S, Chao H J. TriBiCa: trie bitmap content analyzer for high-speed network intrusion detection. In: Proceedings of IEEE INFOCOM 2007. Anchorage: IEEE Press, 2007

    Google Scholar 

  16. Song H, Dharmapurikar S, Turner J, et al. Fast hash table lookup using extended Bloom filter: an aid to network processing. In: Proceedings of ACM SIGCOMM 2005. Philadelphia: ACM Press, 2005

    Google Scholar 

  17. Fan L, Cao P, Almeida J, et al. Summary cache: a scalable wide-area web cache sharing protocol. IEEE/ACM Trans Network, 2000, 8: 281–293

    Article  Google Scholar 

  18. Tan L, Brotherton B, Sherwood T. Bit-split string-matching engines for intrusion detection and prevention. ACM Trans Architect Code Opt, 2006, 3: 3–34

    Article  Google Scholar 

  19. Brodie B C, Cytron R K, Taylor D E. A scalable architecture for high-throughput regular-expression pattern matching. In: Proceedings of ISCA 2006. Boston: IEEE Press, 2006

    Google Scholar 

  20. Lunteren J. High performance pattern-matching for intrusion detection. In: Proceedings of IEEE INFOCOM 2006. Barcelona: IEEE Press, 2006

    Google Scholar 

  21. Yu F, Chen Z, Diao Y, et al. Fast and memory-efficient regular expression matching for deep packet inspection. In: Proceedings of ACM/IEEE ANCS 2006. San Jose: ACM Press, 2006

    Google Scholar 

  22. Tuck N, Sherwood T, Calder B, et al. Deterministic memory-efficient string matching algorithms for intrusion detection. In: Proceedings of IEEE INFOCOM 2004. Hong Kong: IEEE Press, 2004

    Google Scholar 

  23. Aho A V, Corasick M J. Efficient string matching: an aid to bibliographic search. Commun ACM, 1975, 18: 333–340

    Article  MATH  MathSciNet  Google Scholar 

  24. Dharmapurikar S, Lockwood J. Fast and scalable pattern matching for content filtering. In: Proceedings of ACM/IEEE ANCS 2005. Princeton: ACM Press, 2005

    Google Scholar 

  25. Hua N, Song H, Lakshman T V. Variable-stride multi-pattern matching for scalable deep packet inspection. In: Proceedings of IEEE INFOCOM 2009. Rio de Janeiro: IEEE Press, 2009

    Google Scholar 

  26. Song T, Zhang W, Wang D, et al. A memory efficient multiple pattern matching architecture for network security. In: Proceedings of IEEE INFOCOM 2008. Phoenix: IEEE Press, 2008

    Google Scholar 

  27. Kumar S, Dharmapurikar S, Yu F, et al. Algorithms to accelerate multiple regular expressions matching for deep packet inspection. In: Proceedings of ACM SIGCOMM 2006. Pisa: ACM Press, 2006

    Google Scholar 

  28. Kumar S, Turner J, Williams J. Advanced algorithms for fast and scalable deep packet inspection. In: Proceedings of ACM/IEEE ANCS 2006. San Jose: ACM Press, 2006

    Google Scholar 

  29. Becchi M, Cadambi S. Memory-efficient regular expression search using state merging. In: Proceedings of IEEE INFOCOM 2007. Anchorage: IEEE Press, 2007

    Google Scholar 

  30. Smith R, Estan C, Jha S. XFA: Faster signature matching with extended automata. In: Proceedings of IEEE Symposium on Security and Privacy 2008. Oakland: IEEE Press, 2008

    Google Scholar 

  31. Smith R, Estan C, Jha S, et al. Deflating the big bang: fast and scalable deep packet inspection with extended finite automata. In: Proceedings of ACM SIGCOMM 2008. Seattle: ACM Press, 2008

    Google Scholar 

  32. Kumar S, Chandrasekaran B, Turner J, et al. Curing regular expressions matching algorithms from insomnia, amnesia, and acalculia. In: Proceedings of ACM/IEEE ANCS 2007. Orlando: ACM Press, 2007

    Google Scholar 

  33. Broder A, Mitzenmacher M. Network applications of Bloom filters: A survey. Internet Math, 2004, 1: 485–509

    MATH  MathSciNet  Google Scholar 

  34. Dharmapurikar S, Krishnamurthy P, Sproull T S, et al. Deep packet inspection using parallel Bloom filters. IEEE Micro, 2004, 24: 52–61

    Article  Google Scholar 

  35. Broder A, Mitzenmacher M. Using multiple hash functions to improve IP lookups. In: Proceedings of IEEE INFOCOM 2001. Anchorage: IEEE Press, 2001

    Google Scholar 

  36. Kirsch A, Mitzenmacher M. Simple summaries for hashing with choices. IEEE/ACM Trans Network, 2008, 16: 218–231

    Article  Google Scholar 

  37. Kumar S, Crowley P. Segmented hash: an efficient hash table implementation for high performance networking subsystems. In: Proceedings of ACM/IEEE ANCS 2005. Princeton: ACM Press, 2005

    Google Scholar 

  38. Kumar S, Turner J, Crowley P. Peacock hashing: deterministic and updatable hashing for high performance networking. In: Proceedings of IEEE INFOCOM 2008. Phoenix: IEEE Press, 2008

    Google Scholar 

  39. Kirsch A, Mitzenmacher M. The power of one move: hashing schemes for hardware. In: Proceedings of IEEE INFOCOM 2008. Phoenix: IEEE Press, 2008

    Google Scholar 

  40. Yu H, Mahapatra R. A memory-efficient hashing by multi-predicate Bloom filters for packet classification. In: Proceedings of IEEE INFOCOM 2008. Phoenix: IEEE Press, 2008

    Google Scholar 

  41. Hua N, Lin B, Xu J. Rank-indexed hashing: a compact construction of Bloom filters and variants. In: Proceedings of IEEE ICNP 2008. Orlando: IEEE Press, 2008

    Google Scholar 

  42. Varghese G. Network algorithms: an interdisciplinary approach to designing fast network devices. San Fransisco, CA: Morgan Kaufmann Publishers, 2004

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to DaFang Zhang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Huang, K., Zhang, D. An index-split Bloom filter for deep packet inspection. Sci. China Inf. Sci. 54, 23–37 (2011). https://doi.org/10.1007/s11432-010-4132-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11432-010-4132-4

Keywords

Navigation