Fang Hao
ABSTRACT
Real-time IP flow estimation has many potential applications in network management, monitoring, security, and traffic engineering. Existing techniques typically rely on flow definitions being constrained as subsets of the fields in packet headers. This makes flow-membership tests relatively inexpensive. In this paper, we consider a more general flow estimation problem that needs complex packet-payload based tests for flow-membership. An example is to estimate traffic with common strings in the payload and detect potential virus signatures for early alarm generation. We develop a fast, memory efficient algorithm for solving this problem as a variant of the longest common subsequence problem. This is done via an application of Rabin fingerprinting in combination with Bloom Filters. Both analysis and simulation show the effectiveness of the developed method.
- Broder, A.Z., "Some Applications of Rabin's Fingerprinting Method", in Sequences II: Methods in Communications, Security and Computer Science, R. Capocelli, A. De Santis, U. Vaccaro (eds) 1993.]]Google Scholar
- Rao, C.R., Linear Statistical Inference and its Applications, John Wiley, 1973.]]Google Scholar
- T.S. Ferguson., A Course in Large Sampling Theory, Chapman and Hall, 1996.]]Google Scholar
- Chan, C, and Lu, H., "Fingerprinting using Polynomials: Rabin's Method", CMPUT690 Technical Report, University of Alberta, 2001.]]Google Scholar
- H. Kim and B. Karp, "Autograph: Toward Automated, Distributed Work Signature Detection", Proceedings of the 13th Usenix Security Symposium (Security 2004), 2004.]] Google ScholarDigital Library
- C. Kreibich and and J. Crowcroft, "Honeycomb- Creating Intrusion Detection Signatures using Honeypots", HotnetsII, 2003.]]Google Scholar
- C.J. Coit, S. Staniford, and J. McAlerny, "Towards Faster String Matching for Pattern Detection or Exceeding the Speed of Snort", Proceedings of the 2nd DARPA Information Survivability Conference and Exposition (DISCEX II), 2001.]]Google ScholarCross Ref
- S. Dharmapurikar, P. Krishnamurthy, T. Sproull, and J. Lockwood, "Deep Packet Inspection using Parallel Bloom Filters", Symposium on High Performance Interconnects (HotI), 2003.]]Google ScholarCross Ref
- S. Singh, C. Estan, G. Varghese and S. Savage, "The EarlyBird System for Real-Time Detection of Unknown Works", Technical Report CS2003-0761, UCSD, 2003.]]Google Scholar
- C. Estan, S. Savage and G. Varghese, "Automatically Inferring Patterns of Resource Consumption in Network Traffic", Proceedings of ACM SIGCOMM, Aug. 2003.]] Google ScholarDigital Library
- C. Estan, and G. Varghese, "New Directions in Traffic Measurement and Accounting", Proceedings of ACM SIGCOMM, Aug. 2002.]] Google ScholarDigital Library
- C. Estan, G. Varghese, and M. Fisk, "Bitmap Algorithms for Counting Active Flows on High Speed Links", Internet Measurement Conference, Oct. 2003.]] Google ScholarDigital Library
- U. Manber, "Finding Similar Files in a Large File System", USENIX Winter 1994 Technical Conference, PP. 1--10, 1994.]] Google ScholarDigital Library
- M.O. Rabin "Fingerprinting by Random Polynomials", Center for Research in Computing Technology, Harvard University, Tech Report TR-CSE-03-01, 1981 .]]Google Scholar
- J. Twycross and M.M. Williamson, , "Implementing and Testing a New Virus Throttle", 12th USENIX Security Symposium, 2003.]] Google ScholarDigital Library
- Sprint ATL, http://ipmon.sprint.com.]]Google Scholar
- http://www.caida.org/analysis/security/witty.]]Google Scholar
- http://www.clamav.net.]]Google Scholar
- Hao, F., Kodialam, M., Lakshman, T. V., and Zhang, H., "Fast, Memory-Efficient Traffic Estimation by Coincidence Counting", Proceedings of INFOCOM'2005.]]Google Scholar
- Hao, F., Kodialam, M., and Lakshman, T. V., "Real-Time Detection of Hidden Traffic Patterns", Proceedings of ICNP'2004.]] Google ScholarDigital Library
Index Terms
- Fast payload-based flow estimation for traffic monitoring and network security
Recommendations
An adaptive QoS control scheme based on traffic estimation for efficient multimedia services in wireless access networks
ICHIT'11: Proceedings of the 5th international conference on Convergence and hybrid information technologyIn this paper, an enhanced bandwidth allocation scheme according to traffic estimation for adaptive QoS control is proposed with the use of the IEEE 802.11e standard in the wireless access network. The interface between the core network and the access ...
A new scheme for traffic estimation and resource allocation for bandwidth brokers
This paper is motivated by the concern of a multi-service network provider who plans to offer quality of service guarantees to users. A bandwidth broker acts as the resource manager for each network provider. Neighboring bandwidth brokers communicate ...
Performance of estimated traffic matrices in traffic engineering
SIGMETRICS '03: Proceedings of the 2003 ACM SIGMETRICS international conference on Measurement and modeling of computer systemsWe consider the performance of estimated traffic matrices in traffic engineering. More precisely, we first optimize the routing in an IP backbone to minimize congestion with the estimated traffic matrix. We then test the performance of the resulting ...
Comments