skip to main content
article

Flash crowd mitigation via adaptive admission control based on application-level observations

Published: 01 August 2005 Publication History

Abstract

We design an adaptive admission control mechanism, network early warning system (NEWS), to protect servers and networks from flash crowds and maintain high performance for end-users. NEWS detects flash crowds from performance degradation in responses and mitigates flash crowds by admitting incoming requests adaptively. We evaluate NEWS performance with both simulations and testbed experiments. We first investigate a network-limited scenarion in simulations. We find that NEWS detects flash crowds within 20 seconds. By discarding 32% of incoming requests, NEWS protects the target server and networks from overloading, reducing the response packet drop rate from 25% to 2%. For admitted requests, NEWS increases their response rate by two times. This performance is similar to the best static rate limiter deployed in the same scenario. We also investigate the impact of detection intervals on NEWS performance, showing it affects both detection delay and false alarm rate. We further consider a server memory-limited scenario in testbed experiments, confirming that NEWS is also effective in this case. We also examine the runtime cost of NEWS traffic monitoring in practice and find that it consumes little CPU time and relatively small memory. Finally, we show NEWS effectively protects bystander traffic from flash crowds.

References

[1]
Adler, S. 1999. The slashdot effect, an analysis of three Internet publications. Available at http://ssadler.phy.bnl.gov/adler/SDE/SlashDotEffect.html.]]
[2]
Arlitt, M. and Jin, T. 2000. A workload characterization study of the 1998 World Cup Web site. IEEE Network, Special Issue on Web Performance 14, 3 (May), 30--37.]]
[3]
Balakrishnan, H., Rahul, H., and Seshan, S. 1999. An integrated congestion management architecture for internet hosts. In Proceedings of the ACM SIGCOMM. Cambridge, MA, 175--187.]]
[4]
Barford, P. and Plonka, D. 2001. Characteristics of network traffic flow anomalies. In Proceedings of the ACM SIGCOMM Internet Measurement Workshop. San Francisco, CA, 2--13.]]
[5]
Basseville, M. and Nikiforov, I. V. 1993. Detection of Abrupt Changes---Theory and Application, 1st Ed. Prentice-Hall, Englewood Cliffs, NJ.]]
[6]
Berners-Lee, T., Fielding, R., and Frystyk, H. 1996. Hypertext transfer protocol---HTTP/1.0. RFC 1945. IETF. Available at ftp://ftp.isi.edu/in-notes/rfc1945.txt.]]
[7]
Blazek, R. B., Kim, H., Rozovskii, B., and Tartakovsky, A. 2001. A novel approach to detection of denial-of-service attacks via adaptive sequential and batch-sequential change-point detection methods. In Proceedings of the IEEE Systems, Man, and Cybernetics Information Assurance Workshop. West Point, NY.]]
[8]
Breslau, L., Knightly, E. W., Shenker, S., Stoica, I., and Zhang, H. 2000. Endpoint admission control: Architectural issues and performance. In Proceedings of the ACM SIGCOMM. Stockholm, Sweden, 57--69.]]
[9]
Cetinkaya, C. and Knightly, E. 2000. Egress admission control. In Proceedings of the IEEE Infocom. Tel-Aviv, Israel, 1471--1480.]]
[10]
Chen, H. and Mohapatra, P. 2002. Session-based overload control in qos-aware Web servers. In Proceedings of the IEEE Infocom. New York, NY.]]
[11]
Chen, X. and Heidemann, J. 2003. Preferential treatment for short flows to reduce Web latency. Comput. Netw. 41, 6 (April), 779--794.]]
[12]
Clark, D. and Fang, W. 1998. Explicit allocation of best effort packet delivery service. ACM/IEEE Trans. Netw. 6, 4 (Aug.), 362--373.]]
[13]
Clark, D. D., Shenker, S., and Zhang, L. 1992. Supporting real-time applications in an integrated services packet network: Architecture and mechanism. In Proceedings of the ACM SIGCOMM. Baltimore, MD. 14--26. Available at citeseer.nj.nec.com/clark92supporting.html.]]
[14]
Cormen, T. H., Leiserson, C. E., Rivest, R. L., and Stein, C. 2001. Introduction to Algorithms. 2nd Ed. The MIT Press, Cambridge, MA.]]
[15]
Demers, A., Keshav, S., and Shenker, S. 1989. Analysis and simulation of a fair-queueing algorithm. In Proceedings of the ACM SIGCOMM. Austin, TX. 1--12.]]
[16]
Eggert, L. and Heidemann, J. 1999. Application-level differentiated services for Web servers. World-Wide Web J. 2, 3 (Aug.), 133--142.]]
[17]
Estan, C. and Varghese, G. 2002. New directions in traffic measurement and accounting. In Proceedings of the ACM SIGCOMM. Pittsburgh, PA. 323--336.]]
[18]
Fang, W., Seddigh, N., and Nandy, B. 2000. A time-sliding window three colour marker (TSWTCM). RFC 2859. IETF.]]
[19]
Floyd, S. and Fall, K. 1999. Promoting the use of end-to-end congestion control in the Internet. ACM/IEEE Trans. Netw. 7, 4 (Aug.), 458--473.]]
[20]
Floyd, S. and Jacobson, V. 1993. Random early detection gateways for congestion avoidance. ACM/IEEE Trans. Netw. 1, 4 (Aug.), 397--413.]]
[21]
Floyd, S. and Paxson, V. 2001. Difficulties in simulating the Internet. ACM/IEEE Trans. Netw. 9, 4 (Aug.), 392--403.]]
[22]
Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and Berners-Lee, T. 1999. Hypertext transfer protocol---HTTP/1.1. RFC 2616. IETF. Available at http://www.w3.org/Protocols/rfc2616/rfc2616.html.]]
[23]
Gibbens, R., Kelly, F. P., and Key, P. 1995. A decision-theoretic approach to call admission control in ATM networks. IEEE J. Select. Areas Comm. 13, 6 (Aug.), 30--37.]]
[24]
Guo, L. and Matta, I. 2001. The war between mice and elephants. In Proceedings of the IEEE International Conference on Network Protocols. Riverside, CA.]]
[25]
Jacobson, V. 1988. Congestion avoidance and control. In Proceedings of the ACM SIGCOMM. Stanford, CA. 314--329.]]
[26]
Jamin, S., Danzig, P., Shenker, S., and Zhang, L. 1995. Measurement-based admission control algorithms for controlled-load service. In Proceedings of the ACM SIGCOMM. Cambridge, MA. 2--13.]]
[27]
Jamjoom, H. and Shin, K. G. 2003. Persistent dropping: An efficient control of traffic aggregates. In Proceedings of the ACM SIGCOMM. Karlsruhe, Germany, 287--298.]]
[28]
Jung, J., Krishnamurthy, B., and Rabinovich, M. 2002. Flash crowds and denial of service attacks: Characterization and implications for CDNs and Web sites. In Proceedings of the International World Wide Web Conference. Hawaii, 252--262.]]
[29]
Kim, M. and Noble, B. 2001. Mobile network estimation. In Proceedings of the ACM/IEEE International Conference on Mobile Computing and Networking. Rome, Italy, 298--309.]]
[30]
Labs, L.-B. 1992. The Internet Traffic Archive. Available at http://ita.ee.lbl.gov/.]]
[31]
Lan, K. and Heidemann, J. 2002. Rapid model parameterization from traffic measurement. ACM Trans. Model. Comput. Simul. 12, 3 (July), 201--229.]]
[32]
Lever, C., Eriksen, M. A., and Molloy, S. P. 2000. An analysis of the TUX Web server. Tech. rep., (Nov.) Center for Information Technology Integration, University of Michigan.]]
[33]
Lin, D. and Morris, R. 1997. Dynamics of random early detection. In Proceedings of the ACM SIGCOMM. Cannes, France, 127--137.]]
[34]
Lu, C., Abdelzaher, T. F., Stankovic, J. A., and Son, S. H. 2001. A feedback control approach for guaranteeing relative delays in Web servers. In Proceedings of the IEEE Real-Time Technology and Applications Symposium. Taipei, Taiwan.]]
[35]
Mahajan, R., Bellovin, S., Floyd, S., Ioannidis, J., Paxson, V., and Shenker, S. 2001. Controlling high bandwidth aggregates in the network. Tech. rep. International Computer Science Institute (ICSI), Berkeley, CA. (July).]]
[36]
Mahajan, R. and Floyd, S. 2001. Controlling high bandwidth flows at the congested router. In Proceedings of the IEEE International Conference on Network Protocols. Riverside, CA, 1--12.]]
[37]
Mundur, P., Simon, R., and Sood, A. 1999. Integrated admission control in hierarchical video-on-demand systems. In Proceedings of the IEEE International Conference on Multimedia Computing and Systems. Florence, Italy, 220--225.]]
[38]
Netfilter/iptables. 2003. Netfilter/iptables project. Available at http://www.netfilter.org/.]]
[39]
NetFlow. 2003. NetFlow performance analysis. White paper. Cisco Systems, Inc. Available at http://www.cisco.com/warp/public/cc/pd/iosw/prodlit/ntfo_wp.htm.]]
[40]
NIST. 1998. NIST net network emulator. Available at http://is2.antd.nist.gov/itg/nistnet/.]]
[41]
Pan, R., Prabhakar, B., and Psounis, K. 2000. CHOKE, a stateless active queue management scheme for approximating fair bandwidth allocation. In Proceedings of the IEEE Infocom. Tel-Aviv, Israel, 942--951.]]
[42]
Parekh, A. and Gallagher, R. G. 1993. A generalized processor sharing approach to flow control in integrated services networks: the single-node case. ACM/IEEE Trans. Netw. 1, 3 (June), 344--357.]]
[43]
Park, K. and Lee, H. 2001. On the effectiveness of route-based packet filtering for distributed DoS attack prevention in power-law Internets. In Proceedings of the ACM SIGCOMM. San Diego CA, 15--26.]]
[44]
Pieda, P., Ethridge, J., Baines, M., and Shallwani, F. 2000. A network simulator, differentiated services implementation. Available at http://www7.nortel.com:8080/CTL/.]]
[45]
Rahin, M. A. and Kara, M. 1998. Call admission control algorithms in ATM networks: A performance comparison and research directions. Research rep. (Feb.) University of Leeds.]]
[46]
Savage, S., Wetherall, D., Karlin, A., and Anderson, T. 2000. Practical network support for IP traceback. In Proceedings of the ACM SIGCOMM. Stockholm, Sweden, 295--306.]]
[47]
Shenker, S. and Wroclawski, J. 1997. General characterization parameters for integrated service network elements. RFC 2215. IETF.]]
[48]
Shoeten, A. C., Partridge, C., Sanchez, L. A., Jones, C. E., Tchakountio, F., Kent, S. T., and Strayer, W. T. 2001. Hash-based IP traceback. In Proceedings of the ACM SIGCOMM. San Diego, CA. 3--14.]]
[49]
Stoica, I., Shenker, S., and Zhang, H. 1998. Core-stateless fair queueing: Achieving approximately fair bandwidth allocations in high speed networks. In Proceedings of the ACM SIGCOMM. Vancouver, British Columbia, Canada, 118--130.]]
[50]
VINT. 1997. UCB/LBNL/VINT network simulator---ns (version 2). Available at http://www. isi.edu/nsnam/ns/.]]
[51]
Welsh, M. and Culler, D. 2003. Adaptive overload control for busy Internet servers. In Proceedings of the USENIX Symposium on Internet Technologies and Systems. Seattle, WA.]]
[52]
Wolski, R., Spring, N. T., and Hayes, J. 1999. The network weather service: A distributed resource performance forecasting service for metacomputing. J. Fut. Gener. Comput. Syst. 15, 6 (Oct.), 757--768.]]

Cited By

View all
  • (2024)Targeting Tail Latency in Replicated Systems with Proactive RejectionProceedings of the 25th International Middleware Conference10.1145/3652892.3700775(327-340)Online publication date: 2-Dec-2024
  • (2021)Mitigation of Flash Crowd in Web Services By Providing Feedback Information to UsersIEICE Transactions on Information and Systems10.1587/transinf.2020MPP0003E104.D:1(63-75)Online publication date: 1-Jan-2021
  • (2021)Distributed Denial of Service Attack Prevention from Traffic Flow for Network Performance Enhancement2021 2nd International Conference on Smart Electronics and Communication (ICOSEC)10.1109/ICOSEC51865.2021.9591974(406-413)Online publication date: 7-Oct-2021
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Internet Technology
ACM Transactions on Internet Technology  Volume 5, Issue 3
August 2005
132 pages
ISSN:1533-5399
EISSN:1557-6051
DOI:10.1145/1084772
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 August 2005
Published in TOIT Volume 5, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Flash crowds
  2. admission control
  3. experimentation with testbeds
  4. simulations

Qualifiers

  • Article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Targeting Tail Latency in Replicated Systems with Proactive RejectionProceedings of the 25th International Middleware Conference10.1145/3652892.3700775(327-340)Online publication date: 2-Dec-2024
  • (2021)Mitigation of Flash Crowd in Web Services By Providing Feedback Information to UsersIEICE Transactions on Information and Systems10.1587/transinf.2020MPP0003E104.D:1(63-75)Online publication date: 1-Jan-2021
  • (2021)Distributed Denial of Service Attack Prevention from Traffic Flow for Network Performance Enhancement2021 2nd International Conference on Smart Electronics and Communication (ICOSEC)10.1109/ICOSEC51865.2021.9591974(406-413)Online publication date: 7-Oct-2021
  • (2019)Fuzzy based energy efficient workload management system for flash crowdComputer Communications10.1016/j.comcom.2019.08.020Online publication date: Aug-2019
  • (2019)Detecting and confronting flash attacks from IoT botnetsThe Journal of Supercomputing10.1007/s11227-019-03005-2Online publication date: 14-Oct-2019
  • (2018)DDoS detection and discrimination from flash events: a compendious review2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC)10.1109/ICSCCC.2018.8703335(518-524)Online publication date: Dec-2018
  • (2017)Estimating Relative User Expertise for Content Quality Prediction on RedditProceedings of the 28th ACM Conference on Hypertext and Social Media10.1145/3078714.3078720(55-64)Online publication date: 4-Jul-2017
  • (2017)Alleviation of DDoS attack using advance technique2017 International Conference on Innovative Mechanisms for Industry Applications (ICIMIA)10.1109/ICIMIA.2017.7975595(172-176)Online publication date: Feb-2017
  • (2017)Flash crowd prediction in Twitter2017 4th International Conference on Advanced Computing and Communication Systems (ICACCS)10.1109/ICACCS.2017.8014676(1-6)Online publication date: Jan-2017
  • (2017)Web Service Flash Crowd Mitigation Using Feedback to Users2017 IEEE 31st International Conference on Advanced Information Networking and Applications (AINA)10.1109/AINA.2017.121(213-219)Online publication date: Mar-2017
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media