Skip to main content

Encoding Function Pointers and Memory Arrangement Checking against Buffer Overflow Attack

  • Conference paper
  • First Online:
Information and Communications Security (ICICS 2002)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2513))

Included in the following conference series:

Abstract

Program counter is the only mechanism for processor to access instruction to execute. Protecting program counter is the fundamental defense for securing computer system. This paper presents a scheme of protecting program counter by encoding function pointers. In the scheme, every function address is encoded by linker. Compiler inserts instructions for decoding function addresses before call instruction. Encoding code pointers, function addresses as well as return addresses in stack frame, provides substantial coverage of protecting program counter. Several suggestions are also made to detect compromised code pointers at run-time without memory space for sensor mechanism. A demo Linux system has been under construction with the proposed scheme. Experimental data shows performance slowdown less than 10% when all return addresses and function addresses are encoded. With a Pentium III processor of 866MHz, the overhead for each function call is on the order of nanoseconds. We plan to migrate parts of our code pointer encoding scheme from linker to dynamic linker, which should improve security and performance.

This work was supported in part by NSF (CCR-0113409)

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aleph One. Smashing The Stack For Fun And Profit. Phrack Magazine, 7(49):File 14, 1996.

    Google Scholar 

  2. Todd M. Austin, Scott E. Breach, and Gurindar S. Sohi. Efficient Detection of All Pointer and Array Access Errors. In Proceedings of the SIGPLAN’ 94 Conference on Programming Language Design and Implementation, 1994.

    Google Scholar 

  3. Arash Baratloo, Navjot Singh, and Timothy Tsai. Transparent Run-Time Defense Against Stack Smashing Attacks. In Proceedings of the USENIX Annual Technical Conference, June 2000.

    Google Scholar 

  4. Bulba and Kil3r. BYPASSING STACKGUARD AND STACKSHIELD. Phrack Magazine, 10(56), May 2000.

    Google Scholar 

  5. CERT Coordination Center. CERT Advisory CA-1996-26: Deanial-of-Service Attack via ping. http://www.cert.org/advisories/CA-1996-26.html, Dec 1996.

  6. Tzi-Cker Chiueh and Fu-Hau Hsu. RAD: A Compile-Time Solution to Buffer Overflow Attacks. In 21st International Conference on Distributed Computing Systems, 2001.

    Google Scholar 

  7. CORE SECURITY TECHNOLOGIES Inc. Multiple Vulnerabilities in Stack Smashing Protection Technologies. Security Focus Online, Apr. 2002.

    Google Scholar 

  8. Crispin Cowan, Steve Beattie, Ryan Finnin Day, Calton Pu, Perry Wagle, and Erik Walthinsen. Protecting Systems from Stack Smashing Attacks with StackGuard. In the Linux Expo, 1999.

    Google Scholar 

  9. Crispin Cowan, Calton Pu, David Maier, Heather Hinton, Peat Bakke, Steve Beattie, Aaron Grier, Perry Wagle, and Qian Zhang. StackGuard: Automatic Detection and Prevention of Buffer-Overflow Attacks. In the 7th USENIX Security Symposium, 1998.

    Google Scholar 

  10. Crispin Cowan, Perry Wagle, Calton Pu, Steve Beattie, and Jonathan Walpole. Buffer Overflows: Attacks and Defenses for the Vulnerability of the Decade. In the DARPA Information Survivability Conference and Expo DISCEX, 1999.

    Google Scholar 

  11. Roman Danyliw and Allen Householder. CERT Advisory CA-2001-19: Code Red Worm Exploiting Buffer Overflow In IIS Indexing Service DLL. http://www.cert.org/advisories/CA-2001-19.html, Jul 2001.

  12. Roman Danyliw and Allen Householder. CERT Advisory CA-2001-23: Continued Threat of the Code Red Worm. http://www.cert.org/advisories/CA-2001-23.html, Jul 2001.

  13. M. W. Eichin and J. A. Rochlis. With Microscope and Tweezers: An Analysis of the Internet Virus of November 1988. In Proc. IEEE Symposium on Security and Privacy, pages 326–343, 1989.

    Google Scholar 

  14. Samuel P. Harbison and Guy L. Steele Jr. C, A Reference Manual. Prentice Hall, 4th edition, 1995.

    Google Scholar 

  15. R. W. M. Jones and P. H. J. Kelly. Backwards-compatible bounds checking for arrays and pointers in C programs. In Proceedings of the Third International Workshop on Automated Debugging, 1997.

    Google Scholar 

  16. David Larochelle and David Evans. Statically Detecting Likely Buffer Overflow Vulnerabilities. In 2001 USENIX Security Symposium, 2001.

    Google Scholar 

  17. Gyungho Lee and Akhilesh Tyagi. Encoded Program Counter: Self-Protection from Buffer Overflow. In International Conference on Internet Computing, Jun. 2000.

    Google Scholar 

  18. Mindcraft. Webstone standard web server benchmark. http://www.mindcraft.com/webstone.

  19. Mudge. How to Write Buffer Overflows. http://www.insecure.org/stf/mudge_buffer_overflow_tutorial.html, 1995.

  20. Unix System Laboratories. System V Application Binary Interface: Intel386 Architecture Processor Supplement. Prentice Hall, 3rd edition, Jan 1994.

    Google Scholar 

  21. J. Viega, J. T. Bloch, T. Kohno, and G. McGraw. ITS4: A Static Vulnerability Scanner for C and C++ Code. In Proceedings of Annual Computer Security Applications Conference, Dec 2000.

    Google Scholar 

  22. David Wagner, Jeffrey S. Foster, Eric A. Brewer, and Alexander Aiken. A First Step Towards Automated Detection of Buffer Overrun Vulnerabilities. In Network and Distributed System Security Symposium, Feb 2000.

    Google Scholar 

  23. Mariusz Woloszyn. StackGuard Mechanism: Emsi’s Vulnerability. http://immunix.org/StackGuard/emi_vuln.html 1999

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Pyo, C., Lee, G. (2002). Encoding Function Pointers and Memory Arrangement Checking against Buffer Overflow Attack. In: Deng, R., Bao, F., Zhou, J., Qing, S. (eds) Information and Communications Security. ICICS 2002. Lecture Notes in Computer Science, vol 2513. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36159-6_3

Download citation

  • DOI: https://doi.org/10.1007/3-540-36159-6_3

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-00164-5

  • Online ISBN: 978-3-540-36159-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics