Securing heap memory by data pointer encoding

https://doi.org/10.1016/j.future.2011.02.006Get rights and content

Abstract

Since pointer variables frequently cause programs to crash in unexpected ways, they often pose vulnerability abused as immediate or intermediate targets. Although code pointer attacks have been historically dominant, data pointer attacks are also recognized as realistic threats. This paper presents how to secure heap memory from data pointer attacks, in particular, heap overflow attacks. Our protection scheme encrypts the data pointers used for linking free chunks, and decrypts the pointers only before dereferencing. We also present a list structure with duplicate links that is harder to break than the conventional linked list structure. Our experiment shows that the proposed data pointer encoding is effective and has slightly better performance than the integrity check of link pointers in GNU’s standard C library.

Research highlights

► Introduce the concept ”data pointer encoding”. ► Show how to protect heap space by data pointer encoding. ► Enhances the structure of free chunks of heap memory to make dual-linked lists for better security and detection. ► Present the experimental result using the GNU’s standard C library.

Introduction

Since pointer variables frequently cause programs to crash in unexpected ways, they often pose vulnerability abused as immediate or intermediate targets. Although code pointer attacks such as stack smashing [1] have been historically dominant, data pointer attacks are also recognized as realistic threats [2].

Well-known vulnerability of data pointers can be found in the heap area managed by dynamic memory allocators. For instance, the bind8 attack [3] on name servers overflows the heap area compromising two pointers used for linking free chunks. When the dynamic memory manager accesses the compromised data pointers for housekeeping, the target memory location whose address is held in one of the data pointers is overwritten with the value in another data pointer. Another well-known vulnerability in heap space may be observed by freeing a chunk twice [4]. Deallocating an already freed chunk corrupts the free chunk list. Attackers can take advantage of the double-free vulnerability to initiate data pointer attacks.

This paper presents how to secure heap memory from data pointer attacks, in particular, heap overflow attacks. In our scheme, the dynamic memory manager encrypts the pointers linking free chunks, and decrypts the pointers only when it is necessary to know the real addresses before dereferencing. We also present a list structure with duplicate links that is harder to break than the conventional linked list structure with a single link. In addition, lists with duplicate links enable intrusion detection to work under explicit control. We implemented our idea in the GNU C library and compared its effectiveness and runtime overhead with those of the GNU’s standard version in a Linux environment.

In the rest of this paper, Section 2 explains the vulnerabilities of data pointers in heap space and how to attack them. Protection of data pointers by encoding is presented in Section 3. Section 4 is about dual-linked lists. Section 5 demonstrates the effectiveness and performance of data pointer encoding with our implementation in the GNU C library’s dynamic memory manager. Section 6 overviews techniques for data pointer protection and Section 7 concludes this paper.

Section snippets

Vulnerability of data pointers in heap space

Many Linux systems adopt Doug Lea’s memory allocator [5] as the default heap manager. The heap memory space consists of allocated and free chunks shown in Fig. 1. The prev_size and size fields denote the size of previous and current chunks respectively. Using them, physically adjacent chunks can be accessed. For allocated chunks, only the size field is valid, while all fields are valid for free chunks. The P field stands for the PREV_INUSE flag that indicates whether the previous chunk is

Data pointer encoding for linked lists

Bins for free chunks are either singly or doubly linked lists. If a bin is constructed as a singly linked list, only the forward link fd is used. For doubly linked lists, both fd and bk pointers are used. In any case, link pointers can protect themselves by encryption.

In our scheme, the dynamic memory manager encrypts a pointer linking free chunks immediately after it is defined, that is, assigned with an address, and decrypts the pointer only when it is necessary to know the real addresses,

Encoded dual-linked list

Traditional linked lists have a mono-link field for each target. Thus, a singly linked list has a link field for the next element, and a doubly linked list has two link fields for the next and previous elements. With data pointer encoding of mono-linked lists, attacks end up with a segment fault because of memory access violations. To detect and handle actively compromised link pointers, we may use the dual-link structure for free chunks shown in Fig. 8.

Free chunks of dual-linked lists use two

Experiment

We have modified the dynamic memory allocation functions in the GNU’s standard C library version 2.10.1 [8] by replacing the code for checking data pointer integrity with our data pointer encoding. We compared ours with the GNU’s with respect to performance and effectiveness in defending against heap overflow attacks. We confirmed that data pointer encoding is as effective as GNU’s dynamic memory manager. Also, simulated attacks of slapper worm [9] and attacks using double free [4] could not

Related work

Heap server [10] is an independent process responsible for dynamic memory allocation. Applications request memory allocation through interprocess communication. Heap server stores and manages memory chunk’s housekeeping data corresponding to the headers and pointers of Lea’s dlmalloc [5] giving the effect of separating the chunk’s meta-data from program data. Heap server performs better than dlmalloc in symmetric multiprocessor environment, but it performs much differently for a program after

Conclusion

Data pointer encoding is effective in protecting heap memory. Dual-linked lists make it possible to detect actively compromised link pointers. Performance testing shows that data pointer encoding has less overhead than the heap manager functions of GNU’s standard C library.

The idea of data pointer encoding can be applied to self-protection of any pointers whether they are code pointers or data pointers. If a data pointer is involved in program security, the pointer can be protected by data

Acknowledgements

This work was supported by the IT R&D Program of MKE/KEIT [2010-KI002090, Development of Technology Base for Trustworthy Computing] and 2009 Hongik University Research Grant.

Kyungtae Kim is a student of the Master’s program in the Department of Computer Engineering, Hongik University in Seoul, Korea. His research interests include the trustworthiness of programs, tools for program security, virtualization and parallel programming.

References (16)

  • S.-U. Guan et al.

    Pseudorandom number generation based on controllable cellular automata

    Future Generation Computer Systems

    (2004)
  • A. One, Smashing the stack for fun and profit, Phrack Magazine, 49...
  • S. Chen, J. Xu, E.C. Sezer, P. Gauriar, R.K. Iyer, Non-control-data attacks are realistic threats, in: the 14th USENIX...
  • US-CERT, Multiple Vulnerabilities in BIND, 2002....
  • US-CERT, Double Free Bug in zlib Compression Library, 2002....
  • D. Lea, A Memory Allocator, 2009....
  • C. Pyo et al.

    DRAM as source of randomness

    Electronics Letters

    (2009)
  • S. Loosemore, R. Stallman, R. McGrath, A. Oram, U. Drepper, The GNU C library reference manual, Free Software...
There are more references available in the full text version of this article.

Cited by (0)

Kyungtae Kim is a student of the Master’s program in the Department of Computer Engineering, Hongik University in Seoul, Korea. His research interests include the trustworthiness of programs, tools for program security, virtualization and parallel programming.

Changwoo Pyo received his Ph.D. in Computer Science from the University of Illinois at Urbana-Champaign in 1989. He was a research fellow in the US Army Corps of Engineers in 1990. He has been a professor of the Department of Computer Engineering, Hongik University in Seoul, Korea since 1991. During the period, he served Hongik University as the Chief of Information and Computing Office of Hongik University. He is currently a member of the board of trustees, Korea Institute of Information Scientists and Engineers. His research interests include the trustworthiness of programs, tools for program security, and program analysis and transformation.

This work was supported by the IT R&D Program of MKE/KEIT [2010-KI002090, Development of Technology Base for Trustworthy Computing] and 2009 Hongik University Research Grant.

View full text