As market is driving larger-sized TVs and display devices, video compression codec and video image processing algorithm are being developed with higher (4K to 8K) video resolution. It is critical to implement an optimized display system to satisfy the increasing needs for external memory bandwidth. In this paper, we propose an optimized hardware architecture of tile to raster scan line buffer. The tile to raster scan line buffer converts the video image data stored in tiled format to the raster scan order for display devices. Using the pattern of tile to raster scan order, the double buffering used by conventional system is removed and a simplified buffer access algorithm for real-time hardware implementation is proposed. The proposed method reduces buffer size by 50% compared to the conventional architecture.