This paper is focused on the Adaptive Loop Filter (ALF) which is responsible to reduce the distortion between an original image and the encoded image during the video coding process by fixing artifacts from previous stages. It was proposed a novel hardware design for the ALF core which is capable to process all ALF sizes (5×5, 7×7 and 9×9), saving hardware resources consumption through reuse. The design was planned to process QFHD (3840 × 2160 pixels) video sequences in real time at 30 frames per second. The synthesis process was targeted to Altera Cyclone II and Stratix V FPGA devices. The synthesis results show that the designed architecture is capable to process 33 QFHD frames per second, considering the Stratix V implementation.