In this paper, we propose a novel technique for film grain noise removal and synthesis that can be adopted in high fidelity video coding. Film grain noise enhances the natural appearance of high fidelity video, therefore, it should be preserved. However, film grain noise is a burden to typical video compression systems because it has relatively large energy levels in the high frequency region. In order to improve the coding performance while preserving film grain noise, we propose film grain noise removal in the pre-processing step and film grain noise synthesis in the post processing step. In the pre-processing step, the film grain noise is removed by using temporal and inter-color correlations. Specifically, color image denoisng using inter color prediction provides good denoising performance in the noise-concentrated B plane, because film grain noise has inter-color correlation in the RGB domain. In the post-processing step, we present a noise model to generate noise that is close to the actual noise in terms of a couple of observed statistical properties, such as the inter-color correlation and power of the film grain noise. The results show that the coding gain of the denoised video is higher than for previous works, while the visual quality of the final reconstructed video is well preserved.