Targeting to address one of the grandest challenges in HfO$_{2} -$based ferroelectric (FE) materials and devices, we invent a novel, effective, and versatile technique employing replacement electrode solid phase epitaxy (SPE), achieving a significant reduction in the grain size of the HZO layer by $\sim 30$% and enhanced remnant polarization $(P_{r})$ by 42% simultaneously, as compared with the conventional ALD growth technique. $P_{r} (\sim 25 \mu \mathrm{C} /cm^{2})$ among the highest reported at sub$- 450^{\circ}\mathrm{C}$ was realized. We also propose the underlying mechanism of the grain size-dependent ferroelectric properties, guided by thermodynamics-included first-principal simulation for nucleation process and kinetics effects-included analysis for phase change. It was discovered that grain size reduction plays a key role in decrease of the m-phase and the enhancement in grain boundary effects, leading to giant Pr improvement.