Gradient inversion attacks can reconstruct the victim's private data once they have access to the victim's model and gradient. However, existing research is still immature, and many attacks are conducted in ideal conditions. It is unclear how damaging such attacks really are and how they can be effectively defended. In this article, we first summarize the current relevant researches and their limitations. Then we design a general gradient inversion attack framework, which can attack both FedSGD and FedAVG. We propose approaches to enhance the label inference and image restoration, respectively. Our approach surpasses the SOTA attacks, by successfully attacking the batches from ImageNet while other methods fail to attack. Finally, we suggest several defense strategies without any utility loss from extensive experiments. We are confirmed that our work makes people aware of the privacy issues and can actively avoid the potential risks.