Identifying heavy flows is essential for network management. However, it is challenging to detect heavy flow quickly and accurately under the highly dynamic traffic and rapid growth of network capacity. Existing heavy flow detection schemes can make a trade-off in efficiency, accuracy and speed. However, these schemes still require memory large enough to obtain acceptable performance. To address this issue, we propose ChainSketch, which has the advantages of good memory efficiency, high accuracy and fast detection. Specifically, ChainSketch uses the selective replacement strategy to mitigate the over-estimation issue. Meanwhile, ChainSketch utilizes the hash chain and compact structure to improve memory efficiency. We implement the ChainSketch on OVS platform, P4-based testbed and large-scale simulations to process heavy hitter and heavy changer detection. The results of trace-driven tests show that, ChainSketch greatly improves the F1-score by up to $3.43\times$ compared with the state-of-the-art solutions especially for small memory.