DRAM memory suffers from increasingly aggravating refresh penalty, which causes significant performance degradation and power consumption. As memory capacity increases, refresh penalty has become increasingly worse as more rows have to be refreshed. In this work, we propose an effective refresh approach called Compression-Aware Refresh (CAR) to efficiently mitigate refresh overheads. We apply a data compression technique to store data in a compressed format so that the resultant sparse banks need only be partially refreshed. Because of compression, data blocks which are originally distributed across all the constituent chips of a rank only need to be stored in a subset of those chips, leaving banks in the remaining chips not fully occupied. As a result, the memory controller can safely skip refreshing memory rows which contain no useful data without compromising data integrity. Such a compression-aware refresh scheme significantly reduces the refresh and thus improves overall memory performance and energy efficiency. To further take advantage of data compression, we adopt a rank subsetting technique to enable accesses to only those occupied chips for memory requests accessing compressed data blocks. Evaluations using benchmarks from SPEC CPU 2006 and the PARSEC 3.0 on recent DDR4 memory systems have shown that CAR achieves up to 1.66× performance improvement (11.7 percent on average) and up to 45.9 percent energy reduction (27.3 percent on average), and reduce memory traffic by up to 99.9 percent for zero cache lines intensive workloads with an average of 66.3 percent across all benchmarks.