As the complexity of the multicore system grows, the large workload diversity results in serious thermal problems. In practical way, the number of placed thermal sensors is usually limited due to the manufacturing cost. In recent years, the compressive sensing (CS) theory is proven as an efficient way to reconstruct the original signals by using fewer sampling data. However, due to the high computational complexity during signal reconstruction, the compressive sensing theory is not appropriate to apply to the real-time temperature monitoring in the current multicore systems. In this paper, we first propose a grid-based sensor placement approach to allocate the number-limited thermal sensors on the multicore systems. On the other hand, we adopt an algorithmic transformation of Matrix Inversion Bypass (MIB) to reduce the computational complexity of the Orthogonal Matching Pursuit (OMP)-based signal reconstruction approach, which is a wide use signal reconstruction approach in the compressive sensing theory. The experimental results show that the proposed approach can reduce 52% average full-system temperature reconstruction error compared with the previous non-CS-based approaches. Besides, we can also reduce 22%-27% execution time compared with the current CS-based temperature reconstruction algorithm.