With computing systems marching to exascale and big data era, power consumption has become more and more important for the system design. Energy efficiency is becoming one of the critical dimensions in the computer system design space and has been considered from the hardware architecture to software algorithm design. In this paper, we proposed an energy efficient multi-level tiling for dense matrix multiplication on many-core architecture with software-managed memory hierarchy. Based on our energy model, we integrated the two level tiling from off-chip memory to on-chip SRAM and from on-chip SRAM to register into one unified formulation and obtained the optimal tiling sizes of each level for energy efficiency. The experimental results showed that our optimal tiling improved energy cost by 17.9% compared to non-optimal solution and achieved 87.5% perk performance.