Multi-view subspace clustering aims to utilize the comprehensive information of multi-source features to aggregate data into multiple subspaces. Recently, low-rank tensor learning has been applied to multi-view subspace clustering, which explores high-order correlations of multi-view data and has achieved remarkable results. However, these existing methods have certain limitations: 1) The learning processes of low-rank tensor and label indicator matrix are independent. 2) Variable contributions of different views to the consistent clustering results are not discriminated. To handle these issues, we propose a unified framework that integrates low-rank tensor learning and spectral embedding (ULTLSE) for multi-view subspace clustering. Specifically, the proposed model adopts the tensor singular value decomposition (t-SVD) based tensor nuclear norm to encode the low-rank property of the self-representation tensor, and a label indicator matrix via spectral embedding is simultaneously exploited. To distinguish the importance of various views, we learn a quantifiable weighting coefficient for each view. An effective recursion optimization algorithm is also developed to address the proposed model. Finally, we conduct comprehensive experiments on eight real-world datasets with three categories. The experimental results indicate that the proposed ULTLSE is advanced over existing state-of-the-art clustering methods.