In High-Level Synthesis (HLS), the design space of RTL implementations growing exponentially with the number of directives, coupled with the expensive nature of the syn-thesis process to obtain the characteristics of a single RTL implementation, pose challenges in exhaustively exploring the design space to identify Pareto-optimal designs. To find Pareto-optimal designs, most existing studies adopt regression methods or call HLS tools to provide the performance and cost of each design. Considering the essence of finding Pareto-optimal designs, we observed that predicting the relative dominance relation-ship between designs is sufficient while predicting the absolute performance and cost of each design is not a must. We propose an alternative approach to identify Pareto-optimal designs by establishing a Contrastive Learning (CL) framework that determines the dominance relationship between designs based on classification methods. Furthermore, we integrate the CL framework with three state-of-the-art Design Space Exploration (DSE) approaches, namely CL-DSE, with the goal of effectively reducing the number of syntheses. Experimental comparisons of 39 regression methods and 17 CL-assisted classification methods reveal that the classification methods generally overwhelm the regression methods for predicting the dominance relationship between designs. Experimental results also reveal that the CL-DSE approaches can significantly decrease the overall running time by reducing the number of syntheses, while retaining the quality of results in comparison to the DSE approaches. The proposed CL-DSE approaches are publicly available at https://github.com/hong64/CL-DSE.