Hierarchical Clustering as a Dimension Reduction Technique in the Markowitz Portfolio Optimization Problem

Abstract—

Optimal portfolio selection is a common and important solution to an optimization problem. It is often difficult to apply existing methods of optimal portfolio selection in practice, because of the large number of securities available for investment (and, as a consequence, the high dimensionality of the input data). In this article, we propose a method to reduce the dimensionality of the input data based on the hierarchical clustering of the securities available for investment. Many algorithms and methods have already been developed for clustering widely used in computer science. Pearson’s pairwise correlation coefficient is used as a measure of proximity of securities for hierarchical clustering. Next, we investigate the impact of the proposed method on the quality of the optimal solution obtained using several examples of the optimal portfolio selection according to the Markowitz model. The influence of hierarchical clustering parameters (intercluster distance metrics and clustering threshold value) on the change in the quality of the obtained optimal solution is also examined. The dependence between the target portfolio return and dimensionality reduction using the proposed method is analyzed. For each considered example, graphs and tables with the main obtained results of the application of the method are given: dimensionality reduction and return drop (reduction of the quality of the optimal solution) of the portfolio selected using the proposed method as compared to the portfolio selected without the proposed method. The Python programming language and its libraries are used for the experiments: SciPy for clustering and CVXPY for solving the optimization problem (optimal portfolio selection).