Skip to main content
Log in

Fast algorithms for sparse portfolio selection considering industries and investment styles

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

In this paper, we consider a large scale portfolio selection problem with and without a sparsity constraint. Neutral constraints on industries are included as well as investment styles. To develop fast algorithms for the use in the real financial market, we shall expose the special structure of the problem, whose Hessian is the summation of a diagonal matrix and a low rank modification. Specifically, an interior point algorithm taking use of the Sherman–Morrison–Woodbury formula is designed to solve the problem without any sparsity constraint. The complexity in each iteration of the proposed algorithm is shown to be linear with the problem dimension. In the occurrence of a sparsity constraint, we propose an efficient three-block alternating direction method of multipliers, whose subproblems are easy to solve. Extensive numerical experiments are conducted, which demonstrate the efficiency of the proposed algorithms compared with some state-of-the-art solvers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Beller, K.R., Kling, J.L., Levinson, M.J.: Are industry stock returns predictable? Financ. Anal. J. 54(5), 42–57 (1998)

    Article  Google Scholar 

  2. Bertsimas, D., Shioda, R.: Algorithm for cardinality-constrained quadratic optimization. Comput. Optim. Appl. 43(1), 1–22 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bertsimas, D., Cory-Wright, R.: A scalable algorithm for sparse portfolio selection. arXiv:1811.00138v3 [math.OC] (2019)

  4. Cai, X.J., Han, D.R., Yuan, X.M.: On the convergence of the direct extension of ADMM for three-block separable convex minimization models with one strongly convex function. Comput. Optim. Appl. 66(1), 39–73 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  5. Chang, T.J., Meade, N., Beasley, J.E., Sharaiha, Y.M.: Heuristics for cardinality constrained portfolio optimisation. Comput. Oper. Res. 27(13), 1271–1302 (2000)

    Article  MATH  Google Scholar 

  6. Chen, C.H., He, B.S., Ye, Y.Y., Yuan, X.M.: The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent. Math. Program. 155(1–2), 57–79 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  7. Chen, X., Peng, J.M., Zhang, S.Z.: Sparse solutions to random standard quadratic optimization problems. Math. Program. 141(1–2), 273–293 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  8. Cui, X.T., Zheng, X.J., Zhu, S.S., Sun, X.L.: Convex relaxations and MIQCQP reformulations for a class of cardinality-constrained portfolio selection problems. J. Glob. Optim. 56(4), 1409–1423 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  9. Duffie, D., Jorion, P.: An overview of value at risk. J. Deriv. 4, 7–49 (1997)

    Article  Google Scholar 

  10. Facchinei, F., Pang, J.S.: Finite-Dimensional Variational Inequalities and Complementarity Problems. vol. II. Springer. 1, 625-1234 (2003)

  11. Giansante, J.E., Dragun, B.C.: U.S. Patent No. 6,275,814. Washington, DC: U.S. Patent and Trademark Office (2001)

  12. Gondzio, J., Sobral, F.N.C.: Quasi-Newton approaches to interior point methods for quadratic problems. Comput. Optim. Appl. 74, 93–120 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  13. Grauer, R.R., Hakansson, N.H., Shen, F.C.: Industry rotation in the US stock market: 1934–1986 returns on passive, semi-passive, and active strategies. J. Bank. Finance 14(2–3), 513–538 (1990)

    Article  Google Scholar 

  14. Han, D.R., Yuan, X.M.: Local linear convergence of the alternating direction method of multipliers for quadratic programs. SIAM J. Numer. Anal. 51(6), 3446–3457 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  15. He, B.S., Yuan, X.M.: On non-ergodic convergence rate of Douglas–Rachford alternating direction method of multipliers. Numer. Math. 130(3), 567–577 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  16. Hong, M.Y., Luo, Z.Q.: On the linear convergence of the alternating direction method of multipliers. Math. Program. 162(1), 165–199 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  17. Jorion, P.: Value at Risk: The New Benchmark for Controlling Market Risk. McGraw-Hill, Chicago (1997)

    Google Scholar 

  18. Konno, H., Yamazaki, H.: Mean-absolute deviation portfolio optimization model and its applications to Tokyo stock market. Manag. Sci. 37, 519–531 (1991)

    Article  Google Scholar 

  19. Kawadai, N., Konno, H.: Solving large scale mean-variance models with dense non-factorable covariance matrices. J. Oper. Res. Soc. Jpn. 44(3), 251–260 (2001)

    MathSciNet  MATH  Google Scholar 

  20. Li, D., Sun, X.L., Wang, J.: Optimal lot solution to cardinality constrained mean-variance formulation for portfolio selection. Math. Financ. 16(1), 83–101 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  21. Li, H.M., Zhang, K.C.: A decomposition algorithm for solving large-scale quadratic programming problems. Appl. Math. Comput. 173(1), 394–403 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  22. Lu, Z.S., Pong, T.K.: Computing optimal experimental designs via interior point method. SIAM J. Matrix. Anal. A 34(4), 1556–1580 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  23. Markowitz, H.M.: Portfolio selection. J. Finance. 7(1), 77–91 (1952)

    Google Scholar 

  24. Markowitz, H.M.: Portfolio Selection: Efficient Diversification of Investments. Wiley, New York (1959)

    Google Scholar 

  25. Max A.W.: Inverting modified matrices, Memorandum Rept. 42, Statistical Research Group, Princeton University, Princeton, NJ (1950)

  26. Perold, A.F.: Large-scale portfolio optimization. Manag. Sci. 30(10), 1143–1160 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  27. Rockafellar, R.T., Uryasev, S.: Optimization of conditional value-at-risk. J. Risk 2, 21–42 (2000)

    Article  Google Scholar 

  28. Rockafellar, R.T., Uryasev, S.: Conditional value-at-risk for general loss distributions. J. Bank. Finance 26, 1443–1471 (2002)

    Article  Google Scholar 

  29. Rosenberg, B.: Extra-market components of covariance in security returns. J. Financ. Quant. Anal. 9(2), 263–274 (1974)

    Article  Google Scholar 

  30. Sharpe, W.F.: Portfolio theory and capital markets, vol. 217. McGraw-Hill, New York (1970)

    Google Scholar 

  31. Shaw, D., Liu, S., Kopman, L.: Lagrangian relaxation procedure for cardinality-constrained portfolio optimization. Optim. Methods Softw. 23, 411–420 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  32. Sherman, J., Morrison, W.J.: Adjustment of an inverse matrix corresponding to a change in one element of a given matrix. Ann. Math. Stat. 21(1), 124–127 (1950)

    Article  MathSciNet  MATH  Google Scholar 

  33. Sorensen, E.H., Burke, T.: Portfolio returns from active industry group rotation. Financ. Anal. J. 42(5), 43–50 (1986)

    Article  Google Scholar 

  34. Streichert, F., Ulmer, H., Zell, A.: Evolutionary algorithms and the cardinality constrained portfolio optimization problem. In: Operations Research Proceedings 2003, Springer, pp. 253–260 (2004)

  35. Takehara, H.: An interior point algorithm for large scale portfolio optimization. Ann. Oper. Res. 45(1), 373–386 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  36. Vanderbei, R.J., and Bing Y.: On the Symmetric Formulation of Interior-Point Methods. Technical report SOR 94-05, Princeton University, Princeton, NJ (1994)

  37. Wright, S., Ralph, D.: A superlinear infeasible-interior-point algorithm for monotone complementarity problems. Math. Oper. Res. 21(4), 815–838 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  38. Xu, F.M., Lu, Z.S., Xu, Z.B.: An efficient optimization approach for a cardinality-constrained index tracking problem. Optim. Methods Softw. 31(2), 258–271 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  39. Yang, L., Pong, T.K., Chen, X.J.: Alternating direction method of multipliers for a class of nonconvex and nonsmooth problems with applications to background/foreground extraction. SIAM J. Imaging Sci. 10(1), 74–110 (2017)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We would like to thank Professor Zhaosong Lv and Professor Tingkei Pong for helpful discussion on this paper. We also thank the two anonymous referees for their valuable remarks and comments, which helped improve the quality of this article.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Fengmin Xu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The first author was supported by the National Natural Science Foundation of China (Nos. 11801433, 71501155). The second author was supported by the National Natural Science Foundation of China (Nos. 11571271, 11971372). The third author was supported by the National Natural Science Foundation of China (Nos. 11631013, 11991021, 11991020, 11971372) and Beijing Academy of Artificial Intelligence (BAAI).

Appendices

Appendix

Details of the factors used in the paper

The 29 industry factors used in our paper are listed in Table 8.

Table 8 List of the 29 industry factors used from CITIC

The details of the 9 style factors are listed as follows:

  • The Beta factor, which is typically the most important style factor. We compute Beta by time-series regression of excess stock returns against the cap-weighted estimation universe \(R_t\).

    $$\begin{aligned} r_t-r_{ft} = \alpha +\beta R_t +e_t, \end{aligned}$$
    (52)

    where \(r_t\) is the stock return on day t, \(r_{ft}\) is the risk-free return, \(e_t\) is the residual return. The regression coefficients are estimated over the trailing 250 trading days of returns with a half-life of 60 trading days.

  • The Momentum factor differentiates stocks based on their performance over the trailing 6-12 months. When computing Momentum exposures we exclude the last month (21 days) of returns in order to avoid the effects of short-term reversal. It is computed as the sum of excess log returns over the trailing \(T=500\) trading days with a lag of \(L=20\) trading days,

    $$\begin{aligned} RSTR = \sum _{t=L}^{T+L}\omega _t[ln(1+r_t)-ln(1+r_{ft}) ], \end{aligned}$$
    (53)

    where \(\omega _t\) is an exponential weight with a half-life of 120 trading days.

  • The Size factor represents the strongest source of equity return covariance, and captures return differences between large-cap stocks and small-cap stocks. We measure this factor by the log of market capitalization.

    $$\begin{aligned} LNCAP = ln(total\_market\_capitalization). \end{aligned}$$
    (54)
  • The Earnings Yield factor describes return differences based on a company’s earnings relative to its price. Earnings Yield is considered by many investors to be a strong value signal. The most important descriptor in this factor is the analyst-predicted 12-month forward earnings-to-price ratio. The Earnings Yield factor is defined as the convex combination of three small factors EPFWD (predicted earnings-to-price ratio), CETOP (cash earnings-to-price ratio) and ETOP (trailing earnings-to-price ratio). EPFWD is given by the 12-month forward-looking earnings divided by the current market capitalization. Forward-looking earnings are defined as a weighted average between the average analyst-predicted earnings for the current and next fiscal years. CETOP is given by the trailing 12-month cash earnings divided by current price. ETOP is given by the trailing 12-month earnings divided by the current market capitalization. Trailing earnings are defined as the last reported fiscal-year earnings plus the difference between current interim figure and the comparative interim figure from the previous year.

  • The Residual Volatility factor is composed of descriptors that tend to be highly collinear with the Beta factor. The Residual Volatility factor is orthogonalized with respect to the Beta factor as well as the Size factor. This factor is defined as the combination of three small factors DASTD (daily standard deviation), CMRA (cumulative range) and HSIGMA (historical sigma \(\sigma \)). DASTD is computed as the volatility of daily excess returns over the past 250 trading days with a half-life of 40 trading days. The cumulative range is given by

    $$\begin{aligned} CMRA = ln(1+\max _{T=1,2,...,12}\{Z(T)\})-ln(1+\min _{T=1,2,...,12}\{Z(T)\}). \end{aligned}$$
    (55)

    Here,

    $$\begin{aligned} Z(T) = \sum _{\tau =1}^T [ln(1+r_{\tau })-ln(1+r_{f\tau })], \end{aligned}$$
    (56)

    where \(r_{\tau }\) is the stock return for month \(\tau \) (compounded over 20 days), and \(r_{f\tau }\) is the risk free return. HISGMA is computed as the volatility of the residual returns

    $$\begin{aligned} \sigma = std(e_t). \end{aligned}$$
    (57)
  • The Growth factor differentiates stocks based on their prospects for sales or earnings growth. This factor contains forward-looking descriptors in the form of long/short-term analyst predicted earnings growth as well as historical descriptors for sales and earnings growth over the trailing five years. This factor is defined as the convex combination of four small factors, EGRLF (long-term predicted earnings growth), EGRSF (short-term predicted earnings growth), EGRO (earnings growth trailing five years) and SGRO (sales growth trailing five years). EGRLF is computed by the long-term (3-5 years) earnings growth forecasted by analysts. EGRSF is the short-term (1 year) earnings growth forecasted by analysts. If annual reported earnings per share are regressed against time over the past five fiscal years, then the slope coefficient is divided by the average annual earnings per share to obtain the EGRO. If annual reported sales per share are regressed against time over the past five fiscal years, then the slope coefficient is then divided by the average annual sales per share to obtain the SGRO.

  • The Book-to-Price factor (BTOP) is also considered by some to be an indicator of value. This factor is given by the last reported book value of common equity divided by current market capitalization, i.e.

    $$\begin{aligned} BTOP= common\_equity/ current\_market\_capitalization. \end{aligned}$$
    (58)
  • The Leverage factor captures return differences between high-leverage and low-leverage stocks. This factor is defined as the combination of three small factors, MLEV (market leverage), BLEV (book leverage) and DTOA (debt-to-assets ratio). MLEV is computed as

    $$\begin{aligned} MLEV= \frac{ME+PE+LD}{ME}, \end{aligned}$$
    (59)

    where ME is the market value of common equity on the last trading day, PE is the most recent book value of preferred equity, and LD is the most recent book value of long-term debt. DTOA is computed as

    $$\begin{aligned} DTOA = \frac{TD}{TA}, \end{aligned}$$
    (60)

    where TD is the book value of total debt (long-term debt and current liabilities), and TA is most recent book value of total assets. BLEV is computed as

    $$\begin{aligned} BLEV = \frac{BE+PE+LD}{BE}, \end{aligned}$$
    (61)

    where BE is the most recent book value of common equity, PE and LD are the same as above.

  • The Liquidity factor describes return differences due to relative trading activity. The descriptors for this factor are based on the fraction of total shares outstanding that trade over a recent window. The Liquidity factor is orthogonalized with respect to the Size factor. This factor is defined as the combination of three small factors, STOM (share turnover, one month), STOQ (average share turnover, trailing 3 months) and STOA (average share turnover, trailing 12 months). STOM is computed as the log of the sum of daily turnover during the previous 20 trading days,

    $$\begin{aligned} STOM = ln\left( \sum _{t=1}^{20} \frac{V_t}{S_t} \right) , \end{aligned}$$
    (62)

    where \(V_t\) is the trading volume on day t, and \(S_t\) is the number of shares outstanding. Let \(STOM_{\tau }\) be the share turnover for month \(\tau \), with each month consisting of 20 trading days. The quarterly share turnover is defined by

    $$\begin{aligned} STOQ = ln\left[ \frac{1}{T}\sum _{\tau =1}^T exp(STOM_\tau )\right] , \end{aligned}$$
    (63)

    where \(T=3\) months. The annual share turnover is defined by

    $$\begin{aligned} STOA = ln\left[ \frac{1}{T}\sum _{\tau =1}^T exp(STOM_\tau )\right] , \end{aligned}$$
    (64)

    where \(T=12\) months.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dong, ZL., Xu, F. & Dai, YH. Fast algorithms for sparse portfolio selection considering industries and investment styles. J Glob Optim 78, 763–789 (2020). https://doi.org/10.1007/s10898-020-00911-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-020-00911-1

Keywords