Skip to main content
SpringerLink
Log in

Strategy optimization for controlled Markov process with descriptive complexity constraint

  • Published:
Science in China Series F: Information Sciences Aims and scope Submit manuscript

Abstract

Due to various advantages in storage and implementation, simple strategies are usually preferred than complex strategies when the performances are close. Strategy optimization for controlled Markov process with descriptive complexity constraint provides a general framework for many such problems. In this paper, we first show by examples that the descriptive complexity and the performance of a strategy could be independent, and use the F-matrix in the No-Free-Lunch Theorem to show the risk that approximating complex strategies may lead to simple strategies that are unboundedly worse in cardinal performance than the original complex strategies. We then develop a method that handles the descriptive complexity constraint directly, which describes simple strategies exactly and only approximates complex strategies during the optimization. The ordinal performance difference between the resulting strategies of this selective approximation method and the global optimum is quantified. Numerical examples on an engine maintenance problem show how this method improves the solution quality. We hope this work sheds some insights to solving general strategy optimization for controlled Markov process with descriptive complexity constraint.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Puterman M L. Markov Decision Processes: Discrete Stochastic Dynamic Programming. New York: John Wiley and Sons, Inc., 1994

    MATH  Google Scholar 

  2. Bertsekas D P. Dynamic Programming and Optimal Control. Belmont, MA: Athena Scientific, 2007

    Google Scholar 

  3. Li M, Vitányi P. An Introduction to Kolmogorov Complexity and Its Applications. 2nd ed. New York: Springer-Verlag New York Inc., 1997

    MATH  Google Scholar 

  4. Ho Y C, Zhao Q C, Pepyne D L. The no free lunch theorems: complexity and security. IEEE Trans Automat Contr, 2003, 48(5): 783–793

    Article  MathSciNet  Google Scholar 

  5. Vapnik V N. Statistical Learning Theory. New York: John Wiley and Sons, Inc., 1998

    MATH  Google Scholar 

  6. Bertsekas D P, Tsitsiklis J N. Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996

    MATH  Google Scholar 

  7. Gunn S. Support Vector Machines for Classification and Regression. ISIS Technical Report. 1998

  8. Burges C J C. A tutorial on support vector machines for pattern recognition. Data Min Knowl Disc, 1998, 2(2): 955–975

    Article  Google Scholar 

  9. Ma J, Zhao Y, Ahalt S. Osu svm classifier matlab toolbox. Version 3.00. Available: http://www.ece.osu.edu/maj/osu_svm/

  10. Cho D I, Parlar M. A survey of maintenance models for multiunit systems. Eur J Oper Res, 1991, 51: 1–23

    Article  Google Scholar 

  11. Dekker R. Applications of maintenance optimization models: a review and analysis. Reliab Eng Syst Safe, 1996, 51: 229–240

    Article  Google Scholar 

  12. Tan J S, Kramer M A. A general framework for preventive maintenance optimization in chemical process operations. Comput Chem Eng, 1997, 21(12): 1451–1469

    Article  Google Scholar 

  13. Wang H. A survey of maintenance policies of deteriorating systems. Eur J Oper Res, 2002, 139: 469–489

    Article  MATH  Google Scholar 

  14. Xia L, Zhao Q C, Jia Q S. A structure property of optimal policies for maintenance problems with safety-critical components. IEEE Trans Automat Sci Eng, 2008, 5(3): 519–531

    Article  Google Scholar 

  15. Sun T, Zhao Q C, Luh P B, et al. Optimization of joint replacement policies for multi-part systems by a rollout framework. IEEE Trans Automat Sci Eng, 2008, 5(4): 609–619

    Article  Google Scholar 

  16. Dekker R, Wildeman R E, Van Der Duyn Schouten F A. A review of multi-component maintenance models with economic dependence. Math Method Oper Res, 1997, 45: 411–435

    Article  MATH  Google Scholar 

  17. Van Der Duyn Schouten F A, Vanneste S G. Two simple control policies for a multi-component maintenance system. Oper Res, 1993, 41: 1125–1136

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Intelligent and Networked Systems, Department of Automation, TNLIST, Tsinghua University, Beijing, 100084, China

    QingShan Jia & QianChuan Zhao

Authors
  1. QingShan Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. QianChuan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to QianChuan Zhao.

Additional information

Supported by the National Natural Science Foundation of China (Grant Nos. 60274011, 60574067, 60704008, 60736027, 60721003, 90924001), the New Century Excellent Talents in University (Grant No. NCET-04-0094), the Specialized Research Fund for the Doctoral Program of Higher Education (Grant No. 20070003110), and the Programme of Introducing Talents of Discipline to Universities (the National 111 International Collaboration Projects) (Grant No. B06002)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jia, Q., Zhao, Q. Strategy optimization for controlled Markov process with descriptive complexity constraint. Sci. China Ser. F-Inf. Sci. 52, 1993–2005 (2009). https://doi.org/10.1007/s11432-009-0192-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11432-009-0192-8

Keywords

Search

Navigation