Skip to main content
SpringerLink
Log in

Ergodic and adaptive control of nearest-neighbor motions

  • Published:
Mathematics of Control, Signals and Systems Aims and scope Submit manuscript

Abstract

The self-tuning approach to adaptive control is applied to a class of Markov chains called nearest-neighbor motions. These have a countable state space and move from any state to at most finitely many neighboring states. For compact parameter and control spaces, the almost-sure optimality of the self-tuner for an ergodic cost criterion is established under two sets of assumptions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. V. E. Beneš, Existence of optimal strategies based on specified information for a class of stochastic decision problems,SIAM J. Control Optim.,8 (1970), 179–188.

    Article  Google Scholar 

  2. V. S. Borkar, Controlled Markov chains and stochastic networks,SIAM J. Control Optim.,21 (1983), 652–666.

    Article  MathSciNet  Google Scholar 

  3. V. S. Borkar, On minimum cost per unit time control of Markov chains,SIAM J. Control Optim.,22 (1984), 965–978.

    Article  MathSciNet  Google Scholar 

  4. V. S. Borkar, Control of Markov chains with long-run average cost criterion, inProceedings of the I.M.A. Workshop on Stochastic Differential Systems with Application to Electrical/Computer Engineering, Control Theory and Operations Research (W. Fleming and P. L. Lions, eds.), pp. 57–77, Springer-Verlag, New York, 1987.

    Google Scholar 

  5. V. S. Borkar, A convex analytic approach to Markov decision processes,Probab. Theory Related Fields,78 (1988), 583–602.

    Article  MathSciNet  Google Scholar 

  6. V. S. Borkar, Control of Markov chains with long-run average cost criterion: the dynamic programming equations,SIAM J. Control Optim.,27 (1989), 642–657.

    Article  MathSciNet  Google Scholar 

  7. V. S. Borkar, The Kumar-Becker-Lin scheme revisited,J. Optim. Theory Appl. (to appear in August 1990).

  8. V. S. Borkar, and P. Varaiya, Adaptive control of Markov chain, I: Finite parameter set,IEEE Trans. Automat. Control,24 (1979), 953–957.

    Article  MathSciNet  Google Scholar 

  9. V. S. Borkar, and P. Varaiya, Identification and adaptive control of Markov chains,SIAM J. Control Optim.,20 (1982), 470–489.

    Article  MathSciNet  Google Scholar 

  10. B. Hajek, Hitting-time and occupation-time bounds implied by drift analysis with applications,Adv. in Appl. Probab.,14 (1982), 502–525.

    Article  MathSciNet  Google Scholar 

  11. O. Hernandez-Lerma and S. I. Marcus, Adaptive control of service in queueing systems,Systems Control Lett.,3 (1983), 283–289.

    Article  MathSciNet  Google Scholar 

  12. O. Hernandez-Lerma and S. I. Marcus, Optimal adaptive control of priority assignment in queueing systems,Systems Control Lett.,4, (1984), 65–72.

    Article  MathSciNet  Google Scholar 

  13. P. R. Kumar, Survey of results in stochastic adaptive control,SIAM J. Control Optim.,23 (1985), 329–380.

    Article  MathSciNet  Google Scholar 

  14. P. R. Kumar and W. Lin, Optimal adaptive controllers for unknown Markov chains,IEEE Trans. Automat. Control,27 (1982), 765–774.

    Article  MathSciNet  Google Scholar 

  15. P. Mandl, Estimation and control in Markov chains,Adv. in Appl. Probab.,6 (1974), 40–60.

    Article  MathSciNet  Google Scholar 

  16. B. Sagalovsky, Adaptive control and parameter estimation in Markov chains: a linear case,IEEE Trans. Automat. Control,27 (1982), 137–146.

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Indian Institute of Science, 560 012, Bangalore, India

    Vivek S. Borkar

  2. Tata Institute of Fundamental Research, Bangalore Centre, P.O. Box 1234, 560 012, Bangalore, India

    Mrinal K. Ghosh

Authors
  1. Vivek S. Borkar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mrinal K. Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Borkar, V.S., Ghosh, M.K. Ergodic and adaptive control of nearest-neighbor motions. Math. Control Signal Systems 4, 81–98 (1991). https://doi.org/10.1007/BF02551382

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02551382

Key words

Search

Navigation