Skip to main content
SpringerLink
Log in

Convergence of stochastic approximation coupled with perturbation analysis in a class of manufacturing flow control models

  • Published:
Discrete Event Dynamic Systems Aims and scope Submit manuscript

Abstract

This paper deals with a class of piecewise determinstic control systems for which the optimal control can be approximated through the use of an optimization-by-simulation approach. The feedback control law is restricted to belong to an a priori fixed class of feedback control laws depending on a (small) finite set of parameters. Under some general conditions developed in this paper, infinitesimal perturbation analysis (IPA) can be used to estimate the gradient of the objective function with respect to these parameters for finite horizon simulation and the consistency of the IPA estimators, as the simulation length goes to infinity, is assured. Also, the parameters can be optimized through a stochastic approximation (SA) algorithm combined with IPA. We prove that in this context, under appropriate conditions, such an approach converges towards the optimum.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Akella, R.A., and Kumar, P.R. 1986. Optimal control of production rate in failure prone manufacturing systems.IEEE Trans. Autom. Control AC-31, pp. 116–126.

    Google Scholar 

  • Algoet, P.H. 1989. Flow balance equations for the steady-state distribution of a flexible manufacturing system.IEEE Trans. Automat. Control AC-34, 917–921.

    Google Scholar 

  • Bernard, J., Haurie, A., and Missaoui, R. 1981. Modele d'optimisation des courbes d'alerte pour la gestion de réservoirs en vue de l'irrigation et de la production d'électricité.INFOR 4, pp. 347–365.

    Google Scholar 

  • Bielecki, T., and Kumar, P.R. 1988. Optimality of zero-inventory policies for unreliable manufacturing systems.Oper. Res. 36, pp. 532–541.

    Google Scholar 

  • Boukas, E.K., and Haurie, A. 1988. Optimality conditions for continuous-time systems with controlled jump Markov disturbances: application to an FMS planning problem. InAnalysis and Optimization of Systems (A. Bensoussan and J.L. Lions, eds.), Springer-Verlag, pp. 633–676.

  • Boukas, E.K., and Haurie, A. 1990. Planning production and preventive maintenance in flexible manufacturing systems: a stochastic control approach.IEEE Trans. Automat. Control AC-35, pp. 1024–1031.

    Google Scholar 

  • Brock, W.A., and Haurie, A. 1976. On existence of overtaking optimal trajectories over an infinite time horizon.Math. Oper. Res. 1, pp. 337–346.

    Google Scholar 

  • Cao, X.R. 1987. Sensitivity estimates based on one realization of s stochastic system.J. Statist. Comput. Simul. pp. 211–232.

  • Caramanis, M., and Liberopoulos, G. 1992. Perturbation analysis for the design of flexible manufacturing system flow controllers.Oper. Res. 40, pp. 1107–1125.

    Google Scholar 

  • Caramanis, M., and Sharifnia, A. 1989. Design of near optimal flow controllers for flexible manufacturing systems.Int. J. Prod. Res. pp. 1–32.

  • Chong, E.K.P., and Ramadge, P.J. 1992a. Convergence of recursive optimization algorithms using infinitesimal perturbation analysis estimates.Discrete Event Dynamic Systems 1, pp. 339–372.

    Google Scholar 

  • Chong, E.K.P., and Ramadge, P.J. 1992b. Stochastic optimization of regenerative systems using infinitesimal perturbation analysis (submitted).

  • Chong, E.K.P., and Ramadge, P.J. 1993. Optimization of queues using an IPA based stochastic algorithm with general update times.SIAM J. Control Optim. (to appear).

  • Davis, M.H.A. 1984. Piecewise-deterministic Markov processes: a general class of non-diffusion stochastic models.J. R. Statist. Soc. 46, pp. 353–388.

    Google Scholar 

  • Dempster, M. 1989. Optimal control of piecewise deterministic Markov processes.Proc. Imperial College Workshop Appl. Stochastic Anal.

  • Dieudonné, J. 1969.Foundations of Modern Analysis, 2nd ed. New York: Academic Press.

    Google Scholar 

  • Fu, M.C. 1990. Convergence of a stochastic approximation algorithm for the GI/G/1 queue using infinitestimal perturbation analysis.J. Optim. Theory Appl. 65, pp. 149–160.

    Google Scholar 

  • Gershwin, S.B., Akella, R., and Choong, Y. 1985. Short term scheduling of an automated manufacturing facility.IBM J. Res. Dev. 29, pp. 393–400.

    Google Scholar 

  • Glasserman, P. 1991.Gradient Estimation via Perturbation Analysis. Boston: Kluwer.

    Google Scholar 

  • Glynn, P.W. 1986. Stochastic approximation for Monte-Carlo optimization.Proc. Winter Simulation Conf. 1986, IEEE Press, pp. 356-2-364.

  • Gut, A. 1974. On the moments of some first passage times for sums of dependent random variables.Stoch. Process. Appl. 2, pp. 115–126.

    Google Scholar 

  • Gut, A. 1988.Stopped Random Walks. New York: Springer-Verlag.

    Google Scholar 

  • Haurie, A., Hoang, H., and Polis, M. 1975. Reducing energy consumption through trajectory optimization for a metro network.IEEE Trans. Automat. Control 20, pp. 590–595.

    Google Scholar 

  • Haurie, A., and van Delft, C. 1990. Turnpike improvement via simulation in manufacturing flow control models.Proc. 1990 Japan-USA Symp. Flexible Automation, 3, ISCIE, Japan, pp. 1171–1178.

    Google Scholar 

  • Haurie, A., and van Delft, C. 1992. Turnpike properties for a class of piecewise deterministic systems arising in manufacturing flow control.Ann. Oper. Res. 2, pp. 1–27.

    Google Scholar 

  • Ho, Y.-C. 1987. Performance evaluation and perturbation analysis of discrete event dynamic systems.IEEE Trans. Automat. Control AC-32, 563–572.

    Google Scholar 

  • Ho, Y.-C., and Cao, X.-R. 1991.Discrete-Event Dynamic Systems and Perturbation Analysis. Boston: Kluwer.

    Google Scholar 

  • Kimemia, J., and Gershwin, S.B. 1983. An algorithm for computer control of a flexible manufacturing system.IIE Trans. 15, pp. 353–362.

    Google Scholar 

  • Kushner, H.J., and Clark, D.S. 1978.Stochastic Approximation Methods for Constrained and Unconstrained Systems. New York: Springer-Verlag, Applied Math. Sciences, vol. 26.

    Google Scholar 

  • L'Ecuyer, P. 1990. A unified view of the IPA, SF and LR gradient estimation techniques.Management Sci. 36, 1364–1383.

    Google Scholar 

  • L'Ecuyer, P. 1991. An overview of derivative estimation.Proc. 1991 Winter Simulation Conf., IEEE Press, 207–217.

  • L'Ecuyer, P., Giroux, N., and Glynn, P.W. 1992. Stochastic optimization by simulation: numerical experiments with a simple queue in steady-state (submitted).

  • L'Ecuyer, P., and Glynn, P.W. 1992. Stochastic optimization by simulation: convergence proofs for the GI/G/1 queue in steady-state. (submitted)

  • Liberopoulos, G., and Caramanis, M. 1992. Strong consistency of sample path derivatives in the design of manufacturing flow controllers.IEEE Trans. Automat. Control (to appear).

  • Maimon, O.Z., and Gershwin, S.B. 1988. Dynamic scheduling and routing for flexible manufacturing systems that have unreliable machines.Oper. Res. 36, pp. 279–292.

    Google Scholar 

  • Malhamé, R., and Boukas, E.K. 1991. A Markov renewal analysis of the production control of a flexible manufacturing system under hedging point policies.IEEE Trans. Automat. Control, 36, 580–587.

    Google Scholar 

  • Rishel, R. 1975. Dynamic programming and minimum principles for systems with jump Markov disturbances.SIAM J. Control 13, pp. 338–371.

    Google Scholar 

  • Ross, S.M. 1970.Applied Probability Models with Optimization Applications. San Francisco: Holden-Day.

    Google Scholar 

  • Sharifnia, A. 1988. Production control of a manufacturing system with multiple machine states.IEEE Trans. Automat. Control AC-33, pp. 620–625.

    Google Scholar 

  • Tsitsiklis, J. 1984. Convexity and characterization of optimal policies in a dynamic routing problem.J. Optim. Theory Appl. 44, pp. 165–208.

    Google Scholar 

  • Vermes, D., 1985. Optimal control of piecewise deterministic Markov process.Stochastics 14, pp. 165–208.

    Google Scholar 

  • Wolff, R. 1989.Stochastic Modeling and the Theory of Queues. Englewood Cliffs, NJ: Prentice-Hall.

    Google Scholar 

  • Yan, H., Yin, G., and Lou, S.X.C. 1992. Approximating optimal threshold values for unreliable manufacturing systems via stochastic optimization.Proc. 31st CDC Conf., Tucson.

Download references

Author information

Authors and Affiliations

  1. Départment d'économie commerciale et industrielle, Faculté des sciences économiques et sociales, Université de Genève, Suisse

    A. Haurie

  2. GERAD, École des Hautes Études Commerciales, Montréal, Québec, Canada

    A. Haurie

  3. GERAD and Départment d'informatique et de recherche opérationnelle, Université de Montréal, succ. A, C.P. 6128, H3C 3J7, Montréal, Canada

    P. L'ecuyer

  4. Départment de management industriel et logistique, Groupe HEC, 78531, Jouy-en-Josas, France

    Ch. Van Delft

Authors
  1. A. Haurie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P. L'ecuyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ch. Van Delft
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Haurie, A., L'ecuyer, P. & Van Delft, C. Convergence of stochastic approximation coupled with perturbation analysis in a class of manufacturing flow control models. Discrete Event Dyn Syst 4, 87–111 (1994). https://doi.org/10.1007/BF01516011

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01516011

Key Words

Search

Navigation