Summary
This paper presents new results concerning the use of information theoretic inference techniques in system modeling and concerning the widespread applicability of certain simple queuing theory formulas. For the case when an M/G/1 queue provides a reasonable system model but when information about the service time probability density is limited to knowledge of a few moments, entropy maximization and cross-entropy minimization are used to derive information theoretic approximations for various performance distributions such as queue length, waiting time, residence time, busy period, etc. Some of these approximations are shown to reduce to exact M/M/1 results when G = M. For the case when a G/G/1 queue provides a reasonable system model, but when information about the arrival and service distributions is limited to the average arrival and service rates, it is shown that various well known M/M/1 formulas are information theoretic approximations. These results not only provide a new method for approximating the performance distributions, but they help to explain the widespread applicability of the M/M/1 formulas.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Muntz, R.R.: Analytic Modeling of Interactive Systems. Proc. IEEE 63, 946–953 (1975)
Chen, P.P.: Queueing Network Model of Interactive Computing Systems. Proc. IEEE 63, 954–957 (1975)
Kleinrock, L.: Queueing Systems, Vol. II: Computer Applications. New York: John Wiley 1976
Denning, P. J., Buzen, F.P.: The Operational Analysis of Queueing Network Models. Computing Surveys 10, 225–261 (1978)
Shore, J. E., Johnson, R.W.: Axiomatic Derivation of the Principle of Maximum Entropy and the Principle of Minimum Cross-Entropy. IEEE Trans. Information Theory IT-26, 26–37 (1980)
Ferdinand, A.E.: A Statistical Mechanics Approach to Systems Analysis. IBM J. Res. Dev. 539–547 (1970)
Beneš, V.E.: Mathematical Theory of Connecting Networks and Telephone Traffic. New York: Academic Press 1965
Shore, J.E.: Derivation of Equilibrium and Time-Dependent Solutions ot M/M/∞//N and M/M/∞ Queueing Systems Using Entropy Maximization. Proceedings 1978 National Computer Conference, AFIPS, 1978, pp. 483–487
Bard, Y.: A Model of Shared DASD and Multipathing. CACM 23, 564–572 (1980)
Jaynes, E.T.: Information Theory and Statistical Mechanics I. Phys. Rev. 106, 171–190 (1957)
Elsasser, W.M.: On Quantum Measurements and the Role of the Uncertainty Relations in Statistical Mechanics. Phys. Rev. 52, 987–999 (1937)
Kullback, S.: Information Theory and Statistics. New York: Wiley 1959
Csiszàr, I.: I-Divergence Geometry of Probability Distributions and Minimization Problems. Ann. Prob. 3, 146–58 (1975)
Shore, J.E., Johnson, R.W.: Properties of Cross-Entropy Minimization. IEEE Trans. Inf. Theory IT-27, 472–482 (1980)
Johnson, R.W.: Determining Probability Distributions by Maximum Entropy and Minimum Cross-Entropy. Proceedings APL79, (ACM 0-89791-005), May 1979, pp. 24–29
Good, I.J.: Maximum Entropy for Hypothesis Formulation, Especially for Multidimensional Contingency Tables. Annals Math. Stat. 34, 911–934 (1963)
Jaynes, E.T.: Prior Probabilities. IEEE Trans. on Systems Science and Cybernetics SSC-4, 227–241 (1968)
Hobson, A., Cheng, B.: A Comparison of the Shannon and Kullback Information Measures. J. Stat. Phys. 7, 301–310 (1973)
Johnson, R.W.: Axiomatic Characterization of the Directed Divergences and Their Linear Combinations. IEEE Trans. Information Theory IT-25, 709–716 (1979)
Good, I.J.: Probability and the Weighing of Evidence. London: Charles Griffen 1950
Pinsker, M. S.: Information and Information Stability of Random Variables and Processes. Holden-Day: San Francisco 1964 (1981)
Shore, J. E.: Minimum Cross-Entropy Spectral Analysis. IEEE Trans. Acoustics, Speech, & Signal Proc. ASSP-29, 230–237 (1981)
Gray, R.M., Gray, A.H., Jr., Rebolledo, G., Shore, J.E.: Rate-Distortion Speech Coding With a Minimum Discrimination Information Distortion Measure. IEEE Trans. Inf. Theory IT-27, November 1981
Shore, J.E., Gray, R.M.: Minimum Cross-Entropy Pattern Classification and Cluster Analysis. IEEE Trans. Pattern Anal. & Mach. Intell., January 1982
Hobson, A.: A New Theorem of Information Theory. J. Stat. Phys. 1, 383–391 (1969)
Geronimus, L.: Orthogonal Polynomials. New York: Consultants bureau 1961
Gray, R.M., Buzo, A., Gray, A.H., Jr., Matsuyama, Y.: Distortion Measures for Speech Processing. IEEE Trans. Acoustics, Speech, and Signal Processing ASSP-28, 367–376 (1980)
Markel, J.D., Gray, A.H., Jr.: Linear Prediction of Speech. New York: Springer 1976
Cohen, J.W.: The Single Server Queue. Amsterdam: North-Holland 1969
Cox, D.R., Smith, W.L.: Queues. London: Chapman and Hall 1961
Kleinrock, L.: Queueing Systems, Vol I: Theory. New York: John Wiley 1975
Shore, J.E.: Information Theoretic Approximations for M/G/1 and G/G/1 Queueing Systems. Naval Research Laboratory Memorandum Report 4047, July 1979
Shore, J.E.: Accuracy of an Information-Theoretic, Light-Load Approximation for the M/M/1 Busy Period Probability Density. Naval Research Laboratory Memorandum Report 4037, July 1979
Ephremides, A.E.: private communication
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Shore, J.E. Information theoretic approximations for M/G/1 and G/G/1 queuing systems. Acta Informatica 17, 43–61 (1982). https://doi.org/10.1007/BF00262975
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF00262975