Abstract
Information signal from real case and natural complex dynamical systems such as traffic flow are usually specified by irregular motions. Chaotic nonlinear dynamics approach is now the most powerful tool for scientists to deal with complexities in real cases, and neural networks and neuro-fuzzy models are widely used for their capabilities in nonlinear modeling of chaotic systems more than the traditional methods. As mentioned, the traffic flow conditions caused the forecasting values of traffic flow to lack robustness and accuracy. In this paper, the traffic flow forecasting is analyzed with emotional concepts and multi-agent systems (MASs) points of view as a new method in this field. The findings enabled the researchers to develop a newly object-oriented method of forecasting traffic flow. Its architecture is based on a temporal difference (TD) Q-learning with a neuro-fuzzy structure, which is the nonparametric approach. The performance of TD Q-learning is improved by emotional learning. The proposed method on the present conditions and the action of the system according to the criteria could forecast traffic signals so that the objectives are reached in minimum time. The ability of presented learning algorithm to prospect gains from future actions and obtain rewards from its past experiences allows emotional TD Q-learning algorithm to improve its decisions for the best possible actions. In addition, to study in a more practical situation, the neuro-fuzzy behaviors could be modeled by MAS. The proposed method (intelligent/nonparametric approach) is compared by parametric approach, autoregressive integrated moving average (ARIMA) method, which is implemented by multi-layer perceptron neural networks and called ARIMANN. Here, the ARIMANN is updated by backpropagation and temporal difference backpropagation for the first time. The simulation results revealed that the studied forecaster could discover the optimal forecasting by means of the Q-learning algorithm. Difficult to handle through parametric and classic methods, the real traffic flow signals used for fitting the algorithms is obtained from a two-lane street I-494 in Minnesota City.
Similar content being viewed by others
Notes
e i is an error signal between the forecasted signal and desired value; m is the number of each section (in this paper m = 3); c i is the critic signal; r i is the actual value of reward signal for every section; \( \hat{r}_{i} \) is the forecasted value of reward signal for every section, and \( \hat{r} \) is the forecasted value of reward signal for destination section. As it is shown, the emotional multi-critics are composed of the following functional elements:
Critics: providing the emotional signals according to the error situation (This error may increase in peaks which are caused by such a non-recurring traffic incidents).
Learning procedure: modeling the reinforcement signals that are eventually be obtained from the system for the current action determined by neuro-fuzzy forecaster.
Fuzzy network: making decision for weights, \( w_{1} ,w_{2} , \ldots ,w_{m} \), for the m internal reinforcement signals according to m station status signals, \( e_{1} ,e_{2} , \ldots ,e_{m} \) in every iteration.
Normalization unit: putting together the internal reinforcement signals decided by the TDs learning and assigning an appropriate reinforcement signal at each time-step. Consequently, the normalization unit can be combined with internal reinforcement signals at once.
References
Abdi J, Fatourechi M, Lucas C, Sedigh AK, Famil Khalili GH (2004) Control of multivariable systems based on emotional temporal difference learning controller. Int J Eng Natl Centre Sci Res 17(4):363–376
Abdi J, Rashidi F, Lucas C, Sedigh AK (2005) The role of context in emotional temporal difference learning and its applications in control. J Sci Technol Sharif Univ Technol 30:13–21
Abdulhai B, Porwal H, Recker W (2002) Short term traffic-flow forecasting using neuro-genetic algorithms. J Intell Transp Syst Technol Plann Oper 7(1):3–41
Ahmed MS, Cook AR (1979) Analysis of freeway traffic time-series data by using Box-Jenkins techniques. Transp Res Rec 722:1–9
Al-Saba T, El-Amin I (1999) Artificial neural networks as applied to long-term demand forecasting. Artif Intell Eng 13:189–197
Balkenius C, Moren J (2000) A computational model of context processing. In: Meyer JA, Berthoz A, Floreano D, Roitblat HL, Wilson SW (eds) From animals to animats 6: proceedings of the 6th international conference on the simulation of adaptive behaviour. MIT Press, Cambridge
Berenji HR, Khedkar P (1992) Learning and tuning fuzzy logic controller through reinforcements. IEEE Trans Neural Netw 3:724–740
Box GEP, Jenkins GM (1976) Time series analysis: forecasting and control, 2nd edn. Holden Day, San Francisco
Cao L, Hong Y, Fang H, He G (1995) Predicting chaotic time series with wavelet networks. Physica D: Nonlinear Phenom 85(1–2):225–238
Cetin M, Comert G (2006) Short-term traffic flow prediction with regime switching models. Transp Res Rec 1965:23–31
Chen HB, Grant-Muller S (2001) Use of sequential learning for short term traffic-flow predicting. Transp Res Part C 9(5):319–336
Cheng CT, Ou CP, Chau KW (2002) Combining a fuzzy optimal model with a genetic algorithm to solve multiobjective rainfall-runoff model calibration. J Hydrol 268(1–4):72–86
Denton JW (1995) How good are neural networks for causal forecasting. J Bus Forecast Methods Syst 14(2):17–20
Dong H, Jia L, Sun X, Li C, Qin Y (2009) Road traffic-flow forecasting with a time-oriented ARIMA Model. In: Proceedings of the fifth international joint conference on INC, IMS and IDC, 1649–1652
Faruk DO (2010) A hybrid neural network and ARIMA model for water quality time series prediction. Eng Appl Artif Intell 23:586–594
Fatourechi M, Lucas C, Sedigh AK (2001) Reducing control effort by means of emotional learning. In: Proceedings of the 9th Iranian conference on electrical engineering, Iran, 41-1 to 41-8
Fishman GS (1971) Estimating sample size in computing simulation experiments. Manage Sci 18(1):21–38
Gholipour A, Abbaspour A, Lucas C, Araabi BN, Fatourechi M (2003) Enhancing the performance of neuro-fuzzy predictors by emotional learning algorithm. Informatica 27(2):165–174
http://www.d.umn.edu/~tkwon/TDRLSoftware/TravelTimeMetroTC.msi (online)
Hwang HP (2001) Insights into neural-network forecasting time series corresponding to ARMA (p; q) structures. Omega 29:273–289
Hwang JN, Moon S (1991) Temporal difference method for multi-step prediction: application to power load forecasting. First International Forum on Applications of Neural Networks to Power Systems, USA, pp 41–45
Jaakkola T, Jordan MI, Singh SP (1994) On the convergence of stochastic iterative dynamic programming algorithms. Neural Comput 6:1185–1201
Jiang X, Adeli H (2005) Dynamic wavelet neural network model for traffic flow forecasting. J Transp Eng 131(10):771–779
Kaastra I, Boyd M (1996) Designing a neural network for forecasting financial and economic time series. Neurocomputing 10(3):215–236
Karlaftis MG, Vlahogianni EI (2009) Memory properties and fractional integration in transportation time-series. Transp Res Part C Emerg Technol 17(4):444–453
Kentridge RW, Aggleton JP (1990) Emotion: sensory representations, reinforcement and the temporal lobe. Cogn Emot 4:191–208
Li B, De Moor B (2002) Identification of influential observations on total least squares estimate. Linear Algebra Appl 348:23–39
Lin CT, Lee CSG (1991) Neural-network based fuzzy logic control and decision system. IEEE Trans Comput 40(12):1320–1327
Maes P (1991) Designing autonomous agents: theory and practice from biology to engineering and back. The MIT press, London
Markham IS, Rakes TR (1998) The effect of sample size and variability of data on the comparative performance of artificial neural networks and regression. Comput Oper Res 25:251–263
Messer CJ (1993) Advanced freeway system ramp metering strategies for Texas. TTI Research Report 1232–23. Texas Transportation Institute, College Station
Mitchell TM (1997) Machine learning. McGraw-Hill, New York
Okutani I, Stephanedes YJ (1984) Dynamic forecasting of traffic volume through Kalman filtering theory. Transp Res Part B 18(1):1–11
Ong CS, Huang JJ, Tzeng GH (2005) Model identification of ARIMA family using genetic algorithms. Appl Math Comput 164(3):885–912
Pai PF, Lin CS (2005) A hybrid ARIMA and support vector machines model in stock price forecasting. Omega 33(6):497–505
Park B (2002) Hybrid neuro-fuzzy application in short term freeway traffic volume predicting. Transp Res Rec 1802:190–196
Rashidi F, Rashidi M, Hashemi-Hosseini A (2003) Emotional temporal difference learning based intelligent controller. In: Proceedings of the IEEE conference on control applications, 1:341–346
Roger J (1993) ANFIS: adaptive-network-based fuzzy inference system. IEEE Trans Syst Man Cybern 23(3):665–685
Rojas I, Valenzuela O, Rojas F, Guillen A, Herrera LJ, Pomares H, Marquez L, Pasadas M (2008) Soft-computing techniques and ARMA model for time series prediction. Neurocomputing 71:519–537
Rolls ED (1998) The brain and emotion. Oxford University Press, Oxford
Rummery GA, Niranjan M (1994) On-line q-learning using connectionist systems. Cambridge University Engineering Department Technical Report CUED/F-INFENG/TR 166
Russel S, Norwig P (1995) A modern approach to artificial intelligence. Prentice-Hall, Englewood Cliffs
Russel S, Norwig P (2003) Artificial intelligence—a modern approach, 2nd edn. Pearson Education, Inc., New Jersey
SAS 9 (2004) Equations for the smoothing models, Jan. 2004. http://support.sas.com/91doc/getDoc/hpfug.hlp/hpfdet_sect7.htm
Schriber TJ, Andrews RW (1984) ARMA-based confidence intervals for simulation output analysis. Am J Math Manage Sci 4(3–4):345–373
Sharda R, Patil RB (1990) Neural networks as forecasting experts: an empirical test. In: Proceedings of the International joint conference on neural networks, Washington, DC, 2:491–494
Smith BL, Demetsky MJ (1994) Short-term traffic-flow prediction: neural network approach. Transp Res Rec 1453:98–104
Smith BL, Demetsky MJ (1997) Traffic-flow predicting: comparison of modeling approaches. J Trans Eng 123(4):261–266
Smith BL, Williams BM, Oswald RK (2002) Comparison of parametric and nonparametric models for traffic-flow forecasting. Transp Res Part C 10:303–321
Stathopoulos A, Karlaftis MG (2003) A multivariate state space approach for urban traffic flow modeling and prediction. Transp Res Part C Emerg Technol 11(2):121–135
Sutton RS (1988) Learning to predict by the methods of temporal differences. Mach Learn 3:9–44
Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. The MIT Press, England
Tan G, Deng Q, Tian Z, Yang J (2007) A Dish Parallel BP for Traffic-flow Forecasting. In: Proceedings of the international conference on computational intelligence and security, pp 546–549
Tan MC, Feng LB, Xu JM (2007) Traffic flow prediction based on hybrid ARIMA and ANN model. Chin J Highway Transp 20(4):118–121
Tan G, Shi H, Wang F, Deng C (2009) Short-term traffic-flow forecasting based on parallel quasi-newton neural network. In: International conference on measuring technology and mechatronics automation, ICMTMA’09 3:305–308
Tan MC, Wong SC, Xu JM, Guan ZR, Zhang P (2009) An aggregation approach to short-term traffic flow prediction. IEEE Trans Intell Transp Syst 10(1):60–69
Tang Z, Fishwick PA (1993) Feedforward neural nets as models for time series forecasting. ORSA J Comput 5:374–385
Tang Z, Almeida C, Fishwick PA (1991) Time series forecasting using neural networks vs Box–Jenkins methodology. Simulation 57:303–310
Thomakos DD, Guerard JB, Naïve J (2004) ARIMA, nonparametric, transfer function and VAR models: a comparison of forecasting performance. Int J Forecast 20(1):53–67
Valenzuela O, Rojas I, Rojas F, Pomares H, Herrera LJ, Guillen A, Marquez L, Pasadas M (2008) Hybridization of intelligent techniques and ARIMA models for time series prediction. Fuzzy Sets Syst 159:821–845
Van Der Voort M, Dougherty M, Watson S (1996) Combining Kohonen maps with ARIMA time series models to forecast traffic-flow. Transp Res Part C 4(5):307–318
Vlahogianni EI (2009) Enhancing predictions in signalized arterials with information on short-term traffic flow dynamics. J Intell Transp Syst 13(2):73–84
Vlahogianni EI, Karlaftis MG (2010) Advanced computational approaches for predicting tourist arrivals: the case of charter air-travel. In: Todd Evans (ed) Nonlinear dynamics, ISBN: 978-953-7619-61-9, Chap. 13, pp 309–324, INTECH, Croatia
Vlahogianni EI, Golias JC, Karlaftis MG (2004) Short-term traffic forecasting: overview of objectives and methods. Transp Rev 24(5):533–557
Vlahogianni EI, Karlaftis MG, Golias JC (2005) Optimized and meta-optimized neural networks for short-term traffic flow prediction: a genetic approach. Transp Res Part C 13:211–234
Vlahogianni EI, Karlaftis MG, Golias JC (2006) Statistical methods for detecting nonlinearity and non-stationarity in univariate short-term time-series of traffic volume. Transp Res Part C Emerg Technol 14(5):351–367
Vlahogianni EI, Karlaftis MG, Golias JC (2007) Spatio-temporal urban traffic volume forecasting using genetically-optimized modular networks. Computer-aided Civil Infrastruct Eng 22(5):317–325
Vlahogianni EI, Karlaftis MG, Golias JC (2008) Temporal evolution of short-term urban traffic flow: a nonlinear dynamics approach. Computer-Aided Civil Infrastruct Eng 22(5):317–325
Wang WC, Chau KW, Cheng CT, Qiu L (2009) A comparison of performance of several artificial intelligence methods for forecasting monthly discharge time series. J Hydrol 374:294–306
Watkins PD (1992) Technical note Q-learning. Mach Learn 8:279–292
Wedding DK, Cios KJ (1996) Time series forecasting by combining RBF networks, certainty factors, and the Box–Jenkins model. Neurocomputing 10:149–168
Wei CH, Lee Y (2007) Sequential forecast of incident duration using Artificial Neural Network models. J Accident Anal Prev 39:944–954
Wooldridge M, Jennings N (1995) Intelligent agents: theory and practice. Knowl Eng Rev J 10(2):115–152
Xie JX, Cheng CT, Chau KW, Pei YZ (2006) A hybrid adaptive time-delay neural network model for multi-step-ahead prediction of sunspot activity. Int J Environ Pollut 28(3–4):364–381
Zhang GP (2003) Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing 50:159–175
Zhang X, Rice J (2001) Short-term travel time prediction using a time-varying coefficient linear model. Transp Res Part C 11:187–210
Zhang Y, Ye Z (2008) Short term traffic-flow forecasting using fuzzy logic system methods. J Intell Transp Syst 12(3):102–112
Zhang GP, Patuwo EB, Hu MY (2001) A simulation study of artificial neural networks for nonlinear time-series forecasting. Comput Oper Res 28:381–396
Zhong M, Sharma S, Lingras P (2005) Short-term traffic prediction on different types of roads with genetically designed regression and time delay neural network models. J Comput Civil Eng 19(1):94–103
Acknowledgments
The author would like to express our sincere appreciations to Professor Caro Lucas for his invaluable contributions to the overall analysis of emotional learning theory in this project. We also would like to thank three anonymous referees for their helpful comments that greatly enhanced the article’s quality.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Abdi, J., Moshiri, B., Abdulhai, B. et al. Short-term traffic flow forecasting: parametric and nonparametric approaches via emotional temporal difference learning. Neural Comput & Applic 23, 141–159 (2013). https://doi.org/10.1007/s00521-012-0977-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-012-0977-3