Abstract
This paper considers the accident images and develops a deep learning method for feature extraction together with a mixture of experts for classification. For the first task, the outputs of the last max-pooling layer of a Convolution Neural Network (CNN) are used to extract the hidden features automatically. For the second task, a mixture of advanced variations of Extreme Learning Machine (ELM) including basic ELM, constraint ELM (CELM), On-Line Sequential ELM (OSELM) and Kernel ELM (KELM), is developed. This ensemble classifier combines the advantages of different ELMs using a gating network and its accuracy is very high while the processing time is close to real-time. To show the efficiency, the different combinations of the traditional feature extraction and feature selection methods and the various classifiers are examined on two kinds of benchmarks including accident images’ data set and some general data sets. It is shown that the proposed system detects the accidents with 99.31% precision, recall and F-measure. Besides, the precisions of accident-severity classification and involved-vehicle classification are 90.27% and 92.73%, respectively. This system is suitable for on-line processing on the accident images that will be captured by Unmanned Aerial Vehicles (UAV) or other surveillance systems.
Similar content being viewed by others
Notes
Broyden–Fletcher–Goldfarb–Shanno algorithm (BFGS) is one of the best quasi-Newton methods that has been proposed for unconstraint nonlinear programming.
References
Bisht, N., Siddhi, P., Kashyap, H.: Monitoring road accidents using sensors and providing medical facilities. Treatise Electr Magn 2, 68–73 (2012)
Hoose, N., Vicencio, M., Zhang, X.: Incident detection in urban roads using computer image processing. Traffic Eng Control 33(4), 236–244 (1992)
Zifeng, J.: Macro and micro freeway automatic incident detection (aid) methods based on image processing. In: Intelligent Transportation System, ITSC’97 (1997)
Coifman, B., McCord, M., Mishalani, R., Iswalt, M., Ji, Y.: Roadway traffic monitoring from an unmanned aerial vehicle. In: IEE Proceedings-Intelligent Transport Systems (2006)
Cao, X., Lan, J., Yan, P., Li, X.: Vehicle detection and tracking in airborne videos by multi-motion layer analysis. Mach. Vis. Appl. 23(5), 921–935 (2012)
Kim, N., Chervonenkis, M.: Situation control of unmanned aerial vehicles for road traffic monitoring. Modern Appl. Sci. 9(5), 1 (2015)
Srinivasan, D., Jin, X., Cheu, R.: Evaluation of adaptive neural network models for freeway incident detection. IEEE Trans. Intell. Transp. Syst. 5(1), 1–11 (2004)
Chiou, Y.-C., Fu, C.: Modeling crash frequency and severity using multinomial-generalized poisson model with error components. Accid. Anal. Prev. 50, 73–82 (2013)
Anderson, J., Govada, M., Steffen, T., Thorne, C., Varvarigou, V., Kales, S., Burks, S.: Obesity is associated with the future risk of heavy truck crashes among newly recruited commercial drivers. Accid. Anal. Prev. 49, 378–384 (2012)
Ahonen, T., Hadid, A., Pietikainen M.: Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2037–2041 (2006)
Wang, X., Han, T., Yan, S.: An hog-Lbp human detector with partial occlusion handling. In: International Conference on Computer Vision (2009)
Chen, H., Tsai, S., Schroth, G., Chen, D., Grzeszczuk, R., Girod, B.: Robust text detection in natural images with edge-enhanced maximally stable extremal regions. In: 18th IEEE International Conference on Image Processing (ICIP) (2011)
Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput Vis. Image Underst. 110(3), 346–359 (2008)
Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Conference on computer vision and pattern recognition, pp. 779–788 (2016)
Ahn, B.: Real-time video object recognition using convolutional neural network. In: Neural Networks (IJCNN). pp. 1–7 (2015)
LeCun, Y., Kavukcuoglu, K., Farabet, C.: Convolutional networks and applications in vision. In: International Symposium on Circuits and Systems (ISCAS) (2010)
Lee, K., Park, D.C.: Image classification using fast learning convolutional neural networks. Adv. Sci. Technol. Lett. 113, 50–55 (2015)
Sadeky, S., Al-Hamadiy, A., Michaelisy, B., Sayed, U.: “Real-time automatic traffic accident recognition using Hfg. In: 20th International Conference on Pattern Recognition (ICPR) (2010)
Nejjari, F., Benhlima, L., Bah S.: Event traffic detection using heterogenous wireless sensors network. In: 13th International Conference of Computer Systems and Applications (AICCSA) (2016)
Kahaki, S., Nordin, M.: Highway traffic incident detection using high-resolution aerial remote sensing imagery. J. Comput. Sci. 7(6), 949 (2011)
Jiansheng, F.: Vision-based real-time traffic accident detection. In: 11th World Congress on Intelligent Control and Automation, WCICA (2014)
Chen, L., Cao, Y., Ji, R.: Automatic incident detection algorithm based on support vector machine. In: Sixth International Conference on Natural Computation (ICNC) (2010)
Prabha, C., Sunitha, R., Anitha, R.: Automatic vehicle accident detection and messaging system using GSM and GPS modem. Int. J. Adv. Res Electr. Electron. Instrum. Eng. 3(7), 10723–10727 (2014)
Kagesawa, M., Nakamura, A., Ikeuchi, K., Saito H.: Vehicle type classification in infra-red image using parallel vision board. ITSWC (2000)
Zhou, Y., Nejati, H., Do, T.-T., Cheung, N.-M., Cheah, L.: Image-based vehicle analysis using deep neural network: a systematic study. In: International Conference on Digital Signal Processing (DSP) (2016)
Chen, Z., Ellis, T.: Semi-automatic annotation samples for vehicle type classification in urban environments. IET Intel. Transp. Syst. 3(9), 240–249 (2014)
Wang, X., Zhang, W., Wu, X., Xiao, L., Qian, Y., Fang, Z.: Real-time vehicle type classification with deep convolutional neural networks. J. Real-Time Image Proc. (2017). https://doi.org/10.1007/s11554-017-0712-5
Zheng, Z., Lu, P., Lantz, B.: Commercial truck crash injury severity analysis using gradient boosting data mining model. J. Saf. Res. 65, 115–124 (2018)
Delen, D., Sharda, R., Bessonov, M.: Identifying significant predictors of injury severity in traffic accidents using a series of artificial neural networks. Accid. Anal. Prev. 38(3), 434–444 (2006)
Chang, L.-Y., Wang, H.-W.: Analysis of traffic injury severity: an application of non-parametric classification tree techniques. Accid. Anal. Prev. 38(5), 1019–1027 (2006)
Nguyen, C.H., Cai, Chen, F.: Automatic classification of traffic incident’s severity using machine learning approaches. IET Intel. Transp. Syst. 11, 615–623 (2017)
Kheradpisheh, S., Sharifizadeh, F., Nowzari-Dalini, A., Ganjtabesh, M., Ebrahimpour, R.: Mixture of feature specified experts. Inf. Fusion 20, 242–251 (2014)
Li, L., Zou, B., Hu, Q., Wu, X., Yu, D.: Dynamic classifier ensemble using classification confidence. Neurocomputing 99, 581–591 (2013)
Yu, J.S., Chen, J., Xiang, Z.Q., Zou, Y.X.: A hybrid convolutional neural networks with extreme learning machine for WCE image classification. In: IEEE International Conference on Robotics and Biomimetics (ROBIO) (2015)
McDonnell, M.D., Tissera, M.D., Vladusich, T., Van Schaik, A., Tapson, J.: Fast, simple and accurate handwritten digit classification by training shallow neural network classifiers with the ‘extreme learning machine’algorithm. PLoS One 10(8), e0134254 (2015)
Liu, Y., Yao, X.: Ensemble learning via negative correlation. Neural Netw. 12(10), 1399–1404 (1999)
Masoudnia, S., Ebrahimpour, R., Arani, S.: Incorporation of a regularization term to control negative correlation in mixture of experts. Neural Process. Lett. 36(1), 31–47 (2012)
Islam, M., Yao, X., Nirjon, S., Islam, M., Murase, K.: Bagging and boosting negatively correlated neural networks. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 38(3), 771–784 (2008)
Liu, Y., Yao, X.: Simultaneous training of negatively correlated neural networks in an ensemble. IEEE Trans.Syst. Man Cybern. Part B (Cybern.) 29(6), 716–725 (1999)
Ebrahimpour, R., Sadeghnejad, N., Masoudnia, S., Arani, S.: Boosted pre-loaded mixture of experts for low-resolution face recognition. Int. J. Hybrid Intell. Syst. 9(3), 145–158 (2012)
Lotfi, M., Motamedi, S., Sharifian, S.: Time-based feedback-control framework for real-time video surveillance systems with utilization control. J. Real-Time Image Proc. (2016). https://doi.org/10.1007/s11554-016-0637-4
Zarándy, Á, Nemeth, M., Nagy, Z., Kiss, A., Santha, L., Zsedrovits: A real-time multi-camera vision system for UAV collision warning and navigation. J. Real-Time Image Proc. 4, 709–724 (2016)
Puri, A.: A survey of unmanned aerial vehicles (UAV) for traffic surveillance. Department of computer science and engineering, University of South Florida (2005)
Pearson, K.: Liii. On lines and planes of closest fit to systems of points in space. Lond. Edinb. Dublin Philos. Mag. J. Sci. 2(11), 559–572 (1901)
Hall, M.: Correlation-based feature selection for machine learning. PhD thesis, Department of Computer Science, University of Waikato Hamilton (1999)
Hamon, J.: Optimisation Combinatoire Pour La Sélection De Variables En Régression En Grande Dimension: Application En Génétique Animale. Université des Sciences et Technologie de Lille-Lille I. (2013)
Mofarreh-Bonab, M., Mofarreh-Bonab, M.: Color image compression using PCA. Int. J. Comput. Appl. 111(5):16–19 (2015)
Zhang, L., Yang, F., Zhang, Y.D., Zhu, Y.J.: Road crack detection using deep convolutional neural network. In: International Conference on Image Processing (ICIP), pp. 3708–3712 (2016)
Weng, Q., Mao, Z., Lin, J., Liao, X.: Land-use scene classification based on a cnn using a constrained extreme learning machine. Int. J. Remote Sens. pp. 1–19 (2018)
Martinel, N., Piciarelli, C., Foresti, G., Micheloni C.: Mobile food recognition with an extreme deep tree. In: Proceedings of the 10th International Conference on Distributed Smart Camera (2016)
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)
Abbasi, E., Shiri, M., Ghatee, M.: A regularized root–quartic mixture of experts for complex classification problems. Knowl.-Based Syst. 110, 98–109 (2016)
Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: International Joint Conference on Neural Networks (2004)
Huang, G.-B., Zhou, H., Ding, X., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 42(2), 513–529 (2012)
Huang, G.-B., Liang, N.-Y., Rong, H.-J., Saratchandran, P., Sundararajan N.: “On-line sequential extreme learning machine. Comput. Intell. 2005, 232–237 (2005)
Zhu, W., Miao, J., Qing, L.: Constrained extreme learning machine: a novel highly discriminative random feedforward neural network. In: International Joint Conference on Neural Networks (IJCNN) (2014)
Tian, H.X., Mao, Z.Z.: An ensemble ELM based on modified AdaBoost. RT algorithm for predicting the temperature of molten steel in ladle furnace. IEEE Trans. Autom. Sci. Eng. 7(1), 73–80 (2010)
Huang, Y., Suen, C.: The behavior-knowledge space method for combination of multiple classifiers. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (1993)
Kuncheva, L., Bezdek, J., Duin, R.: Decision templates for multiple classifier fusion: an experimental comparison. Pattern Recognit. 34(2), 299–314 (2001)
Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. John Wiley & Sons (2004)
Pashaei, A., Ghatee, M., Sajedi, H.: Accident images analysis dataset. Amirkabir University of Technology, 2018. (Online). https://github.com/mghatee/Accident-Images-Analysis-Dataset. Accessed 2018
Yann, L., Corinna, C., Christopher, J.: The Mnist Database of Handwritten Digits (Online). http://yhann.lecun.com/exdb/mnist (1998)
Vahdatpour, M., Sajedi, H., Ramezani, F.: Air pollution forecasting from sky images with shallow and deep classifiers. Earth Sci. Inf. 11(3), 413–422 (2018)
Cheng, J., Huang, W., Cao, S., Yang, R., Yang, W., Yun, Z., Wang, Z., Feng, Q.: Enhanced performance of brain tumor classification via tumor region augmentation and partition. PLoS One 10(10), 0140381 (2015)
Arróspide, J., Salgado, L., Nieto, M.: Video analysis based vehicle detection and tracking using an MCMC sampling framework. EURASIP J. Adv. Signal Process. (2012)
Wan, L., Zeiler, M., Zhang, S., Le Cun, Y., Fergus R.: Regularization of neural networks using dropconnect. In: International Conference on Machine Learning (2013)
Haut, J., Paoletti, M., Plaza, J., Plaza, A.: Fast dimensionality reduction and classification of hyperspectral images with extreme learning machines. J. Real-Time Image Proc. 15(3), 439–462 (2018)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Pashaei, A., Ghatee, M. & Sajedi, H. Convolution neural network joint with mixture of extreme learning machines for feature extraction and classification of accident images. J Real-Time Image Proc 17, 1051–1066 (2020). https://doi.org/10.1007/s11554-019-00852-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-019-00852-3