Abstract
In a dynamic optimization problem (DOP) the optima can change either in a sequential or in a recurrent manner. In sequential DOPs, the optima change gradually over time while in recurrent DOPs, previous optima reappear over time. The common strategy to tackle recurrent DOPs is to employ an archive of solutions along with information allowing to associate them with their respective problem instances. In this paper, a memory-based Dynamic Particle Swarm Optimization (DPSO) approach which relies on a dual-purpose memory for fast optimization of streams of recurrent problems is proposed. The dual-purpose memory is based on a Gaussian Mixture Model (GMM) of candidate solutions estimated in the optimization space which provides a compact representation of previously-found PSO solutions. This GMM is estimated over time during the optimization phase. Such memory operates in two modes: generative and regression. When operating in generative mode, the memory produces solutions that in many cases allow avoiding costly re-optimizations over time. When operating in regression mode, the memory replaces costly fitness evaluations with Gaussian Mixture Regression (GMR). For proof of concept simulation, the proposed hybrid GMM-DPSO technique is employed to optimize embedding parameters of a bi-tonal watermarking system on a heterogeneous database of document images. Results indicate that the computational burden of this watermarking problem is reduced by up to 90.4 % with negligible impact on accuracy. Results involving the use of the memory of GMMs in regression mode as a mean of replacing fitness evaluations (surrogate-based optimization) indicate that such learned memory also provides means of decreasing computational burden in situations where re-optimization cannot be avoided.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Nickabadi, A., Ebadzadeh, M.M.. Safabakhsh, R.: DNPSO: A dynamic niching particle swarm optimizer for multi-modal optimization. In: IEEE World Congress on Computational Intelligence, pp. 26-32 (2008)
Farina, M., Deb, K., Amato, P.: Dynamic multiobjective optimization problems: test cases, approximations, and applications. IEEE Trans. Evol. Comput. 8(5), 425-442 (2004)
Blackwell, T., Branke, J.: Multiswarms, exclusion, and anti-convergence in dynamic environments. IEEE Trans. Evol. Comput. 10(4), 459-472 (2006)
Carlisle, A., Dozier, G.: Tracking changing extrema with adaptive particle swarm optimizer. In: Proceedings of the 5th Biannual World Automation Congress, 2002, vol. 13, pp. 265-270 (2002)
Hu, X., Eberhart, R.: Adaptive particle swarm optimization: detection and response to dynamic systems. In: Proceedings of the 2002 Congress on Evolutionary Computation. CEC ’02, vol. 2, pp. 1666-1670 (2002)
Wang, H., Wang, D., Yang, S.: Triggered memory-based swarm optimization in dynamic environments, In: EvoWorkshops, pp. 637-646 ( 2007)
Barlow, G.J., Smith, S.F.: Using memory models to improve adaptive efficiency in dynamic problems, In: IEEE Symposium on Computational Intelligence in Scheduling, 2009. CI-Sched ’09, pp. 7-14 (2009)
Yang, S., Yao, X.: Population-based incremental learning with associative memory for dynamic environments. IEEE Trans. Evol. Comput. 12(5), 542-561 (2008)
Li, X., Dam, K.H.: Comparing particle swarms for tracking extrema in dynamic environments. In: Proceedings of the 2003 Congress on Evolutionary Computation. CEC ’03, vol. 3, pp. 1772-1779 (2003)
Kapp, M.N., Sabourin, R., Maupin, P.: A dynamic model selection strategy for support vector machine classifiers. Appl. Soft Comput. 12(8), 2550-2565 (2012)
Vellasques, E., Sabourin, R., Granger, E.: A high throughput system for intelligent watermarking of bi-tonal images. Appl. Soft Comput. 11(8), 5215-5229 (2011)
Connolly, J.F., Granger, E., Sabourin, R.: Evolution of heterogeneous ensembles through dynamic particle swarm optimization for video-based. Pattern Recognit. 45(7), 2460-2477 (2012)
Connolly, J.F., Granger, E., Sabourin, R.: An adaptive classification system for video-based face recognition. Inf. Sci. 192, 50-70 (2012)
Connolly, J.F., Granger, E., Sabourin, R.: Dynamic multi-objective evolution of classifier ensembles for video-based face recognition. Appl. Soft Comput. 13(6), 3149-3166 (2013)
Vellasques, E., Sabourin, R., Granger, E.: Gaussian mixture modeling for dynamic particle swarm optimization of recurrent problems. In: Proceedings of the Genetic and Evolutionary Computation Conference GECCO ’12, pp. 73-80. ACM (2012)
Vellasques, E., Sabourin, R., Granger, E.: Fast intelligent watermarking of heterogeneous image streams through mixture modeling of PSO populations. Appl. Soft Comput. 13(6), 3130-3148 (2013)
Parno, M.D., Hemker, T., Fowler, K.R.: Applicability of surrogates to improve efficiency of particle swarm optimization for simulation-based problems. Eng. Optim. 44(5), 521-535 (2012)
Kennedy, J.: Some issues and practices for particle swarms. In: Swarm Intelligence Symposium, 2007, SIS 2007, pp. 162-169. IEEE (2007)
Blackwell, M.: Particle swarms and population diversity. Soft Comput. 9(11), 793-802 (2005)
Poli, R., Kennedy, J., Blackwell, T.: Particle swarm optimisation: an overview. Swarm Intell. J. 1(1), 33-57 (2007). June
Nguyen, T.T., Yang, S., Branke, J.: Evolutionary dynamic optimization: A survey of the state of the art. Swarm Evol. Comput. 6, 1-24 (2012)
Blackwell, T.: Particle swarm optimization in dynamic environments. Evolutionary Computation in Dynamic Environments, pp. 29-49. Springer, Berlin (2007)
Yang, S.: Population-based incremental learning with memory scheme for changing environments. In: Proceedings of the 2005 conference on Genetic and evolutionary computation, GECCO ’05, pp. 711-718. ACM, New York, NY, USA (2005)
Parrott, D.. Li, X.: A particle swarm model for tracking multiple peaks in a dynamic environment using speciation. In: Proceedings of the 2004 Congress on Evolutionary Computation. CEC ’04, vol. 1, pp. 98-103 (2004)
Pelikan, M., Goldberg, D.E., Lobo, F.G.: A survey of optimization by building and using probabilistic models. Comput. Optim. Appl. 21(1), 5-20 (2002)
Figueiredo, M.A.T., Jain, A.K.: Unsupervised learning of finite mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 24, 381-396 (2000)
Sfikas, G., Constantinopoulos, C., Likas, A., Galatsanos, N.P.: An analytic distance metric for gaussian mixture models with application in image retrieval. In: Proceedings of the 15th international conference on Artificial neural networks: formal models and their applications–vol. Part II, ICANN’05, pp. 835-840. Springer, Berlin (2005)
Hennig, C.: Methods for merging gaussian mixture components. Adv. Data Anal. Classif. 4, 3-34 (2010)
Queipo, N.V., Haftka, R.T., Shyy, W., Goel, T., Vaidyanathan, R., Tucker, P.K.: Surrogate-based analysis and optimization. Prog. Aerosp. Sci. 41(1), 1-28 (2005)
Sung, H.G.: Gaussian mixture regression and classification, Ph.D. thesis, Rice University (2004)
V. Torczon, M. W. Trosset, Using approximations to accelerate engineering design optimization, in: Proceedings of the 7th AIAA/USAF/NASA/ISSMO Multidisciplinary Analysis and Optimization Symposium, Saint Louis, USA, 1998, pp. 738-748
Muharemagic, E.: Adaptive two-level watermarking for binary document images, Ph.D. thesis, Florida Atlantic University (2004)
Lu, H., Kot, A.C., Shi, Y.Q.: Distance-reciprocal distortion measure for binary document images. IEEE Signal Process. Lett. 11(2), 228-231 (2004)
Collette, Y., Siarry, P.: On the sensitivity of aggregative multiobjective optimization methods. CIT 16(1), 1-13 (2008)
Sauvola, J., Kauniskangas, H.: MediaTeam Document Database II, A CD-ROM Collection of Document Images. University of Oulu, Finland (1999)
Acknowledgments
This work has been supported by National Sciences and Engineering Research Council of Canada and BancTec Canada Inc.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Vellasques, E., Sabourin, R., Granger, E. (2016). A Dual-Purpose Memory Approach for Dynamic Particle Swarm Optimization of Recurrent Problems. In: Abielmona, R., Falcon, R., Zincir-Heywood, N., Abbass, H. (eds) Recent Advances in Computational Intelligence in Defense and Security. Studies in Computational Intelligence, vol 621. Springer, Cham. https://doi.org/10.1007/978-3-319-26450-9_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-26450-9_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26448-6
Online ISBN: 978-3-319-26450-9
eBook Packages: EngineeringEngineering (R0)