Skip to main content
SpringerLink
Log in

Evolutive Approaches for Variable Selection Using a Non-parametric Noise Estimator

  • Chapter
Book cover Parallel Architectures and Bioinspired Algorithms

Part of the book series: Studies in Computational Intelligence ((SCI,volume 415))

Abstract

The design of a model to approximate a function relies significantly on the data used in the training stage. The problem of selecting an adequate set of variables should be treated carefully due to its importance. If the number of variables is high, the number of samples needed to design the model becomes too large and the interpretability of the model is lost. This chapter presents several methodologies to perform variable selection in a local or a globalmanner using a non-parametric noise estimator to determine the quality of a subset of variables. Several methods that apply parallel paradigms in different architecures are compared from the optimization and efficiency point of view since the problem is computationally expensive.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Al-Sultan, K.S., Al-Fawzan, M.A.: A tabu search hooke and jeeves algorithm for unconstrained optimization. European Journal of Operational Research 103(1), 198–208 (1997)

    Article  MATH  Google Scholar 

  2. Alba, E., Luna, F., Nebro, A.J.: Advances in parallel heterogeneous genetic algorithms for continuous optimization. Int. J. Appl. Math. Comput. Sci. 14, 317–333 (2004)

    MathSciNet  MATH  Google Scholar 

  3. Alba, E., Tomassini, M.: Parallelism and evolutionary algorithms. IEEE Trans. on Evolutionary Computation 6(5), 443–462 (2002)

    Article  Google Scholar 

  4. Baker, J.E.: Reducing bias and inefficiency in the selection algorithm. In: Grefenstette, J.J. (ed.) Proceedings of the Second International Conference on Genetic Algorithms, pp. 14–21. Lawrence Erlbaum Associates, Hillsdale (1987)

    Google Scholar 

  5. Brandao, J.: A tabu search algorithm for the open vehicle routing problem. European Journal of Operational Research 157(3), 552–564 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  6. Cantú-Paz, E.: Efficient and Accurate Parallel Genetic Algorithms. Kluwer Academic Publishers, Massachusetts (2000)

    Book  MATH  Google Scholar 

  7. Cantú-Paz, E.: Markov chain of parallel genetic algorithms. IEEE Trans. Evolutionary Computation 4, 216–226 (2000)

    Article  Google Scholar 

  8. Chakraborty, U.K., Deb, K., Chakraborty, M.: Analysis of selection algorithms: A markov chain approach. Evol. Comput. 4(2), 133–167 (1996)

    Article  Google Scholar 

  9. De Jong, K.A.: An analysis of the behavior of a class of genetic adaptive systems, Ph.D. thesis, University of Michigan (1975)

    Google Scholar 

  10. De Jong, K.A.: Evolutionary computation: Recent developments and open issues. In: Goodman, E.D., Punch, B., Uskov, V. (eds.) Proceedings of the First International Conference on Evolutionary Computation and Its Applications, Moscow, pp. 7–17 (1996)

    Google Scholar 

  11. Deb, K., Goel, T.: Controlled Elitist Non-dominated Sorting Genetic Algorithms for Better Convergence. In: Zitzler, E., Deb, K., Thiele, L., Coello Coello, C.A., Corne, D.W. (eds.) EMO 2001. LNCS, vol. 1993, pp. 67–81. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  12. DeJong, K.A., Spears, W.M.: An Analysis of the Interacting Roles of Population Size and Crossover in Genetic Algorithms. In: Schwefel, H.-P., Männer, R. (eds.) PPSN 1990. LNCS, vol. 496, pp. 38–47. Springer, Heidelberg (1991)

    Chapter  Google Scholar 

  13. DellÁmico, M., Trubian, M.: Applying tabu search to the job-shop scheduling problem. Ann. Oper. Res. 41(1-4), 231–252 (1993)

    Article  Google Scholar 

  14. Eirola, E., Liitiäinen, E., Lendasse, A., Corona, F., Verleysen, M.: Using the delta test for variable selection. In: European Symposium on Artificial Neural Networks, ESANN 2008, Bruges, Belgium, pp. 25–30 (April 2008)

    Google Scholar 

  15. Eshelman, L.J., Schaffer, J.D.: Real-coded genetic algorithms and interval schemata. In: Darrell Whitley, L. (ed.) Foundation of Genetic Algorithms, vol. 2, pp. 187–202. Morgan-Kauffman Publishers, Inc. (1993)

    Google Scholar 

  16. Fogarty, T.C.: Varying the probability of mutation in the genetic algorithms. In: Schaffer, J.D. (ed.) Proc. of the Third International Conference on Genetic Algorithms, pp. 104–109. Morgan-Kauffman Publishers, Inc. (June 1989)

    Google Scholar 

  17. Garcia, V., Debreuve, E., Barlaud, M.: Fast k nearest neighbor search using GPU. In: CVPR Workshop on Computer Vision on GPU (2008)

    Google Scholar 

  18. Glover, F.: Future paths for integer programming and links to artificial intelligence. Comput. Oper. Res. 13(5), 533–549 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  19. Glover, F.: Tabu search part i. ORSA Journal on Computing 1(3), 190–206 (1989)

    Article  MATH  Google Scholar 

  20. Glover, F.: Tabu search part ii. ORSA Journal on Computing 2, 4–32 (1990)

    Article  MATH  Google Scholar 

  21. Glover, F.: Parametric tabu-search for mixed integer programs. Comput. Oper. Res. 33(9), 2449–2494 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  22. Goldberg, D.E.: Genetic algorithms in search, optimization and machine learning. Addison Wesley (1989)

    Google Scholar 

  23. Goldberg, D.E.: Optimal initial population size for binary-coded genetic algorithms, Technical Report TCGA 85001, Department of Engineering Mechanics, University of Alabama, Tuscaloosa, AL 35486 (November 1985)

    Google Scholar 

  24. Goldberg, D.E.: Sizing populations for serial and parallel genetic algorithms. In: Schaffer, J.D. (ed.) Proc. of the Third International Conference on Genetic Algorithms, pp. 398–405. Morgan-Kauffman Publishers, Inc. (June 1989)

    Google Scholar 

  25. Goldberg, D.E., et al.: Genetic algorithms, noise and the sizing of populations. Complex Systems 6, 333–362 (1992)

    MATH  Google Scholar 

  26. Grefenstette, J.J.: Parallel adaptive algorithms for function optimization, Technical Report TCGA CS-81-19, Department of Engineering Mechanics, University of Alabama, Vanderbilt University (1981)

    Google Scholar 

  27. Grefenstette, J.J.: Optimization of control parameters for genetic algorithms. IEEE Trans. Systems, Man and Cybernetics 16(1), 122–128 (1992)

    Article  Google Scholar 

  28. Guillén, A., González, J., Rojas, I., Pomares, H., Herrera, L.J., Valenzuela, O., Rojas, F.: Output Value-Based Initialization For Radial Basis Function Neural Networks. Neural Processing Letters (2007)

    Google Scholar 

  29. Guillén, A., Pomares, H., González, J., Rojas, I., Herrera, L.J., Prieto, A.: Parallel multi-objective memetic rbfnns design and feature selection for function approximation problems. Neurocomputing, 3541–3555 (2009)

    Google Scholar 

  30. Guillén, A., Pomares, H., González, J., Rojas, I., Valenzuela, O., Prieto, B.: Parallel multiobjective memetic rbfnns design and feature selection for function approximation problems. Neurocomputing 72(16-18), 3541–3555 (2009)

    Article  Google Scholar 

  31. Guillén, A., Rojas, I., González, J., Pomares, H., Herrera, L.J., Paechter, B.: Improving the Performance of Multi-objective Genetic Algorithm for Function Approximation Through Parallel Islands Specialisation. In: Sattar, A., Kang, B.-h. (eds.) AI 2006. LNCS (LNAI), vol. 4304, pp. 1127–1132. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  32. Guillén, A., Rojas, I., González, J., Pomares, H., Herrera, L.J., Paechter, B.: Boosting the Performance of a Multiobjective Algorithm to Design RBFNNs Through Parallelization. In: Beliczynski, B., Dzielinski, A., Iwanowski, M., Ribeiro, B. (eds.) ICANNGA 2007. LNCS (LNAI), vol. 4431, pp. 85–92. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  33. Guillén, A., Rojas, I., González, J., Pomares, H., Herrera, L.J., Valenzuela, O., Prieto, A.: Improving Clustering Technique for Functional Approximation Problem Using Fuzzy Logic: ICFA algorithm. Neurocomputing 70(16-18), 2853–2860 (2007)

    Article  Google Scholar 

  34. Guillén, A., Garcia-Arenas, M., Herrera, L.J., Pomares, H., Rojas, I.: GPU Cluster with MATLAB. In: International Conference on Parallel and Distributed Processing Techniques and Applications, pp. 37–46 (2011)

    Google Scholar 

  35. Guillén, A., Sovilj, D., Lendasse, A., Mateo, F., Rojas, I.: Minimising the delta test for variable selection in regression problems. Int. J. High Perform. Syst. Archit. 1, 269–281 (2008)

    Article  Google Scholar 

  36. Guillén, A., van Heeswijk, M., Sovilj, D., Arenas, M.G., Herrera, L.J., Pomares, H., Rojas, I.: Variable Selection in a GPU Cluster Using Delta Test. In: Cabestany, J., Rojas, I., Joya, G. (eds.) IWANN 2011, Part I. LNCS, vol. 6691, pp. 393–400. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  37. Guyon, I., Gunn, S., Nikravesh, M., Zadeh, A.: Feature extraction: Foundations and applications. STUDFUZZ (studies in fuzziness and soft computing). Springer-Verlag New York, Secaucus (2006)

    Google Scholar 

  38. Hedar, A.-R., Fukushima, M.: Tabu search directed by direct search methods for nonlinear global optimization. European Journal of Operational Research 170(2), 329–349 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  39. Herrera, F., Lozano, M.: Gradual distributed real-coded genetic algorithms. IEEE Transactions on Evolutionary Computation 4(1), 43 (2000)

    Article  Google Scholar 

  40. Herrera, L.J., Pomares, H., Rojas, I., Verleysen, M., Guilén, A.: Effective Input Variable Selection for Function Approximation. In: Kollias, S.D., Stafylopatis, A., Duch, W., Oja, E. (eds.) ICANN 2006. LNCS, vol. 4131, pp. 41–50. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  41. Herrera, L.J., Pomares, H., Rojas, I., Guillén, A., Valenzuela, O.: The TaSe-NF model for function approximation problems: Approaching local and global modelling. Fuzzy Sets and Systems 171(1), 1–21 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  42. Holland, J.J.: Adaption in natural and artificial systems. University of Michigan Press (1975)

    Google Scholar 

  43. Ishibuchi, H., Yoshida, T., Murata, T.: Balance between genetic search and local search in memetic algorithms for multiobjective permutation flowshop scheduling. IEEE Trans. on Evolutionary Computation 7, 204–223 (2003)

    Article  Google Scholar 

  44. Jones, A.: New tools in non-linear modelling and prediction. Computational Management Science 1(2), 109–149 (2004)

    Article  MATH  Google Scholar 

  45. Kosko, B.: Fuzzy systems as universal approximators. IEEE Transactions on Computers 43(11), 1329–1333 (1994)

    Article  MATH  Google Scholar 

  46. Lee, S.-W., Verri, A. (eds.): SVM 2002. LNCS, vol. 2388. Springer, Heidelberg (2002)

    MATH  Google Scholar 

  47. Mantawy, A.H., Soliman, S.A., El-Hawary, M.E.: A new tabu search algorithm for the long-term hydro scheduling problem. In: 2002 Large Engineering Systems Conference on Power Engineering, LESCOPE 2002, pp. 29–34 (2002)

    Google Scholar 

  48. Mateo, F., Lendasse, A.: A variable selection approach based on the delta test for extreme learning machine models. In: Proceedings of the European Symposium on Time Series Prediction, pp. 57–66 (2008)

    Google Scholar 

  49. Michalewicz, Z.: Genetic algorithms + Data structures = Evolution programs, 3rd edn. Springer, Heidelberg (1996)

    MATH  Google Scholar 

  50. Hiroyasu, T., Miki, M., Negami, M.: Distributed genetic algorithms with randomized migration rate. In: Proceedings of the IEEE Conf. Systems, Man and Cybernetics, pp. 689–694 (1999)

    Google Scholar 

  51. Mitchell, M., Forrest, S.: Genetic algorithms and artificial life. Artificial Life 1(3), 267–289 (1995)

    Article  Google Scholar 

  52. Oh, I.-S., Lee, J.-S., Moon, B.-R.: Local search-embedded genetic algorithms for feature selection. In: Proceedings of 16th International Conference on Pattern Recognition, vol. 2, pp. 148–151 (2002)

    Google Scholar 

  53. Oh, I.-S., Lee, J.-S., Moon, B.-R.: Hybrid genetic algorithms for feature selection. IEEE Trans. on Pattern Analysis and Machine Intelligence 26(11), 1424–1437 (2004)

    Article  Google Scholar 

  54. Pi, H., Peterson, C.: Finding the embedding dimension and variable dependencies in time series. Neural Computation 6(3), 509–520 (1994)

    Article  Google Scholar 

  55. Poggio, T., Girosi, F.: A theory of networks for approximation and learning, Tech. Report AI-1140, MIT Artificial Intelligence Laboratory, Cambridge, MA (1989)

    Google Scholar 

  56. Punch, W.F., Goodman, E.D., Pei, M., Chia-Shun, L., Hovland, P., Enbody, R.: Further research on feature selection and classification using genetic algorithms. In: Forrest, S. (ed.) Proc. of the Fifth Int. Conf. on Genetic Algorithms, pp. 557–564. Morgan Kaufmann, San Mateo (1993)

    Google Scholar 

  57. Raymer, M.L., Punch, W.F., Goodman, E.D., Kuhn, L.A., Jain, A.K.: Dimensionality reduction using genetic algorithms. IEEE Transactions on Evolutionary Computation 4(2), 164–171 (2000)

    Article  Google Scholar 

  58. Reeves, C.R.: Using genetic algorithms with small populations. In: Forrest, S. (ed.) Proceedings of the Fifth International Conference on Genetic Algorithms, pp. 92–99. Morgan Kaufmann (1993)

    Google Scholar 

  59. Reeves, C.R.: Using genetic algorithms with small populations. In: Forrest, S. (ed.) Proc. of the Fifth International Conference on Genetic Algorithms, pp. 92–99. Morgan-Kauffman Publishers, Inc. (July 1993)

    Google Scholar 

  60. Rubio, G., Herrera, L.J., Pomares, H., Rojas, I., Guillén, A.: Design of specific-to-problem kernels and use of kernel weighted K-nearest neighbours for time series modelling. Neurocomputing 73(10-12), 1965–1975 (2010)

    Article  Google Scholar 

  61. Saeys, Y., Inza, I., Larranaga, P.: A review of feature selection techniques in bioinformatics. Bioinformatics 23(19), 2507–2517 (2007)

    Article  Google Scholar 

  62. Schaffer, J.D.: A study of control parameters affecting online performance of genetic algorithms for function optimization. In: Schaffer, J.D. (ed.) Proc. of the Third International Conference on Genetic Algorithms, pp. 51–60. Morgan-Kauffman Publishers, Inc. (June 1989)

    Google Scholar 

  63. Scheuerer, S.: A tabu search heuristic for the truck and trailer routing problem. Comput. Oper. Res. 33(4), 894–909 (2006)

    Article  MATH  Google Scholar 

  64. Sywerda, G.: Uniform crossover in genetic algorithms. In: Proceedings of the Third International Conference on Genetic Algorithms, pp. 2–9. Morgan Kaufmann Publishers Inc., San Francisco (1989)

    Google Scholar 

  65. Thierens, D., Goldberg, D.E.: Mixing in genetic algorithms. In: Forrest, S. (ed.) Proc. of the Fifth International Conference on Genetic Algorithms, pp. 38–45. Morgan-Kauffman Publishers, Inc. (July 1993)

    Google Scholar 

  66. Wang, L., Kazmierski, T.J.: Vhdl-ams based genetic optimization of a fuzzy logic controller for automotive active suspension systems. In: Proceedings of the 2005 IEEE International Behavioral Modeling and Simulation Workshop, BMAS 2005, pp. 124–127 (2005)

    Google Scholar 

  67. Xu, J., Chiu, S., Glover, F.: A probabilistic tabu search for the telecommunications network design. Journal of Combinatorial Optimization, Special Issue on Topological Network Design 1, 69–94 (1996)

    Google Scholar 

  68. Xu, J., Chiu, S., Glover, F.: Using tabu search to solve steiner tree-star problem in telecommunications network design. Telecommunication Systems 6, 117–125 (1996)

    Article  Google Scholar 

  69. Zhang, C., Li, P., Guan, Z., Rao, Y.Y.: A tabu search algorithm with a new neighborhood structure for the job shop scheduling problem. Computers & Operations Research 34(11), 3229–3242 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  70. Zhang, J., Li, S., Shen, S.: Extracting minimum unsatisfiable cores with a greedy genetic algorithm. In: Proc. ACAI 2006, pp. 847–856 (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Technology and Architecture, University of Granada, Granada, Spain

    Alberto Guillén, Luis Javier Herrera, Héctor Pomares & Ignacio Rojas

  2. Department of Information and Computer Science, Aalto University School of Science, Espoo, Finland

    Dušan Sovilj, Mark van Heeswijk & Amaury Lendasse

Authors
  1. Alberto Guillén
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dušan Sovilj
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mark van Heeswijk
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Luis Javier Herrera
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Amaury Lendasse
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Héctor Pomares
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ignacio Rojas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alberto Guillén .

Editor information

Editors and Affiliations

  1. Centro Universitario de Mérida, Universidad de Extremadura, C/ Sta Teresa de Jornet 38, Mérida, 06800, Spain

    Francisco Fernández de Vega

  2. Facultad de Informática, Universidad Complutense de Madrid, Calle del Profesor José García santesmases s/n, Madrid, 28040, Spain

    José Ignacio Hidalgo Pérez

  3. Facultad de Informática, Universidad Complutense de Madrid, Calle del Profesor José García santesmases s/n, Madrid, 28040, Spain

    Juan Lanchares

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer Berlin Heidelberg

About this chapter

Cite this chapter

Guillén, A. et al. (2012). Evolutive Approaches for Variable Selection Using a Non-parametric Noise Estimator. In: Fernández de Vega, F., Hidalgo Pérez, J., Lanchares, J. (eds) Parallel Architectures and Bioinspired Algorithms. Studies in Computational Intelligence, vol 415. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28789-3_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-28789-3_11

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-28788-6

  • Online ISBN: 978-3-642-28789-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation