Abstract
Learning from data is virtually universally useful. Master it and you will be welcomed nearly everywhere! Looking back, my own strange path into data mining effectiveness was guided as much by setbacks as successes. In the end, the answer is found by and through serving others.
Editor’s note. Dr. Elder heads Elder Research, one of the earliest and largest consultancies in data mining and predictive analytics. He is an Adjunct Professor in the Systems Engineering Department at the University of Virginia, where he periodically teaches graduate courses in optimization or data mining. With ERI colleague Andrew Fast and others, he has written a book on Practical Text Mining, published by Elsevier in January 2012. He is grateful to be a follower of Christ and the father of five children.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
One could do that a generation ago, when the ratio of initial salary to tuition was probably four to five times better than it is today! College tuitions have grown so fast relative to other costs—partly due to being a “high touch” (not easily automated) commodity, but mostly due to subsidies—that college is increasingly becoming a money-losing investment for those not entering a profession valued by our economy. Fortunately, data mining has a very healthy return on learning investment.
- 2.
Then, the Museum of History and Technology, nicknamed by some “America’s Attic.”
- 3.
Polynomial networks are something of a cross between regression and neural networks, but their structure adapts to the complexity of the problem rather than being preestablished [2].
- 4.
For Tolkien fans, imagine building a Palantir, which can reveal what might be the future, though at great cost!
- 5.
I had long wanted to use inductive modeling to improve global optimization. By analogy, imagine you seek the location of the deepest part of a large lake. You can probe the lake anywhere (by lowering an anchor, say), but each such experiment is costly, so you want to get to the overall bottom (not some local minimum) as efficiently as possible. You would also like some idea of the probability that a better result is still out there. The key idea is to model the response surface from the known probe results to guide the location of the next probe and improve the model with each new experiment’s result. The required properties of the surface model are very different than those for normal prediction. My search algorithm—for Global R d Optimization when Probes are Expensive (GROPE)—generalized Kushner’s two-dimensional search [4] and was the world champ for many years by the metric of requiring the fewest probes [1].
- 6.
From the Chinese fable of a wise father urging his quarreling sons to stand together, while each could easily break a single stick, none could break them when bundled together.
- 7.
The quantitative system was fearless. We were not!
- 8.
Our tasks continued the groundbreaking and successful work by Colleen McCue in using data mining for public safety (see Chap. “Operational Security Analytics: My Path of Discovery”).
- 9.
Video excerpts on four of the top mistakes can be found on YouTube, starting with http://www.youtube.com/watch?v=Rd60vmoMMRY
- 10.
The book (http://www.tinyurl.com/bookERI) won the 2009 PROSE award for Mathematics (no doubt helped by full color figures and charts). An interview about it is at http://www.youtube.com/watch?v=4B3SritCxSk
- 11.
In the Bible, the book by Matthew, Chap. 5, Verses 43–48, Jesus says: “You have heard that it was said, ‘Love your neighbor and hate your enemy.’ But I tell you, love your enemies and pray for those who persecute you, that you may be children of your Father in heaven. He causes his sun to rise on the evil and the good, and sends rain on the righteous and the unrighteous. If you love those who love you, what reward will you get? Are not even the tax collectors doing that? And if you greet only your own people, what are you doing more than others? Do not even pagans do that? Be perfect, therefore, as your heavenly Father is perfect.”
- 12.
The Las Vegas talk went over well; the post-talk interview by SAS is at http://www.youtube.com/watch?v=mVzbEtobb2E
References
J.F. Elder, Efficient optimization through response surface modeling: a GROPE algorithm. Dissertation, School of Engineering and Applied Sciences, University of Virginia, 1993
J.F. Elder, D. Brown, in Network Models for Control and Processing (Chapter 6), ed. by M.D.Fraser. Induction and Polynomial Networks, Intellect, 2000
J.F. Elder, D. Pregibon, in Advances in Knowledge Discovery and Data Mining, eds. by U.Fayyad, G. Piatetsky-Shapiro, P. Smyth, R. Uthurusamy. A Statistical Perspective on Knowledge Discovery in Databases, Chapter 4 (American Association for Artificial Intelligence, Palo Alto, California, 1996)
H.J. Kushner, A new method of locating the maximum of an arbitrary multipeak curve in the presence of noise. J. Basic Engineer. 86, 97–106 (1964) (March)
R. Nisbet, J. Elder, G. Miner, Handbook of Statistical Analysis and Data Mining Applications (Elsevier’s Academic, San Diego, CA, 2009)
W.H. Press, B.P. Flannery, S.A. Teukolsky, W.T. Vetterling, Numerical Recipes in C: The Art of Scientific Computing (Cambridge University Press, Cambridge, 1988)
G. Seni, J.F. Elder, in Ensemble Methods in Data Mining: Improving Accuracy Through Combining Predictions (Morgan and Claypool, FL, 2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Elder, J.F. (2012). Driving Full Speed, Eyes on the Rear-View Mirror. In: Gaber, M. (eds) Journeys to Data Mining. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28047-4_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-28047-4_6
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28046-7
Online ISBN: 978-3-642-28047-4
eBook Packages: Computer ScienceComputer Science (R0)