Abstract
Web Spam is one of the main difficulties that crawlers have to overcome. According to Gyöngyi and Garcia-Molina it is defined as “any deliberate human action that is meant to trigger an unjustifiably favourable relevance or importance of some web pages considering the pages’ true value”. There are several studies on characterising and detecting Web Spam pages. However, none of them deals with all the possible kinds of Web Spam. This paper shows an analysis of different kinds of Web Spam pages and identifies new elements that characterise it. Taking them into account, we propose a new Web Spam detection system called SAAD, which is based on a set of heuristics and their use in a C4.5 classifier. Its results are also improved by means of Bagging and Boosting techniques. We have also tested our system in some well-known Web Spam datasets and we have found it to be very effective.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Amitay, E., Carmel, D., Darlow, A., Lempel, R., Soffer, A.: The connectivity sonar: detecting site functionality by structural patterns. In: Proceedings of the Fourteenth ACM Conference on Hypertext and Hypermedia, pp. 38–47. ACM Press (2003)
Benczur, A.A., Csalogany, K., Sarlos, T., Uher, M., Uher, M.: Spamrank - fully automatic link spam detection. In: Proceedings of the First International Workshop on Adversarial Information Retrieval on the Web (AIRWeb) (2005)
Breiman, L., Breiman, L.: Bagging predictors. In: Machine Learning, pp. 123–140 (1996)
Castillo, C., Davison, B.D.: Adversarial Web Search 4(5), 377–486 (2010)
Chellapilla, K., Maykov, A.: A taxonomy of javascript redirection spam. In: Proceedings of the 3rd International Workshop on Adversarial Information Retrieval on the Web, AIRWeb 2007, pp. 81–88. ACM, New York (2007)
Cova, M., Kruegel, C., Vigna, G.: Detection and analysis of drive-by-download attacks and malicious javascript code. In: Proceedings of the 19th International Conference on World Wide Web, WWW 2010, pp. 281–290. ACM, New York (2010)
Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. Commun. ACM 51, 107–113 (2008)
Fetterly, D., Manasse, M., Najork, M.: Spam, damn spam, and statistics: using statistical analysis to locate spam web pages. In: Proceedings of the 7th International Workshop on the Web and Databases: colocated with ACM SIGMOD/PODS, WebDB 2004, pp. 1–6. ACM, New York (2004)
Fetterly, D., Manasse, M., Najork, M.: Detecting phrase-level duplication on the world wide web. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 170–177. ACM Press (2005)
Gonzalez Jesus, B.W., Cristina, A.: Implementacion y evaluacion de un detector masivo de web spam (2009)
Gyongyi, Z., Garcia-Molina, H.: Web spam taxonomy. Technical Report 2004-25, Stanford InfoLab (March 2004)
Gyöngyi, Z., Garcia-Molina, H.: Link spam alliances. In: Proceedings of the 31st International Conference on Very Large Data Bases, VLDB 2005, pp. 517–528. VLDB Endowment (2005)
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The weka data mining software: an update. SIGKDD Explor. Newsl. 11, 10–18 (2009)
Henzinger, M.R., Motwani, R., Silverstein, C.: Challenges in web search engines. SIGIR Forum 36, 11–22 (2002)
Hidalgo, J.M.G.: Evaluating cost-sensitive unsolicited bulk email categorization (2002)
Jansen, B.J., Spink, A.: An analysis of web documents retrieved and viewed (2003)
Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection, pp. 1137–1143. Morgan Kaufmann (1995)
Ntoulas, A., Manasse, M.: Detecting spam web pages through content analysis. In: Proceedings of the World Wide Web Conference, pp. 83–92. ACM Press (2006)
Quinlan, J.R.: Bagging, boosting, and c4.5. In: Proceedings of the Thirteenth National Conference on Artificial Intelligence, pp. 725–730. AAAI Press (1996)
Quinlan, J.R.: Improved use of continuous attributes in c4.5. Journal of Artificial Intelligence Research 4, 77–90 (1996)
Quinlan, J.R.: C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc., San Francisco (1993)
Sahami, M., Dumais, S., Heckerman, D., Horvitz, E.: A bayesian approach to filtering junk e-mail (1998)
Webb, S.: Introducing the webb spam corpus: Using email spam to identify web spam automatically. In: Proceedings of the 3rd Conference on Email and AntiSpam (CEAS) (2006) (Mountain View)
Webb, S.: Webb Spam Corpus (2011), http://www.cc.gatech.edu/projects/doi/WebbSpamCorpus.html
Wu, B., Davison, B.D.: Cloaking and redirection: A preliminary study (2005)
Wu, B., Davison, B.D.: Identifying link farm spam pages. In: Special Interest Tracks and Posters of the 14th International Conference on World Wide Web, WWW 2005, pp. 820–829. ACM, New York (2005)
Yahoo!: Web spam challenge (2011), http://webspam.lip6.fr/wiki/pmwiki.php
Yahoo!: Web Spam Detection - Resources for Research on Web Spam (2011), http://barcelona.research.yahoo.net/webspam/
Zhang, H., Goel, A., Govindan, R., Mason, K., Van Roy, B.: Making Eigenvector-Based Reputation Systems Robust to Collusion. In: Leonardi, S. (ed.) WAW 2004. LNCS, vol. 3243, pp. 92–104. Springer, Heidelberg (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Prieto, V.M., Álvarez, M., López-García, R., Cacheda, F. (2012). Analysis and Detection of Web Spam by Means of Web Content. In: Salampasis, M., Larsen, B. (eds) Multidisciplinary Information Retrieval. IRFC 2012. Lecture Notes in Computer Science, vol 7356. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31274-8_4
Download citation
DOI: https://doi.org/10.1007/978-3-642-31274-8_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-31273-1
Online ISBN: 978-3-642-31274-8
eBook Packages: Computer ScienceComputer Science (R0)