Skip to main content
SpringerLink
Log in

Evaluations of context-based co-citation searching

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Since machine-readable documents have become widespread, some recent studies have proposed retrieval methods using a combination of citation linkage and its context. In the case of co-citation linkage, there have been attempts to discern ‘strong’ co-citations from ‘weak’ ones by examining the positions of citations in a document. However, this promising concept has not yet been sufficiently evaluated, and it remains unclear whether search performance is significantly improved. Therefore, this paper explores the effects of using co-citation context more deeply and more widely by comparing the search performance of six retrieval methods, which differ as to whether co-citation context and normalization using cited frequency are used. For empirically evaluating the effects, a special test collection was created from CiteSeer Metadata, and the search performances of the six retrieval methods were compared by two IR metrics (AP and nDCG). The main conclusions of this paper are: (1) co-citation context has a positive effect on co-citation searching; (2) the normalization technique using cited frequency is useful for context-based co-citation searching; (3) approaches of using co-citation context tend to affect the characteristics of search performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. http://portal.acm.org/dl.cfm.

  2. http://www.sciencedirect.com/.

  3. http://citeseer.ist.psu.edu/.

  4. http://www.citebase.org/.

  5. http://www.openarchives.org/OAI/openarchivesprotocol.html.

  6. http://citeseer.ist.psu.edu/oai.html.

References

  • Badran, O. M. (1984). An alternative search strategy to improve information retrieval. In Proceedings of the 47th ASIS annual meeting (pp. 137–140). Philadelphia: Knowledge Industry.

  • Bichteler, J., & Eaton, E. A., III. (1980). The combined use of bibliographic coupling and cocitation for document retrieval. Journal of the American Society for Information Science, 31(4), 278–282.

    Article  Google Scholar 

  • Braam, R. R., Moed, H. F., & van Raan, A. F. J. (1988). Mapping of science: Critical elaboration and new approaches, a case study in agricultural biochemistry In Proceedings of the 1st international conference on bibliometrics and theoretical aspects of information retrieval (pp. 15–18). Diepenbeek: Elsevier.

  • Bradshaw, S. (2003). Reference directed indexing: Redeeming relevance for subject search in citation indexes. In Proceedings of the 7th European conference on digital libraries (pp. 499–510). Trondheim: Springer.

  • Brody, T. (2006). Evaluating research impact through open access to scholarly communication. PhD dissertation. Hampshire: University of Southampton.

  • Callahan, A., Hockema, S., & Eysenbach, G. (2010). Contextual cocitation: Augmenting cocitation analysis and its applications. Journal of the American Society for Information Science and Technology, 61(6), 1130–1143.

    Google Scholar 

  • Chakrabarti, S., Dom, B., Raghavan, P., Rajagopalan, S., Gibson, D., & Kleinberg, J. (1998). Automatic resource list compilation by analyzing hyperlink structure and associated text. In Proceedings of the 7th international conference on world wide web (pp. 65–74). Brisbane: Elsevier Science.

  • Chapman, J., & Subramanyam, K. (1981). Cocitation search strategy. In Proceedings of the 2nd national online meeting (pp. 97–102). Medford: Learned Information.

  • Cronin, B. (1984). The citation process: The role and significance of citations in scientific communication. London: Taylor Graham.

    Google Scholar 

  • Duncan, E. B., Anderson, F. D., & McAleese, R. (1981). Qualified citation indexing: Its relevance to educational technology. In Proceedings of the 1st symposium on information retrieval in educational technology (pp. 70–79). Aberdeen: Aberdeen University.

  • Egghe, L., & Rousseau, R. (1990). Introduction to informetrics: Quantitative methods in library, documentation and information science. Amsterdam: Elsevier Science.

    Google Scholar 

  • Elkiss, A., Shen, S., Fader, A., Erkan, G., States, D., & Radev, D. (2008). Blind men and elephants: What do citation summaries tell us about a research article? Journal of the American Society for Information Science and Technology, 59(1), 51–62.

    Article  Google Scholar 

  • Eto, M. (2007). Multivalued co-citation measure based on semantic distance between co-cited papers in a citing paper: A case study focused on enumeration of citations. Library and Information Science, 58, 49–67. (in Japanese).

    Google Scholar 

  • Eto, M. (2008a). A new co-citation measure based on structures of citing papers. IPSJ Transactions on Databases, 47(SIG 7(TOD 37)), 1–15. (in Japanese).

    Google Scholar 

  • Eto, M. (2008b). Document retrieval method using context based co-citation relationship. IPSJ SIG Notes, 2008(105), 39–46. (in Japanese).

    Google Scholar 

  • Garzone, M., & Mercer, R. E. (2000). Towards an automated citation classifier. In Proceedings of the 13th Biennial conference of the canadian society on computational studies of intelligence (pp. 337–346). London: Springer.

  • Gipp, B., & Beel. J. (2009a). Citation proximity analysis (CPA)—A new approach for identifying related work based on co-citation analysis. In Proceedings of the 12th international conference on scientometrics and informetrics (Vol. 2, pp. 571–575). Rio de Janeiro: International Society for Scientometrics and Informetrics.

  • Gipp, B., & Beel, J. (2009b). Identifying related documents for research paper recommender by CPA and COA. In Proceedings of international conference on education and information technology (Vol. 1, pp. 636–639), Berkeley: International Association of Engineers.

  • Hamers, L., Hemeryck, Y., Herweyers, G., Janssen, M., Keters, H., Rousseau, R., et al. (1989). Similarity measures in scientometric research: the Jaccard index versus Salton’s cosine formula. Information Processing and Management, 25(3), 125–131.

    Article  Google Scholar 

  • Järvelin, K., & Kekäläinen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4), 422–446.

    Article  Google Scholar 

  • Kessler, M. M. (1963). Bibliographic coupling between scientific papers. American Documentation, 14(1), 10–25.

    Article  Google Scholar 

  • Knapp, S. D. (1984). Cocitation searching: Some useful strategies. Online, 8(4), 43–48.

    Google Scholar 

  • Krampen, G., Becker, R., Wahner, U., & Montada, L. (2007). On the validity of citation counting in science evaluation: content analyses of references and citations in psychological publications. Scientometrics, 71(2), 191–202.

    Article  Google Scholar 

  • Larsen, B. (2004). References and citations in automatic indexing and retrieval systems: experiments with the boomerang effect. PhD dissertation. Copenhagen: Royal School of Library and Information Science.

  • Larsen, B., & Ingwersen, P. (2006). Using citations for ranking in digital libraries, In Proceedings of the 6th ACM/IEEE-CS joint conference on digital libraries (p. 370). Chapel Hill: ACM.

  • Le, M. H., Ho, T. B., & Nakamori, Y. (2006). Detecting citation types using finite-state machines. In Proceedings of the 10th Pacific–Asia conference on knowledge discovery and data mining (pp. 265–274). Singapore: Springer.

  • Lipetz, B. A. (1965). Improvement of the selectivity of citation indexes to science literature through inclusion of citation relationship indicators. American Documentation, 16(2), 81–90.

    Article  Google Scholar 

  • MacRoberts, M., & MacRoberts, B. (1989). Problems of citation analysis: A critical review. Journal of the American Society for Information Science, 40(5), 342–349.

    Article  Google Scholar 

  • Marshakova, I. (1973). System of document connections based on references. Nauchno-Tekhnicheskaya Informatsiya Seriya, 2(6), 3–8.

    Google Scholar 

  • McBryan, O.A. (1994). GENVL and WWWW: Tools for Taming theWeb. In Proceedings of the first international world wide web conference (pp. 79–90). Geneva: Elsevier Science.

  • McSherry, F., & Najork, M. (2008). Computing information retrieval performance measures efficiently in the presence of tied scores. In Proceedings of the 30th European conference on IR research (pp. 414–421). Glasgow: Springer.

  • Moise, G., Sander, J., & Rafiei, D. (2003). Focused co-citation: Improving the retrieval of related pages on the web. In Proceedings of the 12th international world wide web conference. Budapest: ACM Press.

  • Nadel, E. (1981). Citation and co-citation indicators of a phased impact of the BCS theory in the physics of superconductivity. Scientometrics, 3(3), 203–221.

    Article  MathSciNet  Google Scholar 

  • Nanba, H., Kando, N., & Okumura, M. (2000). Classification of research papers using citation links and citation types: Towards automatic review article generation. In Proceedings of the American society for information science/the 11th SIG classification research workshop, classification for user support and learning (pp. 117–134). Chicago: ASIS.

  • O’Connor, J. (1982). Citing statements: Computer recognition and use to improve retrieval. Information Processing and Management, 18(3), 125–131.

    Article  Google Scholar 

  • O’Connor, J. (1983). Biomedical citing statements: Computer recognition and use to aid full-text retrieval. Information Processing and Management, 19(6), 361–368.

    Article  Google Scholar 

  • Pao, M. L. (1989). Concepts of information retrieval. Englewood: Libraries Unlimited.

    Google Scholar 

  • Pao, M. L. (1993). Term and citation retrieval: A field study. Information Processing and Management, 29(1), 95–112.

    Article  MathSciNet  Google Scholar 

  • Pham, S. B., & Hoffmann, A. G. (2003). A new approach for scientific citation classification using cue phrases. In Proceedings of the 16th Australian conference on AI (pp. 759–771). Perth: Springer.

  • Ritchie, A., Robertson, S., & Teufel, S. (2008a). Comparing citation contexts for information retrieval. In Proceedings of the 17th ACM conference on information and knowledge management (pp. 213–222). Napa Valley: ACM Press.

  • Ritchie, A., Teufel, S., & Robertson, S. (2006). How to find better index terms through citations. In Proceedings of the workshop on how can computational linguistics improve information retrieval? (pp. 25–32). Sydney: Association for Computational Linguistics.

  • Ritchie, A., Teufel, S., & Robertson, S. (2008b). Using terms from citations for ir: Some first results. In Proceedings of the 30th European conference on IR research (pp. 211–221). Glasgow: Springer.

  • Salton, G. (1963). Associative document retrieval techniques using bibliographic information. Journal of the ACM, 10(4), 440–457.

    Article  MATH  Google Scholar 

  • Salton, G., & Bergmark, D. (1979). A citation study of computer science literature. IEEE Transactions on Professional Communication, 22(3), 146–158.

    Google Scholar 

  • Salton, G., & McGill, M. J. (1983). Introduction to modern information retrieval. New York: McGraw-Hill.

    MATH  Google Scholar 

  • Shibata, N., Kajikawa, Y., Takeda, Y., & Matsushima, K. (2009). Comparative study on methods of detecting research fronts using different types of citation. Journal of the American Society for Information Science and Technology, 60(3), 571–580.

    Article  Google Scholar 

  • Small, H. (1973). Co-citation in the scientific literature: A new measure of the relationship between two documents. Journal of the American Society for Information Science, 24(4), 265–269.

    Article  Google Scholar 

  • Small, H. (1982). Citation context analysis. In Progress in communication sciences (pp. 287–310). Norwood: Ablex.

  • Small, H. (1997). Update on science mapping: Creating large document spaces. Scientometrics, 38(2), 275–293.

    Article  MathSciNet  Google Scholar 

  • Small, H. (1999). Visualizing science by citation mapping. Journal of the American Society for Information Science, 50(9), 799–813.

    Article  Google Scholar 

  • Small, H. (2009). Critical thresholds for co-citation clusters and emergence of the giant component. Journal of Informetrics, 3(4), 332–340.

    Article  Google Scholar 

  • Small, H., & Garfield, E. (1985). The geography of science: Disciplinary and national mappings. Journal of Information Science, 11(4), 147–159.

    Article  Google Scholar 

  • Small, H., & Griffith, B. C. (1974). The structure of scientific literatures I: Identifying and graphing specialties. Science Studies, 4(1), 17–40.

    Article  Google Scholar 

  • Small, H., & Sweeney, E. (1985). Clustering the science citation index using co-citations I. A comparison of methods. Scientometrics, 7(3–6), 391–409.

    Article  Google Scholar 

  • Smith, L. C. (1981). Citation analysis. Library Trends, 30(1), 83–106.

    Google Scholar 

  • Spärck Jones, K. (1972). A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation, 28(1), 11–21.

    Article  Google Scholar 

  • Teufel, S., Siddharthan, A., & Tidhar, D. (2006). Automatic classification of citation function. In Proceedings of the 2006 conference on empirical methods in natural language processing (pp. 103–110). Sydney: Association for Computational Linguistics.

  • Van, T. T., & Beigbeder, M. (2007). Web co-citation: Discovering relatedness between scientific papers. In Proceedings of the 5th Atlantic web intelligence conference (pp. 343–348). Fontainebleau: Springer.

  • Voos, K., & Dagaev, H. (1976). Are all citations equal? or, did we op. cit. your idem? Journal of Academic Librarianship, 1(6), 19–21.

    Google Scholar 

  • White, H. (2004). Citation analysis and discourse analysis revisited. Applied Linguistics, 25(1), 89–116.

    Article  Google Scholar 

  • White, H., & Griffith, B. (1981). Author cocitation: A literature measure of intellectual structure. Journal of the American Society for Information Science, 32(3), 163–171.

    Article  Google Scholar 

Download references

Acknowledgments

The author is indebted to Prof. Kazuaki Kishida, Prof. Motomichi Toyama and Prof. Shunsaku Tamura, Keio University. This work was supported by a Grant-in-Aid for Scientific Research for Young Scientists (B) (No. 23700289).

Author information

Authors and Affiliations

  1. Gakushuin Women’s College, 3-20-1 Toyama, Shinjuku-ku, Tokyo, 1628650, Japan

    Masaki Eto

Authors
  1. Masaki Eto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Masaki Eto.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Eto, M. Evaluations of context-based co-citation searching. Scientometrics 94, 651–673 (2013). https://doi.org/10.1007/s11192-012-0756-z

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-012-0756-z

Keywords

Search

Navigation