Skip to main content
SpringerLink
Log in

A 3-dimensional analysis for evaluating technology emergence indicators

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Technology emergence has become a hot topic in R&D policy and management communities. Various methods of measuring technology emergence have been developed. However, there is little literature discussing how to evaluate the results identified by different methods. This research sharpens a promising Technology Emergence Indicator (TEI) set by assessing alternative formulations on three distinct datasets: Dye-Sensitized Solar Cells, Non-Linear Programming, and Nano-Enabled Drug Delivery. Our TEIs derive from a conceptual foundation including three attributes of emergence: persistence, community, and growth that we systematically address through a 3-dimensional evaluation framework. Comparing TEI behavior through sensitivity analyses shows good robustness for the measures. The TEI serve to distinguish emerging R&D topics in the field under study. They can further be used to identify highly active players publishing on those topics. Importantly, results show that identified emerging terms and topics persist to a strong degree; thus, they serve to predict highly active R&D foci within the technical domain under study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. The Emergence script provided in VantagePoint enables one to vary many parameters of EScoring. To alter the 1.77 threshold (chosen based on empirical comparisons—see Porter et al. 2018), one just needs to scan the list of terms ordered by EScore in VantagePoint or MS Excel.

  2. We also tried calculations based on the later two-year test period and last 2 years of the validation period as the prior period to test the predictive utility. Results are similar.

  3. As of January, 2020, we have adjusted the specialized scope procedure. We abandon the cross-dataset comparison as problematic in requiring a representative population or random sample outside the test dataset. We tighten the within-dataset filter to use IDF ≥ 1.

References

  • Arora, S. K., Porter, A. L., Youtie, J., & Shapira, P. (2013). Capturing new developments in an emerging technology: an updated search strategy for identifying nanotechnology research outputs. Scientometrics,95(1), 351–370.

    Article  Google Scholar 

  • Bettencourt, L., Kaiser, D., Kaur, J., Castillo-Chavez, C., & Wojick, D. (2008). Population modeling of the emergence and development of scientific fields. Scientometrics,75(3), 495–518.

    Article  Google Scholar 

  • Breitzman, A., & Thomas, P. (2015). The Emerging Clusters Model: A tool for identifying emerging technologies across multiple patent systems. Research Policy,44(1), 195–205.

    Article  Google Scholar 

  • Burmaoglu, S., Porter, A. L., & Souminen, A. What is technology emergence? A micro level definition for improving tech mining practice. In Portland International Conference on Management of Engineering and Technology (PICMET), Honolulu, 2018.

  • Burmaoglu, S., Sartenaer, O., & Porter, A. (2019a). Conceptual definition of technology emergence: A long journey from philosophy of science to science policy. Technology in Society. https://doi.org/10.1016/j.techsoc.2019.04.002.

    Article  Google Scholar 

  • Burmaoglu, S., Sartenaer, O., Porter, A., & Li, M. (2019b). Analysing the theoretical roots of technology emergence: an evolutionary perspective. Scientometrics,119(1), 97–118.

    Article  Google Scholar 

  • Carley, S. F., Newman, N. C., Porter, A. L., & Garner, J. G. (2017). A measure of staying power: Is the persistence of emergent concepts more significantly influenced by technical domain or scale? Scientometrics,111(3), 2077–2087.

    Article  Google Scholar 

  • Carley, S. F., Newman, N. C., Porter, A. L., & Garner, J. G. (2018). An indicator of technical emergence. Scientometrics,115(1), 35–49.

    Article  Google Scholar 

  • Carley, S. F., Porter, A. L., & Youtie, J. L. (2019). A multi-match approach to the author uncertainty problem. Journal of Data and Information Science,4(2), 1–18.

    Article  Google Scholar 

  • Cheng, A. C., Chen, C. J., & Chen, C. Y. (2008). A fuzzy multiple criteria comparison of technology forecasting methods for predicting the new materials development. Technological Forecasting and Social Change,75(1), 131–141. https://doi.org/10.1016/j.techfore.2006.08.002.

    Article  Google Scholar 

  • Coates, V., Farooque, M., Klavans, R., Lapid, K., Linstone, H. A., Pistorius, C., et al. (2001). On the future of technological forecasting. Technological Forecasting and Social Change,67(1), 1–17.

    Article  Google Scholar 

  • Corning, P. A. (2002). The re-emergence of “emergence”: A venerable concept in search of a theory. Complexity,7(6), 18–30.

    Article  MathSciNet  Google Scholar 

  • Corrocher, N., Malerba, F., & Montobbio, F. (2003). The emergence of new technologies in the ICT field: Main actors, geographical distribution and knowledge sources. Varese: Department of Economics, University of Insubria.

    Google Scholar 

  • Cozzens, S., Gatchair, S., Kang, J., Kim, K. S., Lee, H. J., Ordóñez, G., et al. (2010). Emerging technologies: Quantitative identification and measurement. Technology Analysis & Strategic Management,22(3), 361–376.

    Article  Google Scholar 

  • Crutchfield, J. P. (2013). Is anything ever new? Considering emergence. In M. A. Bedau & P. Humphreys (Eds.), Emergence: Contemporary readings in philosophy and science. Cambridge: MIT Press Scholarship Online, The MIT Press.

    Google Scholar 

  • Daim, T. U., Rueda, G., Martin, H., & Gerdsri, P. (2006). Forecasting emerging technologies: Use of bibliometrics and patent analysis. Technological Forecasting and Social Change,73(8), 981–1012.

    Article  Google Scholar 

  • Esmaelian, M., Tavana, M., Di Caprio, D., & Ansari, R. (2017). A multiple correspondence analysis model for evaluating technology foresight methods. Technological Forecasting and Social Change,125, 188–205. https://doi.org/10.1016/j.techfore.2017.07.022.

    Article  Google Scholar 

  • Foster, J., & Metcalfe, J. S. (2012). Economic emergence: An evolutionary economic perspective. Journal of Economic Behavior & Organization,82(2–3), 420–432.

    Article  Google Scholar 

  • Goldspink, C., & Kay, R. (2010). Emergence in organizations: The reflexive turn. Emergence: Complexity and Organization,12(3), 47–63.

    Google Scholar 

  • Guo, Y., Xu, C., Huang, L., & Porter, A. (2012). Empirically informing a technology delivery system model for an emerging technology: Illustrated for dye-sensitized solar cells. R&D Management,42(2), 133–149.

    Article  Google Scholar 

  • Guston, D. H., & Sarewitz, D. (2002). Real-time technology assessment. Technology in Society,24(1–2), 93–109.

    Article  Google Scholar 

  • Halaweh, M. (2013). Emerging technology: What is it. Journal of Technology Management and Innovation,8(3), 108–115.

    Article  Google Scholar 

  • Harper, D. A., & Endres, A. M. (2012). The anatomy of emergence, with a focus upon capital formation. Journal of Economic Behavior & Organization,82(2–3), 352–367.

    Article  Google Scholar 

  • Huang, Y., Ma, J., Porter, A. L., Kwon, S., & Zhu, D. (2015). Analyzing collaboration networks and developmental patterns of nano-enhanced drug delivery (NEDD) for brain cancer. Beilstein Journal of Nanotechnology,6(Special issue on Nanoinformatics), 1666–1676.

    Article  Google Scholar 

  • Jun, S., & Lee, S. J. (2012). Emerging technology forecasting using new patent information analysis. International Journal of Software Engineering and Its Applications,6(3), 107–116.

    Google Scholar 

  • Kajikawa, Y., Yoshikawa, J., Takeda, Y., & Matsushima, K. (2008). Tracking emerging technologies in energy research: Toward a roadmap for sustainable energy. Technological Forecasting and Social Change,75(6), 771–782.

    Article  Google Scholar 

  • Kim, D. H., Lee, H., & Kwak, J. (2017). Standards as a driving force that influences emerging technological trajectories in the converging world of the Internet and things: An investigation of the M2M/IoT patent network. Research Policy,46(7), 1234–1254. https://doi.org/10.1016/j.respol.2017.05.008.

    Article  Google Scholar 

  • Li, M., Porter, A. L., & Suominen, A. (2018). Insights into relationships between disruptive technology/innovation and emerging technology: A bibliometric perspective. Technological Forecasting and Social Change,129, 285–296.

    Article  Google Scholar 

  • Martin, B. R. (1995). Foresight in science and technology. Technology Analysis & Strategic Management,7(2), 139–168.

    Article  Google Scholar 

  • Martin, R., & Sunley, P. (2012). Forms of emergence and the evolution of economic landscapes. Journal of Economic Behavior & Organization,82(2–3), 338–351. https://doi.org/10.1016/j.jebo.2011.08.005.

    Article  Google Scholar 

  • Metcalfe, B. (1995). Metcalfe’s law: A network becomes more valuable as it reaches more users. Infoworld,17(40), 53.

    Google Scholar 

  • Porter, A. L. (2010). Technology foresight: types and methods. International Journal of Foresight and Innovation Policy,6(1–3), 36–45.

    Article  Google Scholar 

  • Porter, A. L., & Cunningham, S. W. (2005). Tech mining: Exploiting new technologies for competitive advantage. New York: Wiley. (Chinese edition, Tsinghua University Press, 2012).

    Google Scholar 

  • Porter, A. L., Garner, J., Carley, S. F., & Newman, N. C. (2018). Emergence scoring to identify frontier R&D topics and key players. Technological Forecasting and Social Change,146, 628–643. https://doi.org/10.1016/j.techfore.2018.04.016.

    Article  Google Scholar 

  • Porter, A. L., Roessner, J. D., Jin, X. Y., & Newman, N. C. (2002). Measuring national ‘emerging technology’ capabilities. Science and Public Policy,29(3), 189–200.

    Article  Google Scholar 

  • Porter, A. L., Rossini, F. A., Carpenter, S. R., & Roper, A. T. (1980). A guidebook for technology assessment and impact analysis. New York: North Holland.

    Google Scholar 

  • Rader, M., & Porter, A. (2008). Fitting future-oriented technology analysis methods to study types. In C. Cagnin, M. Keenan, R. Johnston, F. Scapolo, & R. Barre (Eds.), future-oriented technology analysis (pp. 25–40). Berlin: Springer.

    Chapter  Google Scholar 

  • Roper, A. T., Cunningham, S. W., Porter, A. L., Mason, T. W., Rossini, F. A., & Banks, J. (2011). Forecasting and management of technology (2d ed.). New York: Wiley.

    Book  Google Scholar 

  • Rotolo, D., Hicks, D., & Martin, B. R. (2015). What is an emerging technology? Research Policy,44(10), 1827–1843.

    Article  Google Scholar 

  • Saritas, O., & Burmaoglu, S. (2015). The evolution of the use of Foresight methods: A scientometric analysis of global FTA research output. Scientometrics,105(1), 497–508. https://doi.org/10.1007/s11192-015-1671-x.

    Article  Google Scholar 

  • Sawyer, R. K. (2001). Emergence in sociology: Contemporary philosophy of mind and some implications for sociological theory. American Journal of Sociology,107(3), 551–585.

    Article  MathSciNet  Google Scholar 

  • Small, H. (1999). Visualizing science by citation mapping. Journal of the American Society for Information Science,50(9), 799–813.

    Article  Google Scholar 

  • Small, H., Boyack, K. W., & Klavans, R. (2014). Identifying emerging topics in science and technology. Research Policy,43(8), 1450–1467.

    Article  Google Scholar 

  • Sohn, S. Y., & Ahn, B. J. (2003). Multigeneration diffusion model for economic assessment of new technology. Technological Forecasting and Social Change,70(3), 251–264. https://doi.org/10.1016/S0040-1625(02)00200-7.

    Article  Google Scholar 

  • Srinivasan, R. (2008). Sources, characteristics and effects of emerging technologies: Research opportunities in innovation. Industrial Marketing Management,37(6), 633–640.

    Article  Google Scholar 

  • Van Merkerk, R. O., & Smits, R. E. (2008). Tailoring CTA for emerging technologies. Technological Forecasting and Social Change,75(3), 312–333.

    Article  Google Scholar 

  • Wang, Q. (2018). A bibliometric model for identifying emerging research topics. Journal of the Association for Information Science and Technology,69(2), 290–304.

    Article  Google Scholar 

  • Wang, Z., Porter, A. L., Kwon, S., Youtie, J., Shapira, P., Carley, S. F., et al. (2019). Updating a search strategy to track emerging nanotechnologies. Journal of Nanoparticle Research. https://doi.org/10.1007/s11051-019-4627-x.

    Article  Google Scholar 

  • Zhou, X., Porter, A. L., Robinson, D. K., Shim, M. S., & Guo, Y. (2014). Nano-enabled drug delivery: A research profile. Nanomedicine: Nanotechnology, Biology and Medicine,10(5), e889–e896.

    Article  Google Scholar 

Download references

Acknowledgements

This work was undertaken with support from the US National Science Foundation (Award #1759960 – “Indicators of Technological Emergence”) to Search Technology, Inc. and Georgia Tech. The findings and observations contained in this work are those of the authors and do not necessarily reflect the views of the National Science Foundation. We wish to express our appreciation to Jan Youtie for sharing pearls of wisdom with us and to Seokbeom Kwon who provided the sensitivity tests of TEI.

Author information

Authors and Affiliations

  1. School of Management and Economics, Beijing Institute of Technology, Beijing, 10086, China

    Xiaoyu Liu

  2. School of Public Policy, Georgia Institute of Technology, Atlanta, GA, 30332-0345, USA

    Alan L. Porter

  3. Search Technology, Inc, Norcross, GA, 30092, USA

    Alan L. Porter

Authors
  1. Xiaoyu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alan L. Porter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoyu Liu.

Electronic supplementary material

Below is the link to the electronic supplementary material.

11192_2020_3432_MOESM1_ESM.docx

Extensive supplemental materials are available at: http://….. These materials provide the sensitivity test of TEIs in NLP dataset and DSSCs dataset, and the full lists of emerging terms identified by both the new method and the previous method for all datasets. (DOCX 359 kb)

Appendix

Appendix

Detailed steps for the term cleaning process are listed below.

  1. 1.

    Use VantagePoint’s natural language processing to extract noun phrases of abstracts and titles. [We have experimented with use of WOS Keywords-Plus and Keywords-Authors. We don’t use those here so as to make our evaluation processes more generalizable to other data resources lacking such fields.]

  2. 2.

    Merge the two fields (abstract noun phrases and title noun phrases) together and remove the terms with instances fewer than 2.

  3. 3.

    Apply Cluster Suite (a VantagePoint script available at www.VPInstitute.org, under Resources) to

    1. a.

      eliminate single characters,

    2. b.

      remove keywords beginning with non-alpha numeric characters,

    3. c.

      remove XML tags,

    4. d.

      consolidate chemical compounds and their abbreviations,

    5. e.

      consolidate multiple keywords common to scientific and academic publications into one header.

  4. 4.

    Run a general fuzzy routine in VantagePoint to consolidate name variations in the list.

  5. 5.

    Divide the terms into unigrams and multigrams:

    1. a.

      For unigrams, run a WOS stopwords list to remove common academic words.

    2. b.

      For multigrams, use VantagePoint’s Folding NLP macro to consolidate the phrases.

  6. 6.

    Combine the unigrams and multigrams, and finally get the consolidated terms field as input to identify emerging terms and calculate their emergence scores.

Here is our process to generate annual, random sample datasets for WOS:

  1. 1.

    Split the total records into several search queries with under 100,000 records each. In recent years, there have been more than 2 million records in the WOS database for each year. Since the WOS could show no more than 100,000 records in one search query, we needed to split the records into several search queries. We first split the data by the capital letters of organizations. If the number of records for one capital letter was also more than 100,000, then we would separate the search by the Web of Science Categories (WCs). This way, we could split the total records for each year into 39 search queries, which constitute the whole record set.

  2. 2.

    Calculate the number of records we should download for each query. The data were split into 39 search queries, and the number of selections for each query was calculated according to the share of the number of records for each query out of the total number of records. For example, there are 90,079 records that are assigned to organizations beginning with the letter “k”, making 3.52% of the total 2,559,592 records in 2014. Thus, we would download 3.52% * 5000 = 176 records for this query.

  3. 3.

    Generate a specific amount of random numbers and download the corresponding records. For example, for the search query “OG = K* AND PY = 2014”, we generate 176 random numbers between 1 and 90079, and download the relative records.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, X., Porter, A.L. A 3-dimensional analysis for evaluating technology emergence indicators. Scientometrics 124, 27–55 (2020). https://doi.org/10.1007/s11192-020-03432-6

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-020-03432-6

Keywords

JEL Classification

Mathematics Subject Classification

Search

Navigation