Abstract
Policy documents are the carriers of policy and provide a channel for researchers to observe the main contents of a policy and the policy process. Policy documents are different from traditional scientific texts (including papers and patents) because they serve the function of governance and blueprint planning. This makes it impossible to accurately describe the content of policy texts by relying solely on traditional word-based bibliometric methods. In this paper, we propose a new bibliometric method for detecting changes in policy themes based on policy target–policy instrument patterns. We collected relevant policy documents under specific target topics, identified policy target–policy instrument patterns implied in those documents, and built a policy target–policy instrument network. Then, based on the eigenvector centrality features of network nodes, we identified the core “policy target” and core “policy instrument” in different time periods and ultimately identified the evolution of policy instruments and policy target, and also the continuity of policy targets. A case study of China’s nuclear energy policies was used to demonstrate the reliability of our method, and the results reflect the practical value of this method in quantitative analysis on policy documents.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bakhtin, P., Saritas, O., Chulok, A., Kuzminov, I., & Timofeev, A. (2017). Trend monitoring for linking science and strategy. Scientometrics, 111(3), 2059–2075. https://doi.org/10.1007/s11192-017-2347-5.
Bhatia, A. (2006). Critical discourse analysis of political press conferences. Discourse & Society, 17(2), 173–203. https://doi.org/10.1177/0957926506058057.
Bonacich, P. (1987). Power and centrality: A family of measures. American Journal of Sociology, 92(5), 1170–1182.
Bornmann, L., Haunschild, R., & Marx, W. (2016). Policy documents as sources for measuring societal impact: How often is climate change research mentioned in policy-related documents? Scientometrics, 109(3), 1477–1495. https://doi.org/10.1007/s11192-016-2115-y.
Chowdhury, G., & Koya, K. (2017). Information practices for sustainability: Role of iSchools in achieving the UN sustainable development goals (SDGs). Journal of the Association for Information Science and Technology, 68(9), 2128–2138. https://doi.org/10.1002/asi.23825.
Connolly, W. E. (1983). The terms of political discourse. Princeton: Princeton University Press.
Edwards-Schachter, M., & Wallace, M. L. (2017). ‘Shaken, but not stirred’: Sixty years of defining social innovation. Technological Forecasting and Social Change, 119, 64–79. https://doi.org/10.1016/j.techfore.2017.03.012.
Featherston, C. R., & O’Sullivan, E. (2017). Enabling technologies, lifecycle transitions, and industrial systems in technology foresight: Insights from advanced materials FTA. Technological Forecasting and Social Change, 115, 261–277. https://doi.org/10.1016/j.techfore.2016.06.025.
Fisher, E., Slade, C. P., Anderson, D., & Bozeman, B. (2010). The public value of nanotechnology? Scientometrics, 85(1), 29–39. https://doi.org/10.1007/s11192-010-0237-1.
Gagliardi, D., Niglia, F., & Battistella, C. (2014). Evaluation and design of innovation policies in the agro-food sector: An application of multilevel self-regulating agents. Technological Forecasting and Social Change, 85, 40–57. https://doi.org/10.1016/j.techfore.2013.10.015.
Gao, Z., & Tisdell, C. (2004). China’s reformed science and technology system: An overview and assessment. Prometheus, 22(3), 311–331. https://doi.org/10.1080/0810902042000255741.
Haunschild, R., & Bornmann, L. (2017). How many scientific papers are mentioned in policy-related documents? An empirical investigation using web of science and altmetric data. Scientometrics, 110(3), 1209–1216. https://doi.org/10.1007/s11192-016-2237-2.
Hayashi, M., Rothberg, D., & Hayashi, C. R. M. (2010). Scientific knowledge and digital democracy in Brazil: How to assess public health policy debate with applied scientometrics. Scientometrics, 83(3), 825–833. https://doi.org/10.1007/s11192-009-0125-8.
Heidrich, O., Hill, G. A., Neaimeh, M., Huebner, Y., Blythe, P. T., & Dawson, R. J. (2017). How do cities support electric vehicles and what difference does it make? Technological Forecasting and Social Change, 123, 17–23. https://doi.org/10.1016/j.techfore.2017.05.026.
Howlett, M. (2009). Governance modes, policy regimes and operational plans: A multi-level nested model of policy instrument choice and policy design. Policy Sciences, 42(1), 73–89. https://doi.org/10.1007/s11077-009-9079-1.
Huang, C. (2016). Quantitative research on policy literature. Ottawa: Science Press.
Huang, C., Su, J., Xie, X., & Li, J. (2014). Basic research is overshadowed by applied research in China: A policy perspective. Scientometrics, 99(3), 689–694. https://doi.org/10.1007/s11192-013-1199-x.
Huang, C., Su, J., Xie, X. A., Ye, X. T., Li, Z., Porter, A., et al. (2015). A bibliometric study of China’s science and technology policies: 1949–2010. Scientometrics, 102(2), 1521–1539. https://doi.org/10.1007/s11192-014-1406-4.
Huang, C., Su, J., Zhao, X. Y., Sui, J. G., Ru, P., Zhang, H. W., et al. (2012). Government funded renewable energy innovation in China. Energy Policy, 51, 121–127. https://doi.org/10.1016/j.enpol.2011.08.069.
Jordana, J., Fernandez, X., Sancho, D., & Welp, Y. (2005). Which internet policy? Assessing regional initiatives in Spain. Information Society, 21(5), 341–351. https://doi.org/10.1080/01972240500253509.
Lasswell, H. D., & Kaplan, A. (2013). Power and society: A framework for political inquiry. Piscataway: Transaction Publishers.
Laver, M., Benoit, K., & Garry, J. (2003). Extracting policy positions from political texts using words as data. American Political Science Review, 97(2), 311–331.
Lehtoranta, S., Nissinen, A., Mattila, T., & Melanen, M. (2011). Industrial symbiosis and the policy instruments of sustainable consumption and production. Journal of Cleaner Production, 19(16), 1865–1875. https://doi.org/10.1016/j.jclepro.2011.04.002.
Li, F., Butel, L., & Wang, P. (2017). Innovation policy configuration—A comparative study of Russia and China. Policy Studies, 38(4), 311–338. https://doi.org/10.1080/01442872.2017.1308477.
Lindstad, B. H., Pistorius, T., Ferranti, F., Dominguez, G., Gorriz-Mifsud, E., Kurttila, M., et al. (2015). Forest-based bioenergy policies in five European countries: An explorative study of interactions with national and EU policies. Biomass and Bioenergy, 80(Supplement C), 102–113. https://doi.org/10.1016/j.biombioe.2015.04.033.
McWilliam, W., Brown, R., Eagles, P., & Seasons, M. (2015). Evaluation of planning policy for protecting green infrastructure from loss and degradation due to residential encroachment. Land Use Policy, 47, 459–467. https://doi.org/10.1016/j.landusepol.2015.05.006.
Namenwirth, J. Z. (1967). Some long and short term trends in one american political value: A computer analysis of concern with wealth in 62 party platforms. Historical Methods Newsletter, 1(1), 6. https://doi.org/10.1080/00182494.1967.10593839.
Nilsson, M., Zamparutti, T., Petersen, J. E., Nykvist, B., Rudberg, P., & McGuinn, J. (2012). Understanding policy coherence: Analytical framework and examples of sector–environment policy interactions in the EU. Environmental Policy and Governance, 22(6), 395–423. https://doi.org/10.1002/eet.1589.
O’Leary, R., & Bingham, L. (2009). The collaborative public manager: New ideas for the twenty-first century. Washington, DC: Georgetown University Press.
Porter, M. E. (1998). The competitive advantage of nations. Mumbai: Free Press.
Provan, K. G., & Kenis, P. (2008). Modes of network governance: Structure, management, and effectiveness. Journal of Public Administration Research and Theory, 18(2), 229–252. https://doi.org/10.1093/jopart/mum015.
Quental, N., & Lourenco, J. M. (2012). References, authors, journals and scientific disciplines underlying the sustainable development literature: A citation analysis. Scientometrics, 90(2), 361–381. https://doi.org/10.1007/s11192-011-0533-4.
Ritter, A., & Lancaster, K. (2013). Measuring research influence on drug policy: A case example of two epidemiological monitoring systems. International Journal of Drug Policy, 24(1), 30–37. https://doi.org/10.1016/j.drugpo.2012.02.005.
Roberts, I., Wentz, R., & Edwards, P. (2006). Car manufacturers and global road safety: A word frequency analysis of road safety documents. Injury Prevention, 12(5), 320–322. https://doi.org/10.1136/ip.2006.012849.
Rothwell, R., & Zegveld, W. (1985). Reindustrialization and technology. Armonk: Longman, M.E. Sharpe.
Saidi, T., Salie, F., & Douglas, T. S. (2017). Towards understanding the drivers of policy change: A case study of infection control policies for multi-drug resistant tuberculosis in South Africa. Health Research Policy and Systems. https://doi.org/10.1186/s12961-017-0203-y.
Schader, C., Lampkin, N., Muller, A., & Stolze, M. (2014). The role of multi-target policy instruments in agri-environmental policy mixes. Journal of Environmental Management, 145(Supplement C), 180–190. https://doi.org/10.1016/j.jenvman.2014.06.016.
Snijders, T. A. (2005). Models for longitudinal network data. In P. J. Carrington, J. Scott, & S. Wasserman (Eds.), Models and methods in social network analysis (pp. 215–247). Cambridge: Cambridge University Press.
Su, J. (2014). Public policy for science and technology: An introduction. Ottawa: Science Press.
Van Dijk, T. A. (1997). Discourse as social interaction (Vol. 2). Thousand Oaks: Sage.
Vilkins, S., & Grant, W. J. (2017). Types of evidence cited in Australian Government publications. Scientometrics, 113(3), 1681–1695. https://doi.org/10.1007/s11192-017-2544-2.
Wu, G. (1995). Documentary politics’: Hypotheses, process, and case studies. In C. L. Hamrin, & S. Zhao (Eds.), Decision-Making in Deng’s China: Perspectives from Insiders (pp. 24–38). M. E. Sharpe Incorporated
Acknowledgements
We acknowledge support from the Innovative Research Group Project of the National Natural Science Foundation of China (Grant No. 71721002), Excellent Youth Project of the National Natural Science Foundation of China (Grant No. 71722002) and the General Program of National Natural Science Foundation of China (Grant No. 71673164). The findings and observations contained in this paper are those of the authors and do not necessarily reflect the views of the funding agency.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Huang, C., Yang, C. & Su, J. Policy change analysis based on “policy target–policy instrument” patterns: a case study of China’s nuclear energy policy. Scientometrics 117, 1081–1114 (2018). https://doi.org/10.1007/s11192-018-2899-z
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-018-2899-z