Hao-En Kao
ABSTRACT
Since the Obama administration encouraged open government data (ODG) in 2008, the OGD movement has been prevailed in the whole world. Efforts for evaluating OGD in global or regional scale also show progress throughout the years. Along the way, academia tried to grasp this trend by comparing these benchmarking efforts, whether by international organizations, private companies or regional authorities. This research aims to compare the review articles of international OGD benchmarks, reviews and analyze these benchmarks, and then review the benchmarking framework created by academia in the recent year and how these benchmark efforts from academia can integrate into international benchmarks.
After reviewing six review articles about international OGD benchmarks, this research selects five repeated, and most frequently reviewed benchmarks and analyzes the methodology, level of analysis and indices. Then, a systematic literature review of recent OGD benchmark in academia were performed, the result shows that data quality and use/usage are the most frequent topics in academia, and there is a need for creating benchmarking efforts on OGD impacts, especially on social and economic. Finally, several recommendations about how academia can integrate benchmarks into practice are made. This research contributes to OGD research in three ways: (1) It stands on giants’ should, reviews and compares the review by academia for the OGD benchmark efforts. (2) It reveals the most frequent reviewed benchmarks by comparing these reviews of benchmark and shows the recommendation by these reviews. (3) It compares and reveals the current benchmarks conducted by academia, and how these research can be integrated by benchmarks by international organizations.
Supplemental Material
Available for Download
- Rosenbaum, S., & Berger, M. (2011). The Freedom of Information Act: Implications for Public Health Policy and Practice. Public Health Reports, 126(3), 428–432. https://doi.org/10.1177/003335491112600317Google Scholar
Cross Ref
- Longhi, C., Titz, J.-B., & Viallis, L. (2014). Open data: Challenges and opportunities for the tourism industry. Tourism Management, Marketing, and Development, 57–76. https://doi.org/10.1057/9781137354358_4Google ScholarCross Ref
- White House (2009) Transparency and Open Government MEMORANDUM FOR THE HEADS OF EXECUTIVE DEPARTMENTS AND AGENCIES https://obamawhitehouse.archives.gov/the-press-office/transparency-and-open-governmentGoogle Scholar
- Berners-Lee, T. (2006) Linked data: Design http://www.w3.org/DesignIssues/LinkedData.htmlGoogle Scholar
- Charalabidis, Y., Zuiderwijk, A., Alexopoulos, C., Janssen, M., Lampoltshammer, T., Ferro, E., (2018). Open Data Evaluation Models: Theory and Practice. In: Charalabidis, Y., Zuiderwijk, A., Alexopoulos, C., Janssen, M., Lampoltshammer, T., Ferro, E. (Eds.), The World of Open Data, Vol. 28. Springer, Cham.Google ScholarCross Ref
- Davies, T. (2013). Notes on open government data evaluation and assessment frameworks. http://www.opendataimpacts.net/2013/02/506/Google Scholar
- Susha, I., Zuiderwijk, A., Janssen, MFWHA., & Grönlund, Å. (2015). Benchmarks for evaluating the progress of open data adoption usage, limitations, and lessons learned. Social Science Computer Review, 33(5), 613-630. https://doi.org/10.1177/0894439314560852Google ScholarDigital Library
- Chu, P.-Y., & Tseng, H.-L. (2016). A theoretical framework for evaluating Government open data platform. Proceedings of the International Conference on Electronic Governance and Open Society: Challenges in Eurasia. https://doi.org/10.1145/3014087.3014099Google ScholarDigital Library
- Kawashita, I., Baptista, A.A., & Soares, D.S. (2020). An assessment of open government data benchmark instruments. In EGOV-CeDEM-ePart-* 2020: Link¨ oping, Sweden (pp. 29-40). CEUR Workshop Proceedings. Retrieved from http://ceur-ws.org/Vol-2797/paper2.pdfGoogle Scholar
- Attard, J., Orlandi, F., Scerri, S., & Auer, S. (2015). A systematic review of Open Government Data Initiatives. Government Information Quarterly, 32(4), 399–418. https://doi.org/10.1016/j.giq.2015.07.006Google ScholarCross Ref
- Opengovdata.org (2007). Eight principles of open government data. https://opengovdata.org/Google Scholar
- Sayogo, D., Pardo, T., & Cook, M. (Jan 2014). A framework for benchmarking open government data efforts. System Sciences (HICSS), 2014 47th Hawaii International Conference on (pp. 1896–1905).Google ScholarDigital Library
- Martin, S., Foulonneau, M., & Turki, S. (2013). 1-5 stars: Metadata on the openness level of open data sets in Europe. Communications in Computer and Information Science, 234–245. https://doi.org/10.1007/978-3-319-03437-9_24Google ScholarCross Ref
- Crusoe, J. R., & Ahlin, K. (2019). Users’ activities for using Open Government Data – a process framework. Transforming Government: People, Process and Policy, 13(3/4), 213–236. https://doi.org/10.1108/tg-04-2019-0028Google ScholarCross Ref
- Wirtz, B. W., Weyerer, J. C., Becker, M., & Müller, W. M. (2022). Open government data: A systematic literature review of empirical research. Electronic Markets, 1-24.Google Scholar
- Jetzek, T., Avital, M., & Bjorn-Andersen, N. (2012, December). The value of open government data: A strategic analysis framework. In SIG eGovernment pre-ICIS workshop, Orlando.Google Scholar
- Zhang, H., & Xiao, J. (2020). Quality Assessment Framework for Open Government Data. The Electronic Library, 38(2), 209–222. https://doi.org/10.1108/el-06-2019-0145Google ScholarCross Ref
- Khurshid, M. M., Zakaria, N. H., Rashid, A., Ahmad, M. N., Arfeen, M. I., & Faisal Shehzad, H. M. (2020). Modeling of open government data for public sector organizations using the potential theories and determinants—a systematic review. Informatics, 7(3), 24. https://doi.org/10.3390/informatics7030024Google ScholarCross Ref
- Bachtiar, A., Suhardi, & Muhamad, W. (2020). Literature review of open government data. 2020 International Conference on Information Technology Systems and Innovation (ICITSI). https://doi.org/10.1109/icitsi50517.2020.9264960Google ScholarCross Ref
- Brandusescu, A., & Lämmerhirt, D. (2018). Open Data Charter Measurement Guide. Available at SSRN 3331162Google Scholar
- Vancauwenberghe, G. (2018). Assessing open data. Open Data Exposed, 149–172. https://doi.org/10.1007/978-94-6265-261-3_8Google ScholarCross Ref
- Zuiderwijk, A., Pirannejad, A., & Susha, I. (2021). Comparing open data benchmarks: Which metrics and methodologies determine countries’ positions in the ranking lists? Telematics and Informatics, 62, 101634. https://doi.org/10.1016/j.tele.2021.101634Google ScholarCross Ref
- Craveiro, G., Santana, M., Albuquerque, J.P.(2013) Assessing Open Government Budgetary Data in Brazil. In: ICDS 2013, The Seventh International Conference on Digital Society.Google Scholar
- Davies, T. (2014). Towards Common Methods for Assessing Open Data. World Wide Web Foundation. https://webfoundation.org/2014/06/towards-common-methods-for-assessing-open-data/Google Scholar
- Open Data Charter. (2013). Principles International Open Data Charter. https://opendatacharter.net/principles/Google Scholar
- Lnenicka, M., Luterek, M., & Nikiforova, A. (2022). Benchmarking open data efforts through indices and rankings: Assessing development and contexts of use. Telematics and Informatics, 66, 101745. https://doi.org/10.1016/j.tele.2021.101745Google ScholarDigital Library
- Shaharudin A.A.A. (2020) “Open Government Data: Principles, Benefits and Evaluations.” Discussion Paper. Kuala Lumpur: Khazanah Research Institute. http://www.krinstitute.org/Discussion_Papers-@-Open_Government_Data- ;_Principles,_Benefits_and_Evaluations.aspx.Google Scholar
- Shekhar, S and Padmanabhan, V. (2016) How to support the capacity of open data initiatives with assessment tools. Open Data Institute 2016 https://theodi.org/wp-content/uploads/2016/04/How-to-support-the-capacity-of-open-data-initiatives-with-assessment-tools.pdfGoogle Scholar
- Tinholt, D. (2013). The Open Data Economy. Unlocking Economic Value by Opening Government and Public Data. Retrieved from https://www.capgemini.com/wp-content/uploads/2017/07/the_open_data_economy_unlocking_economic_value_by_opening_government_and_public_data.pdfGoogle Scholar
- OKF, 2017. Global Open Data Index: Methodology. Retrieved from. http://index.okfn.org/methodology/Google Scholar
- W3C (2017a). Methodology of The Open Data Barometer – Leaders Edition. https://opendatabarometer.org/leadersedition/methodology/Google Scholar
- W3C (2017b). Open Data Barometer - Leaders Edition ODB Methodology - v1.0. http://opendatabarometer.org/doc/leadersEdition/ODB-leadersEdition-Methodology.pdfGoogle Scholar
- Carsaniga, G., Arriëns, E.N.L., Dogger, J. van Assen, M., Cecconi, G. (2022) Data Maturity Report 2022. data.europa.eu https://data.europa.eu/sites/default/files/landscaping_insight_report_n8_2022.pdfGoogle Scholar
- OECD (2018), “OECD Open Government Data Survey 4.0”, Unpublished, OECD, Paris.Google Scholar
- Lafortune, G., & Ubaldi, B. (2018). OECD 2017 OURdata Index: Methodology and Results (OECD Working Papers on Public Governance No. 30). https://doi.org/10.1787/2807d3c8-enGoogle ScholarCross Ref
- P´erez, J. A. R., & Emilsson, C. (2020). OECD Open, Useful and Re-usable data (OURdata) Index: 2019. Retrieved from https://www.oecd.org/governance/digitalgovernment/ourdata-index-policy-paper-2020.pdfGoogle Scholar
- Crowell, J., Swanson, E., 2020. Open Data Inventory 2020/21: Methodology Guide. Retrieved from. https://docs.google.com/document/d/1MBK0hN6MoQrii7_E1bmRXmsUcE8Fbb-Q32nxm8d8qTw/editGoogle Scholar
- Çaldağ, M. T., & Gökalp, E. (2022). The maturity of open government data maturity: a multivocal literature review. Aslib Journal of Information Management.Google Scholar
- Izdebski, W., Zwirowicz-Rutkowska, A., & Nowak da Costa, J. (2021). Open data in Spatial Data Infrastructure: The practices and experiences of Poland. International Journal of Digital Earth, 14(11), 1547–1560. https://doi.org/10.1080/17538947.2021.1952323Google ScholarCross Ref
- Wu, D., Xu, H., Yongyi, W., & Zhu, H. (2021). Quality of government health data in COVID-19: Definition and testing of an open government health data quality evaluation framework. Library Hi Tech, 40(2), 516–534. https://doi.org/10.1108/lht-04-2021-0126Google ScholarCross Ref
- Chokki, A. P., Alexopoulos, C., Saxena, S., Frénay, B., Vanderose, B., & Ali, M. (2022). Metadata Quality Matters in open government data (OGD) evaluation! an empirical investigation of OGD portals of the GCC constituents. Transforming Government: People, Process and Policy. https://doi.org/10.1108/tg-09-2022-0118Google ScholarCross Ref
- Penteado, B., Carlos, M. J., & Isotani, S. (2021). Process model with quality control for the production of high quality linked open government data. IEEE Latin America Transactions, 19(3), 421-429.Google ScholarCross Ref
- Raca, V., Velinov, G., Cico, B., Kon-Popovska, M. (2021a) Development framework of monitoring and evaluation of open government data quality, International Journal on Information Technologies and Security, 13(2): 87-100.Google Scholar
- Raca, V., Velinov, G., Cico, B., & Kon-Popovska, M. (2021b). Application-based framework for analysis, monitoring and evaluation of National Open Data Portals. International Journal of Advanced Computer Science and Applications, 12(11). https://doi.org/10.14569/ijacsa.2021.0121104Google ScholarCross Ref
- Raca, V., Velinov, G., Dzalev, S., & Kon-Popovska, M. (2022). A framework for evaluation and improvement of open government data quality: Application to the Western Balkans National Open Data Portals. SAGE Open, 12(2), 215824402211048. https://doi.org/10.1177/21582440221104813Google ScholarCross Ref
- Saxena, S. (2021). Invoking embeddedness and cybernetic theories for evaluating the National Open Government Data Portal of India. Information Discovery and Delivery, 49(4), 328–334. https://doi.org/10.1108/idd-09-2020-0107Google ScholarCross Ref
- Máchová, R., Hub, M., & Lnenicka, M. (2018). Usability evaluation of Open Data Portals. Aslib Journal of Information Management, 70(3), 252–268. https://doi.org/10.1108/ajim-02-2018-0026Google ScholarCross Ref
- Wang, D., Richards, D., Bilgin, A. A., & Chen, C. (2021). Advancing Open Government Data Portals: A Comparative Usability Evaluation Study. Library Hi Tech. https://doi.org/10.1108/lht-10-2020-0250Google ScholarCross Ref
- Mutambik, I., Almuqrin, A., Lee, J., Zhang, J. Z., Alomran, A., Omar, T., Floos, A., & Homadi, A. (2021). Usability of the G7 open government data portals and lessons learned. Sustainability, 13(24), 13740. https://doi.org/10.3390/su132413740Google ScholarCross Ref
- Ansari, B., Barati, M., & Martin, E. G. (2022). Enhancing the usability and usefulness of open government data: A comprehensive review of the state of open government data visualization research. Government Information Quarterly, 39(1) doi:10.1016/j.giq.2021.101657Google ScholarCross Ref
- Lnenicka, M., & Saxena, S. (2021). Re-defining open government data standards for smart cities’ websites: A case study of selected cities. Digital Policy, Regulation and Governance, 23(4), 398–411. https://doi.org/10.1108/dprg-12-2020-0174Google ScholarCross Ref
- Zhao, Y., Liang, Y., Yao, C., & Han, X. (2022). Key factors and generation mechanisms of open government data performance: A mixed methods study in the case of China. Government Information Quarterly, 39(4), 101717. https://doi.org/10.1016/j.giq.2022.101717Google ScholarCross Ref
- Habib, A., Prybutok, V. R., & Philpot, D. (2021). Assessing and building municipal open data capability. Information Systems and e-Business Management, 20(1), 1–25. https://doi.org/10.1007/s10257-021-00539-yGoogle ScholarDigital Library
- Park, J. J., Kim, E., & Jun, H. J. (2022). Development of an open government data (OGD) evaluation framework for BIM. Buildings, 12(4), 490. https://doi.org/10.3390/buildings12040490Google ScholarCross Ref
- Nikiforova, A., & McBride, K. (2021). Open government data portal usability: A user-centred usability analysis of 41 Open Government Data Portals. Telematics and Informatics, 58, 101539. https://doi.org/10.1016/j.tele.2020.101539Google ScholarCross Ref
- Bhandari, S., Ranjan, N., Kim, Y.-C., Park, J.-D., Hwang, K.-I., Kim, W.-H., Hong, Y.-S., & Kim, H. (2021). An automatic data completeness check framework for Open Government Data. Applied Sciences, 11(19), 9270. https://doi.org/10.3390/app11199270Google ScholarCross Ref
- Luo, Y., Tang, Z., & Fan, P. (2021). Could government data openness enhance urban innovation capability? an evaluation based on multistage did method. Sustainability, 13(23), 13495. https://doi.org/10.3390/su132313495Google ScholarCross Ref
- Miasayedava, L., McBride, K., & Tuhtan, J. A. (2022). Automated Environmental Compliance Monitoring of rivers with IOT and open government data. Journal of Environmental Management, 303, 114283. https://doi.org/10.1016/j.jenvman.2021.114283Google ScholarCross Ref
- Zhang, H., Bi, Y., Kang, F. and Wang, Z. (2022), "Incentive mechanisms for government officials' implementing open government data in China", Online Information Review, Vol. 46 No. 2, pp. 224-243. https://doi.org/10.1108/OIR-05-2020-0154Google ScholarCross Ref
- W3C(2011). Open Government Data Feasibility Studies. https://webfoundation.org/our-work/projects/open-government-data-feasibility-studies/Google Scholar
- van der Waal, S., Wecel, K., Ermilov, I., Janev, V., Milošević, U., & Wainwright, M. (2014). Lifting open data portals to the data web. In S. Auer, V. Bryl, & S. Tramp (Eds.), Linked Open Data – Creating Knowledge Out of Interlinked Data. Lecture Notes in Computer Science. (pp. 175–195). Springer International Publishing.Google ScholarCross Ref
- Sanchez-Nielsen, E., Morales, A., Mendo, O., & Chavez-Gutierrez, F. (2021). Sudama: stainable open government nagement framework for long-term publishing and consumption. IEEE Access, 9, 151841–151863. https://doi.org/10.1109/access.2021.3127472Google ScholarCross Ref
Recommendations
The Use of Open Government Data to Citizen Empowerment
ICEGOV '15-16: Proceedings of the 9th International Conference on Theory and Practice of Electronic GovernanceTo this date, governments, civil society organizations, and private sector representatives consider Open Government Data as a building block for open government, as they see it as a key enabler of improved service delivery, transparency, and public ...
The DaCapo benchmarks: java benchmarking development and analysis
OOPSLA '06: Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applicationsSince benchmarks drive computer science research and industry product development, which ones we use and how we evaluate them are key questions for the community. Despite complex runtime tradeoffs due to dynamic compilation and garbage collection ...
Open Government Portals Assessment: A Transparency for Accountability Perspective
EGOV 2013: Proceedings of the 12th IFIP WG 8.5 International Conference on Electronic Government - Volume 8074Dataset portals such as Data.gov and Data.uk.gov have become flagship initiatives of open government and open data strategies. These portals aim to fulfill the open government objectives of promoting re-use of public sector information to develop new ...
Comments