Skip to main content
SpringerLink
Log in

A fly in the ointment: an empirical study on the characteristics of Ethereum smart contract code weaknesses

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Context

Smart contracts are programs that are automatically executed on the blockchain. Code weaknesses in their implementation have led to severe loss of cryptocurrency. It is essential to understand the nature of code weaknesses in Ethereum smart contracts to prevent them in the future. Existing classifications are limited in several ways, e.g., in the breadth of data sources, and the generality of proposed categories.

Objective

We aim to characterize code weaknesses in Ethereum smart contracts written in Solidity, and provide an overview of existing classification schemes in relation to this characterization.

Method

We extracted code weaknesses in Ethereum smart contracts from two public coding platforms and two vulnerability databases and categorized them using an open card sorting approach. We devised a classification scheme of smart contract code weaknesses according to their error source and impact. Afterwards, we mapped existing classification schemes to our classification.

Results

The resulting classification consists of 11 categories describing the error source of code weaknesses and 13 categories describing potential impacts. Our findings show that the language specific coding and the structural data flow categories are the dominant categories, but that the frequency of occurrence differs substantially between the data sources.

Conclusions

Our findings enable researchers to better understand smart contract code weaknesses by defining various dimensions of the problem and supporting our classification with mappings with literature-based classifications and frequency distributions of the defined categories.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

The datasets generated during and/or analysed during the current study are available in the Zenodo repository, https://doi.org/10.5281/zenodo.6388179.

Notes

  1. https://www.coindesk.com/understanding-dao-hack-journalists

  2. https://swcregistry.io/

  3. https://cve.mitre.org/

  4. https://eips.ethereum.org/

  5. https://nvd.nist.gov/vuln/detail/CVE-2018-13783

  6. https://nvd.nist.gov/vuln/detail/CVE-2018-17968

  7. Ethereum corresponding cryptocurrency

  8. https://ethereum.org/en/

  9. https://docs.soliditylang.org/

  10. https://monax.io/

  11. https://cve.mitre.org/cve/

  12. http://nvd.nist.gov/

  13. https://www.securityfocus.com/

  14. http://cwe.mitre.org

  15. https://eips.ethereum.org/

  16. https://samate.nist.gov/BF/

  17. https://standards.ieee.org/standard/1044-2009.html

  18. https://scrapy.org

  19. The first author of this paper.

  20. An Ethereum token can represent anything, including lottery tickets, financial assets, a fiat currency like USD, an ounce of gold, etc.

  21. https://github.com/ethereum/EIPs/blob/master/EIPS/eip-150.md

  22. The actual gas costs are stated in the Solidity documentation and depend on numerous factors, such as the executed functions and the used data types.

  23. https://blog.openzeppelin.com/on-the-parity-wallet-multisig-hack-405a8c12e8f7/

  24. https://swcregistry.io/docs/SWC-122

  25. https://swcregistry.io/docs/SWC-121

  26. https://swcregistry.io/docs/SWC-117

  27. https://cve.mitre.org/docs/vuln-trends/index.html

  28. https://github.com/OpenZeppelin/openzeppelin-contracts/blob/master/contracts/utils/math/SafeMath.sol

  29. https://github.com/ethereum/EIP

References

  • Ahasanuzzaman M, Asaduzzaman M, Roy CK, Schneider KA (2016) Mining duplicate questions of stack overflow. In: 2016 IEEE/ACM 13th Working conference on mining software repositories (MSR), pp. 402–412. IEEE

  • Alharby M, Aldweesh A, van Moorsel A (2018) Blockchain-based smart contracts: A systematic mapping study of academic research (2018). In: 2018 International conference on cloud computing, big data and blockchain (ICCBB), pp 1–6. IEEE

  • Alharby M, Van Moorsel A (2017) Blockchain-based smart contracts: A systematic mapping study. arXiv preprint arXiv:1710.06372

  • Atzei N, Bartoletti M, Cimoli T (2016) A survey of attacks on ethereum smart contracts. IACR Cryptol. ePrint Arch. 2016:1007

    Google Scholar 

  • Atzei N, Bartoletti M, Cimoli T (2017) A survey of attacks on ethereum smart contracts (sok). In: International conference on principles of security and trust, pp 164–186. Springer

  • Ayman A, Aziz A, Alipour A, Laszka A (2019) Smart contract development in practice: Trends, issues, and discussions on stack overflow. arXiv preprint arXiv:1905.08833

  • Ayman A, Roy S, Alipour A, Laszka A (2020) Smart contract development from the perspective of developers: Topics and issues discussed on social media. In: Cryptography Financial, Security Data (eds) Bernhard M, Bracciali A, Camp LJ, Matsuo S, Maurushat A, Rønne PB, Sala M. Springer International Publishing, Cham, pp 405–422

    Google Scholar 

  • Badawi E, Jourdan GV (2020) Cryptocurrencies emerging threats and defensive mechanisms: A systematic literature review. IEEE Access

  • Bajaj K, Pattabiraman K, Mesbah A (2014) Mining questions asked by web developers. In: Proceedings of the 11th working conference on mining software repositories, pp 112–121

  • Bhat M, Vijayal S (2017) A probabilistic analysis on crypto-currencies based on blockchain. In: 2017 International conference on next generation computing and information systems (ICNGCIS), pp 69–74. IEEE

  • Calefato F, Lanubile F, Marasciulo MC, Novielli N (2015) Mining successful answers in stack overflow. In: 2015 IEEE/ACM 12th Working conference on mining software repositories, pp 430–433. IEEE

  • Chen C, Xing Z (2016) Mining technology landscape from stack overflow. In: Proceedings of the 10th ACM/IEEE international symposium on empirical software engineering and measurement, pp 1–10

  • Chen H, Pendleton M, Njilla L, Xu S (2020) A survey on ethereum systems security: Vulnerabilities, attacks, and defenses. ACM Comput Surv (CSUR) 53(3):1–43

    Article  Google Scholar 

  • Chen J, Xia X, Lo D, Grundy J, Luo X, Chen T (2020) Defining smart contract defects on ethereum. IEEE Trans Softw Eng

  • Cosentino V, Izquierdo JLC, Cabot J (2017) A systematic mapping study of software development with github. IEEE Access 5:7173–7192

    Article  Google Scholar 

  • Daian P (2016) Analysis of the dao exploit. Hacking, Distributed, p 6

    Google Scholar 

  • Dingman W, Cohen A, Ferrara N, Lynch A, Jasinski P, Black PE, Deng L (2019) Defects and vulnerabilities in smart contracts, a classification using the nist bugs framework. Int J Netw Distrib Comput 7(3):121–132

    Article  Google Scholar 

  • Durieux T, Ferreira JF, Abreu R, Cruz P (2020) Empirical review of automated analysis tools on 47,587 ethereum smart contracts. In: Proceedings of the ACM/IEEE 42nd international conference on software engineering, pp 530–541

  • Ethereum: Solidity by example (2021). https://docs.soliditylang.org/en/latest/solidity-by-example.html

  • Ethereum Improvement Proposals (EIPs): EIP-1470: Smart Contract Weakness Classification (SWC). (2020) https://github.com/ethereum/EIPs/issues/1469. [Online] Accessed 01 April 2020

  • Hewa T, Ylianttila M, Liyanage M (2020) Survey on blockchain based smart contracts: Applications, opportunities and challenges. J Netw Comput Appl 102857

  • Huang Y, Bian Y, Li R, Zhao JL, Shi P (2019) Smart contract security: A software lifecycle perspective. IEEE Access 7:150184–150202

    Article  Google Scholar 

  • Idelberger F, Governatori G, Riveret R, Sartor G (2016) Evaluation of logic-based smart contracts for blockchain systems. In: International symposium on rules and rule markup languages for the semantic web, pp 167–183. Springer

  • Kalra S, Goel S, Dhawan M, Sharma S (2018) Zeus: Analyzing safety of smart contracts. In: Ndss, pp 1–12

  • Khan SN, Loukil F, Ghedira-Guegan C, Benkhelifa E, Bani-Hani A (2021) Blockchain smart contracts: Applications, challenges, and future trends. Peer-to-peer Netw App 14(5):2901–2925

    Article  Google Scholar 

  • Khan ZA, Namin AS (2020) Ethereum smart contracts: Vulnerabilities and their classifications. In: 2020 IEEE International conference on big data (big data), pp 1–10. IEEE

  • Khan ZA, Namin AS (2020) A survey on vulnerabilities of ethereum smart contracts. arXiv preprint arXiv:2012.14481

  • Maiden N (2009) Card sorts to acquire requirements. IEEE Softw 26(3):85–86

    Article  Google Scholar 

  • Soud M, Liebel G, MH (2021) Dataset: A fly in the ointment: An empirical study on the characteristics of Ethereum smart contracts code weaknesses and vulnerabilities. https://doi.org/10.5281/zenodo.4441254. https://doi.org/10.5281/zenodo.6388179

  • Mavridou A, Laszka A, Stachtiari E, Dubey A (2019) Verisolid: Correct-by-design smart contracts for ethereum. In: International conference on financial cryptography and data security, pp 446–465. Springer

  • Mehar MI, Shier CL, Giambattista A, Gong E, Fletcher G, Sanayhie R, Kim HM, Laskowski M (2019) Understanding a revolutionary and flawed grand experiment in blockchain: the dao attack. J Case Inform Technol (JCIT) 21(1):19–32

    Article  Google Scholar 

  • Nawaz A (2012) A comparison of card-sorting analysis methods. 10th Asia Pacific conference on computer human interaction (Apchi 2012). Matsue-city, Shimane, Japan, pp 28–31

    Google Scholar 

  • Norvill R, Fiz B, State R, Cullen A (2019) Standardising smart contracts: Automatically inferring erc standards. In: 2019 IEEE International conference on blockchain and cryptocurrency (ICBC), pp 192–195. IEEE

  • Perez D, Livshits B (2019) Smart contract vulnerabilities: Vulnerable does not imply exploited. arXiv preprint arXiv:1902.06710

  • Permenev A, Dimitrov D, Tsankov P, Drachsler-Cohen D, Vechev M (2020) Verx: Safety verification of smart contracts. In: 2020 IEEE symposium on security and privacy (SP), pp 1661–1677. IEEE

  • Philosophy G (2019) Solidity by example. https://consensys.github.io/smart-contract-best-practices/general_philosophy/

  • Ponzanelli L, Bavota G, Di Penta M, Oliveto R, Lanza M (2014) Mining stackoverflow to turn the ide into a self-confident programming prompter. In: Proceedings of the 11th working conference on mining software repositories, pp 102–111

  • Praitheeshan P, Pan L, Yu J, Liu J, Doss R (2019) Security analysis methods on ethereum smart contract vulnerabilities: a survey. arXiv preprint arXiv:1908.08605

  • Ralph P (2018) Toward methodological guidelines for process theories and taxonomies in software engineering. IEEE Trans Softw Eng 45(7):712–735

    Article  Google Scholar 

  • Rameder H (2021) Systematic review of ethereum smart contract security vulnerabilities, analysis methods and tools. Ph.D. thesis, Wien

  • Rugg G, McGeorge P (1997) The sorting techniques: a tutorial paper on card sorts, picture sorts and item sorts. Expert Syst 14(2):80–93

    Article  Google Scholar 

  • Sakai R, Aerts J (2015) Card sorting techniques for domain characterization in problem-driven visualization research. In: EuroVis (Short Papers), pp 121–125

  • Samreen NF, Alalfi MH (2021) A survey of security vulnerabilities in ethereum smart contracts. arXiv preprint arXiv:2105.06974

  • Sánchez DC (2018) Raziel: Private and verifiable smart contracts on blockchains. arXiv preprint arXiv:1807.09484

  • Seacord RC, Householder AD (2005) A structured approach to classifying security vulnerabilities. Carnegie-mellon university pittsburgh pa software engineering inst, Tech. rep

    Book  Google Scholar 

  • Sillaber C, Waltl B, Treiblmaier H, Gallersdörfer U, Felderer M (2021) Laying the foundation for smart contract development: an integrated engineering process model. IseB 19(3):863–882

    Article  Google Scholar 

  • Soud, M., Qasse, I., Liebel, G., Hamdaqa, M (2022) Automesc: Automatic framework for mining and classifying ethereum smart contract vulnerabilities and their fixes. arXiv preprint arXiv:2212.10660

  • Spencer D (2009) Card sorting: Designing usable categories. Rosenfeld Media

  • Usman M, Britto R, Börstler J, Mendes E (2017) Taxonomies in software engineering: A systematic mapping study and a revised taxonomy development method. Inf Softw Technol 85:43–59

    Article  Google Scholar 

  • Vacca A, Di Sorbo A, Visaggio CA, Canfora G (2021) A systematic literature review of blockchain and smart contract development: Techniques, tools, and open challenges. J Syst Softw 174:110891

    Article  Google Scholar 

  • Vegas S, Juristo N, Basili VR (2009) Maturing software engineering knowledge through classifications: A case study on unit testing techniques. IEEE Trans Softw Eng 35(4):551–565

    Article  Google Scholar 

  • Viera AJ, Garrett JM et al (2005) Understanding interobserver agreement: the kappa statistic. Fam med 37(5):360–363

    Google Scholar 

  • Wan Z, Xia X, Lo D, Chen J, Luo X, Yang X (2021) Smart contract security: A practitioners’ perspective. In: 2021 IEEE/ACM 43rd International conference on software engineering (ICSE), pp 1410–1422. IEEE

  • Wood JR, Wood LE (2008) Card sorting: current practices and beyond. J Usability Stud 4(1):1–6

    Google Scholar 

  • Yamashita K, Nomura Y, Zhou E, Pi B, Jun S (2019) Potential risks of hyperledger fabric smart contracts. In: 2019 IEEE International workshop on blockchain oriented software engineering (IWBOSE), pp 1–10. IEEE

  • Zhang P, Xiao F, Luo X (2020) A framework and dataset for bugs in ethereum smart contracts. In: 2020 IEEE International conference on software maintenance and evolution (ICSME), pp 139–150. IEEE

  • Zheng Z, Xie S, Dai HN, Chen W, Chen X, Weng J, Imran M (2020) An overview on smart contracts: Challenges, advances and platforms. Futur Gener Comput Syst 105:475–491

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Mohammad Alsarhan, a security expert, for participating in the card sorting and inter-rater agreement discussions. This work was supported by the Icelandic Research Fund (Rannís) grant number 207156-051.

Author information

Authors and Affiliations

  1. Department of Computer Science, Reykjavik University, Reykjavik, Iceland

    Majd Soud, Grischa Liebel & Mohammad Hamdaqa

  2. Department of Computer and Software Engineering, Polytechnique Montreal, Montreal, Quebec, Canada

    Mohammad Hamdaqa

Authors
  1. Majd Soud
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Grischa Liebel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad Hamdaqa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Majd Soud.

Ethics declarations

Competing Interests

The authors declared that they have no conflict of interest.

Additional information

Communicated by: Xin Xia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Soud, M., Liebel, G. & Hamdaqa, M. A fly in the ointment: an empirical study on the characteristics of Ethereum smart contract code weaknesses. Empir Software Eng 29, 13 (2024). https://doi.org/10.1007/s10664-023-10398-5

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10664-023-10398-5

Keywords

Search

Navigation