Skip to main content

Cybersecurity Education and Formal Methods

  • Conference paper
  • First Online:
Formal Methods – Fun for Everybody (FMFun 2019)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1301))

Included in the following conference series:

Abstract

Formal methods have been largely thought of in the context of safety-critical systems, where they have achieved major acceptance. Tens of millions of people trust their lives every day to such systems, based on formal proofs rather than “we haven’t found a bug” (yet!); but why is “we haven’t found a bug” an acceptable basis for systems trusted with hundreds of millions of people’s personal data?

This paper looks at some of these issues in cybersecurity, and the extent to which formal methods, ranging from “fully verified” to better tool support, could help. More importantly, recent policy reports and curricula initiatives appear to recommended formal methods in the limited context of “safety critical applications”; we suggest this is too limited in scope and ambition. Not only are formal methods needed in cybersecurity, the repeated and very public weaknesses of the cybersecurity industry provide a powerful motivation for formal methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The precise definition of cybersecurity is debatable: we can take is as failures of security, generally defined as “preserving the CIA—Confidentiality, Integrity and Availability” of digital information, where computer system played a critical part in the failure.

  2. 2.

    Generally called “Marriott”, but in fact due to the Starwood chain before Marriott took it over.

  3. 3.

    In military parlance, Equifax is being found not to have “defence in depth”. Defence in depth is certainly valuable: [8] described how Google was saved from the consequences of an ‘awesome’ attack on gmail by defence in depth. But the front line is still the first defence: in this case correct code.

  4. 4.

    [16]: “Midsize and small companies frequently install WAFs just to satisfy a compliance requirement. They don’t really care about practical security, and obviously won’t care about maintaining their WAF.” This is backed up by [14], whose survey states “43% run their WAF in alert-only mode!”.

  5. 5.

    Actually, Embedded Systems are a comparatively neglected, but important, cybersecurity area. See, for example, [32] for a description of a pervasive design fault in the “home security” market.

  6. 6.

    A point made in the context of XP and Agile in 2004 [33].

References

  1. Jacquel, M., Berkani, K., Delahaye, D., Dubois, C.: Verifying B proof rules using deep embedding and automated theorem proving. In: Barthe, G., Pardo, A., Schneider, G. (eds.) SEFM 2011. LNCS, vol. 7041, pp. 253–268. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24690-6_18

    Chapter  MATH  Google Scholar 

  2. Bloomberg: Equifax Hack Lasted for 76 Days, Compromised 148 Million People, Government Report Says (2018). http://fortune.com/2018/12/10/equifax-hack-lasted-for-76-days-compromised-148-million-people-government-report-says/

  3. Irwin, L.: Marriott downgrades severity of 2018 data breach: 383 million customers affected (2019). https://www.itgovernance.co.uk/blog/marriott-downgrades-severity-of-2018-data-breach-383-million-customers-affected

  4. Ford, N.: Medical debt collection agency files for bankruptcy protection after data breach (2019). https://www.itgovernance.co.uk/blog/medical-debt-collection-agency-files-for-bankruptcy-protection-after-data-breach

  5. The Guardian: BA faces & #x00A3;183m fine over passenger data breach (2019). https://www.theguardian.com/business/2019/jul/08/ba-fine-customer-data-breach-british-airways

  6. Royal Society: Progress and research in cybersecurity: supporting a resilient and trustworthy system for the UK (2016). http://royalsociety.org/cybersecurity

  7. United States Government Accountability Office: Actions Taken by Equifax and Federal Agencies in Response to the 2017 Breach (2018). https://www.gao.gov/assets/700/694158.pdf

  8. Osborne, C.: Google patches ‘awesome’ XSS vulnerability in Gmail dynamic email feature (2019). https://www.zdnet.com/article/google-patches-awesome-xss-vulnerability-in-gmail/

  9. Lenart, L.: Security Bulletin S2-045 (2017). https://cwiki.apache.org/confluence/display/WW/S2-045

  10. Open Web Application Security Project (OWASP): The Ten Most Critical Web Application Security Risks (2017). https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project#tab=Main

  11. Livshits, V., Lam, M.: Finding security vulnerabilities in Java applications with static analysis. In: Proceedings USENIX Security Symposium, pp. 271–286 (2005)

    Google Scholar 

  12. McGraw, G.: Software Security—Building Security In. Addison-Wesley, Boston (2006)

    Book  Google Scholar 

  13. Payment Card Industry Security Standards Council (PCI SSC): Requirements and Security Assessment Procedures Version 3.2.1 (2018). https://www.pcisecuritystandards.org/documents/PCI_DSS_v3-2-1.pdf

  14. Ponemon Institute: The State of Web Application Firewalls. Ponemon Institute (2019)

    Google Scholar 

  15. Krebs, B.: What We Can Learn from the Capital One Hack (2019). https://krebsonsecurity.com/tag/capital-one-breach/

  16. Kolochenko, I.: Web Application Firewall: a must-have security control or an outdated technology? (2016). https://www.csoonline.com/article/3032743/web-application-firewall-a-must-have-security-control-or-an-outdated-technology.html

  17. Barth, B.: No fly-by-night operation: Researchers suspect Magecart group behind British Airways breach (2018). https://www.scmagazine.com/home/security-news/no-fly-by-night-operation-researchers-suspect-magecart-group-behind-british-airways-breach/

  18. Klein, G., et al.: seL4: formal verification of an OS kernel. In: Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, pp. 207–220 (2009)

    Google Scholar 

  19. The Guardian: Hacking risk leads to recall of 500,000 pacemakers due to patient death fears (2017). https://www.theguardian.com/technology/2017/aug/31/hacking-risk-recall-pacemakers-patient-death-fears-fda-firmware-update

  20. Newman, L.: Hackers Made an App That Kills to Prove a Point (2019). https://www.wired.com/story/medtronic-insulin-pump-hack-app

  21. Evans, M., Loftus, P.: Rattled by Cyberattacks, Hospitals Push Device Makers to Improve Security (2019). https://www.wsj.com/articles/rattled-by-cyberattacks-hospitals-push-device-makers-to-improve-security-11557662400

  22. Food and Drug Administration: FDA Informs Patients, Providers and Manufacturers About Potential Cybersecurity Vulnerabilities in Certain Medical Devices with Bluetooth Low Energy (2020). https://www.fda.gov/news-events/press-announcements/fda-informs-patients-providers-and-manufacturers-about-potential-cybersecurity-vulnerabilities-0

  23. Heiser, G.: What’s new in the world of seL4 (2019). https://archive.fosdem.org/2019/schedule/event/world_of_sel4/

  24. Rescorla, E.: The Transport Layer Security (TLS) Protocol Version 1.3. RFC-8446 (2018)

    Google Scholar 

  25. Beck, K., et al.: The Agile Manifesto (2001). http://agilemanifesto.org/

  26. Blodget, H.: Mark Zuckerberg on Innovation (2009). https://www.businessinsider.com/mark-zuckerberg-innovation-2009-10

  27. Lane, A.: Security + Agile = FAIL (2018). https://securosis.com/assets/library/presentations/Security/AgileFAIL_OWASP.ppt_.pdf

  28. Bartsch, S.: Practitioners’ perspectives on security in agile development. In: International Conference on Availability Reliability and Security, pp. 479–484 (2011)

    Google Scholar 

  29. van der Heijden, A., Broasca, C., Serebrenik, A.: An empirical perspective on security challenges in large-scale agile software development. In: Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2018, pp. 45:1–45:4. ACM, New York (2018)

    Google Scholar 

  30. Tahaei, M., Vaniea, K.: A Survey on Developer-Centred Security (2019). https://groups.inf.ed.ac.uk/tulips/papers/A_Survey_on_Developer_Centred_Security.pdf

  31. Chapman, R.: Industrial experience with Agile in high-integrity software development. In: Parsons, M., Anderson, T. (eds.) Developing Safe Systems: Proceedings of the Twenty-fourth Safety-critical Systems Symposium, Safety-Critical Systems Club, pp. 143–154 (2016)

    Google Scholar 

  32. O’Connor, T., Enck, W., Reaves, B.: Blinded and confused: uncovering systemic flaws in device telemetry for smart-home Internet of Things. In: Proceedings of the 12th Conference on Security and Privacy in Wireless and Mobile Networks, pp. 140–150 (2019)

    Google Scholar 

  33. Wäyrynen, J., Bodén, M., Boström, G.: Security engineering and eXtreme programming: an impossible marriage? In: Zannier, C., Erdogmus, H., Lindstrom, L. (eds.) XP/Agile Universe 2004. LNCS, vol. 3134, pp. 117–128. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-27777-4_12

    Chapter  Google Scholar 

  34. Statt, N.: Zuckerberg: ‘Move fast and break things’ isn’t how Facebook operates anymore (2014). https://www.cnet.com/news/zuckerberg-move-fast-and-break-things-isnt-how-we-operate-anymore/

  35. Salz, R.: Software engineering and OpenSSL is not an oxymoron (presentation at Real World Cryptography 2017) (2017). https://rwc.iacr.org/2017/Slides/rich.saltz.pdf

  36. Seggelmann, R., Tuexen, M., Williams, M.: Transport Layer Security (TLS) and Datagram Transport Layer Security (DTLS) Heartbeat Extension (2012). https://tools.ietf.org/html/rfc6520

  37. Brain, M., Schanda, F.: A lightweight technique for distributed and incremental program verification. In: Joshi, R., Müller, P., Podelski, A. (eds.) VSTTE 2012. LNCS, vol. 7152, pp. 114–129. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-27705-4_10

    Chapter  MATH  Google Scholar 

  38. Chapman, R., Moy, Y.: AdaCore Technologies for Cyber Security (2018). https://www.adacore.com/books/adacore-tech-for-cyber-security

  39. Distefano, D., Fähndrich, M., Logozzo, F., O’Hearn, P.: Scaling static analyses at Facebook. Commun. ACM 62, 62–70 (2019)

    Article  Google Scholar 

  40. Vogels, W.: Proving security at scale with automated reasoning (2019). https://www.allthingsdistributed.com/2019/05/proving-security-at-scale-with-automated-reasoning.html

  41. Sadowski, C., Aftandilian, E., Eagle, A., Miller-Cushion, L., Jaspan, C.: Lessons from building static analysis tools at Google. Commun. ACM 61(4), 58–66 (2018)

    Article  Google Scholar 

  42. Open Web Application Security Project (OWASP): DefectDojo: OpenSource Application Security Management (2019). https://www.defectdojo.org

  43. Chapman, R.: Development and Formal Verification of Secure Updates for Embedded Systems (slides from Verification 2018) (2018). http://www.testandverification.com/conferences/verification-futures/vf2018/

  44. Google (Project Zero): 0day “In the Wild” (2019). https://googleprojectzero.blogspot.com/p/0day.html

  45. Thomas, G.: A proactive approach to more secure code (2019). https://msrc-blog.microsoft.com/2019/07/16/a-proactive-approach-to-more-secure-code/

  46. Centre for the Protection of National Infrastructure: Rail Code of Practice for Security-Informed Safety. CPNI (2019)

    Google Scholar 

  47. ISO/IEC: TS 17961:2013, Information technology—Programming languages, their environments & system software interfaces—C Secure Coding Rules (2013). https://www.iso.org/standard/61134.html

  48. Chong, N., et al.: Code-level model checking in the software development workflow. In: ICSE-SEIP 2020 (2020, to appear)

    Google Scholar 

  49. Cavalcanti, A., Miyazawa, A., Wellings, A., Woodcock, J., Zhao, S.: Java in the safety-critical domain. In: Bowen, J.P., Liu, Z., Zhang, Z. (eds.) SETSS 2016. LNCS, vol. 10215, pp. 110–150. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-56841-6_4

    Chapter  Google Scholar 

  50. Meng, N., Nagy, S., Yao, D., Zhuang, W., Arango Argoty, G.: Secure coding practices in Java: challenges and vulnerabilities. In: 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE), pp. 372–383 (2018)

    Google Scholar 

  51. Google (Chris Povirk): Denial of Service vulnerability for servers that use Guava and deserialize attacker data (2018). https://groups.google.com/forum/#!topic/guava-announce/xqWALw4W1vs/discussion

  52. Guarnieri, S., Livshits, B.: GATEKEEPER: mostly static enforcement of security and reliability policies for JavaScript code. In: USENIX Security Symposium, vol. 10, pp. 76–85 (2009)

    Google Scholar 

  53. Meyerovich, L., Livshits, B.: ConScript: specifying and enforcing fine-grained security policies for JavaScript in the browser. In: 2010 IEEE Symposium on Security and Privacy, pp. 481–496. IEEE (2010)

    Google Scholar 

  54. Maffeis, S., Mitchell, J.C., Taly, A.: Isolating JavaScript with filters, rewriting, and wrappers. In: Backes, M., Ning, P. (eds.) ESORICS 2009. LNCS, vol. 5789, pp. 505–522. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04444-1_31

    Chapter  Google Scholar 

  55. Maffeis, S., Taly, A.: Language-based isolation of untrusted JavaScript. In: Proceedings 22nd IEEE Computer Security Foundations Symposium, pp. 77–91 (2009)

    Google Scholar 

  56. Kotowicz, K.: Trusted Types help prevent Cross-Site Scripting (2019). https://developers.google.com/web/updates/2019/02/trusted-types

  57. Cable, J.: Every Computer Science Degree Should Require a Course in Cybersecurity (2019). https://hbr.org/2019/08/every-computer-science-degree-should-require-a-course-in-cybersecurity

  58. Brown, N.C.C., Sentance, S., Crick, T., Humphreys, S.: Restart: the resurgence of computer science in UK schools. ACM Trans. Comp. Sci. Edu. 14(2), 1–22 (2014). https://doi.org/10.1145/2602484

    Article  Google Scholar 

  59. Davenport, J.H., Crick, T., Hourizi, R.: The institute of coding: a university-industry collaboration to address the UK’s digital skills crisis. In: Proceedings of IEEE Global Engineering Education Conference, pp. 1400–1408. IEEE Press (2020). https://doi.org/10.1109/EDUCON45650.2020.9125272

  60. Davenport, J.H., Hayes, A., Hourizi, R., Crick, T.: Innovative pedagogical practices in the craft of computing. In: Proceedings of 4th International Conference on Learning and Teaching in Computing and Engineering (2016). https://doi.org/10.1109/LaTiCE.2016.38

  61. Murphy, E., Crick, T., Davenport, J.H.: An analysis of introductory programming courses at UK universities. Art Sci. Eng. Prog. 1(2), 18 (2017). https://doi.org/10.22152/programming-journal.org/2017/1/18

    Article  Google Scholar 

  62. Crick, T., Davenport, J.H., Hanna, P., Irons, A., Prickett, T.: Overcoming the challenges of teaching cybersecurity in UK computer science degree programmes. In: Proceedings of 50th Annual Frontiers in Education Conference, IEEE Press (2020). https://doi.org/10.1109/FIE44824.2020.9274033

  63. Crick, T., Davenport, J., Irons, A., Prickett, T.: A UK case study on cybersecurity education and accreditation. In: Proceedings of FIE 2019 (2019)

    Google Scholar 

  64. Crick, T., Davenport, J.H., Hanna, P., Irons, A., Pearce, S., Prickett, T.: Repositioning BCS degree accreditation. ITNOW 62(1), 50–51 (2020). https://doi.org/10.1093/itnow/bwaa023

    Article  Google Scholar 

  65. Naiakshina, A., Danilova, A., Tiefenau, C., Smith, M.: Deception task design in developer password studies: exploring a student sample. In: Fourteenth Symposium on Usable Privacy and Security (SOUPS 2018), pp. 297–313. USENIX Association (2018)

    Google Scholar 

  66. Naiakshina, A., Danilova, A., Gerlitz, E., von Zezschwitz, E., Smith, M.: “If you want, I can store the encrypted password”: a password-storage field study with freelance developers. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 140:1–140:12. ACM (2019)

    Google Scholar 

  67. Cimpanu, C.: 7-Eleven Japanese customers lose \$500,000 due to mobile app flaw (2019). https://www.zdnet.com/article/7-eleven-japanese-customers-lose-500000-due-to-mobile-app-flaw/

  68. Biscoe, C.: MyFitnessPal data breach: 150 million app users affected (2018). https://www.itgovernance.co.uk/blog/myfitnesspal-data-breach-150-million-app-users-affected/

  69. Blackmon, A.: Macy’s hit by data breach (2018). https://eu.freep.com/story/money/business/2018/07/06/macys-data-breach-online/763074002/

  70. Inbenta (CEO): Inbenta and the Ticketmaster Data Breach (2018). http://web.archive.org/web/20181121184620/

  71. Taylor, C., Sakharkar, S.: ’);DROP TABLE textbooks;–: an argument for SQL injection coverage in database textbooks. In: Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE 2019), pp. 191–197. ACM (2019)

    Google Scholar 

  72. Fischer, F., et al.: Stack overflow considered harmful? The impact of copy&paste on Android application security. In: 38th IEEE Symposium on Security and Privacy (SP), pp. 121–136 (2017)

    Google Scholar 

  73. Open Web Application Security Project (OWASP): The Ten Most Critical Web Application Security Risks (2013). https://www.owasp.org/images/f/f8/OWASP_Top_10_-_2013.pdf

  74. Fischer, F., et al.: Stack overflow considered helpful! Deep learning security nudges towards stronger cryptography. In: 28th USENIX Security Symposium (USENIX Security 2019), pp. 339–356 (2019)

    Google Scholar 

  75. Chen, M., Fischer, F., Meng, N., Wang, X., Grossklags, J.: How reliable is the crowdsourced knowledge of security implementation? https://arxiv.org/abs/1901.01327 (2019)

  76. Zhang, M., Meng, W., Lee, S., Lee, B., Xing, X.: All Your Clicks Belong to Me: Investigating Click Interception on the Web (2019). https://www.microsoft.com/en-us/research/uploads/prod/2019/03/zhang-observer.pdf

  77. Austin, A., Williams, L.: One technique is not enough: a comparison of vulnerability discovery techniques. In: Proceedings 2011 International Symposium on Empirical Software Engineering and Measurement, pp. 97–106 (2011)

    Google Scholar 

Download references

Acknowledgements

A predecessor of this paper was given at the 2019 Working Formal Methods Symposium (FROM2019) in Timișoara, Romania. The authors are grateful to the referees and audiences of FROM2019 and FMFun2019 for useful comments. The first author is grateful to the Fulbright Programme for a Cybersecurity Scholarship at New York University in 2017, and to many correspondents and discussions, notably Alastair Irons, Tom Prickett and Tim French. This paper was partially supported by the Institute of Coding, which received £20m of funding from the Office for Students (OfS), as well as support from the Higher Education Funding Council for Wales (HEFCW).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to James H. Davenport .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Davenport, J.H., Crick, T. (2021). Cybersecurity Education and Formal Methods. In: Cerone, A., Roggenbach, M. (eds) Formal Methods – Fun for Everybody. FMFun 2019. Communications in Computer and Information Science, vol 1301. Springer, Cham. https://doi.org/10.1007/978-3-030-71374-4_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-71374-4_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-71373-7

  • Online ISBN: 978-3-030-71374-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics