1 Introduction

On-line hands-on cybersecurity competitions and courses are an effective way to educate students in the field of ICT and to raise security awareness [3, 19]. Today, cyber competencies and awareness are a must for all ICT students. Yet many ICT curricula contain perhaps only one theory-oriented course about security which is not sufficient today [11]. Developers need knowledge and skills to design secure information systems. System engineers need to know how to protect the system and how to act in an actual crisis. In order to study secure system design and programming we need a practical hands-on course. In order to train people to defend the system in a critical situation, a cybersecurity competition as part of the course can be a powerful educational tool [3].

We may ask why are practical hands-on cybersecurity courses and competitions not part of every ICT curriculum and why do most curricula contain only one course which is not of hands-on type? Even practical classes tend to be limited to designing and programming a secure system but do not cover live attacks and defense [19]. One possible reason for this is that creating a practical security course demands much more resources than a theoretical one, and it has to be updated every term due to rapid changes in the field. Moreover, if the course is designed by only a small group of people there may be not enough diverse competencies to build a practical course containing a live exercise. In addition, a practical and nontrivial cybersecurity course needs a realistic sandboxed information system with servers, services and workstations for each student or group. This makes such an approach more expensive compared to traditional courses.

Today we see data breaches every day, from banks to healthcare to popular websites to governmental institutions. Cyber security is a rapidly changing and growing field. We expect that every ICT specialist is security aware, being able to design secure systems and defend existing systems. The growing amount of malicious activities online causes increasing need for security experts - yet ICT specialists sometimes expect that their security department will find all problems in company’s systems by penetration testing and security event monitoring. However, the security problems cannot be solved by one department because security should be designed along with the system rather than added later.

We believe that every ICT specialist should be aware of cyber threats and able to avoid known security mistakes when developing a new system. Today, we use IT systems that were developed years ago with little security in mind, which means that those systems are exploitable by cyber criminals using common attack methods. Thus a skill of defending IT systems is needed in addition to knowledge of how to design a secure system.

Security awareness for ICT specialists should be an integral oart of their education - every ICT student needs to know how to design a secure system and how to defend existing systems against cyber criminals.

Our solution is an open-source virtual sandboxed simulator for practical cybersecurity classes and competitions. The platform is called i-tee (in Estonian it can be interpreted as ‘information way’ or ‘information path’) the main features of which are (a) automated grading/scoring for competitions, (b) automated attacks to simulate cyber criminals and malicious activities, (c) immediate feedback using a virtual teaching assistant, and (d) virtual computer user simulator to make the simulation more realistic [4].

The i-tee platform, released under the MIT license, is available in GithubFootnote 1. At the time of writing, it is in use at four institutions. The platform has also been tested in cybersecurity competitions in Estonia (24 students) and Moldova (21 students).

The aim of this paper is to provide one possible starting point for academics who want to integrate hands-on cyber security elements into existing courses without having to develop their own tools.

2 Background

As the role of cyber security is increasing, every ICT curriculum should provide necessary coverage of cyber field [11]. Usually the studies consist of lectures, practical classes and independent work. The distribution between practical classes and lectures varies but cyber security courses tend to be more theoretical and usually focus on the design and development of new system, this aspect also prevails in practical classes and homework. The traditional lecture-based approach is still dominant [19]. However, we feel that this approach is not suitable to teach the defense of IT systems which should rather focus on a simulation of an actual critical situation. It is possible to simulate cyber attacks using roleplay and groupwork (e.g. tabletop exercises) but defending complex IT systems is a skill that can be mastered only with hands-on, practical training. The hands-on approach has also proven an effective teaching method for science [17].

The approach used in the Cyber Defense Exercises (CDX) is an efficient way to study cyber security and increase information assurance awareness [3, 14]. Students often get bored when learning mere theory but the realistic element of cyber attacks that occurs in CDX provides excitement and demands full commitment from the students. However, using CDX as learning tool does not replace lectures or other learning approaches such as tutorials, projects, mentoring etc. However, using the CDX yields best results when combining different approaches [19]. Also, preparing for the exercise motivates students to learn as they see a practical output for the knowledge [1]. The CDX’es are widely used as teaching tools at universities [6] and private companies, e.g. SANS NetWars [13].

Some examples of cyber security exercises are Defcon’s Capture the Flag (CtF) [2], Cyber Defense Exercise (CDX) [14], International Capture the Flag Contest (iCTF) [18], Locked Shields [9] and Collegiate Cyber Defense Competition [16]. The number of exercises is constantly increasing - the European Union Agency for Network and Information Security has identified over 200 national and international cyber security exercises [10].

2.1 Types of Cyber Security Exercises

The exercises can feature different goals: develop capabilities; evaluate capabilities of individuals, organisations and systems; measure knowledge, ability, endurance and/or capacity; train the participants and provide an opportunity to gain knowledge, understanding and skills [10].

One possible taxonomy of exercises and distribution [10]:

  • Simulation – 35 %

  • Tabletop – 26 %

  • Workshop – 20 %

  • Red-team/Blue-team – 11 %

  • Drill – 3 %

  • Discussion based game – 2 %

  • Seminar – 1 %

  • Capture the flag – 1 %

  • Other – 3 %

Cyber security exercises may be individual (develop skills) or team competitions (develop skills and teamwork). Based on size they can be divided into

  • small-scale, suitable for students and universities, and

  • large-scale, such as international live exercises with complex infrastructure

Based on objectives, exercises may be divided into defense-oriented, offense-oriented and mixed approaches.

To promote hands-on experience and situation awareness we focus on the live exercises such as Red-team/Blue-team and Capture the Flag.

Red-team/blue-team type of exercises contain attacking (red) and defending (blue) teams. Attackers may compete with each other or just provide attacks for all blue teams.

In the Capture the Flag type of exercises, an offending team should gain control over the attacked system, acquiring a critical piece of information (known as the flag) as a proof of success. For example, a flag can be a database record, a passwords or its hash, or some files or hashes from the targetted file system. This type of exercises usually focuses on offensive but may also contain defensive parts as well - e.g. at iCTF where each team defends their own vulnerable network and tries to compromise other team’s networks. The vulnerabilities are same for all teams - thus, after discovering and mitigating security issues at their own systems, competitors can use the found vulnerabilities to attack other teams [18].

Various types of exercises have been used in academic setting: [7]

  • Defensive cyberexercise – defending a vulnerable infrastructure against Red-team attacks. Some examples are: Cyber Defense Exercise (CDE) [15], Locked Shields [9].

  • Small-scale, internal exercises – usually standalone rather than integrated into the curriculum.

  • National Capture-the-Flag exercise – may feature additional teams besides red and blue, such as the green team are responsible for providing and maintain an infrastructure of the game, the white team handling aspects of gameplay, scoring and rules, or the yellow team dealing with situation awareness and providing background information.

  • Semester-long class exercise – they are integrated into curricula and have learning objectives, but tend to be less competitive compared to the CTF type of exercises [7].

2.2 Typical Design of the Cyber Security Exercise

Designing cyber security exercise can be divided into seven steps: Objectives, Approach, Topology, Scenario, Rules, Metrics and Lessons Learned [6].

Objectives are defined according to the goal of the exercise: participants are able to implement security configurations and defend the systems; participants are able to find security vulnerabilities and improve penetration testing skills; students are able to perform reconnaissance and defense in depth approach; students are able to mitigate common web, network and system attacks etc. Design of objectives gives pedagogical value to the cyber exercise and allows reuse of existing work [5].

Approach can be either defense oriented; offense oriented or mixed [6]. In practice the mixed approach is preferred because knowing the attack methods gives advantage to defense. However, knowledge about attacks and tools do not provide skills and qualities to work under pressure in a stressful situation [9].

Topology is designed to support objectives of the exercise and consider technical capabilities of the platform used.

Scenario should support objectives, consider topology restrictions as well as provide a mission and an engaging story for the students.

Rules must address different aspects: who can participate - some exercises exclude students already working in the field of security; what methods are allowed for competitors; which parts are graded (e.g. functionality and uptime of services, successful attacks, recovery time etc.). Rules must be known to the participants.

Metrics and Lessons Learned include collection and processing of feedback, description of exercise system failures (in a complex environment some parts are likely to fail). A published post-activity report containing a Lessons Learned section can be a valuable source of information for designing next exercises. A good example of post-activity report is presented by the Locked Shields competition [9]. The metrics determine how to measure of effectiveness of the exercise and how are the objectives achieved [6].

2.3 Components of Competition Platform

We will look at common components of cyber security platforms and their usage in different exercises.

VM Provisioning. Cyber exercises are usually executed on dedicate virtualized platform to provide a sandbox for attacks and vulnerable systems. The tools used by students are common and used by cyber criminals as well. Therefore, a sandboxed isolated environment is a must for exercises. Some platforms use virtualization technologies, e.g. VMware vSphere used at Locked Shields [9], KVM was used at early stages of Locked Shields in early stages and VirtualBox was used at iCTF [18] and CyberOlympics [4].

Network Provisioning. In order to provide sandboxed network environments for each team the cyber exercise platform should able to configure several virtual networks for each team.

Attacking Systems. Red-team/Blue team exercises may use live attack traffic from a dedicated Red team, alternately all teams may perform offensive operations or attacks can be made by competition system itself.

Attacks must be provided for all teams/competitors in a coherent way to ensure equal chances, they must stay within the environment and not hit public hosts by accident.

Locked Shields uses a red team with more than 50 members to provide consistent traffic for all teams [9]. However, live fire from the red team can be expensive and for small grade exercises it can be replaced with automated attack engine [4].

Sometimes the attacking systems have a Command and Control module similar to the ones found in botnets [4].

Scoring. The exercise objectives should be graded by a scoring system. The iCTF used scoring mechanism called scorebot which periodically tests service functionality to ensure that mitigation methods used by teams did not break any services [18].

Network Traffic Generator. The system contains responsive, centrally managed network traffic generator to provide realistic operational experience during CDX’s and trainings. Realistic network traffic and user emulation in system is a important part of every CDX platform. Without any ‘background noise’ it would be too easy for the participants to isolate the attacking traffic.

2.4 Platforms for Exercises

Internet-Scale Event and Attack Generation Environment (ISEAGE) is a testbed that focuses on hands-on laboratory exercises. The ISEAGE provides a sandboxed environment enabling controlled attacks against students’ systems and networks [12]. The architecture of ISEAGE contains a background traffic generator, an attack module, an attack amplification module as well as a Command & Control module. It can be used in classroom and in cyber exercises, as some attack targets are publicly availableFootnote 2. However, as of 2015, the complete system is not freely available.

The Information Warfare Analysis and Research Laboratory Range (IWAR Range) is developed by US Military Academy and used in the IA curriculum at West Point. The Range uses a virtualized environment with base infrastructure of sample organization and several networks such as the attack network called a Gray Network, the research network called Black Network to support development of the Range and IA course, and the Gold Network for targets. The IWAR Range is isolated from outside networks. For virtualization a VMWare Workstation was used for the Gray Network [15].

The Locked Shields Range is developed by Estonian Defense Forces with NATO Cooperative Cyber Defence Centre of Excellence (NATO CCD COE) and used for the biggest annual international technical exercise called Locked Shields (over 400 participants from 16 NationsFootnote 3. The competition is designed for professional defenders of national critical IT infrastructure but Locked Shields has also a test instance where Cyber Security Master students act as blue team members; this is organized as an elective part of the curriculum at Tallinn University of Technology. For VM provisioning, the proprietary Virtual Lab Manager (VLM) is used to configure VMware VMs. Live red team is used to perform a wide range of attacks against 40 targets per team. The attackers are divided into network, client-side, web-applications, and SCADA teams [9]. No public repositories or competition designs are publicly available.

The Blunderdome platform is designed for teaching students using linear paths to break into a web application which simulates a grade management system, the goal for the students is to change their grade. The students need to break into a vulnerable Linux server. This system is designed to provide a symmetric and linear learning experience with one deterministic path for students [8]. The platform is not publicly available.

The SANS Net Wars is a proprietary platform with relevant content [13].

The survey from European Union Agency For Network And Information Security identified more then 200 different exercises [10]. Yet while the number of cyber exercise platforms can be in the same rank, to our knowledge no open source, publicly available platform with relevant cyber content exists.

3 Our Cyber Simulator Solution

In a student’s perspective, using the cyber simulator starts with login page in a web browser. After successful login, the student can choose a mission (in case of ordinary studies) or join a competition if a cyber competition event is opened. The student will read the scenario of the exercise and then start his/her own sandbox with VM-s and virtual networks. The student can log in to each workstation or server using Remote Desktop Protocol (RDP) included in MS Windows. For Mac users several RDP clients exist, such as Cord or Microsoft RDP client. Linux users can easily use rdesktop or xfreerdp.

Sometimes, access to the environment needs setting up a Virtual Private Network (VPN) [9, 14]. However, we decided that access to the environment should be as easy as possible and not require special software or settings at the students computer. Therefore, we believe that time needed to start with cyber mission for first time user should not exceed 15 min including viewing a video introduction.

When in exercise mode, the Scoreboard displays the current state of the services and historical count for states: service OK counter (green), service interruption counter (red), service vulnerable or in warning state (yellow) as seen on Fig. 1

The Leaderboard gives information about competition leaders - missions completed and score for the first 20 users as seen on Fig. 1. When the exercise starts, the objectives are opened according to the scenario and students can read detailed description of objectives and see the score in real time.

Fig. 1.
figure 1

The architecture of the system (Color figure online)

Full size image

Each mission contains a network topology, virtual machines and objectives that are graded automatically or based on feedback from students. For example, in the introductory mission called Angels and scooters the scoreboard automatically detects whether the objective Configure apache virtualhost is achieved, but in the mission of Hackerloo the system asks information that the student should obtain from a web application using a SQL injection vulnerability.

The web interface is built with Ruby on Rails. Users can be authenticated using LDAP, Active Directory or SAMBA4 services. For virtualization layer we use VirtualBox Headless due to its capability to provide console access using common RDP protocol. Low-level RDP access allows to design a scenarios where VM is broken and boots only from CD image; the student should fix the problem. For networking we use VirtualBox’s internal networks created when the mission starts [4].

Fig. 2.
figure 2

The control panel of botnets

Full size image

The scoring is done by a configurable botnet which performs checks for each objective opened. For competition missions the checks are switched on manually, using a control panel of botnets as seen on Fig. 2. When using non-competition mode missions, the scoring switched on automatically according to the mission scenario and the progress of the student.

For attacks a separate botnet is used because in most of the missions the students are allowed to block the attacker’s IP address using a firewall - if scoring and attacks were done by the same botnet with same addresses, students would be unable to use common practices to block the attackers. The nature of the attack depends on the mission.

Both botnets are controlled using a GUI application seen on Fig. 2 which uses a Fabric frameworkFootnote 4 to control bots and execute scripted scoring and attacks.

Virtual Machines are provisioned using custom script and configured using declarative configuration management via PuppetFootnote 5 with serverless setup.

This platform provides a responsive application level traffic generator that simulates real users and covers the attacks. The network traffic is initiated from hosts from several network segments. It uses an IRC-based and encrypted communication channel between bots and CnC server to receive commands and send status messages. The network traffic generator is tested on Linux (Ubuntu, Debian), Windows (XP, Vista, 8, 8.1, Server 2008, Server 2012, 10) but is designed to work on every platform supported by Python with YAML libraries.

Traffic generators are controlled by the central control server (CnC) and the amount of traffic is tunable during the exercise. The platform allows for simultaneous run of different labs (missions or virtual learning spaces) [4].

4 Discussion

Various sources (and also our experience) suggest that students are better motivated to learn cyber security via live exercises. Therefore, cyber exercises are increasedly used by universities [10]. We can just ask why aren’t they a part of every ICT curriculum - but as suggested above, designing a cyber exercise and virtual lab platform is expensive and time consuming task [3].

We tried to use different existing open source projects like OWASP WebCoatFootnote 6, Damn Vulnerable Web App (DVWA)Footnote 7 and OWASP HackademicFootnote 8 - even if we had some success, those projects are vulnerable targets and need a laboratory system to run them. We tried to find an open-source cyber exercise and competition platform that can be integrated into curriculum. However, such a system did not exist for the time being.

For raising security awareness and motivate students and lecturers we implemented a novel cyber security platform i-tee. The main contribution is the platform itself because it allows to implement cyber exercises into curriculum with relatively low effort compared to developing an exercise from ground up. We believe that the i-tee system can lower the cost and setup time to level that single instructor/lecturer/teaching assistant can install this platform and start using it as part of a semester-long class or one-time event.

To install and use the i-tee platform, a certain level of Linux knowledge is needed. We tested the installation guide on students and those who managed to complete basic Linux course were also able to install a new i-tee instance. For running a full cyber competition, the organizer will need additional knowledge about attacks and vulnerabilities implemented in the open source missions.

We developed several missions for i-tee and open sourced a sample mission implementation: “The Kingdom of Banania” is available from bitbucketFootnote 9 This mission is suitable for competitions, workshops and semester-long courses.

Mission: The Kingdom of Banania – According to the storyline, the participants are offered an internship in the Kingdom of Banania as a sysadmin of Bananian e-Government. They will discover that the websites are riddled with various well-known vulnerabilities which are already being actively exploited for different pranks and web defacement. The participants are then tasked to restore the websites and patch the vulnerabilities.

During the repairs the participants will discover that the website of the largest newspaper of Banania, bandemia.ban, displays ever more nonsensical news stories. The involved journalists ensure that they have never seen those (and some hint a possible political diversion by the neighbouring Empire of Pineapplia). Finally, King Bananius XII Magnus appoints all participants to clean up the situation. If successful, they are promised a lifetime supply of bananas from the Royal Banana Garden. Those who fail will instead face a lifetime of hard labour at the Banana Curvature Measurement Tool Calibration Office.

In addition, the scenario features a hacker group called Acronymous attacking all websites. The infrastructure of each student is seen on Fig. 3.

Fig. 3.
figure 3

Network topology of the mission

Full size image

The goals are restoration of disrupted systems and patching various security holes (every participant will have a small private network that will be constantly under attack during the competition; see Network topology for details).

The graded metrics are: availability and functionality of web applications; resistance for attacks; recovery time after successful attacks. This mission implements eleven learning objectives for example: student installs and configures proper web application firewall; student finds attacks from log files and implements security configurations for web servers; student are able to reconnaissance and enumerate all web applications and databases and backup them.

Vulnerabilities and attacks used include HeartBleed, SQL injection, XSS, ShellShock, DDOS, path traversal, weak security configuration.

5 Future Work

The main goal for the future is to build a community of academics who use i-tee platform and contribute missions and updates. We also plan to involve students in the content creating process and experiment with different learning methodologies. For community we plan to develop a web interface for content contribution and mission design.

6 Conclusions

Setting up a cyber exercise environment requires involvement of multiple people due to the time spent on tasks. Also, the needed skillset is rarely possessed by just one person.

We developed an open source live virtual simulator i-tee that can be integrated into existing curriculum. The time needed for downloading, installation and executing a cyber simulator i-tee is a fraction compared to the time it takes to design the whole system by themselves.

The i-tee can be used in online learning and for instructor lead classroom studies. The platform is suitable for different cyber exercise approaches (Red team/Blue team; Workshop and Capture the Flag). The platform was successfully tested on 10 events with more then 200 users and as of 2016, is used by three higher education institutes in Estonia. We raised security awareness amongst Estonian students and developed system allows other universities and countries do the same.

The platform allows to integrate practical hands-on learning approach to ICT curricula without developing a solution from scratch. We hope that the platform will get contributions from community, as it is open source and publicly available. In future we will experiment with different learning approaches and develop an automated virtual teaching assistant system for supporting students.