Best Practices and Guidelines for Cyber Security Training - version 1.3c
Purpose
The purpose of this document is to provide common best practices and guidelines for Cyber Security Training. It is established as part of a collaborative project within the cyber security training sector in Denmark and supported by NCC-DK (The National Coordination Center for Cyber Security), co-funded by The European Union.
Background and partners
The main partners behind the project are ICS Range (icsrange.com), SagaLabs (sagalabs.dk), Campfire Security (campfiresecurity.com) and Dansk IT (dit.dk). However, we encourage all providers and other stake holders within cyber security training to contribute to the guidelines.
We are proud to work together on a set of common guidelines. We believe that when Danish providers of cyber security training work together, we both improve the level of cyber security training in Denmark – eventually making Denmark more cyber secure – and we improve the competitiveness and quality of the Danish cyber security training sector.
About the guidelines
The guidelines are not meant as a checklist to follow, but rather a set of best practices and guidelines to consider when designing cyber security training. Not all practices/guidelines are relevant for all aspects of cyber security training, and there can also be valid reasons for making different choices.
We hope the guidelines can serve as a quality mark for cyber security training made in Denmark, and we encourage providers of cyber security training to refer to the guidelines if they have been used.
In our context, we are mainly covering cyber security training for IT professionals, including software and security professionals. We do not aim to cover e.g. standard awareness training for non-IT employees, but focus on trainings that provides the participants with skills and/or competences in addition to naturally raining awareness. Of course everyone is welcome to gain inspiration from the document.
The guidelines and best practices are organized as a number of principles.
This is version 1.3 of the guidelines.
Principles
1. Explain the purpose
To make sure the learners are motivated and see the value in the training, it is crucial to explain the purpose from the beginning: Why is this training important for them (and/or their company), what will they learn, and how will it help them? For example, a person in the management role could learn about basic cyber attack techniques in order to (1) gain a better understanding of the cyber threat and (2) learn the terminology to communicate with more technical IT people – and eventually be able to collaborate across the organization to be better in preventing cyber attacks.
2. Explain the teaching methodology
A common question from learners is: “Why do we have to do this?”. It is always recommended to explain why you have designed the learning experience as you have. When using gamification and challenges it is particularly important that the learning goals are clear when developing the challenges, and that the learning goals and context is also clear for the learner – it might be fun, but it should be clear for the learning that it is more than that. For example, when learning how a dictionary attack works on a password login, the point is not to master dictionary attacks, but to understand how to create better password policies.
3. Active learning
Training should be based on active learning, rather than simply reading materials, or watching videos. Active learning not only improves retention and learning outcomes, but it is also more engaging and fun for the learners. Active learning can take many shapes and forms: Exercises, competitions, discussion sessions, and much more.
4. Taking into account different learning styles
There are different theories on learning styles, but the short summary is that different people learn best in different ways. This is something that should always be considered when designing learning materials. One models is that of Neil Fleming: The VAR/VARK model, which include four sensory modalities, i.e. visual learning, aural learning, reading/writing learning and kinesthetic learning – while also emphasizing that people might prefer a mix of these. When creating training materials, it is recommended to ensure that as many learning preferences as possible are accommodated - ideally by having some redundancy. For example, making sure that the same concept is explained both visually, by text and by practical assignments.
5. Use a well known framework for measuring learning outcomes
Academia often uses taxonomies such as Blooms taxonomy, which refers to knowledge, skills and competences. While this can be hard to use for shorter trainings – measured in hours or days rather than in months or years – it is recommended to use a well-known framework for formulating learning outcomes – for example the NICE framework, or variations of it.
6. Role based training – and also collaboration across roles
Training should be designed according to people’s different roles, so that the training is both relevant and fit the backgrounds of the participants and the challenges they experience. When working in teams during a training session or a course, the experience should ideally be designed so that people with different roles need to work together – this supports not only team building, but also the ability to collaborate across roles. For example, a challenge could require network knowledge to do network scanning and find a vulnerable service, whereas programming skills would be required to either exploit or patch the vulnerability. There is also a need for thinking across attack (red) and defense (blue) – as to master one side, you also need to understand the other. For this reason, platforms and courses should be designed to train for both sides.
7. Combining theory and practice
The best learning happens when theory and practice is combined, and in general we recommend breaking down the learning blocks in small parts: It of course depends on topics, but in general it is good to have practical/hands-on problems, experiments or cases every time a new tool, concept or method is introduced. This reinforces the learning of this new tool/concept/method and makes the learner ready to build further knowledge on top. People learn differently and have different preferences in terms of learning styles, so the exact pattern and mix of theory and practice will always depend on the individual. An example could be to build training paths with components, where each component contains (1) introduction/theory, (2) practical challenge and (3) reflection/feedback.
8. One thing at a time
For many learners who are new to cyber security, the initial learning steps can be overwhelming with many words, methods, concepts, and tools. Add on top that the learners also might need to accustom themselves to a new platform, including how to access the materials, different kind of cyber ranges or virtual labs, and point/scoring systems. We recommend that material is designed with this observation in mind, so the learning curve becomes manageable. On the other hand, it is also important that the learners feel they are learning something new: Therefore, it is recommended to think outside the “usual toolbox”. An example of an easy learning curve could be that the first challenge is merely to insert a flag given in the challenge description, the next is to find a flag on a simple website, and the next again to solve a simple challenge that does not require any tools. Thus, the learner will learn a small new step for every challenge, and based on this be able to solve more advanced tasks.
9. Ease of use
A learning platform or a cyber range should be easy and intuitive to use for the learner. This means that it should be simple for inexperienced users to access resources and navigate the platform or cyber range. If platforms become too unpolished or too complex, the learner may use more energy navigating the platform or cyber range, rather than obtaining the skills and knowledge planned. For example, it should be intuitive which steps to follow for a beginner – especially if the training is without (much) supervision.
10. ChatGPT (and other LLMs) safe challenges
ChatGPT and other Large Language Models are amazing for many purposes – including for learning. However, challenges and exercises need to be “GPT Safe” in the sense that they should not be solvable by merely providing the challenge description. It can be just too hard for participants to resist from cheating when easy points are in sight.
11. Collaborative learning
Learning together is always a great way of learning: Instead of solving exercises and problems alone, it is recommended to let learners work together: This often leads to good discussions which support the learning process, and situations where learners are helping each other is also good learning for all parties involved. Compared to working individually, it decreases the risk of being stuck and increases motivation and fun for the learner. As a bonus, learners get to know each other better. For most tasks, we recommend working in smaller groups of 2-4 learners, but this depends of course on the tasks at hand. Another way to motivate for collaborative learning would be to give an option of earning badges by helping others.
12. Gamification with a clear mission
Gamification is often setup as solving problems and getting points – or in the cyber security world to solve challenges and get flags – which then give points. This often works well, and it is impressive to see how much engagement and motivation gamification creates. However, the effects of gamification can be further boosted by working with missions and storylines instead of “just” getting flags or points. For example, building a storyline of infiltrating a malicious actor through their website might be more engaging for the learners than to simply find a website vulnerability (even though the actual task is the same). This principle is not to make the tasks more complex or less explained, but to build the right storylines where the goals are larger than simply getting a flag.
13. Clearly described tasks
In the cyber security community, there are many implicit understandings. When training people outside of this specific community, it is easy to get lost. A description of an exercise, where the learners were supposed to exploit a samba vulnerability, gain access to a server, and find a file on that server with the name flag.txt, once had the description “Have you ever tried dancing samba? Neither have we, but it might come in handy here”. With this, the learner would have no clue where to start and what to achieve. It might be a fun description for a CTF competition, where part of the competition is to decode that specific language, but it is not suitable in a training context. This principle depends a bit on how supervised the learners are: The more they are expected to work independently, the more important it is that descriptions are clear. It is important to explain “implicit” terminology, like 0-day, SIEM etc. – at least it should always be carefully considered to write out abbreviations in full.
14. Hints are important
While some level of frustration is okay, it is important not to be completely blocked without any possibility to advance. That is why hints are important. On the other hand, there is also a lot of learning in figuring things out, and much satisfaction in succeeding with this. This makes it non-trivial to provide the right hints – also, it can be difficult to automatize, since the hints ideally are depending on the individual learner (or group of learners) and their progress. AI and the use of Large Language Models is something that should be considered in this context, as it might be an efficient and scalable way to provide context-based hints.
15. A proper debrief
After finishing a course or learning lecture it is important to debrief and go through the learnings. This helps the learner retain what is learned. It is also a good opportunity to collect feedback for future improvements. Examples of debrief questions could be: What was the most important learnings for you from this course? Which new skills did you acquire? How can what you learned be applied in your everyday life, personally and professionally?
16. Making achievements visible
It is also important to equip the learners with visible proofs of their achievements such as certificates and/or badges. Not only does it give the user a more satisfactory experience, it also provides a valuable recognition for external purposes, and can be used on e.g. social media and in CVs.
17. Taking into account self-efficacy
Self-efficacy is a person’s belief in their ability to perform a task or achieve a goal, and plays an important role for both performance and motivation. Some people might start out with a low self-efficacy, making it important to give them positive experiences with their own achievements from the beginning. And for those starting out with a high self-efficacy, it is also important that they are able to accomplish their expectations. In general, it is just important that the learners can complete the tasks they start out with, instead of being stuck. A particular consequence is that mechanisms should be in place to help everybody starting out at the right level. Also, this related to having a hint system so people are not being stuck.
18. Real-world context and experience
Design training scenarios based on real-world cybersecurity threats and use cases, using current trends such as ransomware, supply chain attack, and cloud security challenges. This should help the student gain familiarity with concepts that are applicable to real threats. Ideally, the scenarios are as close to the everyday life of the participants as possible – including the use of same or similar systems, software and operating systems.
19. Training for the challenges of tomorrow
Cyber Security is moving fast forward, and threats and challenges evolve. It is important that trainings are updated, and that new trends are quickly incorporated into the training materials. For example, as LLM's have become more common place a new type of attack has appeared called "Training Data Poisoning". Even though this is a new concept, at the time of writing, it should be incorporated into the learning, so participants understand the underlying rules and dangers of the attack.
20. Training for incidents
Much training is focused on prevention and detection of cyber-attacks. However, it is equally important also to train for incident management, including aspects such as responsibilities, crisis management, communications and human burnouts.