TY - JOUR
AU - Claire La Fleur
AU - Blaine Hoffman
AU - C. Benjamin Gibson
AU - Norbou Buchler
AB - A critical component to any modern cybersecurity endeavor is effective use of its human resources to secure networks, maintain services and mitigate adversarial events. Despite the importance of the human cyber- analyst and operator to cybersecurity, there has not been a corresponding rise in data-driven analytical approaches for understanding, evaluating, and improving the effectiveness of cybersecurity teams as a whole. Fortunately, cyber defense competitions are well-established and provide a critical window into what makes a cybersecurity team more or less effective. We examined data collected at the national finals and four regional events of the Collegiate Cyber Defense Competition and posited that experience, access to simulation-based training, and functional role composition by the teams would predict team performance on four scoring dimensions relevant to the application of information assurance skills and defensive cyber operations: (a) maintaining services, (b) help-desk customer support, (c) handling scenario injects, and (d) mitigating red team attacks. Bayesian analysis highlighted that experience was a strong predictor of service availability, scenario injects, and red team defense. Simulation training was also associated with good performance along these scoring dimensions. High-performing and experienced teams clustered with one another based on the functional role composition of team skills. These results are discussed within the context of stages of team development, the efficacy of challenge-based learning events, and reinforce previous analytical results from cyber competitions.
BT - Elsevier BV
DA - 2021-05
DO - 10.1016/j.cose.2021.102229
N1 - Organizations are increasingly dependent upon teams of cybersecurity professionals to defend their networked resources and services. The objective of our research is to examine what makes a cybersecurity team more or less effective, which is currently not well understood. Overall, our findings are consistent with work on team development (Tuckman, 1965; Kozlowski and Bell, 2003) and our previous work at Collegiate Cyber Defense Competitions (Buchler et al. 2018) by demonstrating that simulation training and functional role specialization are associated with successful performance for many cyber-defense teams across the competitions. In our Bayesian analysis models, these emerged as critical predictors of performance in the team competition. These results clearly demonstrate the value of strategically composed and experienced teams and training pipelines that involve the application of cyber skills as ’hands-on keyboard’ simulation- based exercises (Crichigno et al., 2019).Any attempt to address the problem of data-driven team optimization needs to understand the human information workflows and specific contexts of cybersecurity operations. This requires a mix of high-level qualitative approaches to capture the work context, and low-level data-driven quantitative approaches needed for empirical model building.
N2 - A critical component to any modern cybersecurity endeavor is effective use of its human resources to secure networks, maintain services and mitigate adversarial events. Despite the importance of the human cyber- analyst and operator to cybersecurity, there has not been a corresponding rise in data-driven analytical approaches for understanding, evaluating, and improving the effectiveness of cybersecurity teams as a whole. Fortunately, cyber defense competitions are well-established and provide a critical window into what makes a cybersecurity team more or less effective. We examined data collected at the national finals and four regional events of the Collegiate Cyber Defense Competition and posited that experience, access to simulation-based training, and functional role composition by the teams would predict team performance on four scoring dimensions relevant to the application of information assurance skills and defensive cyber operations: (a) maintaining services, (b) help-desk customer support, (c) handling scenario injects, and (d) mitigating red team attacks. Bayesian analysis highlighted that experience was a strong predictor of service availability, scenario injects, and red team defense. Simulation training was also associated with good performance along these scoring dimensions. High-performing and experienced teams clustered with one another based on the functional role composition of team skills. These results are discussed within the context of stages of team development, the efficacy of challenge-based learning events, and reinforce previous analytical results from cyber competitions.
PY - 2021
T2 - Elsevier BV
TI - Team performance in a series of regional and national US cybersecurity defense competitions: Generalizable effects of training and functional role specialization
UR - https://www.sciencedirect.com/science/article/pii/S0167404821000535
SN - 0167-4048
ER -