Crisis Exercises: Measure, Don't Just Play

Start with the capability to be examined

An exercise can end with operations restored in the scenario and still leave its learning objective unresolved. The team may have benefited from an accommodating inject, an assumed resource or exercise control’s intervention. To interpret the result, the exercise lead needs to specify which capability the exercise was intended to develop and what observations would show its use.

Learning objectives are more specific than a scenario label. “Manage a cyberattack” names the event. “Prioritize conflicting recovery demands and assign responsibility for implementation” identifies work that can be observed. That distinction determines both scenario design and evaluation.

Relate the scenario to the objective

Decision clarity, role discipline, prioritization and communication under ambiguity matter across different crisis types. Their application nevertheless depends on the scenario. Prioritization during a supply interruption involves different constraints from prioritization during a personnel shortage. Likewise, a ransomware scenario creates different technical and time-related demands from a supply-chain disruption. The exercise must give the team a realistic opportunity to demonstrate the capability being examined.

I recommend linking each learning objective to a small set of observable questions. Did the team explain its priorities? Were assignments directed to roles able to act? Did it revisit an assumption when contradictory information arrived? The debrief then examines episodes that answer those questions rather than treating exhaustion or a satisfactory ending as evidence of progress.

Describe the demands placed on the team

Comparisons between exercises also require a description of their demands. Two useful planning dimensions are the number of parallel problem clusters and the density of injects over time. Several simultaneous problems place different coordination demands on a team from a single developing problem. Ten injects in two hours allow less processing time than ten in six hours.

These dimensions do not exhaust exercise difficulty. Familiarity, information quality, dependencies and available personnel also matter. An index based on problem clusters and inject density can document selected design features; it should not silently become a universal difficulty score.

Use evidence that explains the finding

My manuscript “Prototype-Based Evaluation of Communication Dynamics in Simulated Crisis Exercises (Germany): A DSR Approach” (Rico Kerstan, revised submission dated 12 May 2026) distinguishes capturing communication activity from interpreting its strategic relevance. Call counts and durations make activity traceable. They do not establish whether the team addressed the right problem or reached the intended recipient.

Transparent criteria make both qualitative observations and numerical summaries more useful. A documented episode can explain a coordination failure without being converted into a score. Where exercise runs are compared, the report should identify changes in the conditions as well as changes in the team’s work.

For the next exercise, I recommend agreeing the learning objectives, scenario demands and observation tasks together. That gives the debrief a clear basis for selecting improvements and the next exercise a specific question to revisit.

→ Crisis exercise design: Services → Methodological foundation: Approach