What Structured Crisis Exercise Review Can Measure

Measurement begins by defining the observed unit

A count of decisions, a participant rating and a communication log describe different aspects of an exercise. Combining them under the heading “crisis management quality” does not make them equivalent. Each requires a defined unit, a reference and an explanation of what the result means.

My work began with paper lists and coding of events and outward-facing actions. The practical attraction was straightforward: a traceable record could give an exercise debrief more substance than an overall impression. The methodological task was harder: determining which statements that record could support.

Explain the two coding dimensions

In my manuscript “Prototype-Based Evaluation of Communication Dynamics in Simulated Crisis Exercises (Germany): A DSR Approach” (Rico Kerstan, revised submission dated 12 May 2026), the illustrative communication coding distinguishes effective from proactive action. Effective means plausibly contributing to structuring, prioritizing or handling the scenario situation. Proactive means initiated in anticipation of emerging demands rather than merely responding to external prompts.

The unit is a communication event initiated by the crisis management team. Its time, sender, intended recipient and topic provide the basis for interpretation. The two questions are independent: an event may plausibly contribute without being proactive, or be proactive without being suitable for the problem. This is illustrative, scenario-dependent coding for after-action review, not a validated instrument or a set of comparable performance indicators.

If those categories are summarized as percentages, the denominator must be the explicitly defined set of coded team-initiated events under review. Each numerator counts the events assigned to the respective category. Missing or ambiguous records need to remain identifiable. The reasons for individual classifications must accompany the summary: a change can reflect different action, scenario conditions or coding. Closely connected communications may also serve the same assignment and must not silently become independent performance successes. Without these distinctions, “two percentages” explains neither the observation nor its significance.

Communication activity and decision work remain distinct

The prototype described in that manuscript captures call metadata, including timing, duration and participating endpoints. Its application demonstrated technical feasibility but also exposed a limitation: a call trace can show that contact was attempted without revealing the intended recipient or the purpose. More calls do not necessarily mean better coordination.

My separate conference manuscript “The Decision Corridor: An Exploratory Rubric for Reviewing Temporal Actionability in Crisis Management Team Decisions” (Rico Kerstan, exploratory conference manuscript dated 29 May 2026), therefore examines decision episodes in scenario context. It asks whether proactivity, scenario fit and targetedness are clearly secured. Its demonstration covers 83 episodes, of which 10 meet all three prompts. This is an exploratory review classification, not a validated measure of overall team performance.

Use the result for a specific debrief question

I recommend stating the observed unit and coding rule before presenting any aggregate. Select an episode that explains the pattern, reconstruct what was knowable and discuss the associated assignment. Participant perceptions can be compared with these observations without assuming that either is a complete performance measure.

This approach supports structured exercise review. It does not determine how much of a crisis outcome is causally attributable to the team rather than to other influences.

→ Experimental research: Insights → Crisis exercises as diagnostic instruments: Services