Whitepaper: Human-in-the-Loop Decomposer

An Operational Design Pattern for AI-Assisted Strategic Intelligenceby Thorsten Bill

Executive Summary / Abstract

In The Fortune-Teller Dilemma: Mastering CI/MI in the Age of AI, we exposed the primary risk of using Large Language Models (LLMs) for strategic intelligence: conversational fluency masks methodology failure, and polished answers are routinely mistaken for analytical truth. Because open-world market strategy lacks a pre-execution test bench, delegating reasoning to non-deterministic AI models exposes decision-makers to unexamined hallucinations, cognitive bias, and a total void of human accountability.

Building directly on The Fortune-Teller Dilemma, this white paper presents the Human-in-the-Loop (HITL) Decomposer: an operational framework that bridges AI safety governance and traditional intelligence tradecraft. Aligning with foundational principles of cognitive bias mitigation (Heuer, 1999; Pherson & Heuer, 2020), the framework segregates Mechanical Steps (verifiable computations delegated to AI) from Judgment-Laden Steps (context-dependent calls owned strictly by human analysts).

By categorizing decisions based on risk and reversibility (low-stakes iteration vs. high-stakes, irreversible bets), this paper provides:

  • A 4-point prerequisite checklist for enterprise AI methodology selection.
  • A standardized 5-column classification architecture for auditing any Structured Analytic Technique (SAT).
  • Concrete criteria for designing actionable, human-executable verification guardrails.
  • An executable, production-ready system prompt that decomposes SATs while enforcing human ownership over strategic judgment.

The resulting framework transforms black-box LLM outputs into an audited, defensible decision trail required for high-stakes, board-level strategic choices.

Prerequisite Checklist

Context Alignment: This framework assumes you have already identified and customized the appropriate SAT for your specific CI/MI problem. As established in The Fortune-Teller Dilemma, selecting and adapting a methodology to fit your internal workflows is itself the critical first HITL checkpoint.

Avoid the Legacy Trap: Do not fall into the optimization trap of merely AI-enabling the legacy techniques you have relied on for decades. Because AI collapses the cost and effort of mechanical steps toward zero, the economic math of methodology selection has fundamentally changed. Highly complex, hyper-rigorous SATs (or heavy variants) that were previously discarded due to their massive operational overhead are suddenly fully viable, and entirely new, AI-native analytic frameworks are beginning to emerge.

Build for High-Stakes, Irreversible Decisions: This framework is engineered specifically for high-consequence, low-reversibility intelligence, such as evaluating an M&A target, entering a heavily regulated market, or responding to a systemic black-swan threat. While low-stakes, reversible decisions (where a 90% success rate across fast iterations is acceptable) can be safely left to fully autonomous AI, high-stakes bets require human accountability and audited, step-by-step verification.

The Accountability Boundary: Autonomous AI platforms create a "responsibility void": models cannot be fired, sued, or held legally liable. Delegating reasoning to a black box isn't automation; it is an abdication of fiduciary duty. By enforcing explicit human ownership over judgment (Column 4) and verifiable guardrails (Column 5), this framework transforms black-box outputs into an audited, defensible decision trail when board-level accountability is required.

The Core Principle: The Mechanical vs. Judgment Boundary

Any Structured Analytic Technique, whether Porter’s Five Forces, Analysis of Competing Hypotheses (ACH), PESTLE, or Scenario Planning, is comprised of discrete steps. To use AI safely, every technique must be decomposed along a strict functional boundary:

  • Mechanical Steps (Delegate & Verify): Steps whose correctness can be verified against a rule, computation, or external data source without requiring situated knowledge of your specific business or market. Examples include brainstorming raw candidate lists, clustering datasets, or computing pairwise consistency matrices.
  • Judgment-Laden Steps (Human-Owned): Steps whose validity depends on domain expertise, organizational context, strategic intent, and risk tolerance. Examples include defining system boundaries, weighting factor plausibility, and evaluating source credibility or deceptive intent.

The Decision Heuristic:
Could you verify this step's output against an external rule or computation without needing to know this specific business, market, or decision?

  • Yes: Mechanical. Safe to draft or compute automatically if it yields an inspectable artifact.
  • No: Judgment-Laden. Must be made, justified, and owned explicitly by the human analyst.
  • Unsure? Classify as Judgment-Laden. Overclaiming a judgment call as mechanical is how unexamined guesses hide inside polished tables.

The 5-Column Classification

When auditing or executing an AI-assisted SAT, every step must map cleanly across five standard columns:

  1. Step: The official step name in the method's own terminology.
  2. What it does: A one-sentence, purely functional description.
  3. AI's Role (Mechanical): The part of the step safe to compute or draft automatically, yielding an inspectable artifact.
  4. Judgment AI Can't Supply: The situated, domain-specific calls that require human context.
  5. Guardrail: One concrete, specific check a human performs on this exact step's output.

Guardrail Quality Check:

  • Incorrect Guardrail: "Review the output for accuracy." (Too generic; catches nothing).
  • Correct Guardrail: "Spot-check 3 pairwise consistency ratings against analyst raw notes." (Concrete, actionable human verification).

Execution Instructions

  1. Copy the system prompt below.
  2. Paste it into your AI session.
  3. Click the button or copy the text to trigger decomposition:

Why It Works

While engineered to tackle the epistemic risks of AI in strategic analysis, this framework aligns directly with the tradecraft principles pioneered by Heuer (1999) and Pherson & Heuer (2020). It operationalizes Heuer’s foundational mandate, that analytical reasoning must be decomposed and externalized to prevent cognitive overconfidence, by establishing a strict functional boundary between AI-driven mechanical computation and human-owned contextual judgment.

Executable System Prompt

[SYSTEM INSTRUCTION: Human-in-the-Loop Decomposer for Structured Analytic Techniques]

Role: You are an expert Competitive Intelligence & Market Intelligence (CI/MI) methodology assistant, AI researcher, and prompt engineer familiar with the capabilities and limitations of frontier AI models. Your task is to decompose any Structured Analytic Technique (SAT) into a precise Human-in-the-Loop (HITL) classification table based on "The Fortune-Teller Dilemma" framework.

CORE METHODOLOGY RULES:
1. Procedural rigor is not epistemic rigor.
2. A step is MECHANICAL only if its output can be verified against a rule, computation, or external data source WITHOUT needing situated business context.
3. A step is JUDGMENT-LADEN if its correctness depends on domain expertise, organizational context, or risk tolerance.
4. If in doubt, classify as JUDGMENT-LADEN.
5. Every mechanical step must produce an inspectable, checkable artifact (e.g., raw matrix, source list).
6. Do not merge mechanical computation and final judgment into a single row. If a step contains both, split it into sub-steps.
7. COMPLIANCE & FORMATTING MANDATE: You must generate every step of the specified SAT completely. Do not combine rows, summarize, abbreviate, or truncate Column 5 (Guardrails) to save output tokens.

INTERACTION FLOW:

PHASE 1: INTAKE & OPTIONAL CONTEXT
Ask the user:
1. "Which Structured Analytic Technique (SAT) would you like to decompose?" (e.g., Porter's Five Forces, ACH, PESTLE, Scenario Analysis)
2. "(Optional) What is the specific business decision or market context for this analysis? (If left blank, I will generate a generalized draft.)"

PHASE 2: EXECUTION & OUTPUT
Once the user responds, produce a table with EXACTLY these five columns in order:
1. Step (Name in official method terminology)
2. What it does (1 sentence, functional description only)
3. AI's role (mechanical) (Safe to compute/draft; outputs inspectable artifact)
4. Judgment AI can't supply (Requires domain/business context)
5. Guardrail (Concrete, specific human check: never just "review it")

PHASE 3: MANDATORY AUDIT & META-WARNING
At the bottom of the table, you MUST print this exact notice block:
--------------------------------------------------------------------------------
⚠️ HUMAN-IN-THE-LOOP (HITL) META-AUDIT MANDATE:
This decomposition table is an AI-generated proposal, NOT a finished methodology.
- You (the analyst) must review and sign off on this step classification before using it.
- You personally OWN every judgment call listed in Column 4.
- You must EXPLICITLY EXECUTE the checks in Column 5 before finalizing any analysis.
--------------------------------------------------------------------------------
- If context was provided: "Tailored draft generated based on provided business context."
- If NO context was provided: "⚠️ GENERALIZED DRAFT WARNING: Because no specific business context was provided, Column 4 contains generic judgment categories. You must adapt these judgment calls for your specific strategic situation."

What to Watch for in the AI's Output

Model Requirement: Use a frontier AI model (e.g., Claude 3.5 Sonnet, GPT-4o, Gemini 1.5 Pro, or equivalent).

When reviewing the classification table generated by the model, watch for these common failure modes:

  • Vague Guardrails: A "guardrail" that merely says "verify" or "double-check" is worthless. Force the model to specify how to check it (e.g., "Spot-check 3 pairwise consistency ratings against analyst notes").
  • Overclaimed Mechanical Steps: Steps marked mechanical where the guardrail actually describes a qualitative judgment call.
  • Suspiciously Short Judgment Columns: Real judgment steps are usually where the AI has the least to say. A brief Column 4 is an indicator of human ownership, not a gap to fill.
  • Fluent Confidence: The AI will draft the table with total fluency in every cell, regardless of accuracy. Fluency is never the signal to trust.

Enforcing Step-by-Step (CoT) Execution & Intercept Checkpoints

Never allow an LLM to execute an entire Structured Analytic Technique from start to finish in a single, un-gated generation. Doing so collapses mechanical processing and contextual judgment into an un-auditable narrative black box.

Necessary but not sufficient: To support genuine Human-in-the-Loop governance, design the AI-assisted SAT as a stepwise, artifact-based process governed by explicit checkpoints and human approval gates. Prompt-level STOP commands can structure the interaction, but enforcement should be provided by the surrounding workflow or application.

  • Sequential stepwise execution: Instruct the model to execute strictly one analytical step at a time. Require it to externalize the analytical workflow by presenting the relevant inputs, sources, assumptions, transformations, uncertainties, and mechanical artifacts before moving forward. Do not require disclosure of private chain-of-thought.
  • Mandatory STOP Commands: Program explicit [STOP & WAIT FOR ANALYST] directives at the end of every mechanical phase. The AI must halt generation completely until you inspect the output and authorize the next step.
  • Artifact Inspection Checkpoints: Require the AI to present raw, unpolished intermediate artifacts at each pause (e.g., raw source extraction tables, pairwise consistency matrices, or explicit assumption lists).
  • Human Judgment Injection: Use the pause to apply Column 4 (Judgment) and Column 5 (Guardrails). The human analyst reviews the intermediate artifact, signs off on or adjusts the underlying logic, and manually inputs the situated

Remember: The generated decomposition table is itself an AI-drafted proposal. By enforcing step-by-step pauses and demanding inspectable intermediate artifacts, you retain complete operational control over every link in the analytical chain.

References

 

Connect

Call me  ­   Newsletter  ­   j  YouTube  ­   s  LinkedIn

Log in