As AI makes many CI/MI analysis steps faster and cheaper, practitioners might reasonably ask themselves: is CI/MI turning into just an AI-prompting task, or will it be absorbed entirely by data science and automation?

That conclusion is understandable, but wrong. And the answer isn't in AI or prompt-engineering publications. It's in Jan Herring's work on Key Intelligence Topics. Identifying the actual intelligence need — asking the right question — is the ultimate prerequisite for good competitive and market intelligence. But it is only the first of three. The second is choosing the right method to answer that question. The third, easiest to skip under time pressure, is making sure a human can actually check that the AI performed the chosen method correctly, step by step, rather than trusting one fluent-sounding final answer. In high-stakes intelligence work, that third step is not optional.

1. Asking the Right Question

Herring's Key Intelligence Topics / Key Intelligence Questions (KIT/KIQ) process exists because intelligence failures are rarely a data problem — they are usually the result of answering a question nobody actually needed answered (Herring, 1999). His fix was procedural: interview the decision-maker, propose a formulation back, get corrected, and repeat, until the real need behind the request is on the table.

This matters more, not less, now that execution is cheap. AI can research "what is Competitor X planning" in minutes. But if the VP asking is really looking for ammunition to defend a market-entry decision already made — not a neutral competitor assessment — a fast, technically correct answer to the literal question is still the wrong deliverable. Speed only pays off once the question is right.

2. Choosing the Right Method

Getting the question right does not automatically get the method right. Two analysts can take the same, correctly diagnosed question — "how exposed are we to Competitor X" — and legitimately diverge: one optimizes for short-term defensibility, the other for long-term market share. Both produce valid answers to slightly different versions of the question, and the results can look contradictory even though neither analyst made an error.

This is where an AI tool is dangerous if left to its own devices. Asked a single open-ended prompt, a model will pick an implicit method and commit to it silently — it will analyze a volatile, adversarial market with the same fluent confidence it uses for a stable one, without flagging that the two situations call for different treatment. Choosing the method — and stating it — has to remain a deliberate, human decision. The AI's job is to execute the chosen method well, not to choose it.

3. Why Human-in-the-Loop Is Essential for High-Stakes Intelligence

A model's fluency is not a signal of correctness. If an error enters at an intermediate step — the wrong data source, an unstated assumption, a silently substituted method — it does not surface as a warning. It propagates invisibly into a confident, well-written final answer. For a routine, low-stakes task that risk is tolerable. For intelligence feeding a board decision, an M&A assessment, or a market-entry call, an undetected error at step two can drive a costly decision at step five, and unlike a human analyst, the model will not reliably flag its own uncertainty.

The practical fix is to decompose the analysis into the discrete steps of the method chosen in step 2, and to require that each step produce a checkable, human-legible output — the sources used, the comparison actually run, the assumption applied — rather than accepting one synthesized narrative at the end. This is not a CI-specific idea; it is now closer to regulatory doctrine for high-stakes AI use generally. The EU AI Act, for instance, requires that high-risk AI systems be built so that human overseers can understand what the system is doing, monitor it, and intervene, precisely because unchecked reliance on a system's output — automation bias — is itself treated as a foreseeable failure mode to be designed against (Regulation (EU) 2024/1689, Art. 14). Intelligence work that feeds high-stakes decisions deserves the same discipline, whether or not a regulator requires it.

Practical Takeaways

• Treat every incoming request as a starting point for an interview, not a finished brief — run Herring's KIT/KIQ elicitation before opening any tool.
• When two AI-assisted analyses disagree, check for a hidden difference in the underlying question or method before assuming an error.
• Name the method before you prompt, and prompt in steps — a single end-to-end prompt hides the method-selection decision instead of exposing it.
• For high-stakes outputs, require a checkable intermediate trail — sources, method, assumptions — not just a polished final answer.

References

Herring, J. P. (1999). Key intelligence topics: A process to identify and define intelligence needs. Competitive Intelligence Review, 10(2), 4–14. https://doi.org/10.1002/(SICI)1520-6386(199932)10:2%3C4::AID-CIR3%3E3.0.CO;2-C

Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act), Art. 14. https://eur-lex.europa.eu/eli/reg/2024/1689/oj