Friction as a Feature: Why AI-Driven Delphi Panels Demand Intentional Resistance

by Thorsten Bill
Guess what AI researchers are borrowing to deal with uncertainty, unreliable AI output and hallucinations? 
A method CI/MI practitioners already know: Delphi.

The idea is tempting. Large Language Models (LLMs) could make Delphi panels much faster and easier to scale.

Traditional Delphi panels are slow. They require several rounds of expert input and take considerable effort to organize. They are also vulnerable to analyst fatigue.

Recent research shows that LLMs can reproduce parts of the Delphi process. They can work with expert personas, run multiple rounds, and produce forecasts that can be compared with human panels.

But there is a problem.

In Competitive and Market Intelligence (CI/MI), speed can become a trap. Fast consensus can create an illusion of certainty.

So, can we trust an AI-driven Delphi panel?

Not by default. It needs engineered friction.

The Illusion of Scale: Persona vs. Evidence

A typical approach is easy to imagine.

Ask one LLM to act as a Competitor CEO. Give another the role of a Regulatory Analyst. A third becomes a Supply Chain Expert. Then ask them to discuss the issue and produce a common forecast.

It looks like a diverse expert panel. But are the perspectives really independent?

Recent multi-agent research points to an important limitation. If the agents work from the same underlying information, adding more agents does not necessarily add more knowledge.

Changing the persona does not create true diversity. The result can be a very convincing echo chamber.

Engineering Friction into the Intelligence Pipeline

The job of CI/MI is not to produce agreement. It is to test assumptions, expose blind spots and reduce decision risk.

That requires deliberate friction.

• Data-Routing over Persona Prompting: Give agents different evidence. One might work with company filings. Another could examine regulatory sources. A third could focus on technical publications. Different evidence creates the possibility of genuinely different conclusions.
• Red Teaming & Defeaters: Ask agents to attack the emerging conclusion. Look for the conditions that would make the hypothesis fail. This connects directly to structured analytic techniques and Assurance 2.0.
• The Math of Confidence: Consider a chain of seven conjunctive components. To reach 95% confidence in the overall claim, each component would need to be about 99.27% confident under a simple independence assumption. Small weaknesses can therefore have a large effect on the final conclusion.
• Human-in-the-Loop Validation: LLMs can lose context, propagate errors and rely on outdated information. Analysts still need to assess the evidence and make the final judgment.

The Takeaway for CI/MI Leadership

Can we trust an AI Delphi panel? Only as a structured sparring partner, never as an autonomous oracle.

AI can make Delphi much easier to scale. That is useful. But the goal should not be effortless consensus.

The real value comes from testing assumptions, challenging conclusions and finding evidence that could prove the analysis wrong.

Friction is not a bug in the intelligence process. It is part of the quality control.

Learn More

Want to learn more about the Delphi method and its role in strategic market analysis and intelligence?

Explore the ICI-31 Workshop: Strategic Market Analysis & Intelligence:

References

• Mueller, R. M., Thoring, K., Klöckner, H. W., & Larsen, K. R. (2024). Crafting Future Scenarios with the Help of AI: Potentials of a Hybrid Delphi Expert Panel. In T. X. Bui (Ed.), Proceedings of the 57th Annual Hawaii International Conference on System Sciences (HICSS 2024) (pp. 6458–6467). IEEE Computer Society. https://doi.org/10.24251/HICSS.2024.774
• Bertolotti, F., & Mari, L. (2025). An LLM-based Delphi Study to Predict GenAI Evolution. arXiv:2502.21092. https://arxiv.org/abs/2502.21092
• Barrett, S., Fox, P., Krook, J., Mondal, T., Mylius, S., & Tlaie, A. (2025). Assessing Confidence in Frontier AI Safety Cases. arXiv:2502.05791. https://arxiv.org/abs/2502.05791
• Lorenz, T., & Fritz, M. (2026). Scalable Delphi: Large Language Models for Structured Risk Estimation. arXiv:2602.08889. https://arxiv.org/abs/2602.08889