Built an autonomous system where 5 AI models argue about geopolitical crisis outcomes: Here's what I learned about model behavior

Key takeaways

• Multi-model consensus systems reveal significant disagreement (25+ points) between leading AI models on identical scenarios, with Grok showing bias toward OSINT signals
• Models anchor to their own previous outputs when shown historical context, requiring 'blind' operation to maintain independent reasoning
• Grounding/RAG prevents source hallucination but not content hallucination—models can fabricate specific data while correctly citing authoritative sources
• Named rules in prompts become reasoning shortcuts that models cite instead of performing actual analysis, degrading output quality
• 15-day continuous operation of autonomous multi-agent system provides real-world validation of ensemble AI approaches for complex forecasting