Add comprehensive orchestrator test case with complex multi-specialist scenario

ORCHESTRATOR_TEST_CASE.md

@@ -0,0 +1,336 @@
+# 🎼 Comprehensive Orchestrator Test Case
+
+## Test Scenario: Complex Multi-Drug Resistant Infection
+
+This test case demonstrates the orchestrator's ability to coordinate multiple specialist agents to solve a complex infectious disease case requiring expertise in multiple domains.
+
+---
+
+## 🎯 Setup: Create Your Agent Team
+
+### Step 1: Create Specialist Subagents
+
+Create these **4 specialist agents** first (they will be the orchestrator's team):
+
+#### Agent 1: Stewardship Specialist
+```
+Agent Type: 🎯 Specialist
+Agent Name: Stewardship Expert
+Mission: Expert in antimicrobial stewardship, antibiotic selection, de-escalation strategies, and optimizing duration of therapy
+Skills:
+  ✓ recommend_deescalation
+  ✓ alert_prolonged_antibiotic_use
+  ✓ search_pubmed (if you want literature support)
+```
+
+#### Agent 2: ID Diagnostician
+```
+Agent Type: 🎯 Specialist
+Agent Name: ID Diagnostician
+Mission: Expert in infectious disease diagnosis, differential diagnosis, culture interpretation, and diagnostic workup
+Skills:
+  ✓ differential_diagnosis
+  ✓ generate_ddx_mermaid
+  ✓ search_pubmed
+```
+
+#### Agent 3: ICU Consultant
+```
+Agent Type: 🎯 Specialist  
+Agent Name: ICU Sepsis Consultant
+Mission: Expert in critical care infectious diseases, sepsis management, hemodynamic support, and ICU-specific considerations
+Skills:
+  ✓ Any relevant tools you want
+  ✓ search_pubmed
+```
+
+#### Agent 4: Infection Prevention Specialist
+```
+Agent Type: 🎯 Specialist
+Agent Name: IPC Specialist
+Mission: Expert in infection prevention and control, isolation precautions, outbreak management, and transmission-based precautions
+Skills:
+  ✓ ipc_reporting (if available)
+  ✓ Any other relevant tools
+```
+
+### Step 2: Create the Orchestrator
+```
+Agent Type: 🎼 Orchestrator
+Agent Name: ID Maestro
+Mission: Coordinate multiple ID specialists to provide comprehensive infectious disease consultation for complex cases requiring multidisciplinary expertise
+Skills:
+  (Orchestrators have access to all your other agents automatically)
+API Key: Your OpenAI API key
+```
+
+---
+
+## 📋 Test Case: Complex Septic Patient
+
+### Patient Presentation:
+
+Copy and paste this detailed case to the **ID Maestro** orchestrator:
+
+```
+I need help with a complex case:
+
+Patient: 68-year-old male
+PMH: Type 2 diabetes (A1C 9.2%), ESRD on hemodialysis (MWF), recent hospitalization 3 weeks ago for MRSA bacteremia from infected dialysis catheter
+
+Current Presentation:
+- Day 3 ICU admission for septic shock
+- Fever 39.8°C, HR 118, BP 85/50 on norepinephrine 0.15 mcg/kg/min
+- WBC 24,000 with 18% bands, procalcitonin 8.5
+- Lactate 4.2 → 2.8 after 4L crystalloid
+
+Cultures:
+- Blood cultures (×2 sets): Gram-positive cocci in clusters at 14 hours
+- Preliminary: MRSA (same strain as 3 weeks ago)
+- Sensitivities pending (results in 24 hours)
+- Urine culture: Negative
+- CXR: Right lower lobe infiltrate
+
+Current Antibiotics (started 72 hours ago):
+- Vancomycin 1g IV q12h (trough pending)
+- Piperacillin-tazobactam 3.375g IV q6h
+
+Additional Info:
+- New tunneled dialysis catheter placed 2 weeks ago
+- Patient on contact precautions
+- Last vancomycin trough (from previous admission): 18 mcg/mL
+- CrCl: Not applicable (on dialysis)
+- Patient improving clinically: BP improving, lactate trending down
+
+Questions:
+1. Is current antibiotic coverage appropriate?
+2. Should we de-escalate or change therapy?
+3. What's the optimal duration?
+4. Any diagnostic workup needed?
+5. Are isolation precautions correct?
+6. What are the key stewardship considerations?
+```
+
+---
+
+## ✅ Expected Orchestrator Behavior
+
+### Phase 1: Planning
+The orchestrator should:
+1. **Analyze the request** and identify multiple distinct tasks
+2. **Create an execution plan** listing which agents to invoke:
+   - ID Diagnostician (for differential diagnosis and culture interpretation)
+   - Stewardship Expert (for antibiotic optimization and de-escalation)
+   - ICU Sepsis Consultant (for critical care considerations)
+   - IPC Specialist (for isolation precautions)
+3. **Display the plan** with numbered steps
+4. **Wait for your confirmation** ("proceed")
+
+### Phase 2: Execution (after you say "proceed")
+The orchestrator should:
+1. **Invoke each specialist agent** sequentially or in parallel
+2. **Show progress**: "🚀 Invoking [Agent Name]..."
+3. **Collect responses** from each specialist
+4. **Display intermediate results** as they come in
+
+### Phase 3: Synthesis
+The orchestrator should:
+1. **Synthesize all specialist inputs** into a comprehensive recommendation
+2. **Address all 6 questions** from the original query
+3. **Highlight agreements** between specialists
+4. **Resolve conflicts** if specialists disagree
+5. **Provide prioritized recommendations**
+6. **Include specific details** from each specialty area
+
+---
+
+## 📊 What to Look For
+
+### ✅ Good Orchestrator Performance:
+
+**Planning Phase:**
+- [ ] Identifies 4-6 distinct tasks in the case
+- [ ] Plans to invoke 3-4 specialist agents
+- [ ] Explains rationale for each agent selection
+- [ ] Asks for confirmation before proceeding
+
+**Execution Phase:**
+- [ ] Shows "Invoking [Agent Name]" messages
+- [ ] Displays each agent's response as it arrives
+- [ ] Maintains context between agent calls
+- [ ] Handles any agent errors gracefully
+
+**Synthesis Phase:**
+- [ ] Comprehensive final answer addressing all questions
+- [ ] Specific recommendations from each specialist:
+  - **Diagnostician**: Culture interpretation, workup recommendations
+  - **Stewardship**: De-escalation plan, duration recommendations
+  - **ICU Consultant**: Hemodynamic considerations, monitoring
+  - **IPC**: Isolation precautions, transmission prevention
+- [ ] Prioritized action items (urgent vs routine)
+- [ ] Clinical reasoning and evidence-based rationale
+
+### ❌ Signs of Issues:
+
+- Orchestrator doesn't invoke any subagents (just gives generic answer)
+- Shows execution plan but doesn't actually call the agents
+- Can't find subagents (error: "agent not found")
+- Synthesis doesn't incorporate subagent responses
+- Generic response without specialty-specific details
+
+---
+
+## 🔍 Detailed Expected Outputs
+
+### From ID Diagnostician:
+```
+Expected content:
+- MRSA bacteremia recurrence vs new infection
+- Differential diagnosis for persistent bacteremia
+- Recommendations for:
+  * Echo to rule out endocarditis
+  * Consider removing/replacing dialysis catheter
+  * Imaging for metastatic foci
+- Discussion of complicated vs uncomplicated bacteremia
+```
+
+### From Stewardship Expert:
+```
+Expected content:
+- Vancomycin optimization (check trough, AUC/MIC)
+- Piperacillin-tazobactam: likely unnecessary (can de-escalate)
+- Alternative options: daptomycin, ceftaroline, linezolid
+- Duration: 14 days for uncomplicated, longer if endocarditis
+- Monitoring: Weekly vancomycin troughs, renal function
+- De-escalation timeline: After sensitivities available
+```
+
+### From ICU Sepsis Consultant:
+```
+Expected content:
+- Hemodynamic status: Improving (decreasing vasopressor needs)
+- Source control: Consider catheter removal
+- Fluid resuscitation: Adequate (lactate improving)
+- Monitoring: Daily blood cultures until clearance
+- ICU-specific considerations: Dialysis timing with antibiotics
+- Prognosis: Good if source controlled
+```
+
+### From IPC Specialist:
+```
+Expected content:
+- Contact precautions: Appropriate for MRSA
+- Isolation duration: Until cultures negative
+- Staff education: Hand hygiene, PPE compliance
+- Cohorting considerations
+- Decolonization protocols (if recurrent MRSA)
+- Environmental cleaning protocols
+```
+
+### Orchestrator's Synthesis:
+```
+Expected structure:
+1. Summary of case (MRSA bacteremia, improving septic shock)
+2. Answers to each question with specialist input:
+   Q1: Coverage - vanc appropriate, pip-tazo can stop
+   Q2: De-escalation - stop pip-tazo, optimize vanc dosing
+   Q3: Duration - 14+ days depending on source control
+   Q4: Workup - echo, consider catheter removal
+   Q5: Isolation - contact precautions correct
+   Q6: Stewardship - multiple opportunities to optimize
+3. Prioritized action plan:
+   - Urgent (today): Echo, check vanc trough, stop pip-tazo
+   - Within 24h: Review sensitivities, daily blood cultures
+   - Ongoing: Monitor clinical response, consider catheter removal
+4. Key takeaways and follow-up plan
+```
+
+---
+
+## 🧪 Alternative Test Cases
+
+### Quick Test (Simpler):
+```
+"Patient with pneumonia needing antibiotic choice and stewardship guidance. 
+How should I treat and what's the optimal duration?"
+```
+
+### Complex Multi-System Test:
+```
+"78F with UTI, pneumonia, and C. diff. Multiple antibiotics on board. 
+Need help with antibiotic optimization, infection prevention, and diagnostic workup."
+```
+
+### Outbreak Scenario:
+```
+"3 patients in ICU with carbapenem-resistant Enterobacterales. 
+Need infection control measures, treatment options, and stewardship guidance."
+```
+
+---
+
+## 📝 Testing Checklist
+
+### Before Testing:
+- [ ] All 4 specialist agents created
+- [ ] Orchestrator agent created
+- [ ] NCBI_EMAIL and API keys set (for PubMed searches)
+- [ ] Browser ready in builder panel
+
+### During Test:
+- [ ] Paste complete case to orchestrator
+- [ ] Wait for execution plan
+- [ ] Type "proceed" to start execution
+- [ ] Watch for agent invocation messages
+- [ ] Note any errors or missing agents
+
+### After Test:
+- [ ] Review final synthesis
+- [ ] Verify all questions answered
+- [ ] Check if recommendations are actionable
+- [ ] Confirm specialty-specific details included
+- [ ] Test passed: ✅ or needs debugging: ❌
+
+---
+
+## 🐛 Troubleshooting
+
+### Problem: Orchestrator doesn't invoke subagents
+**Solution**: Verify you created the subagents first and they're visible in your "Active Agents" list
+
+### Problem: "Agent not found" errors
+**Solution**: Check agent names match exactly (case-sensitive). Recreate agents if needed.
+
+### Problem: Generic response without specialist details
+**Solution**: Orchestrator might not have access to subagents. Verify per-user isolation working correctly.
+
+### Problem: Execution plan created but agents not invoked
+**Solution**: Make sure to type "proceed" after the plan is shown
+
+### Problem: Only one agent invoked instead of multiple
+**Solution**: Case might not be complex enough. Use the detailed test case above.
+
+---
+
+## 🎓 Learning Objectives
+
+This test demonstrates:
+1. ✅ **Multi-agent coordination** - Orchestrator managing 4+ specialists
+2. ✅ **Complex reasoning** - Breaking down multifaceted clinical case
+3. ✅ **Information synthesis** - Combining multiple expert opinions
+4. ✅ **Conflict resolution** - Handling differing recommendations
+5. ✅ **Prioritization** - Urgent vs routine actions
+6. ✅ **Comprehensive coverage** - Addressing all aspects of care
+7. ✅ **Clinical applicability** - Actionable recommendations
+
+---
+
+## 📚 Documentation Reference
+
+- **Orchestrator Setup**: See `FIX_ORCHESTRATOR_SUBAGENTS.md`
+- **Agent Isolation**: See `AGENT_ISOLATION_COMPLETE.md`
+- **Testing Guide**: See `TEST_AGENT_ISOLATION.md`
+
+---
+
+**Ready to test?** Start by creating your 4 specialist agents, then the orchestrator, then paste the complex case! 🎼🔬