# Graph-Driven Agentic System MVP

## Overview
An intelligent agent system that reads instructions from Neo4j, queries PostgreSQL databases, pauses for human review, and maintains a complete audit trail. The system demonstrates agentic workflow orchestration with human-in-the-loop controls.

## Key Features

🤖 **Autonomous Agent**: Processes instructions sequentially with configurable pause durations  
📊 **Graph Database**: All workflow metadata stored in Neo4j for complete traceability  
🔍 **Natural Language SQL**: Converts questions to SQL using LLM integration  
⏸️ **Human-in-the-Loop**: 5-minute pauses allow instruction editing during execution  
🎯 **Single API Gateway**: All Neo4j operations routed through MCP server  
📈 **Real-time Visualization**: Live workflow progress in browser interface  
🔄 **Complete Audit Trail**: Every action logged with timestamps and relationships  

## Architecture

```
┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   Frontend  │────│ MCP Server  │────│    Neo4j    │
│  (Next.js)  │    │ (FastAPI)   │    │  (Graph)    │
└─────────────┘    └─────────────┘    └─────────────┘
                           │
                    ┌─────────────┐    ┌─────────────┐
                    │    Agent    │────│ PostgreSQL  │
                    │  (Python)   │    │   (Data)    │
                    └─────────────┘    └─────────────┘
```

### Components

- **Neo4j**: Graph database storing workflows, instructions, and execution metadata
- **MCP Server**: Single gateway for all Neo4j operations with parameter fixing
- **Agent**: Executes instructions with configurable pause periods for human review
- **PostgreSQL**: Sample data source for testing SQL generation
- **Frontend**: React/Next.js chat interface with real-time workflow visualization

## Quick Start

### Prerequisites
- Docker & Docker Compose
- OpenAI or Anthropic API key (for LLM integration)

### Setup
1. **Clone and configure**:
   ```bash
   git clone <repository-url>
   cd <repository-name>
   cp .env.example .env
   ```

2. **Add your LLM API key** to `.env`:
   ```bash
   # For OpenAI
   LLM_API_KEY=sk-your-openai-key-here
   LLM_MODEL=gpt-4

   # For Anthropic  
   LLM_API_KEY=your-anthropic-key-here
   LLM_MODEL=claude-3-sonnet-20240229
   ```

3. **Start the system**:
   ```bash
   make up
   ```

4. **Seed demo data**:
   ```bash
   make seed
   ```

5. **Open the interface**:
   - Frontend: http://localhost:3000
   - Neo4j Browser: http://localhost:7474 (neo4j/password)

## Usage

### Basic Workflow
1. **Ask a question** in the chat interface:
   - "How many customers do we have?"
   - "Show me all customers who have placed orders"
   - "What's the total revenue?"

2. **Watch the agent process**:
   - Creates workflow with 3 instructions
   - Discovers database schema
   - Generates SQL from your question
   - Reviews and formats results

3. **Human intervention** (during 5-minute pauses):
   - Edit instructions in Neo4j Browser
   - Change parameters or questions
   - Stop workflows if needed

### Editing Instructions During Pause

When the agent pauses, you can modify instructions in Neo4j Browser:

```cypher
// Change the question being asked
MATCH (i:Instruction {status: 'pending'}) 
SET i.parameters = '{"question": "Show me customers from the last month"}'

// Stop the entire workflow
MATCH (w:Workflow {status: 'active'}) 
SET w.status = 'stopped'
```

### Monitoring

Check system health:
```bash
make health
```

View real-time logs:
```bash
make logs
```

Check specific service:
```bash
make debug-agent    # Agent logs
make debug-mcp      # MCP server logs  
make debug-frontend # Frontend logs
```

## Commands Reference

| Command | Description |
|---------|-------------|
| `make up` | Start all services |
| `make down` | Stop all services |
| `make clean` | Remove all data and containers |
| `make health` | Check service health |
| `make seed` | Create demo data |
| `make logs` | View all logs |
| `make demo` | Full clean + setup + seed |
| `make test` | Run integration test |

## Configuration

### Environment Variables

All configuration is in `.env` file:

- **Neo4j**: Database connection and auth
- **PostgreSQL**: Sample data source  
- **MCP**: API keys and server settings
- **Agent**: Polling interval and pause duration
- **LLM**: API key and model selection

### Pause Duration

Default: 5 minutes (300 seconds)
Configurable via `PAUSE_DURATION` in `.env`

### Polling Interval  

Default: 30 seconds
Configurable via `AGENT_POLL_INTERVAL` in `.env`

## Development

### File Structure
```
├── agent/          # Python agent that executes instructions
├── frontend/       # Next.js chat interface  
├── mcp/           # FastAPI server for Neo4j operations
├── neo4j/         # Neo4j configuration and data
├── postgres/      # PostgreSQL setup and sample data
├── ops/           # Operational scripts (seeding, etc.)
├── docker-compose.yml
├── Makefile
└── README.md
```

### Adding New Instruction Types

1. **Define handler in agent**:
   ```python
   def handle_new_instruction_type(instruction):
       # Implementation
       return {"status": "success", "result": "..."}
   ```

2. **Add to agent main loop**:
   ```python
   elif instruction['type'] == 'new_instruction_type':
       exec_result = handle_new_instruction_type(instruction)
   ```

3. **Update frontend** to create new instruction types in workflows.

### Database Schema

The system uses two databases:

**Neo4j** (Workflow metadata):
- `Workflow` nodes with status and metadata
- `Instruction` nodes with type, sequence, parameters
- `Execution` nodes with results and timestamps  
- Relationships: `HAS_INSTRUCTION`, `EXECUTED_AS`, `NEXT_INSTRUCTION`

**PostgreSQL** (Sample data):
- `customers` table
- `orders` table
- Sample data for testing SQL generation

## Troubleshooting

### Common Issues

**Services not starting**:
```bash
make down
make clean  
make up
```

**Agent not processing**:
```bash
make restart-agent
make debug-agent
```

**Frontend not loading**:
```bash
make restart-frontend
make debug-frontend  
```

**Database connection issues**:
```bash
make health
# Check .env configuration
```

### Debug Mode

For detailed logging, check individual service logs:
```bash
docker-compose logs -f agent
docker-compose logs -f mcp
docker-compose logs -f frontend
```

### Reset Everything

Complete clean slate:
```bash
make clean
cp .env.example .env  
# Edit .env with your API key
make demo
```

## Contributing

1. Fork the repository
2. Create a feature branch
3. Test with `make demo`
4. Submit a pull request

## License

MIT License - see LICENSE file for details.

---

## Quick Demo

Want to see it in action immediately?

```bash
# 1. Clone repo
git clone <repo-url> && cd <repo-name>

# 2. Add your API key  
cp .env.example .env
# Edit .env: LLM_API_KEY=your-key-here

# 3. Start everything
make demo

# 4. Open http://localhost:3000
# 5. Ask: "How many records are in the database?"
# 6. Watch the magic happen! ✨
```

The system will:
- Create a workflow with 3 instructions
- Pause for 5 minutes before each step (editable in Neo4j Browser)
- Generate SQL from your natural language question
- Execute the query and return formatted results
- Show the entire process in a visual graph

🎉 **Welcome to the future of human-AI collaboration!**