hf-data-engine/README_DEPLOYMENT.md

# 🚀 Hugging Face Spaces Deployment Guide

This repository is **production-ready** for deployment on Hugging Face Spaces using Docker runtime.

## ✅ Pre-Deployment Checklist

All requirements are already configured:

- ✅ **FastAPI app**: `api_server_extended.py` defines `app = FastAPI(...)`
- ✅ **Health endpoint**: `/health` returns service status
- ✅ **Dockerfile**: Configured with correct CMD for uvicorn
- ✅ **Requirements**: All dependencies listed in `requirements.txt`
- ✅ **Port handling**: Supports `${PORT}` environment variable
- ✅ **Resilient startup**: Runs in degraded mode if some services fail

## 🎯 Deployment Steps

### 1. Create a New Space on Hugging Face

1. Go to https://huggingface.co/spaces
2. Click **"Create new Space"**
3. Configure:
   - **Space name**: `crypto-monitor-api` (or your choice)
   - **License**: Choose appropriate license
   - **SDK**: Select **Docker**
   - **Hardware**: CPU Basic (minimum) or CPU Upgrade (recommended)
   - **Visibility**: Public or Private

### 2. Push Repository to Space

```bash

# Clone your new Space

git clone https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME

cd YOUR_SPACE_NAME



# Copy all files from this repository

cp -r /path/to/crypto-dt-source-main/* .



# Add and commit

git add .

git commit -m "Initial deployment of Crypto Monitor API"



# Push to Hugging Face

git push

```

### 3. Configure Space Settings (Optional)

In your Space settings, you can add these environment variables:

- `PORT` - Port number (default: 7860 for HF, 8000 for local)
- `ENABLE_AUTO_DISCOVERY` - Enable auto-discovery service (default: false)
- Add any API keys as **Repository secrets** (not in code!)

### 4. Monitor Deployment

1. Go to your Space page
2. Check the **Logs** tab for build progress
3. Wait for "Running" status (usually 2-5 minutes)
4. Access your API at: `https://YOUR_USERNAME-YOUR_SPACE_NAME.hf.space`

## 📊 API Endpoints

Once deployed, your API will be available at:

### Core Endpoints
- **Root**: `https://your-space.hf.space/`
- **API Docs**: `https://your-space.hf.space/docs` (Interactive Swagger UI)
- **Health Check**: `https://your-space.hf.space/health`
- **Status**: `https://your-space.hf.space/api/status`

### Provider Management
- `GET /api/providers` - List all crypto data providers
- `GET /api/providers/{id}` - Get provider details
- `POST /api/providers/{id}/health-check` - Check provider health
- `GET /api/providers/category/{category}` - Filter by category

### Pool Management
- `GET /api/pools` - List all provider pools
- `POST /api/pools` - Create new pool
- `POST /api/pools/{id}/members` - Add provider to pool
- `POST /api/pools/{id}/rotate` - Rotate pool providers

### Real-time Updates
- `WS /ws` - WebSocket connection for live updates

### Monitoring & Diagnostics
- `GET /api/stats` - System statistics
- `GET /api/logs` - Application logs
- `POST /api/diagnostics/run` - Run diagnostics

See `/docs` for complete API documentation with interactive testing.

## 🧪 Local Testing

### Test with Docker (Recommended)

```bash

# Build the image

docker build -t crypto-monitor-test .



# Run the container

docker run -p 8000:8000 crypto-monitor-test



# Test health endpoint

curl http://localhost:8000/health



# Access API docs

open http://localhost:8000/docs

```

### Test with Python

```bash

# Install dependencies

pip install -r requirements.txt



# Run locally

python main.py



# Or with uvicorn directly

uvicorn api_server_extended:app --host 0.0.0.0 --port 8000 --reload

```

## 🔍 Troubleshooting

### Build Fails

**Check logs for specific errors:**
- Missing dependencies? Verify `requirements.txt`
- Import errors? Ensure all local modules exist
- System dependencies? Check Dockerfile `apt-get install` section

**Common fixes:**
```bash

# Rebuild without cache

docker build --no-cache -t crypto-monitor-test .



# Check for syntax errors

python -m py_compile api_server_extended.py

```

### Container Starts but Health Check Fails

**Increase startup time:**
Edit `Dockerfile` and increase `start-period`:
```dockerfile

HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \

    CMD curl -f http://localhost:${PORT:-8000}/health || exit 1

```

**Check logs:**
```bash

docker logs <container_id>

```

### Service Runs in Degraded Mode

This is **normal** if:
- Some external APIs are unavailable
- Network connectivity is limited
- Optional services fail to start

The service will still work with available providers. Check `/health` endpoint for details.

### WebSocket Connection Issues

If WebSocket connections fail:
1. Ensure your client uses `wss://` (not `ws://`) for HTTPS spaces
2. Check CORS settings in `api_server_extended.py`
3. Verify firewall/proxy settings

## 📈 Performance Optimization

### For Better Performance

1. **Upgrade Hardware**: Use CPU Upgrade or GPU in Space settings
2. **Disable Auto-Discovery**: Set `ENABLE_AUTO_DISCOVERY=false` (already default)
3. **Reduce Provider Count**: Edit config files to monitor fewer providers
4. **Enable Caching**: Already enabled by default

### Resource Usage

- **Memory**: ~2-4 GB (depends on active providers)
- **CPU**: Low to moderate (spikes during health checks)
- **Storage**: ~500 MB (includes models and data)

## 🔐 Security Best Practices

1. **Never commit API keys** - Use HF Repository secrets
2. **Use HTTPS** - Hugging Face provides this automatically
3. **Rate limiting** - Already implemented via `slowapi`
4. **CORS** - Configured to allow all origins (adjust if needed)

## 📝 Configuration Files

### Required Files (Already Present)
- `Dockerfile` - Container configuration
- `requirements.txt` - Python dependencies
- `api_server_extended.py` - Main FastAPI application
- `.dockerignore` - Files to exclude from image

### Optional Configuration
- `.env.example` - Environment variable template
- `providers_config_*.json` - Provider configurations
- `crypto_resources_*.json` - Resource definitions

## 🎓 Features

This deployment includes:

✅ **200+ Crypto Data Providers** - Comprehensive coverage
✅ **Provider Pools** - Load balancing and failover
✅ **Real-time WebSocket** - Live updates
✅ **Health Monitoring** - Automatic health checks
✅ **Auto-Discovery** - Find new data sources (optional)
✅ **Diagnostics** - Built-in troubleshooting
✅ **Logging System** - Comprehensive logging
✅ **Resource Management** - Import/export configs
✅ **Rate Limiting** - Prevent abuse
✅ **CORS Support** - Cross-origin requests
✅ **API Documentation** - Interactive Swagger UI

## 🆘 Support

### Check Service Status
```bash

# Health check

curl https://your-space.hf.space/health



# Detailed status

curl https://your-space.hf.space/api/status



# Run diagnostics

curl -X POST https://your-space.hf.space/api/diagnostics/run

```

### Common Issues

1. **Space shows "Building"** - Wait 2-5 minutes for first build
2. **Space shows "Runtime Error"** - Check logs tab for details
3. **API returns 503** - Service starting up, wait 30-60 seconds
4. **Slow responses** - Upgrade hardware or reduce provider count

### Get Help

- Check `/api/diagnostics/run` for automatic issue detection
- Review Space logs for error messages
- Test locally with Docker to isolate issues
- Check Hugging Face Spaces documentation

## 📄 License

See LICENSE file for details.

---

**Ready to deploy!** Follow the steps above to get your Crypto Monitor API running on Hugging Face Spaces.