archive/docs/APP_DEPLOYMENT_GUIDE.md

Crypto Admin Dashboard - Deployment Guide

Overview

This is a REAL-DATA-ONLY Gradio admin dashboard for the Crypto Data Aggregator platform. It provides comprehensive monitoring and management capabilities through a clean web interface.

Features

7 Main Tabs

1. 📊 Status - System health overview, database stats, quick diagnostics
2. 🔌 Providers - API provider management with filtering and reload
3. 📈 Market Data - Live cryptocurrency prices, charts, and market data
4. 🔍 APL Scanner - Auto Provider Loader for discovering and validating providers
5. 🤖 HF Models - HuggingFace model status and testing interface
6. 🔧 Diagnostics - Full system diagnostics with auto-repair capabilities
7. 📋 Logs - System logs viewer with filtering

Key Principles

• NO MOCK DATA: All data comes from real sources (database, APIs, files)
• Real-time: Live updates from actual collectors and services
• Error Handling: Graceful degradation with clear error messages
• HuggingFace Ready: Designed to run as a Gradio Space

Installation

Local Setup

# 1. Install dependencies
pip install -r requirements.txt

# 2. Ensure database is initialized
python -c "import database; db = database.get_database()"

# 3. Run the app
python app.py

The dashboard will be available at: http://localhost:7860

HuggingFace Space Deployment

1. Create a new Space on HuggingFace

2. Choose Space SDK: Gradio

3. Upload all necessary files:

   • app.py (main entrypoint)
   • config.py
   • database.py
   • collectors.py
   • ai_models.py
   • auto_provider_loader.py
   • provider_validator.py
   • requirements.txt
   • providers_config_extended.json
   • Backend services (if needed)

4. HuggingFace will automatically detect app.py and launch the Gradio app

Data Sources

Tab 1: Status

• Database: db.get_database_stats() - real-time DB metrics
• Providers: providers_config_extended.json - actual provider count
• Market: db.get_latest_prices(3) - live top 3 coins

Tab 2: Providers

• Source: providers_config_extended.json
• Categories: Dynamically extracted from provider configs
• Operations: Real file reload, category filtering

Tab 3: Market Data

• Prices: db.get_latest_prices(100) - real database records
• Refresh: collectors.collect_price_data() - live API calls to CoinGecko/CoinCap
• Charts: db.get_price_history(symbol, hours) - historical data from DB
• Plotly: Interactive charts with real price data

Tab 4: APL Scanner

• Scan: auto_provider_loader.AutoProviderLoader().run() - actual APL execution
• Results: Real validation results from HTTP providers and HF models
• Report: Reads actual PROVIDER_AUTO_DISCOVERY_REPORT.md

Tab 5: HF Models

• Status: ai_models.get_model_info() - real model loading status
• Test: ai_models.analyze_sentiment(), ai_models.summarize_text() - actual HF inference
• Initialize: ai_models.initialize_models() - loads real transformers models

Tab 6: Diagnostics

• Run: backend.services.diagnostics_service.DiagnosticsService().run_full_diagnostics()
• Checks: Real dependency checks, network tests, file system validation
• Auto-fix: Actually installs packages, creates directories

Tab 7: Logs

• Source: config.LOG_FILE - actual log file
• Filters: Real-time filtering of ERROR/WARNING/INFO
• Clear: Actually clears log file (with backup)

Testing Checklist

✅ Pre-Flight Checks

# 1. Verify Python version
python --version  # Should be 3.10+

# 2. Install dependencies
pip install -r requirements.txt

# 3. Check database exists
ls -lh data/database/crypto_aggregator.db

# 4. Check config files
ls -lh providers_config_extended.json
ls -lh config.py

✅ Tab-by-Tab Testing

Tab 1: Status

• Click "Refresh Status" - should show real DB stats
• Click "Run Quick Diagnostics" - should show actual issues (if any)
• Verify market snapshot shows real BTC/ETH/BNB prices (if available)
• Check database stats JSON shows actual record counts

Tab 2: Providers

• View providers table - should load from JSON file
• Change category filter - should filter providers
• Click "Reload Providers" - should show success message
• Verify provider count matches file

Tab 3: Market Data

• View market data table - should show real prices from DB
• Enter search term (e.g., "Bitcoin") - should filter results
• Click "Refresh Prices" - should collect new data from APIs
• Enter symbol "BTC" and click "Plot" - should show price chart (if Plotly installed)
• Verify no mock/hardcoded data

Tab 4: APL Scanner

• View last report - should show previous scan or "no report"
• Click "Run APL Scan" - WARNING: This runs a full scan
  • Should validate HTTP providers
  • Should check HF models
  • Should update providers_config_extended.json
• Click "View Last Report" - should show markdown report

Tab 5: HF Models

• View models table - should show configured models
• Click "Initialize Models" - should load transformers (if installed)
• Select "sentiment", enter text "Bitcoin is great!", click "Run Test"
  • Should show real sentiment analysis results
• Try "summarization" with longer text

Tab 6: Diagnostics

• Click "Run Diagnostics" - should check:
  • Dependencies (Python packages)
  • Configuration (env vars, files)
  • Network (API connectivity)
  • Services (provider status)
  • Models (HF model availability)
  • Filesystem (directories, files)
• Click "Run with Auto-Fix" - WARNING: May install packages
  • Should attempt to fix issues automatically

Tab 7: Logs

• Select "recent" - should show last 100 log lines
• Select "errors" - should show only ERROR lines
• Click "Refresh Logs" - should update display
• Click "Clear Logs" - WARNING: Clears log file
  • Should create backup first

✅ Error Handling Tests

# Test with no data
# 1. Empty database
python -c "import database; db = database.get_database(); import os; os.remove(str(db.db_path))"

# 2. Run app - should show "No data available" messages, not crash

# Test with missing config
# 1. Rename providers config
mv providers_config_extended.json providers_config_extended.json.bak

# 2. Run app - should show error messages, not crash

# Test with no internet
# 1. Disconnect network
# 2. Click "Refresh Prices" - should show connection errors, not crash

Configuration

Environment Variables

Optional environment variables for enhanced functionality:

# HuggingFace API token (for model access)
export HF_TOKEN="your_hf_token_here"

# API keys (optional)
export CMC_API_KEY="your_coinmarketcap_key"
export ETHERSCAN_KEY="your_etherscan_key"

Config.py Settings

Key settings in config.py:

LOG_LEVEL = "INFO"  # DEBUG, INFO, WARNING, ERROR
AUTO_REFRESH_INTERVAL = 30  # seconds
GRADIO_SERVER_NAME = "0.0.0.0"
GRADIO_SERVER_PORT = 7860

Troubleshooting

Issue: "gradio not installed"

pip install gradio

Issue: "No market data available"

# Collect initial data
python -c "import collectors; collectors.collect_price_data()"

Issue: "Plotly not available"

pip install plotly
# Charts will work after restart

Issue: "Transformers not available"

pip install transformers torch
# HF model features will work after restart

Issue: "Log file not found"

# Ensure logs directory exists
mkdir -p logs
# Run any collector to create log
python -c "import collectors; collectors.collect_price_data()"

Performance Notes

• Initial Load: First load may be slow as models initialize
• APL Scan: Can take 30-60 seconds to validate all providers
• Diagnostics: Full scan takes ~5-10 seconds
• Charts: Rendering large datasets may take a few seconds

Security Notes

• Database Queries: Only SELECT queries allowed in DB Explorer
• Log Clearing: Creates backup before clearing
• Auto-Fix: Only installs packages and creates directories
• No Shell Access: No direct shell command execution

Development

Adding New Tabs

with gr.Tab("🆕 New Tab"):
    gr.Markdown("### New Feature")
    
    # Your components here
    output = gr.Markdown()
    
    def new_function():
        # MUST use real data only
        return "Real data result"
    
    demo.load(
        fn=new_function,
        outputs=output
    )

Adding New Data Sources

1. Add function to fetch real data (no mock data!)
2. Wire function to Gradio component
3. Add error handling
4. Test with missing/unavailable data

License

Part of the Crypto Data Aggregator project.

---

Last Updated: 2025-11-16 Version: 1.0.0 Maintainer: Crypto DT Source Team