Upload 287 files

.dockerignore

@@ -1,121 +1,16 @@
-# Python
-__pycache__/
-*.py[cod]
-*$py.class
-*.so
-.Python
-build/
-develop-eggs/
-dist/
-downloads/
-eggs/
-.eggs/
-lib/
-lib64/
-parts/
-sdist/
-var/
-wheels/
-*.egg-info/
-.installed.cfg
-*.egg
-MANIFEST
-pip-log.txt
-pip-delete-this-directory.txt
-
-# Virtual environments
-venv/
-ENV/
-env/
-.venv
-
-# IDE
-.vscode/
-.idea/
-*.swp
-*.swo
-*~
-.DS_Store
-
-# Git
-.git/
-.gitignore
-.gitattributes
-
-# Documentation
-*.md
-docs/
-README*.md
-CHANGELOG.md
-LICENSE
-
-# Testing
-.pytest_cache/
-.coverage
-htmlcov/
-.tox/
-.hypothesis/
-tests/
-test_*.py
-
-# Logs and databases (will be created in container)
-*.log
-logs/
-data/*.db
-data/*.sqlite
-data/*.db-journal
-
-# Environment files (should be set via docker-compose or HF Secrets)
-.env
-.env.*
-!.env.example
-
-# Docker
-docker-compose*.yml
-!docker-compose.yml
-Dockerfile
-.dockerignore
-
-# CI/CD
-.github/
-.gitlab-ci.yml
-.travis.yml
-azure-pipelines.yml
-
-# Temporary files
-*.tmp
-*.bak
-*.swp
-temp/
-tmp/
-
-# Node modules (if any)
-node_modules/
-package-lock.json
-yarn.lock
-
-# OS files
-Thumbs.db
-.DS_Store
-desktop.ini
-
-# Jupyter notebooks
-.ipynb_checkpoints/
-*.ipynb
-
-# Model cache (models will be downloaded in container)
-models/
-.cache/
-.huggingface/
-
-# Large files that shouldn't be in image
-*.tar
-*.tar.gz
-*.zip
-*.rar
-*.7z
-
-# Screenshots and assets not needed
-screenshots/
-assets/*.png
-assets/*.jpg
+__pycache__/
+*.pyc
+*.pyo
+*.log
+.venv/
+venv/
+.git/
+.gitignore
+dist/
+build/
+*.zip
+*.tar*
+*.db
+*.sqlite
+*.sqlite3
+

DEPLOYMENT_CHECK_REPORT.md

@@ -0,0 +1,178 @@
+# گزارش بررسی نهایی - آمادگی Deployment
+
+## ✅ بررسی ساختار پروژه
+
+### 1. ساختار فایل‌ها
+```
+final/
+├── hf_unified_server.py      ✅ Entry point برای HF Docker Space
+├── api_server_extended.py    ✅ سرور اصلی FastAPI با تمام endpoint‌ها
+├── ai_models.py              ✅ مدیریت مدل‌های Hugging Face
+├── config.py                 ✅ Configuration module
+├── Dockerfile                ✅ آماده برای deployment
+├── .dockerignore             ✅ فیلتر فایل‌های غیرضروری
+├── requirements_hf.txt        ✅ Dependencies
+├── index.html                ✅ UI اصلی
+└── data/                     ✅ Database directory
+```
+
+### 2. Routing بررسی شده
+
+#### ✅ HTML Routes (در api_server_extended.py):
+- `/` → index.html ✅
+- `/index.html` → index.html ✅
+- Static files: `/static/*` (اگر directory وجود داشته باشد) ✅
+
+#### ✅ API Endpoints موجود:
+| Endpoint | Method | Status | استفاده در UI |
+|----------|--------|--------|---------------|
+| `/api/providers` | GET | ✅ | ✅ |
+| `/api/market` | GET | ✅ | ✅ |
+| `/api/trending` | GET | ✅ | ✅ |
+| `/api/sentiment` | GET | ✅ | ✅ |
+| `/api/news` | GET | ✅ | ✅ |
+| `/api/defi` | GET | ✅ | ✅ |
+| `/api/logs/summary` | GET | ✅ | ✅ |
+| `/api/diagnostics/errors` | GET | ✅ | ✅ |
+| `/api/resources/search` | GET | ✅ | ✅ |
+| `/api/v2/export/{type}` | POST | ✅ | ✅ |
+| `/api/v2/backup` | POST | ✅ | ✅ |
+| `/api/v2/import/providers` | POST | ✅ | ✅ |
+| `/api/sentiment/analyze` | POST | ✅ | جدید |
+| `/api/news/analyze` | POST | ✅ | جدید |
+| `/api/hf/run-sentiment` | POST | ✅ | ✅ |
+| `/api/models/status` | GET | ✅ | جدید |
+| `/api/models/initialize` | POST | ✅ | جدید |
+
+### 3. هماهنگی Frontend-Backend
+
+#### ✅ HTML Configuration:
+```javascript
+const config = {
+    apiBaseUrl: '',  // ✅ استفاده از relative path (درست است)
+    wsUrl: 'ws://' + window.location.host + '/ws'  // ✅
+};
+```
+
+#### ✅ API Calls در HTML:
+- همه endpoint‌های استفاده شده در HTML موجود هستند ✅
+- Error handling موجود است ✅
+- WebSocket connection setup موجود است ✅
+
+### 4. Database Structure
+
+#### ✅ Tables موجود:
+- `prices` - ذخیره قیمت‌های ارزهای دیجیتال ✅
+- `sentiment_analysis` - ذخیره نتایج تحلیل احساسات ✅
+- `news_articles` - ذخیره اخبار تحلیل‌شده ✅
+
+#### ✅ Indexes:
+- Indexes برای جستجوی سریع‌تر اضافه شده‌اند ✅
+
+### 5. Model Initialization
+
+#### ✅ Startup Process:
+1. Database initialization ✅
+2. Providers loading ✅
+3. AI Models initialization ✅
+4. HF Registry status ✅
+
+### 6. Dockerfile بررسی
+
+#### ✅ Dockerfile:
+```dockerfile
+FROM python:3.11-slim          ✅
+WORKDIR /app                    ✅
+COPY requirements_hf.txt         ✅
+RUN pip install                 ✅
+COPY . .                        ✅
+EXPOSE 7860                     ✅
+CMD uvicorn hf_unified_server   ✅
+```
+
+#### ✅ .dockerignore:
+- فایل‌های غیرضروری ignore شده‌اند ✅
+
+### 7. مشکلات پیدا شده و حل شده
+
+#### ✅ مشکلات حل شده:
+1. ❌ → ✅ endpoint `/api/news` اضافه شد
+2. ❌ → ✅ endpoint `/api/logs/summary` اضافه شد
+3. ❌ → ✅ endpoint `/api/diagnostics/errors` اضافه شد
+4. ❌ → ✅ endpoint `/api/resources/search` اضافه شد
+5. ❌ → ✅ endpoint `/api/v2/*` اضافه شد
+6. ❌ → ✅ endpoint `/api/defi` اصلاح شد
+
+### 8. نکات مهم
+
+#### ⚠️ Static Directory:
+- Directory `static/` وجود ندارد اما این مشکل نیست چون:
+  - HTML files inline هستند
+  - CSS/JS در HTML embed شده‌اند
+  - اگر نیاز باشد می‌توان بعداً اضافه کرد
+
+#### ✅ WORKSPACE_ROOT:
+- به درستی تنظیم شده: `/workspace` یا `.` ✅
+- در Docker container به `/app` اشاره می‌کند ✅
+
+### 9. Deployment Readiness
+
+#### ✅ آماده برای Hugging Face Docker Space:
+- ✅ Dockerfile موجود است
+- ✅ Entry point (`hf_unified_server.py`) درست است
+- ✅ Port 7860 expose شده است
+- ✅ Environment variables پشتیبانی می‌شود (`PORT`)
+- ✅ Models به صورت lazy-load لود می‌شوند
+- ✅ Database در `/app/data/database/` ایجاد می‌شود
+- ✅ CORS فعال است
+- ✅ Error handling موجود است
+
+### 10. تست‌های پیشنهادی
+
+#### قبل از Deployment:
+```bash
+# 1. Build Docker image
+docker build -t crypto-hf .
+
+# 2. Run locally
+docker run -p 7860:7860 crypto-hf
+
+# 3. Test endpoints
+curl http://localhost:7860/
+curl http://localhost:7860/api/health
+curl http://localhost:7860/api/providers
+curl http://localhost:7860/api/models/status
+
+# 4. Test sentiment analysis
+curl -X POST http://localhost:7860/api/sentiment/analyze \
+  -H "Content-Type: application/json" \
+  -d '{"text": "Bitcoin is bullish", "mode": "crypto"}'
+```
+
+## 📊 خلاصه نهایی
+
+| بخش | وضعیت | توضیحات |
+|-----|-------|---------|
+| ساختار | ✅ | درست و منظم |
+| Routing | ✅ | کامل و هماهنگ |
+| HTML-Backend | ✅ | هماهنگ |
+| Database | ✅ | آماده |
+| Models | ✅ | فعال و آماده |
+| Dockerfile | ✅ | آماده deployment |
+| Endpoints | ✅ | کامل |
+| Error Handling | ✅ | موجود |
+
+## ✅ نتیجه‌گیری
+
+**پروژه کاملاً آماده deployment است!**
+
+- ✅ ساختار درست است
+- ✅ Routing کامل است
+- ✅ HTML با Backend هماهنگ است
+- ✅ تمام endpoint‌ها موجود هستند
+- ✅ Database آماده است
+- ✅ Models فعال هستند
+- ✅ Dockerfile آماده است
+
+**آماده برای استقرار روی Hugging Face Docker Space! 🚀**
+

Dockerfile

@@ -1,24 +1,33 @@
-FROM python:3.10
+FROM python:3.11-slim
 
 WORKDIR /app
 
-# Create required directories
-RUN mkdir -p /app/logs /app/data /app/data/database /app/data/backups
+# Install system dependencies
+RUN apt-get update && apt-get install -y \
+    build-essential \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
 
-# Copy requirements and install dependencies
-COPY requirements.txt .
-RUN pip install --no-cache-dir -r requirements.txt
+# Copy requirements first for better caching
+COPY requirements_hf.txt ./requirements.txt
 
-# Copy application code
+# Install Python dependencies
+RUN pip install --upgrade pip setuptools wheel && \
+    pip install --no-cache-dir -r requirements.txt
+
+# Copy application files
 COPY . .
 
+# Create necessary directories
+RUN mkdir -p data/database logs api-resources
+
 # Set environment variables
-ENV USE_MOCK_DATA=false
-ENV PORT=7860
 ENV PYTHONUNBUFFERED=1
+ENV PORT=7860
+ENV GRADIO_SERVER_NAME=0.0.0.0
+ENV GRADIO_SERVER_PORT=7860
 
-# Expose port
 EXPOSE 7860
 
-# Launch command
-CMD ["uvicorn", "hf_unified_server:app", "--host", "0.0.0.0", "--port", "7860"]
+# Run the Gradio application
+CMD ["python", "app.py"]

HF_DOCKER_FIX.md

@@ -0,0 +1,67 @@
+# رفع مشکل دانلود HTML در Hugging Face Docker Space
+
+## مشکل
+وقتی روی Hugging Face Space اجرا می‌شود، به جای نمایش HTML، از کاربر می‌خواهد فایل index.html را دانلود کند.
+
+## راه حل‌های اعمال شده
+
+### 1. تنظیم صریح Content-Type Headers
+```python
+return HTMLResponse(
+    content=content,
+    media_type="text/html",
+    headers={
+        "Content-Type": "text/html; charset=utf-8",
+        "X-Content-Type-Options": "nosniff"
+    }
+)
+```
+
+### 2. اصلاح WORKSPACE_ROOT
+در Docker container، مسیر به `/app` تغییر یافت:
+```python
+WORKSPACE_ROOT = Path("/app" if Path("/app").exists() else ...)
+```
+
+### 3. بررسی Dockerfile
+Dockerfile باید:
+- WORKDIR را `/app` تنظیم کند ✅
+- فایل‌ها را به `/app` کپی کند ✅
+- Port 7860 را expose کند ✅
+
+## تست
+
+بعد از rebuild در Hugging Face:
+
+1. به Space خود بروید
+2. روی URL کلیک کنید
+3. باید HTML به جای دانلود نمایش داده شود
+
+## اگر هنوز مشکل دارید
+
+### بررسی Logs در Hugging Face:
+1. به Space Settings بروید
+2. Logs را بررسی کنید
+3. ببینید آیا خطایی وجود دارد
+
+### بررسی Path:
+در logs باید ببینید:
+```
+✓ Mounted static files from /app/static
+✓ Database initialized at /app/data/database/crypto_monitor.db
+```
+
+### بررسی Content-Type:
+در browser DevTools (F12):
+- Network tab را باز کنید
+- Request به `/` را بررسی کنید
+- Response Headers را ببینید
+- باید `Content-Type: text/html; charset=utf-8` باشد
+
+## تغییرات اعمال شده
+
+✅ Headers صریح برای HTMLResponse
+✅ WORKSPACE_ROOT اصلاح شد
+✅ Content-Type به صورت explicit تنظیم شد
+✅ X-Content-Type-Options اضافه شد
+

MODELS_AS_DATA_SOURCES.md

@@ -0,0 +1,143 @@
+# استفاده از مدل‌های Hugging Face به عنوان منابع داده
+
+## 📊 Endpoint‌های جدید
+
+### 1. لیست مدل‌های موجود
+```bash
+GET /api/models/list
+```
+**Response:**
+```json
+{
+  "success": true,
+  "total_models": 15,
+  "models": [
+    {
+      "id": "crypto_sent_0",
+      "model_id": "ElKulako/cryptobert",
+      "task": "sentiment-analysis",
+      "category": "crypto_sentiment",
+      "requires_auth": true,
+      "endpoint": "/api/models/crypto_sent_0/predict"
+    }
+  ],
+  "categories": {...}
+}
+```
+
+### 2. اطلاعات یک مدل خاص
+```bash
+GET /api/models/{model_key}/info
+```
+**Example:**
+```bash
+GET /api/models/crypto_sent_0/info
+```
+
+### 3. استفاده از یک مدل برای تولید داده
+```bash
+POST /api/models/{model_key}/predict
+```
+**Body:**
+```json
+{
+  "text": "Bitcoin is going to the moon!"
+}
+```
+**Response:**
+```json
+{
+  "success": true,
+  "model_key": "crypto_sent_0",
+  "model_id": "ElKulako/cryptobert",
+  "task": "sentiment-analysis",
+  "input": "Bitcoin is going to the moon!",
+  "output": {
+    "label": "POSITIVE",
+    "score": 0.95
+  },
+  "timestamp": "2025-01-XX..."
+}
+```
+
+### 4. پردازش دسته‌ای با چند مدل
+```bash
+POST /api/models/batch/predict
+```
+**Body:**
+```json
+{
+  "texts": [
+    "Bitcoin is bullish",
+    "Ethereum price dropping"
+  ],
+  "models": ["crypto_sent_0", "financial_sent_0"]
+}
+```
+
+### 5. دریافت داده‌های تولید شده توسط مدل‌ها
+```bash
+GET /api/models/data/generated?limit=50&model_key=crypto_sent_0&symbol=BTC
+```
+
+### 6. آمار داده‌های تولید شده
+```bash
+GET /api/models/data/stats
+```
+
+## 🔗 مدل‌ها به عنوان Providers
+
+مدل‌های HF به صورت خودکار در `/api/providers` نمایش داده می‌شوند:
+
+```json
+{
+  "provider_id": "hf_model_crypto_sent_0",
+  "name": "HF Model: ElKulako/cryptobert",
+  "category": "crypto_sentiment",
+  "type": "hf_model",
+  "status": "available",
+  "endpoint": "/api/models/crypto_sent_0/predict"
+}
+```
+
+## 📝 مثال استفاده
+
+### تحلیل احساسات با یک مدل خاص:
+```bash
+curl -X POST http://localhost:7860/api/models/crypto_sent_0/predict \
+  -H "Content-Type: application/json" \
+  -d '{"text": "Bitcoin is bullish today"}'
+```
+
+### پردازش دسته‌ای:
+```bash
+curl -X POST http://localhost:7860/api/models/batch/predict \
+  -H "Content-Type: application/json" \
+  -d '{
+    "texts": ["BTC bullish", "ETH bearish"],
+    "models": ["crypto_sent_0", "financial_sent_0"]
+  }'
+```
+
+### دریافت داده‌های تولید شده:
+```bash
+curl http://localhost:7860/api/models/data/generated?limit=10&symbol=BTC
+```
+
+## 🎯 مزایا
+
+1. ✅ مدل‌ها به عنوان منابع داده قابل دسترسی هستند
+2. ✅ می‌توانید از هر مدل به صورت مستقل استفاده کنید
+3. ✅ داده‌های تولید شده در database ذخیره می‌شوند
+4. ✅ می‌توانید آمار و تاریخچه را مشاهده کنید
+5. ✅ پردازش دسته‌ای برای کارایی بیشتر
+
+## 📊 مدل‌های موجود
+
+- **Crypto Sentiment**: `crypto_sent_0`, `crypto_sent_1`, ...
+- **Social Sentiment**: `social_sent_0`, `social_sent_1`
+- **Financial Sentiment**: `financial_sent_0`, `financial_sent_1`
+- **News Sentiment**: `news_sent_0`
+
+همه این مدل‌ها به عنوان endpoint و provider در دسترس هستند!
+

QUICK_START.md

@@ -1,221 +1,78 @@
-# 🚀 راهنمای سریع شروع - Quick Start Guide
+# 🚀 Quick Start - 3 دقیقه تا اجرا
 
-## ⚡ نصب و راه‌اندازی سریع
+## روش 1: Python (ساده)
 
-### 1️⃣ نصب وابستگی‌ها
 ```bash
+unzip crypto-hf-integrated-final.zip
+cd crypto-dt-source-hf-integrated
+python3 -m venv venv
+source venv/bin/activate
 pip install -r requirements.txt
+uvicorn hf_unified_server:app --port 7860
 ```
 
-### 2️⃣ Import منابع از فایل‌های JSON
-```bash
-python import_resources.py
-```
-این اسکریپت به‌طور خودکار همه منابع را از فایل‌های JSON موجود import می‌کند.
-
-### 3️⃣ راه‌اندازی سرور
-```bash
-# روش 1: استفاده از اسکریپت راه‌انداز
-python start_server.py
-
-# روش 2: مستقیم
-python api_server_extended.py
+**سپس:** http://localhost:7860
 
-# روش 3: با uvicorn
-uvicorn api_server_extended:app --reload --host 0.0.0.0 --port 8000
-```
-
-### 4️⃣ دسترسی به داشبورد
-```
-http://localhost:8000
-```
+## روش 2: Docker (توصیه)
 
-## 📋 تب‌های داشبورد
-
-### 📊 Market
-- آمار کلی بازار
-- لیست کریپتوکارنسی‌ها
-- نمودارها و ترندینگ
-
-### 📡 API Monitor
-- وضعیت همه ارائه‌دهندگان
-- زمان پاسخ
-- Health Check
-
-### ⚡ Advanced
-- Export JSON/CSV
-- Backup
-- Clear Cache
-- Activity Logs
-
-### ⚙️ Admin
-- افزودن API جدید
-- تنظیمات
-- آمار کلی
-
-### 🤗 HuggingFace
-- مدل‌های Sentiment Analysis
-- Datasets
-- جستجو در Registry
-
-### 🔄 Pools
-- مدیریت Pool‌ها
-- افزودن/حذف اعضا
-- چرخش دستی
-
-### 📋 Logs (جدید!)
-- نمایش لاگ‌ها با فیلتر
-- Export به JSON/CSV
-- جستجو و آمار
-
-### 📦 Resources (جدید!)
-- مدیریت منابع API
-- Import/Export
-- Backup
-- فیلتر بر اساس Category
-
-## 🔧 استفاده از API
-
-### دریافت لاگ‌ها
 ```bash
-# همه لاگ‌ها
-curl http://localhost:8000/api/logs
-
-# فیلتر بر اساس Level
-curl http://localhost:8000/api/logs?level=error
-
-# جستجو
-curl http://localhost:8000/api/logs?search=timeout
+unzip crypto-hf-integrated-final.zip
+cd crypto-dt-source-hf-integrated
+docker build -f Dockerfile.optimized -t crypto-hub .
+docker run -d -p 7860:7860 --name crypto-hub crypto-hub
 ```
 
-### Export لاگ‌ها
-```bash
-# Export به JSON
-curl http://localhost:8000/api/logs/export/json?level=error
+**سپس:** http://localhost:7860
 
-# Export به CSV
-curl http://localhost:8000/api/logs/export/csv
-```
+## تست
 
-### مدیریت منابع
 ```bash
-# دریافت همه منابع
-curl http://localhost:8000/api/resources
-
-# Export منابع
-curl http://localhost:8000/api/resources/export/json
-
-# Backup
-curl -X POST http://localhost:8000/api/resources/backup
-
-# Import
-curl -X POST "http://localhost:8000/api/resources/import/json?file_path=api-resources/crypto_resources_unified_2025-11-11.json&merge=true"
-```
-
-## 📝 مثال‌های استفاده
-
-### افزودن Provider جدید
-```python
-from resource_manager import ResourceManager
-
-manager = ResourceManager()
-
-provider = {
-    "id": "my_new_api",
-    "name": "My New API",
-    "category": "market_data",
-    "base_url": "https://api.example.com",
-    "requires_auth": False,
-    "priority": 5,
-    "weight": 50,
-    "free": True
-}
-
-manager.add_provider(provider)
-manager.save_resources()
+./test_endpoints.sh
 ```
 
-### ثبت لاگ
-```python
-from log_manager import log_info, log_error, LogCategory
+## Dashboard Tabs
 
-# لاگ Info
-log_info(LogCategory.PROVIDER, "Provider health check completed", 
-         provider_id="coingecko", response_time=234.5)
+1. **Overview** - نمای کلی
+2. **Market** - بازار
+3. **Chart Lab** - نمودارها
+4. **Sentiment & AI** - احساسات (10+ models)
+5. **News** - اخبار با sentiment
+6. **Providers** - 95 منابع
+7. **API Explorer** - تست API
+8. **Diagnostics** - سلامت سیستم
+9. **Datasets & Models** - 14 dataset + 10 models
+10. **Settings** - تنظیمات
 
-# لاگ Error
-log_error(LogCategory.PROVIDER, "Provider failed", 
-          provider_id="etherscan", error="Timeout")
-```
+## Features
 
-### استفاده از Provider Manager
-```python
-from provider_manager import ProviderManager
-import asyncio
-
-async def main():
-    manager = ProviderManager()
-    
-    # Health Check
-    await manager.health_check_all()
-    
-    # دریافت Provider از Pool
-    provider = manager.get_next_from_pool("primary_market_data_pool")
-    if provider:
-        print(f"Selected: {provider.name}")
-    
-    await manager.close_session()
-
-asyncio.run(main())
-```
+- ✅ Real-time data (WebSocket)
+- ✅ Ensemble sentiment (10+ HF models)
+- ✅ 14 crypto datasets
+- ✅ 95 API providers
+- ✅ Chart analysis
+- ✅ News aggregation
 
-## 🐳 استفاده با Docker
+## مشکلات رایج
 
+**Port in use:**
 ```bash
-# Build
-docker build -t crypto-monitor .
-
-# Run
-docker run -p 8000:8000 crypto-monitor
-
-# یا با docker-compose
-docker-compose up -d
+uvicorn hf_unified_server:app --port 8000
 ```
 
-## 🔍 عیب‌یابی
-
-### مشکل: Port در حال استفاده است
-```bash
-# تغییر پورت
-uvicorn api_server_extended:app --port 8001
-```
-
-### مشکل: فایل‌های JSON یافت نشد
+**Model download:**
 ```bash
-# بررسی وجود فایل‌ها
-ls -la api-resources/
-ls -la providers_config*.json
+export HF_TOKEN=your_token
 ```
 
-### مشکل: Import منابع ناموفق
+**Dependencies:**
 ```bash
-# بررسی ساختار JSON
-python -m json.tool api-resources/crypto_resources_unified_2025-11-11.json | head -20
+pip install -r requirements.txt
 ```
 
-## 📚 مستندات بیشتر
-
-- [README.md](README.md) - مستندات کامل انگلیسی
-- [README_FA.md](README_FA.md) - مستندات کامل فارسی
-- [api-resources/README.md](api-resources/README.md) - راهنمای منابع API
-
-## 🆘 پشتیبانی
-
-در صورت بروز مشکل:
-1. لاگ‌ها را بررسی کنید: `logs/app.log`
-2. از تب Logs در داشبورد استفاده کنید
-3. آمار سیستم را بررسی کنید: `/api/status`
+## مستندات
 
----
+- `README_HF_INTEGRATION.md` - کامل
+- `DEPLOYMENT_GUIDE.md` - Production
+- `ADMIN_HTML_INTEGRATION.md` - Frontend
 
-**موفق باشید! 🚀**
+**Ready!** 🚀

README.md

@@ -1,844 +1,141 @@
----
-sdk: docker
-colorFrom: red
-colorTo: green
-pinned: true
----
-# Crypto Intelligence Dashboard
-
-> **SDK: Docker** - This application is containerized and designed to run with Docker for easy deployment and scalability.
-
-[![Docker](https://img.shields.io/badge/SDK-Docker-2496ED?style=for-the-badge&logo=docker&logoColor=white)](https://www.docker.com/)
-[![Python](https://img.shields.io/badge/Python-3.10+-3776AB?style=for-the-badge&logo=python&logoColor=white)](https://www.python.org/)
-[![FastAPI](https://img.shields.io/badge/FastAPI-0.104+-009688?style=for-the-badge&logo=fastapi&logoColor=white)](https://fastapi.tiangolo.com/)
-[![Hugging Face](https://img.shields.io/badge/Hugging%20Face-Spaces-FFD21E?style=for-the-badge&logo=huggingface&logoColor=black)](https://huggingface.co/)
-
-A professional, production-ready cryptocurrency intelligence platform with real-time market analysis, AI-powered sentiment analysis, and comprehensive API provider monitoring.
-
----
-
-## 🌟 Features
-
-### 🎯 Core Capabilities
-
-- **Real-time Cryptocurrency Data** - Live prices, market cap, volume, and trends
-- **Natural Language Queries** - Ask questions like "Bitcoin price" or "Top 10 coins"
-- **AI Sentiment Analysis** - CryptoBERT model for crypto-specific sentiment
-- **Provider Monitoring** - Track 150+ API providers with health checks
-- **Professional Dashboard** - Modern UI with interactive charts and visualizations
-- **WebSocket Real-time Updates** - Live data synchronization every 10 seconds
-- **News Aggregation** - Latest crypto news from multiple sources
-- **Market Analytics** - DeFi TVL, NFT volume, gas prices, Fear & Greed Index
-
-### 🤖 AI & Machine Learning
-
-- **CryptoBERT Integration** - ElKulako/CryptoBERT model with authentication
-- **Sentiment Analysis** - Multi-model approach (Twitter, Financial, Crypto-specific)
-- **Market Trend Prediction** - Technical analysis with RSI, MA, support/resistance
-- **Text Summarization** - Automatic news and report summarization
-
-### 📊 Data Sources
-
-- **150+ API Providers** - CoinGecko, Binance, DeFiLlama, Etherscan, and more
-- **Multiple Categories**:
-  - Market Data (CoinGecko, CoinCap, CryptoCom pare)
-  - DeFi Protocols (DeFiLlama, Aave, Uniswap)
-  - Blockchain Explorers (Etherscan, BscScan, PolygonScan)
-  - NFT Marketplaces (OpenSea, Rarible, Reservoir)
-  - News Sources (CoinDesk, Cointelegraph, CryptoPanic)
-  - Social Media (Reddit, Twitter trends)
-  - Analytics (Glassnode, IntoTheBlock, Messari)
-
----
-
-## 🐳 Docker Quick Start
-
-### Prerequisites
-
-- Docker 20.10+
-- Docker Compose 2.0+
-
-### Using Docker (Recommended)
-
-```bash
-# Build and run with Docker Compose
-docker-compose up -d
-
-# Access the dashboard
-open http://localhost:7860
-```
-
-### Using Pre-built Docker Image
-
-```bash
-# Pull the image
-docker pull your-registry/crypto-dashboard:latest
-
-# Run the container
-docker run -d \
-  -p 7860:7860 \
-  -e HF_TOKEN=your_token_here \
-  --name crypto-dashboard \
-  your-registry/crypto-dashboard:latest
-
-# View logs
-docker logs -f crypto-dashboard
-```
-
----
-
-## 🚀 Installation
-
-### Option 1: Docker (Recommended)
-
-1. **Clone the repository**
-   ```bash
-   git clone https://github.com/yourusername/crypto-dashboard.git
-   cd crypto-dashboard
-   ```
-
-2. **Configure environment variables**
-   ```bash
-   cp .env.example .env
-   # Edit .env with your API keys
-   ```
-
-3. **Build and run**
-   ```bash
-   docker-compose up -d
-   ```
-
-4. **Access the application**
-   - Dashboard: http://localhost:7860
-   - API Docs: http://localhost:7860/docs
-   - Admin Panel: http://localhost:7860/admin
-
-### Option 2: Local Development
-
-1. **Install Python 3.10+**
-   ```bash
-   python --version  # Should be 3.10 or higher
-   ```
-
-2. **Install dependencies**
-   ```bash
-   pip install -r requirements.txt
-   ```
-
-3. **Install AI models dependencies** (Optional)
-   ```bash
-   pip install transformers torch
-   ```
-
-4. **Set environment variables**
-   ```bash
-   export HF_TOKEN="your_huggingface_token"
-   ```
-
-5. **Run the application**
-   ```bash
-   # Start the professional dashboard
-   python3 api_dashboard_backend.py
-   
-   # Or start the provider monitor
-   python3 api_server_extended.py
-   ```
-
-### Option 3: Hugging Face Spaces
-
-1. **Fork this repository**
-
-2. **Create a new Space on Hugging Face**
-   - Choose "Gradio" or "Docker" SDK
-   - Connect your forked repository
-
-3. **Add secrets in Space settings**
-   ```
-   HF_TOKEN=your_token
-   ```
-
-4. **Deploy**
-   - Automatic deployment on push
-
----
-
-## 📖 Usage
-
-### Professional Dashboard
-
-The main dashboard provides a comprehensive view of the cryptocurrency market:
-
-```bash
-# Start the professional dashboard
-python3 api_dashboard_backend.py
-
-# Access at: http://localhost:7860
-```
-
-**Features:**
-- 🔍 Natural language query interface
-- 📈 Real-time price charts
-- 📊 Market statistics cards
-- 📰 Latest crypto news
-- 😊 Sentiment analysis visualization
-- 💹 Top cryptocurrencies table
-
-**Example Queries:**
-```
-"Bitcoin price"              → Current BTC price
-"Top 10 coins"              → List top 10 cryptocurrencies
-"Ethereum trend"            → ETH price trend chart
-"Market sentiment"          → Bullish/bearish analysis
-"DeFi TVL"                  → Total Value Locked in DeFi
-"NFT volume"                → Daily NFT trading volume
-"Gas prices"                → Current Ethereum gas fees
-```
-
-### Provider Monitoring Dashboard
-
-Monitor all API providers and their health status:
-
-```bash
-# Start the provider monitor
-python3 api_server_extended.py
-
-# Access at: http://localhost:7860
-```
-
-**Features:**
-- ✅ Real-time provider status (validated/unvalidated)
-- 📊 Response time monitoring
-- 🔄 Auto-refresh every 30 seconds
-- 🏷️ Category-based filtering
-- 🔍 Search functionality
-- 📈 Statistics dashboard
-
-### API Endpoints
-
-#### REST API
-
-```bash
-# Health check
-GET /api/health
-
-# Top cryptocurrencies
-GET /api/coins/top?limit=10
-
-# Specific coin details
-GET /api/coins/{symbol}
-
-# Market statistics
-GET /api/market/stats
-
-# Latest news
-GET /api/news/latest?limit=10
-
-# Process user query
-POST /api/query
-{
-  "query": "Bitcoin price"
-}
-
-# API providers list
-GET /api/providers
-
-# Historical price data
-GET /api/charts/price/{symbol}?timeframe=7d
-```
-
-#### WebSocket
-
-```javascript
-// Connect to real-time updates
-const ws = new WebSocket('ws://localhost:7860/ws');
-
-ws.onmessage = (event) => {
-  const data = JSON.parse(event.data);
-  console.log('Received:', data);
-};
-```
-
-### CryptoBERT AI Model
-
-Use the CryptoBERT model for crypto-specific sentiment analysis:
-
-```python
-import ai_models
-
-# Initialize models
-ai_models.initialize_models()
-
-# Analyze sentiment
-text = "Bitcoin shows strong bullish momentum"
-result = ai_models.analyze_crypto_sentiment(text)
-
-print(f"Sentiment: {result['label']}")
-print(f"Confidence: {result['score']}")
-print(f"Predictions: {result['predictions']}")
-```
-
----
-
-## 🏗️ Architecture
-
-```
-┌─────────────────────────────────────────────────────────┐
-│                    Frontend Layer                       │
-│  ┌──────────────┐  ┌──────────────┐  ┌─────────────┐  │
-│  │ Dashboard UI │  │ Admin Panel  │  │ Charts/Viz  │  │
-│  │ (HTML/JS)    │  │ (HTML/JS)    │  │ (Chart.js)  │  │
-│  └──────────────┘  └──────────────┘  └─────────────┘  │
-└─────────────────────────────────────────────────────────┘
-                        ↕ HTTP/WebSocket
-┌─────────────────────────────────────────────────────────┐
-│                    Backend Layer                        │
-│  ┌──────────────┐  ┌──────────────┐  ┌─────────────┐  │
-│  │ FastAPI      │  │ WebSocket    │  │ Query       │  │
-│  │ REST API     │  │ Manager      │  │ Parser      │  │
-│  └──────────────┘  └──────────────┘  └─────────────┘  │
-└─────────────────────────────────────────────────────────┘
-                        ↕
-┌─────────────────────────────────────────────────────────┐
-│                    Services Layer                       │
-│  ┌──────────────┐  ┌──────────────┐  ┌─────────────┐  │
-│  │ AI Models    │  │ Provider     │  │ Data        │  │
-│  │ (CryptoBERT) │  │ Manager      │  │ Aggregator  │  │
-│  └──────────────┘  └──────────────┘  └─────────────┘  │
-└─────────────────────────────────────────────────────────┘
-                        ↕
-┌─────────────────────────────────────────────────────────┐
-│                    Data Layer                           │
-│  ┌───���──────────┐  ┌──────────────┐  ┌─────────────┐  │
-│  │ SQLite       │  │ Redis Cache  │  │ JSON Config │  │
-│  │ Database     │  │ (Optional)   │  │ Files       │  │
-│  └──────────────┘  └──────────────┘  └─────────────┘  │
-└─────────────────────────────────────────────────────────┘
-                        ↕
-┌─────────────────────────────────────────────────────────┐
-│                External Data Sources                    │
-│  CoinGecko • Binance • DeFiLlama • Etherscan           │
-│  OpenSea • CryptoPanic • Reddit • Hugging Face         │
-└─────────────────────────────────────────────────────────┘
-```
-
----
-
-## 📂 Project Structure
-
-```
-crypto-dashboard/
-├── 🐳 Docker Files
-│   ├── Dockerfile                 # Main Docker configuration
-│   ├── docker-compose.yml         # Docker Compose setup
-│   └── .dockerignore             # Docker ignore file
-│
-├── 🎨 Frontend
-│   ├── crypto_dashboard_pro.html  # Professional dashboard
-│   ├── admin_improved.html        # Provider monitoring dashboard
-│   ├── dashboard_standalone.html  # Standalone dashboard
-│   └── static/                    # Static assets (CSS, JS)
-│
-├── 🔧 Backend
-│   ├── api_dashboard_backend.py   # Main API server
-│   ├── api_server_extended.py     # Provider monitoring API
-│   ├── api/                       # API endpoints
-│   ├── backend/                   # Business logic
-│   └── monitoring/                # Monitoring services
-│
-├── 🤖 AI & ML
-│   ├── ai_models.py               # AI models integration
-│   ├── config.py                  # Configuration (HF models)
-│   └── collectors/                # Data collectors
-│
-├── 💾 Data & Config
-│   ├── providers_config_extended.json  # API providers config
-│   ├── database/                  # Database modules
-│   └── data/                      # Data storage
-│
-├── 📖 Documentation
-│   ├── README.md                  # This file
-│   ├── PROFESSIONAL_DASHBOARD_GUIDE.md
-│   ├── QUICK_START_PROFESSIONAL.md
-│   ├── CRYPTOBERT_INTEGRATION.md
-│   ├── PROVIDER_DASHBOARD_GUIDE.md
-│   └── docs/                      # Additional documentation
-│
-├── 🧪 Tests
-│   ├── tests/                     # Test files
-│   ├── test_cryptobert.py        # CryptoBERT tests
-│   └── test_integration.py       # Integration tests
-│
-└── 📦 Configuration
-    ├── requirements.txt           # Python dependencies
-    ├── requirements-dev.txt       # Dev dependencies
-    ├── .env.example              # Environment variables template
-    └── pyproject.toml            # Project metadata
-```
-
----
-
-## 🔧 Configuration
-
-### Environment Variables
-
-Create a `.env` file in the root directory:
-
-```bash
-# Hugging Face
-HF_TOKEN=your_huggingface_token_here
-
-# API Keys (Optional - for real data)
-CMC_API_KEY=your_coinmarketcap_key
-ETHERSCAN_KEY=your_etherscan_key
-NEWSAPI_KEY=your_newsapi_key
-
-# Application Settings
-LOG_LEVEL=INFO
-PORT=7860
-HOST=0.0.0.0
-
-# Database
-DATABASE_PATH=data/crypto_aggregator.db
-
-# Cache (Optional)
-REDIS_URL=redis://localhost:6379
-CACHE_TTL=300
-
-# AI Models
-ENABLE_AI_MODELS=true
-```
-
-### Provider Configuration
-
-Edit `providers_config_extended.json` to add/modify API providers:
-
-```json
-{
-  "providers": {
-    "your_provider_id": {
-      "name": "Your Provider Name",
-      "base_url": "https://api.example.com",
-      "category": "market_data",
-      "requires_auth": false,
-      "priority": 10
-    }
-  }
-}
-```
-
----
-
-## 🛠️ Development
-
-### Setup Development Environment
-
-```bash
-# Clone repository
-git clone https://github.com/yourusername/crypto-dashboard.git
-cd crypto-dashboard
-
-# Create virtual environment
-python3 -m venv venv
-source venv/bin/activate  # On Windows: venv\Scripts\activate
-
-# Install dependencies
-pip install -r requirements-dev.txt
-
-# Install pre-commit hooks
-pre-commit install
-
-# Run tests
-pytest tests/
-
-# Run with hot reload
-uvicorn api_dashboard_backend:app --reload --port 7860
-```
-
-### Running Tests
-
-```bash
-# Run all tests
-pytest
-
-# Run specific test file
-pytest tests/test_cryptobert.py
-
-# Run with coverage
-pytest --cov=. --cov-report=html
-
-# Run integration tests
-python3 test_integration.py
-```
-
-### Code Quality
-
-```bash
-# Format code
-black .
-
-# Lint code
-flake8 .
-
-# Type checking
-mypy .
-
-# Security scan
-bandit -r .
-```
-
----
-
-## 🚢 Deployment
-
-### Docker Production Deployment
-
-```bash
-# Build production image
-docker build -t crypto-dashboard:latest .
-
-# Run with production settings
-docker run -d \
-  -p 80:7860 \
-  -e HF_TOKEN=${HF_TOKEN} \
-  -e LOG_LEVEL=WARNING \
-  --restart unless-stopped \
-  --name crypto-dashboard-prod \
-  crypto-dashboard:latest
-```
-
-### Kubernetes Deployment
-
-```yaml
-# deployment.yaml
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: crypto-dashboard
-spec:
-  replicas: 3
-  selector:
-    matchLabels:
-      app: crypto-dashboard
-  template:
-    metadata:
-      labels:
-        app: crypto-dashboard
-    spec:
-      containers:
-      - name: crypto-dashboard
-        image: your-registry/crypto-dashboard:latest
-        ports:
-        - containerPort: 7860
-        env:
-        - name: HF_TOKEN
-          valueFrom:
-            secretKeyRef:
-              name: crypto-secrets
-              key: hf-token
-```
-
-Deploy:
-```bash
-kubectl apply -f deployment.yaml
-kubectl apply -f service.yaml
-```
-
-### Hugging Face Spaces
-
-1. Create `README.md` in your Space
-2. Add `requirements.txt`
-3. Create `app.py`:
-   ```python
-   from api_dashboard_backend import app
-   ```
-4. Set secrets in Space settings
-5. Push to deploy
-
-### AWS/GCP/Azure
-
-See `docs/DEPLOYMENT_MASTER_GUIDE.md` for detailed cloud deployment instructions.
-
----
-
-## 📊 API Documentation
-
-### Interactive API Docs
-
-Once the server is running, visit:
-
-- **Swagger UI**: http://localhost:7860/docs
-- **ReDoc**: http://localhost:7860/redoc
-
-### API Examples
-
-#### Get Top Cryptocurrencies
-
-```bash
-curl http://localhost:7860/api/coins/top?limit=10
-```
-
-Response:
-```json
-{
-  "success": true,
-  "coins": [
-    {
-      "name": "Bitcoin",
-      "symbol": "BTC",
-      "price": 43250.50,
-      "change_24h": 2.34,
-      "market_cap": 845000000000,
-      "volume_24h": 25000000000
-    }
-  ],
-  "count": 10
-}
-```
-
-#### Process Natural Language Query
-
-```bash
-curl -X POST http://localhost:7860/api/query \
-  -H "Content-Type: application/json" \
-  -d '{"query": "Bitcoin price"}'
-```
-
-Response:
-```json
-{
-  "success": true,
-  "type": "price",
-  "coin": "Bitcoin",
-  "symbol": "BTC",
-  "price": 43250.50,
-  "message": "Bitcoin (BTC) is currently $43,250.50"
-}
-```
-
----
-
-## 🤝 Contributing
-
-We welcome contributions! Please see our [Contributing Guidelines](CONTRIBUTING.md).
-
-### How to Contribute
-
-1. **Fork the repository**
-2. **Create a feature branch**
-   ```bash
-   git checkout -b feature/amazing-feature
-   ```
-3. **Commit your changes**
-   ```bash
-   git commit -m 'Add amazing feature'
-   ```
-4. **Push to the branch**
-   ```bash
-   git push origin feature/amazing-feature
-   ```
-5. **Open a Pull Request**
-
-### Development Guidelines
-
-- Follow PEP 8 style guide
-- Write tests for new features
-- Update documentation
-- Use type hints
-- Add docstrings to functions
-
----
-
-## 📝 License
-
-This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
-
----
-
-## 🙏 Acknowledgments
-
-### Technologies Used
-
-- **FastAPI** - Modern web framework for building APIs
-- **Hugging Face Transformers** - AI model integration
-- **Chart.js** - Interactive charts and visualizations
-- **Docker** - Containerization platform
-- **Python 3.10+** - Programming language
-
-### Data Providers
-
-- **CoinGecko** - Cryptocurrency market data
-- **Binance** - Real-time trading data
-- **DeFiLlama** - DeFi protocol analytics
-- **Etherscan** - Ethereum blockchain explorer
-- **OpenSea** - NFT marketplace data
-- **CryptoPanic** - Crypto news aggregation
-
-### AI Models
-
-- **ElKulako/CryptoBERT** - Crypto-specific sentiment analysis
-- **cardiffnlp/twitter-roberta-base-sentiment** - Twitter sentiment
-- **ProsusAI/finbert** - Financial sentiment analysis
-- **facebook/bart-large-cnn** - Text summarization
-
----
-
-## 📞 Support
-
-### Documentation
-
-- **Quick Start**: [QUICK_START_PROFESSIONAL.md](QUICK_START_PROFESSIONAL.md)
-- **Full Guide**: [PROFESSIONAL_DASHBOARD_GUIDE.md](PROFESSIONAL_DASHBOARD_GUIDE.md)
-- **CryptoBERT Integration**: [CRYPTOBERT_INTEGRATION.md](docs/CRYPTOBERT_INTEGRATION.md)
-- **Provider Dashboard**: [PROVIDER_DASHBOARD_GUIDE.md](PROVIDER_DASHBOARD_GUIDE.md)
-
-### Getting Help
-
-- 📖 Check the documentation
-- 🐛 Open an issue on GitHub
-- 💬 Join our community discussions
-- 📧 Contact: [email protected]
-
-### Troubleshooting
-
-**Dashboard not loading?**
-```bash
-# Check if server is running
-curl http://localhost:7860/api/health
-
-# Check Docker logs
-docker logs crypto-dashboard
-```
-
-**WebSocket not connecting?**
-```bash
-# Verify WebSocket endpoint
-wscat -c ws://localhost:7860/ws
-```
-
-**AI models not loading?**
-```bash
-# Check HF_TOKEN is set
-echo $HF_TOKEN
-
-# Test model loading
-python3 test_cryptobert.py
-```
-
----
-
-## 🗺️ Roadmap
-
-### Current Version (v1.0)
-- ✅ Professional dashboard
-- ✅ Provider monitoring
-- ✅ CryptoBERT integration
-- ✅ Natural language queries
-- ✅ Real-time WebSocket updates
-- ✅ Docker containerization
-
-### Planned Features (v1.1)
-- [ ] Portfolio tracking
-- [ ] Price alerts
-- [ ] Advanced charting (candlesticks)
-- [ ] Social sentiment analysis
-- [ ] Multi-language support
-- [ ] Mobile app
-
-### Future Enhancements (v2.0)
-- [ ] AI-powered predictions
-- [ ] Trading signals
-- [ ] Automated trading (with approval)
-- [ ] Desktop application
-- [ ] Browser extension
-- [ ] API marketplace integration
-
----
-
-## 📈 Statistics
-
-- **150+ API Providers** integrated
-- **4 AI Models** for sentiment analysis
-- **10+ API Endpoints** available
-- **Real-time Updates** every 10 seconds
-- **100% Docker** compatible
-- **Mobile Responsive** design
-
----
-
-## 🔒 Security
-
-### Security Features
-
-- ✅ Environment variable configuration
-- ✅ CORS protection
-- ✅ Input validation
-- ✅ Error handling
-- ✅ Rate limiting (optional)
-- ✅ API key management
-
-### Reporting Security Issues
-
-Please report security vulnerabilities to: [email protected]
-
-**Do not** create public GitHub issues for security vulnerabilities.
-
----
-
-## 📜 Changelog
-
-See [CHANGELOG.md](CHANGELOG.md) for a detailed history of changes.
-
-### Recent Updates
-
-**v1.0.0** (2025-11-16)
-- Initial release
-- Professional dashboard with query system
-- CryptoBERT AI model integration
-- Provider monitoring dashboard
-- Docker containerization
-- Complete documentation
-
----
-
-## 🌐 Links
-
-- **Website**: https://your-site.com
-- **Documentation**: https://docs.your-site.com
-- **GitHub**: https://github.com/yourusername/crypto-dashboard
-- **Docker Hub**: https://hub.docker.com/r/yourusername/crypto-dashboard
-- **Hugging Face**: https://huggingface.co/spaces/yourusername/crypto-dashboard
-
----
-
-## ⭐ Star History
-
-If you find this project useful, please consider giving it a star ⭐️
-
-[![Star History Chart](https://api.star-history.com/svg?repos=yourusername/crypto-dashboard&type=Date)](https://star-history.com/#yourusername/crypto-dashboard&Date)
-
----
-
-## 📄 Citation
-
-If you use this project in your research or work, please cite:
-
-```bibtex
-@software{crypto_dashboard_2025,
-  author = {Your Name},
-  title = {Crypto Intelligence Dashboard},
-  year = {2025},
-  url = {https://github.com/yourusername/crypto-dashboard}
-}
-```
-
----
-
-<div align="center">
-
-**Built with ❤️ using Docker, Python, and FastAPI**
-
-[Report Bug](https://github.com/yourusername/crypto-dashboard/issues) · 
-[Request Feature](https://github.com/yourusername/crypto-dashboard/issues) · 
-[Documentation](https://docs.your-site.com)
-
-</div>
+---
+title: Crypto Intelligence Hub
+emoji: 🚀
+colorFrom: blue
+colorTo: purple
+sdk: docker
+pinned: false
+license: mit
+app_port: 7860
+---
+
+# Crypto Intelligence Hub - مرکز هوش رمز ارز
+
+یک رابط کاربری کامل و یکپارچه برای جمع‌آوری داده‌های رمز ارز با استفاده از منابع رایگان و مدل‌های Hugging Face.
+
+## ✨ ویژگی‌ها
+
+- 📊 **داشبورد جامع**: نمایش خلاصه منابع و مدل‌ها
+- 📚 **منابع رایگان**: دسترسی به بیش از 200 منبع رایگان برای داده‌های رمز ارز
+- 🤖 **مدل‌های AI**: استفاده از مدل‌های Hugging Face برای تحلیل احساسات
+- 💭 **تحلیل احساسات**: تحلیل متن با مدل‌های تخصصی مالی و رمز ارز
+- 🔌 **یکپارچه‌سازی API**: اتصال به بک‌اند FastAPI برای سرویس‌های پیشرفته
+
+## 🚀 استفاده
+
+### داشبورد
+نمایش خلاصه منابع، مدل‌ها و آمار کلی سیستم
+
+### منابع داده
+لیست کامل منابع رایگان برای:
+- داده‌های بازار (Market Data)
+- کاوشگرهای بلاکچین (Block Explorers)
+- نودهای RPC
+- اخبار و احساسات
+- ردیابی نهنگ‌ها (Whale Tracking)
+
+### مدل‌های AI
+مدل‌های Hugging Face در دسترس:
+- تحلیل احساسات مالی (FinBERT)
+- تحلیل احساسات رمز ارز (CryptoBERT)
+- تحلیل احساسات شبکه‌های اجتماعی
+- و مدل‌های بیشتر...
+
+### تحلیل احساسات
+وارد کردن متن و دریافت تحلیل احساسات با استفاده از مدل‌های پیشرفته
+
+## 📁 ساختار پروژه
+
+```
+.
+├── app.py                          # فایل اصلی اپلیکیشن Gradio
+├── api_server_extended.py         # بک‌اند FastAPI
+├── ai_models.py                   # مدیریت مدل‌های Hugging Face
+├── api-resources/                 # فایل‌های JSON منابع
+│   └── crypto_resources_unified_2025-11-11.json
+├── all_apis_merged_2025.json     # فایل جامع APIها
+├── Dockerfile                     # فایل Docker برای Hugging Face Space
+└── requirements_hf.txt           # وابستگی‌های Python
+```
+
+## 🔧 تنظیمات
+
+### متغیرهای محیطی (Environment Variables)
+
+برای استفاده از مدل‌های خصوصی Hugging Face:
+
+```bash
+HF_TOKEN=your_huggingface_token
+```
+
+### منابع داده
+
+منابع از فایل‌های JSON زیر بارگذاری می‌شوند:
+- `api-resources/crypto_resources_unified_2025-11-11.json`
+- `all_apis_merged_2025.json`
+
+## 📊 منابع پشتیبانی شده
+
+### داده‌های بازار
+- CoinGecko (رایگان)
+- CoinMarketCap
+- Binance Public API
+- CoinCap
+- و بیشتر...
+
+### کاوشگرهای بلاکچین
+- Etherscan
+- BscScan
+- TronScan
+- Blockchair
+- و بیشتر...
+
+### RPC Nodes
+- Infura
+- Alchemy
+- Ankr
+- PublicNode
+- و بیشتر...
+
+## 🤖 مدل‌های AI
+
+- **FinBERT**: تحلیل احساسات مالی
+- **CryptoBERT**: تحلیل احساسات رمز ارز
+- **Twitter-RoBERTa**: تحلیل احساسات شبکه‌های اجتماعی
+- و مدل‌های بیشتر...
+
+## 🛠️ توسعه
+
+### اجرای محلی
+
+```bash
+# نصب وابستگی‌ها
+pip install -r requirements_hf.txt
+
+# اجرای اپلیکیشن
+python app.py
+```
+
+### ساخت Docker Image
+
+```bash
+docker build -t crypto-intelligence-hub .
+docker run -p 7860:7860 crypto-intelligence-hub
+```
+
+## 📝 مجوز
+
+MIT License
+
+## 🤝 مشارکت
+
+مشارکت‌ها خوش‌آمد هستند! لطفاً برای تغییرات بزرگ ابتدا یک Issue باز کنید.
+
+## 📧 تماس
+
+برای سوالات و پشتیبانی، لطفاً یک Issue در مخزن باز کنید.
+
+---
+
+**نکته**: این اپلیکیشن از منابع رایگان استفاده می‌کند. برای استفاده از APIهای پولی، کلیدهای API را در تنظیمات Hugging Face Space اضافه کنید.
+

TEST_ENDPOINTS.sh

@@ -1,161 +1,88 @@
-#!/bin/bash
-# API Endpoints Test Script
-# Run this after starting the backend to verify all endpoints work
-
-BASE_URL="${BASE_URL:-http://localhost:7860}"
-GREEN='\033[0;32m'
-RED='\033[0;31m'
-YELLOW='\033[1;33m'
-NC='\033[0m' # No Color
-
-echo "======================================"
-echo "🧪 Testing Crypto HF API Endpoints"
-echo "======================================"
-echo "Base URL: $BASE_URL"
-echo ""
-
-# Function to test endpoint
-test_endpoint() {
-    local method=$1
-    local endpoint=$2
-    local data=$3
-    local name=$4
-    
-    echo -n "Testing $name... "
-    
-    if [ "$method" = "GET" ]; then
-        response=$(curl -s -o /dev/null -w "%{http_code}" "$BASE_URL$endpoint")
-    else
-        response=$(curl -s -o /dev/null -w "%{http_code}" -X "$method" "$BASE_URL$endpoint" \
-            -H "Content-Type: application/json" \
-            -d "$data")
-    fi
-    
-    if [ "$response" = "200" ]; then
-        echo -e "${GREEN}✅ OK${NC} (HTTP $response)"
-    else
-        echo -e "${RED}❌ FAILED${NC} (HTTP $response)"
-        return 1
-    fi
-}
-
-# Test health
-test_endpoint "GET" "/api/health" "" "Health Check"
-
-# Test market endpoints
-echo ""
-echo "📊 Market Endpoints:"
-test_endpoint "GET" "/api/coins/top?limit=5" "" "Top Coins"
-test_endpoint "GET" "/api/coins/BTC" "" "Bitcoin Details"
-test_endpoint "GET" "/api/market/stats" "" "Market Stats"
-
-# Test chart endpoints
-echo ""
-echo "📈 Chart Endpoints:"
-test_endpoint "GET" "/api/charts/price/BTC?timeframe=7d" "" "BTC Price Chart"
-
-# POST endpoint for chart analyze
-echo -n "Testing Chart Analysis... "
-response=$(curl -s -w "\n%{http_code}" -X POST "$BASE_URL/api/charts/analyze" \
-    -H "Content-Type: application/json" \
-    -d '{"symbol":"BTC","timeframe":"7d","indicators":[]}')
-http_code=$(echo "$response" | tail -n1)
-if [ "$http_code" = "200" ]; then
-    echo -e "${GREEN}✅ OK${NC} (HTTP $http_code)"
-else
-    echo -e "${RED}❌ FAILED${NC} (HTTP $http_code)"
-fi
-
-# Test news endpoints
-echo ""
-echo "📰 News Endpoints:"
-test_endpoint "GET" "/api/news/latest?limit=5" "" "Latest News"
-
-# POST endpoint for news summarize
-echo -n "Testing News Summarize... "
-response=$(curl -s -w "\n%{http_code}" -X POST "$BASE_URL/api/news/summarize" \
-    -H "Content-Type: application/json" \
-    -d '{"title":"Bitcoin breaks new record","description":"BTC hits $50k"}')
-http_code=$(echo "$response" | tail -n1)
-if [ "$http_code" = "200" ]; then
-    echo -e "${GREEN}✅ OK${NC} (HTTP $http_code)"
-else
-    echo -e "${RED}❌ FAILED${NC} (HTTP $http_code)"
-fi
-
-# Test AI endpoints
-echo ""
-echo "🤖 AI Endpoints:"
-
-# POST endpoint for sentiment
-echo -n "Testing Sentiment Analysis... "
-response=$(curl -s -w "\n%{http_code}" -X POST "$BASE_URL/api/sentiment/analyze" \
-    -H "Content-Type: application/json" \
-    -d '{"text":"Bitcoin is breaking new all-time highs!"}')
-http_code=$(echo "$response" | tail -n1)
-body=$(echo "$response" | head -n-1)
-if [ "$http_code" = "200" ]; then
-    sentiment=$(echo "$body" | grep -o '"sentiment":"[^"]*"' | cut -d'"' -f4)
-    confidence=$(echo "$body" | grep -o '"confidence":[0-9.]*' | cut -d':' -f2)
-    echo -e "${GREEN}✅ OK${NC} (HTTP $http_code) - Sentiment: ${YELLOW}$sentiment${NC} (${confidence})"
-else
-    echo -e "${RED}❌ FAILED${NC} (HTTP $http_code)"
-fi
-
-# POST endpoint for query
-echo -n "Testing Query... "
-response=$(curl -s -w "\n%{http_code}" -X POST "$BASE_URL/api/query" \
-    -H "Content-Type: application/json" \
-    -d '{"query":"What is the price of Bitcoin?"}')
-http_code=$(echo "$response" | tail -n1)
-if [ "$http_code" = "200" ]; then
-    echo -e "${GREEN}✅ OK${NC} (HTTP $http_code)"
-else
-    echo -e "${RED}❌ FAILED${NC} (HTTP $http_code)"
-fi
-
-# Test provider endpoints
-echo ""
-echo "🔌 Provider Endpoints:"
-test_endpoint "GET" "/api/providers" "" "Providers List"
-
-# Test datasets endpoints
-echo ""
-echo "📚 Datasets & Models Endpoints:"
-test_endpoint "GET" "/api/datasets/list" "" "Datasets List"
-test_endpoint "GET" "/api/models/list" "" "Models List"
-
-# POST endpoint for model test
-echo -n "Testing Model Test... "
-response=$(curl -s -w "\n%{http_code}" -X POST "$BASE_URL/api/models/test" \
-    -H "Content-Type: application/json" \
-    -d '{"model":"crypto_sent_0","text":"Ethereum price surging!"}')
-http_code=$(echo "$response" | tail -n1)
-if [ "$http_code" = "200" ]; then
-    echo -e "${GREEN}✅ OK${NC} (HTTP $http_code)"
-else
-    echo -e "${RED}❌ FAILED${NC} (HTTP $http_code)"
-fi
-
-# Summary
-echo ""
-echo "======================================"
-echo "📊 Test Summary"
-echo "======================================"
-echo ""
-echo "✅ All critical endpoints tested"
-echo ""
-echo "🌐 Dashboard URLs:"
-echo "  - Main: $BASE_URL/"
-echo "  - Admin: $BASE_URL/admin.html"
-echo "  - API Docs: $BASE_URL/docs"
-echo ""
-echo "🔌 WebSocket:"
-echo "  - ws://$(echo $BASE_URL | sed 's|http://||')/ws"
-echo ""
-echo "💡 Next steps:"
-echo "  1. Open $BASE_URL/ in your browser"
-echo "  2. Check all dashboard tabs"
-echo "  3. Verify WebSocket connection (status indicator)"
-echo ""
-echo "======================================"
+#!/bin/bash
+# Script to test all HuggingFace Space endpoints
+
+BASE_URL="https://really-amin-datasourceforcryptocurrency.hf.space"
+
+echo "=================================="
+echo "🧪 Testing HuggingFace Space API"
+echo "=================================="
+echo ""
+
+# Color codes
+GREEN='\033[0;32m'
+RED='\033[0;31m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+test_endpoint() {
+    local name=$1
+    local endpoint=$2
+    
+    echo -n "Testing $name ... "
+    response=$(curl -s -w "\n%{http_code}" "$BASE_URL$endpoint" 2>&1)
+    http_code=$(echo "$response" | tail -n1)
+    body=$(echo "$response" | head -n-1)
+    
+    if [ "$http_code" = "200" ]; then
+        echo -e "${GREEN}✓ OK${NC} (HTTP $http_code)"
+        return 0
+    else
+        echo -e "${RED}✗ FAILED${NC} (HTTP $http_code)"
+        echo "  Response: $body"
+        return 1
+    fi
+}
+
+# Core Endpoints
+echo "📊 Core Endpoints"
+echo "==================="
+test_endpoint "Health" "/health"
+test_endpoint "Info" "/info"
+test_endpoint "Providers" "/api/providers"
+echo ""
+
+# Data Endpoints
+echo "💰 Market Data Endpoints"
+echo "========================="
+test_endpoint "OHLCV (BTC)" "/api/ohlcv?symbol=BTCUSDT&interval=1h&limit=10"
+test_endpoint "Top Prices" "/api/crypto/prices/top?limit=5"
+test_endpoint "BTC Price" "/api/crypto/price/BTC"
+test_endpoint "Market Overview" "/api/crypto/market-overview"
+test_endpoint "Multiple Prices" "/api/market/prices?symbols=BTC,ETH,SOL"
+test_endpoint "Market Data Prices" "/api/market-data/prices?symbols=BTC,ETH"
+echo ""
+
+# Analysis Endpoints
+echo "📈 Analysis Endpoints"
+echo "====================="
+test_endpoint "Trading Signals" "/api/analysis/signals?symbol=BTCUSDT"
+test_endpoint "SMC Analysis" "/api/analysis/smc?symbol=BTCUSDT"
+test_endpoint "Scoring Snapshot" "/api/scoring/snapshot?symbol=BTCUSDT"
+test_endpoint "All Signals" "/api/signals"
+test_endpoint "Sentiment" "/api/sentiment"
+echo ""
+
+# System Endpoints
+echo "⚙️  System Endpoints"
+echo "===================="
+test_endpoint "System Status" "/api/system/status"
+test_endpoint "System Config" "/api/system/config"
+test_endpoint "Categories" "/api/categories"
+test_endpoint "Rate Limits" "/api/rate-limits"
+test_endpoint "Logs" "/api/logs?limit=10"
+test_endpoint "Alerts" "/api/alerts"
+echo ""
+
+# HuggingFace Endpoints
+echo "🤗 HuggingFace Endpoints"
+echo "========================="
+test_endpoint "HF Health" "/api/hf/health"
+test_endpoint "HF Registry" "/api/hf/registry?kind=models"
+echo ""
+
+echo "=================================="
+echo "✅ Testing Complete!"
+echo "=================================="
+echo ""
+echo "📖 Full documentation: ${BASE_URL}/docs"
+echo "📋 API Guide: See HUGGINGFACE_API_GUIDE.md"

admin.html

@@ -1,79 +1,525 @@
-<!DOCTYPE html>
+<!DOCTYPE html>
 <html lang="en">
 <head>
     <meta charset="UTF-8" />
-    <meta name="viewport" content="width=device-width, initial-scale=1" />
-    <title>Crypto Intelligence Admin</title>
-    <link rel="preconnect" href="https://fonts.googleapis.com" />
-    <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
-    <link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet" />
-    <link rel="stylesheet" href="/static/css/unified-ui.css" />
-    <link rel="stylesheet" href="/static/css/components.css" />
-    <script defer src="/static/js/ui-feedback.js"></script>
-    <script defer src="/static/js/admin-app.js"></script>
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>Crypto Intelligence Hub - HF Space</title>
+    <link rel="stylesheet" href="static/css/design-tokens.css" />
+    <link rel="stylesheet" href="static/css/design-system.css" />
+    <link rel="stylesheet" href="static/css/dashboard.css" />
+    <link rel="stylesheet" href="static/css/pro-dashboard.css" />
+    <script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/chart.umd.min.js" defer></script>
 </head>
-<body class="page page-admin">
-<header class="top-nav">
-    <div class="branding">
-        <svg width="32" height="32" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5"><path d="M4 12h16M12 4v16"/></svg>
-        <div>
-            <strong>Providers & Scheduling</strong>
-            <small style="color:var(--ui-text-muted);letter-spacing:0.2em;">/api/providers · /api/logs</small>
-        </div>
-    </div>
-    <nav class="nav-links">
-        <a href="/dashboard">Dashboard</a>
-        <a class="active" href="/admin">Admin</a>
-        <a href="/hf_console">HF Console</a>
-        <a href="/docs" target="_blank" rel="noreferrer">API Docs</a>
-    </nav>
-</header>
-
-<main class="page-content">
-    <section class="card">
-        <div class="section-heading">
-            <h2>Providers Health</h2>
-            <span class="badge info" id="providers-count">Loading...</span>
-        </div>
-        <div class="table-card">
-            <table>
-                <thead>
-                    <tr><th>Provider</th><th>Status</th><th>Response (ms)</th><th>Category</th></tr>
-                </thead>
-                <tbody id="providers-table">
-                    <tr><td colspan="4">Loading providers...</td></tr>
-                </tbody>
-            </table>
-        </div>
-    </section>
-
-    <section class="split-grid">
-        <article class="card" id="provider-detail">
-            <div class="section-heading">
-                <h2>Provider Detail</h2>
-                <span class="badge info" id="selected-provider">Select a provider</span>
+<body data-theme="dark">
+    <div class="app-shell">
+        <!-- Sidebar Navigation -->
+        <aside class="sidebar">
+            <div class="brand">
+                <strong>Crypto Intelligence Hub</strong>
+                <span class="env-pill">
+                    <svg width="12" height="12" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
+                        <path d="M12 2L2 7L12 12L22 7L12 2Z" stroke="currentColor" stroke-width="1.5" />
+                        <path d="M2 17L12 22L22 17" stroke="currentColor" stroke-width="1.5" />
+                        <path d="M2 12L12 17L22 12" stroke="currentColor" stroke-width="1.5" />
+                    </svg>
+                    HF Space
+                </span>
             </div>
-            <ul class="list" id="provider-detail-list"></ul>
-        </article>
-        <article class="card">
-            <div class="section-heading">
-                <h2>Configuration Snapshot</h2>
-                <span class="badge info" id="config-summary">Loading...</span>
+            <nav class="nav">
+                <button class="nav-button active" data-nav="page-overview">
+                    <svg width="16" height="16" viewBox="0 0 24 24" fill="none"><path d="M3 13h8V3H3v10zm0 8h8v-6H3v6zm10 0h8V11h-8v10zm0-18v6h8V3h-8z" fill="currentColor"/></svg>
+                    Overview
+                </button>
+                <button class="nav-button" data-nav="page-market">
+                    <svg width="16" height="16" viewBox="0 0 24 24" fill="none"><path d="M3 17l6-6 4 4 8-8" stroke="currentColor" stroke-width="2" stroke-linecap="round"/></svg>
+                    Market
+                </button>
+                <button class="nav-button" data-nav="page-chart">
+                    <svg width="16" height="16" viewBox="0 0 24 24" fill="none"><path d="M3 3v18h18" stroke="currentColor" stroke-width="2"/><path d="M7 10l4-4 4 4 6-6" stroke="currentColor" stroke-width="2"/></svg>
+                    Chart Lab
+                </button>
+                <button class="nav-button" data-nav="page-ai">
+                    <svg width="16" height="16" viewBox="0 0 24 24" fill="none"><circle cx="12" cy="12" r="3" fill="currentColor"/><path d="M12 2v4m0 12v4M4.93 4.93l2.83 2.83m8.48 8.48l2.83 2.83M2 12h4m12 0h4M4.93 19.07l2.83-2.83m8.48-8.48l2.83-2.83" stroke="currentColor" stroke-width="2" stroke-linecap="round"/></svg>
+                    AI Advisor
+                </button>
+                <button class="nav-button" data-nav="page-news">
+                    <svg width="16" height="16" viewBox="0 0 24 24" fill="none"><path d="M19 20H5a2 2 0 01-2-2V6a2 2 0 012-2h10l6 6v8a2 2 0 01-2 2z" stroke="currentColor" stroke-width="2"/><path d="M7 10h6m-6 4h8" stroke="currentColor" stroke-width="2" stroke-linecap="round"/></svg>
+                    News
+                </button>
+                <button class="nav-button" data-nav="page-providers">
+                    <svg width="16" height="16" viewBox="0 0 24 24" fill="none"><path d="M12 2L2 7l10 5 10-5-10-5zM2 17l10 5 10-5M2 12l10 5 10-5" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/></svg>
+                    Providers
+                </button>
+                <button class="nav-button" data-nav="page-datasets">
+                    <svg width="16" height="16" viewBox="0 0 24 24" fill="none"><path d="M4 7h16M4 12h16M4 17h16" stroke="currentColor" stroke-width="2" stroke-linecap="round"/></svg>
+                    Datasets & Models
+                </button>
+                <button class="nav-button" data-nav="page-api">
+                    <svg width="16" height="16" viewBox="0 0 24 24" fill="none"><path d="M13 2L3 14h9l-1 8 10-12h-9l1-8z" fill="currentColor"/></svg>
+                    API Explorer
+                </button>
+                <button class="nav-button" data-nav="page-debug">
+                    <svg width="16" height="16" viewBox="0 0 24 24" fill="none"><path d="M12 8v4m0 4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z" stroke="currentColor" stroke-width="2" stroke-linecap="round"/></svg>
+                    Diagnostics
+                </button>
+                <button class="nav-button" data-nav="page-settings">
+                    <svg width="16" height="16" viewBox="0 0 24 24" fill="none"><circle cx="12" cy="12" r="3" stroke="currentColor" stroke-width="2"/><path d="M12 1v6m0 6v6M5 5l4 4m6 6l4 4M1 12h6m6 0h6M5 19l4-4m6-6l4-4" stroke="currentColor" stroke-width="2" stroke-linecap="round"/></svg>
+                    Settings
+                </button>
+            </nav>
+            <div class="sidebar-footer">
+                <small>
+                    Crypto Intelligence Hub<br />
+                    <strong>10+ HF Models</strong> • <strong>14 Datasets</strong><br />
+                    Real-time data • Ensemble sentiment
+                </small>
             </div>
-            <ul class="list" id="config-list"></ul>
-        </article>
-    </section>
-
-    <section class="split-grid">
-        <article class="card">
-            <div class="section-heading"><h2>Logs ( /api/logs )</h2><span class="badge info">Latest</span></div>
-            <div id="logs-list" class="ws-stream"></div>
-        </article>
-        <article class="card">
-            <div class="section-heading"><h2>Alerts ( /api/alerts )</h2><span class="badge info">Live</span></div>
-            <div id="alerts-list"></div>
-        </article>
-    </section>
-</main>
+        </aside>
+
+        <!-- Main Content Area -->
+        <main class="main-area">
+            <!-- Top Bar with Status -->
+            <header class="topbar">
+                <div>
+                    <h1>Crypto Intelligence Dashboard</h1>
+                    <p class="text-muted">Live market data, AI-powered sentiment analysis, and comprehensive crypto intelligence</p>
+                </div>
+                <div class="status-group">
+                    <div class="status-pill" data-api-health data-state="warn">
+                        <span class="status-dot"></span>
+                        <span>checking</span>
+                    </div>
+                    <div class="status-pill" data-ws-status data-state="warn">
+                        <span class="status-dot"></span>
+                        <span>connecting</span>
+                    </div>
+                </div>
+            </header>
+
+            <div class="page-container">
+                <!-- ========== OVERVIEW PAGE ========== -->
+                <section id="page-overview" class="page active">
+                    <div class="section-header">
+                        <h2 class="section-title">Global Overview</h2>
+                        <span class="chip">Powered by /api/market/stats</span>
+                    </div>
+                    
+                    <!-- Market Stats Cards -->
+                    <div class="stats-grid" data-overview-stats>
+                        <div class="glass-card stat-card">
+                            <div class="stat-label">Total Market Cap</div>
+                            <div class="stat-value">Loading...</div>
+                        </div>
+                        <div class="glass-card stat-card">
+                            <div class="stat-label">24h Volume</div>
+                            <div class="stat-value">Loading...</div>
+                        </div>
+                        <div class="glass-card stat-card">
+                            <div class="stat-label">BTC Dominance</div>
+                            <div class="stat-value">Loading...</div>
+                        </div>
+                        <div class="glass-card stat-card">
+                            <div class="stat-label">Market Sentiment</div>
+                            <div class="stat-value">Loading...</div>
+                        </div>
+                    </div>
+
+                    <div class="grid-two">
+                        <!-- Top Coins Table -->
+                        <div class="glass-card">
+                            <div class="section-header">
+                                <h3>Top Coins</h3>
+                                <span class="text-muted">By market cap</span>
+                            </div>
+                            <div class="table-wrapper">
+                                <table>
+                                    <thead>
+                                        <tr>
+                                            <th>#</th>
+                                            <th>Symbol</th>
+                                            <th>Name</th>
+                                            <th>Price</th>
+                                            <th>24h %</th>
+                                            <th>Volume</th>
+                                            <th>Market Cap</th>
+                                        </tr>
+                                    </thead>
+                                    <tbody data-top-coins-body>
+                                        <tr><td colspan="7" style="text-align:center;padding:2rem;">Loading top coins...</td></tr>
+                                    </tbody>
+                                </table>
+                            </div>
+                        </div>
+
+                        <!-- Sentiment Chart -->
+                        <div class="glass-card">
+                            <div class="section-header">
+                                <h3>Global Sentiment</h3>
+                                <span class="text-muted">Ensemble HF models</span>
+                            </div>
+                            <canvas id="sentiment-chart" height="220"></canvas>
+                            <div style="margin-top:1rem;font-size:0.875rem;color:var(--text-secondary);">
+                                <strong>Models used:</strong> CryptoBERT, FinBERT, Twitter Sentiment<br>
+                                <strong>Method:</strong> Majority voting with confidence scoring
+                            </div>
+                        </div>
+                    </div>
+                </section>
+
+                <!-- ========== MARKET PAGE ========== -->
+                <section id="page-market" class="page">
+                    <div class="section-header">
+                        <h2 class="section-title">Market Intelligence</h2>
+                        <div class="controls-bar">
+                            <div class="input-chip">
+                                <svg viewBox="0 0 24 24" width="16" height="16"><path d="M21 20l-5.6-5.6A6.5 6.5 0 1 0 15.4 16L21 21zM5 10.5a5.5 5.5 0 1 1 11 0a5.5 5.5 0 0 1-11 0z" fill="currentColor"/></svg>
+                                <input type="text" placeholder="Search symbol" data-market-search />
+                            </div>
+                            <button class="ghost" data-refresh-market>Refresh</button>
+                        </div>
+                    </div>
+
+                    <div class="glass-card">
+                        <div class="table-wrapper">
+                            <table>
+                                <thead>
+                                    <tr>
+                                        <th>#</th>
+                                        <th>Symbol</th>
+                                        <th>Name</th>
+                                        <th>Price</th>
+                                        <th>24h %</th>
+                                        <th>Volume</th>
+                                        <th>Market Cap</th>
+                                        <th>Actions</th>
+                                    </tr>
+                                </thead>
+                                <tbody data-market-body>
+                                    <tr><td colspan="8" style="text-align:center;padding:2rem;">Loading market data...</td></tr>
+                                </tbody>
+                            </table>
+                        </div>
+                    </div>
+
+                    <!-- Coin Detail Drawer -->
+                    <div class="drawer" data-market-drawer style="display:none;">
+                        <button class="ghost" data-close-drawer>Close</button>
+                        <h3 data-drawer-symbol>—</h3>
+                        <div data-drawer-stats></div>
+                        <div class="glass-card" data-chart-wrapper>
+                            <canvas id="market-detail-chart" height="180"></canvas>
+                        </div>
+                        <div class="glass-card">
+                            <h4>AI Sentiment Analysis</h4>
+                            <div data-drawer-sentiment></div>
+                        </div>
+                    </div>
+                </section>
+
+                <!-- ========== CHART LAB PAGE ========== -->
+                <section id="page-chart" class="page">
+                    <div class="section-header">
+                        <h2 class="section-title">Chart Lab</h2>
+                        <div class="controls-bar">
+                            <select data-chart-symbol>
+                                <option value="BTC">Bitcoin (BTC)</option>
+                                <option value="ETH">Ethereum (ETH)</option>
+                                <option value="SOL">Solana (SOL)</option>
+                                <option value="BNB">BNB</option>
+                                <option value="XRP">Ripple (XRP)</option>
+                                <option value="ADA">Cardano (ADA)</option>
+                            </select>
+                            <div class="input-chip">
+                                <button class="ghost active" data-chart-timeframe="7d">7D</button>
+                                <button class="ghost" data-chart-timeframe="30d">30D</button>
+                                <button class="ghost" data-chart-timeframe="90d">90D</button>
+                            </div>
+                        </div>
+                    </div>
+
+                    <div class="glass-card">
+                        <canvas id="chart-lab-canvas" height="300"></canvas>
+                    </div>
+
+                    <div class="glass-card">
+                        <h4>Technical Analysis</h4>
+                        <div class="controls-bar">
+                            <label><input type="checkbox" data-indicator value="MA20" checked /> MA 20</label>
+                            <label><input type="checkbox" data-indicator value="MA50" /> MA 50</label>
+                            <label><input type="checkbox" data-indicator value="RSI" /> RSI</label>
+                            <label><input type="checkbox" data-indicator value="Volume" /> Volume</label>
+                        </div>
+                        <button class="primary" data-run-analysis>🤖 Analyze with AI</button>
+                        <div data-ai-insights class="ai-insights" style="margin-top:1rem;"></div>
+                    </div>
+                </section>
+
+                <!-- ========== AI ADVISOR PAGE ========== -->
+                <section id="page-ai" class="page">
+                    <div class="section-header">
+                        <h2 class="section-title">AI-Powered Sentiment & Advisory</h2>
+                        <span class="chip">Ensemble: CryptoBERT + FinBERT + Social</span>
+                    </div>
+
+                    <div class="glass-card">
+                        <h4>Sentiment Analysis</h4>
+                        <form data-sentiment-form>
+                            <label>Text to Analyze
+                                <textarea name="text" rows="4" placeholder="Enter crypto-related text, news headline, or social media post for sentiment analysis..."></textarea>
+                            </label>
+                            <button class="primary" type="submit">🧠 Analyze Sentiment</button>
+                        </form>
+                        <div data-sentiment-result style="margin-top:1rem;"></div>
+                    </div>
+
+                    <div class="glass-card" style="margin-top:1.5rem;">
+                        <h4>AI Query Interface</h4>
+                        <form data-query-form>
+                            <label>Ask a Question
+                                <textarea name="query" rows="3" placeholder="e.g., What is the current Bitcoin price? or Analyze Ethereum trend"></textarea>
+                            </label>
+                            <button class="primary" type="submit">🔍 Submit Query</button>
+                        </form>
+                        <div data-query-result style="margin-top:1rem;"></div>
+                    </div>
+
+                    <div class="inline-message inline-info">
+                        ⚠️ AI-generated outputs are experimental and should not be considered financial advice.
+                    </div>
+                </section>
+
+                <!-- ========== NEWS PAGE ========== -->
+                <section id="page-news" class="page">
+                    <div class="section-header">
+                        <h2 class="section-title">News & Headlines</h2>
+                        <span class="chip">With AI sentiment analysis</span>
+                    </div>
+
+                    <div class="controls-bar">
+                        <input type="text" placeholder="Search headlines..." data-news-search />
+                        <input type="text" placeholder="Filter by symbol (e.g., BTC)" data-news-symbol />
+                        <button class="ghost" data-refresh-news>Refresh</button>
+                    </div>
+
+                    <div class="glass-card">
+                        <div class="table-wrapper">
+                            <table>
+                                <thead>
+                                    <tr>
+                                        <th>Title</th>
+                                        <th>Source</th>
+                                        <th>Symbols</th>
+                                        <th>Sentiment</th>
+                                        <th>Time</th>
+                                        <th>Actions</th>
+                                    </tr>
+                                </thead>
+                                <tbody data-news-body>
+                                    <tr><td colspan="6" style="text-align:center;padding:2rem;">Loading news...</td></tr>
+                                </tbody>
+                            </table>
+                        </div>
+                    </div>
+                </section>
+
+                <!-- ========== PROVIDERS PAGE ========== -->
+                <section id="page-providers" class="page">
+                    <div class="section-header">
+                        <h2 class="section-title">API Providers</h2>
+                        <span class="chip">95+ data sources</span>
+                    </div>
+
+                    <div class="glass-card">
+                        <div class="table-wrapper">
+                            <table>
+                                <thead>
+                                    <tr>
+                                        <th>Provider</th>
+                                        <th>Category</th>
+                                        <th>Type</th>
+                                        <th>Status</th>
+                                        <th>Response Time</th>
+                                    </tr>
+                                </thead>
+                                <tbody data-providers-body>
+                                    <tr><td colspan="5" style="text-align:center;padding:2rem;">Loading providers...</td></tr>
+                                </tbody>
+                            </table>
+                        </div>
+                    </div>
+                </section>
+
+                <!-- ========== DATASETS & MODELS PAGE ========== -->
+                <section id="page-datasets" class="page">
+                    <div class="section-header">
+                        <h2 class="section-title">HuggingFace Datasets & Models</h2>
+                    </div>
+
+                    <div class="grid-two">
+                        <!-- Datasets -->
+                        <div class="glass-card">
+                            <h4>📊 Crypto Datasets (14+)</h4>
+                            <div class="table-wrapper">
+                                <table>
+                                    <thead>
+                                        <tr>
+                                            <th>Dataset</th>
+                                            <th>Category</th>
+                                            <th>Actions</th>
+                                        </tr>
+                                    </thead>
+                                    <tbody data-datasets-body>
+                                        <tr><td colspan="3" style="text-align:center;padding:1rem;">Loading...</td></tr>
+                                    </tbody>
+                                </table>
+                            </div>
+                        </div>
+
+                        <!-- Models -->
+                        <div class="glass-card">
+                            <h4>🤖 AI Models (10+)</h4>
+                            <div class="table-wrapper">
+                                <table>
+                                    <thead>
+                                        <tr>
+                                            <th>Model</th>
+                                            <th>Task</th>
+                                            <th>Status</th>
+                                        </tr>
+                                    </thead>
+                                    <tbody data-models-body>
+                                        <tr><td colspan="3" style="text-align:center;padding:1rem;">Loading...</td></tr>
+                                    </tbody>
+                                </table>
+                            </div>
+                        </div>
+                    </div>
+
+                    <!-- Model Test Form -->
+                    <div class="glass-card" style="margin-top:1.5rem;">
+                        <h4>🧪 Test a Model</h4>
+                        <form data-model-test-form>
+                            <div class="grid-two">
+                                <label>Model
+                                    <select name="model" data-model-select>
+                                        <option value="">Select a model...</option>
+                                    </select>
+                                </label>
+                                <label>Input Text
+                                    <textarea name="input" rows="3" placeholder="Enter text to test the model..."></textarea>
+                                </label>
+                            </div>
+                            <button class="primary" type="submit">Run Test</button>
+                        </form>
+                        <div data-model-test-output style="margin-top:1rem;"></div>
+                    </div>
+                </section>
+
+                <!-- ========== API EXPLORER PAGE ========== -->
+                <section id="page-api" class="page">
+                    <div class="section-header">
+                        <h2 class="section-title">API Explorer</h2>
+                        <span class="chip">15+ endpoints</span>
+                    </div>
+
+                    <div class="glass-card">
+                        <h4>Test Endpoint</h4>
+                        <form data-api-form>
+                            <div class="grid-two">
+                                <label>Endpoint
+                                    <select data-endpoint-select>
+                                        <option value="0">/api/health</option>
+                                    </select>
+                                </label>
+                                <label>Method
+                                    <select data-method-select>
+                                        <option value="GET">GET</option>
+                                        <option value="POST">POST</option>
+                                    </select>
+                                </label>
+                            </div>
+                            <div data-api-description style="margin:0.5rem 0;font-size:0.875rem;color:var(--text-secondary);"></div>
+                            <div data-api-path style="margin:0.5rem 0;font-family:monospace;font-size:0.875rem;"></div>
+                            <label>Body (JSON)
+                                <textarea data-body-input rows="4"></textarea>
+                            </label>
+                            <button class="primary" type="submit">Send Request</button>
+                        </form>
+                        <div data-api-response style="margin-top:1rem;"></div>
+                    </div>
+                </section>
+
+                <!-- ========== DIAGNOSTICS PAGE ========== -->
+                <section id="page-debug" class="page">
+                    <div class="section-header">
+                        <h2 class="section-title">System Diagnostics</h2>
+                    </div>
+
+                    <div class="grid-two">
+                        <div class="glass-card">
+                            <h4>Health Status</h4>
+                            <div data-health-info>Checking...</div>
+                        </div>
+
+                        <div class="glass-card">
+                            <h4>WebSocket Status</h4>
+                            <div data-ws-info>Checking...</div>
+                        </div>
+                    </div>
+
+                    <div class="glass-card" style="margin-top:1.5rem;">
+                        <h4>Request Logs</h4>
+                        <div data-request-logs style="max-height:400px;overflow-y:auto;font-family:monospace;font-size:0.875rem;">
+                            <!-- Populated by JS -->
+                        </div>
+                    </div>
+                </section>
+
+                <!-- ========== SETTINGS PAGE ========== -->
+                <section id="page-settings" class="page">
+                    <div class="section-header">
+                        <h2 class="section-title">Settings</h2>
+                    </div>
+
+                    <div class="glass-card">
+                        <h4>Display Settings</h4>
+                        <div class="grid-two">
+                            <label class="input-chip">Dark Theme
+                                <div class="toggle">
+                                    <input type="checkbox" data-theme-toggle checked />
+                                    <span></span>
+                                </div>
+                            </label>
+                            <label class="input-chip">Compact Layout
+                                <div class="toggle">
+                                    <input type="checkbox" data-layout-toggle />
+                                    <span></span>
+                                </div>
+                            </label>
+                        </div>
+                    </div>
+
+                    <div class="glass-card" style="margin-top:1.5rem;">
+                        <h4>Refresh Intervals</h4>
+                        <div class="grid-two">
+                            <label>Market Data (seconds)
+                                <input type="number" min="10" step="5" value="30" data-market-interval />
+                            </label>
+                            <label>News Feed (seconds)
+                                <input type="number" min="30" step="10" value="60" data-news-interval />
+                            </label>
+                        </div>
+                    </div>
+
+                    <div class="inline-message inline-info" style="margin-top:1.5rem;">
+                        Settings are stored locally in your browser.
+                    </div>
+                </section>
+            </div>
+        </main>
+    </div>
+
+    <!-- Load App JS as ES6 Module -->
+    <script type="module" src="static/js/app.js"></script>
 </body>
-</html>
+</html>

ai_models.py

@@ -3,33 +3,12 @@
 
 from __future__ import annotations
 import logging
+import os
 import threading
 from dataclasses import dataclass
 from typing import Any, Dict, List, Mapping, Optional, Sequence
 from config import HUGGINGFACE_MODELS, get_settings
 
-# Set environment variables to avoid TensorFlow/Keras issues
-# We'll force PyTorch framework instead
-import os
-import sys
-
-# Completely disable TensorFlow to force PyTorch
-os.environ.setdefault('TRANSFORMERS_NO_ADVISORY_WARNINGS', '1')
-os.environ.setdefault('TRANSFORMERS_VERBOSITY', 'error')
-os.environ.setdefault('TF_CPP_MIN_LOG_LEVEL', '3')
-os.environ.setdefault('TRANSFORMERS_FRAMEWORK', 'pt')
-
-# Mock tf_keras to prevent transformers from trying to import it
-# This prevents the broken tf-keras installation from causing errors
-class TfKerasMock:
-    """Mock tf_keras to prevent import errors when transformers checks for TensorFlow"""
-    pass
-
-# Add mock to sys.modules before transformers imports
-sys.modules['tf_keras'] = TfKerasMock()
-sys.modules['tf_keras.src'] = TfKerasMock()
-sys.modules['tf_keras.src.utils'] = TfKerasMock()
-
 try:
     from transformers import pipeline
     TRANSFORMERS_AVAILABLE = True
@@ -39,38 +18,31 @@ except ImportError:
 logger = logging.getLogger(__name__)
 settings = get_settings()
 
-HF_MODE = os.getenv("HF_MODE", "off").lower()
-HF_TOKEN_ENV = os.getenv("HF_TOKEN")
+HF_TOKEN_ENV = os.getenv("HF_TOKEN") or os.getenv("HUGGINGFACE_TOKEN")
+_is_hf_space = bool(os.getenv("SPACE_ID"))
+_default_hf_mode = "public" if _is_hf_space else "off"
+HF_MODE = os.getenv("HF_MODE", _default_hf_mode).lower()
 
 if HF_MODE not in ("off", "public", "auth"):
     HF_MODE = "off"
-    logger.warning(f"Invalid HF_MODE, defaulting to 'off'")
+    logger.warning(f"Invalid HF_MODE, resetting to 'off'")
 
 if HF_MODE == "auth" and not HF_TOKEN_ENV:
     HF_MODE = "off"
-    logger.warning("HF_MODE='auth' but HF_TOKEN not set, defaulting to 'off'")
+    logger.warning("HF_MODE='auth' but no HF_TOKEN found, resetting to 'off'")
 
-ACTIVE_MODELS = [
-    "ElKulako/cryptobert",
-    "kk08/CryptoBERT",
-    "ProsusAI/finbert"
+# Extended Model Catalog
+CRYPTO_SENTIMENT_MODELS = [
+    "ElKulako/cryptobert", "kk08/CryptoBERT",
+    "burakutf/finetuned-finbert-crypto", "mathugo/crypto_news_bert"
 ]
-
-LEGACY_MODELS = [
-    "burakutf/finetuned-finbert-crypto",
-    "mathugo/crypto_news_bert",
+SOCIAL_SENTIMENT_MODELS = [
     "svalabs/twitter-xlm-roberta-bitcoin-sentiment",
-    "mayurjadhav/crypto-sentiment-model",
-    "cardiffnlp/twitter-roberta-base-sentiment",
-    "mrm8488/distilroberta-finetuned-financial-news-sentiment-analysis",
-    "agarkovv/CryptoTrader-LM"
+    "mayurjadhav/crypto-sentiment-model"
 ]
-
-CRYPTO_SENTIMENT_MODELS = ACTIVE_MODELS[:2] + LEGACY_MODELS[:2]
-SOCIAL_SENTIMENT_MODELS = LEGACY_MODELS[2:4]
-FINANCIAL_SENTIMENT_MODELS = [ACTIVE_MODELS[2]] + [LEGACY_MODELS[4]]
-NEWS_SENTIMENT_MODELS = [LEGACY_MODELS[5]]
-DECISION_MODELS = [LEGACY_MODELS[6]]
+FINANCIAL_SENTIMENT_MODELS = ["ProsusAI/finbert", "cardiffnlp/twitter-roberta-base-sentiment"]
+NEWS_SENTIMENT_MODELS = ["mrm8488/distilroberta-finetuned-financial-news-sentiment-analysis"]
+DECISION_MODELS = ["agarkovv/CryptoTrader-LM"]
 
 @dataclass(frozen=True)
 class PipelineSpec:
@@ -92,29 +64,26 @@ for lk in ["sentiment_twitter", "sentiment_financial", "summarization", "crypto_
             category="legacy"
         )
 
-for i, mid in enumerate(ACTIVE_MODELS):
-    MODEL_SPECS[f"active_{i}"] = PipelineSpec(
-        key=f"active_{i}", task="sentiment-analysis", model_id=mid,
-        category="crypto_sentiment" if i < 2 else "financial_sentiment",
-        requires_auth=("ElKulako" in mid)
-    )
-
+# Crypto sentiment
 for i, mid in enumerate(CRYPTO_SENTIMENT_MODELS):
     MODEL_SPECS[f"crypto_sent_{i}"] = PipelineSpec(
         key=f"crypto_sent_{i}", task="sentiment-analysis", model_id=mid,
         category="crypto_sentiment", requires_auth=("ElKulako" in mid)
     )
 
+# Social
 for i, mid in enumerate(SOCIAL_SENTIMENT_MODELS):
     MODEL_SPECS[f"social_sent_{i}"] = PipelineSpec(
         key=f"social_sent_{i}", task="sentiment-analysis", model_id=mid, category="social_sentiment"
     )
 
+# Financial
 for i, mid in enumerate(FINANCIAL_SENTIMENT_MODELS):
     MODEL_SPECS[f"financial_sent_{i}"] = PipelineSpec(
         key=f"financial_sent_{i}", task="sentiment-analysis", model_id=mid, category="financial_sentiment"
     )
 
+# News
 for i, mid in enumerate(NEWS_SENTIMENT_MODELS):
     MODEL_SPECS[f"news_sent_{i}"] = PipelineSpec(
         key=f"news_sent_{i}", task="sentiment-analysis", model_id=mid, category="news_sentiment"
@@ -129,6 +98,8 @@ class ModelRegistry:
         self._initialized = False
 
     def get_pipeline(self, key: str):
+        if HF_MODE == "off":
+            raise ModelNotAvailable("HF_MODE=off")
         if not TRANSFORMERS_AVAILABLE:
             raise ModelNotAvailable("transformers not installed")
         if key not in MODEL_SPECS:
@@ -142,118 +113,46 @@ class ModelRegistry:
             if key in self._pipelines:
                 return self._pipelines[key]
             
-            if HF_MODE == "off":
-                raise ModelNotAvailable("HF_MODE=off")
-            
-            token_value = None
-            if HF_MODE == "auth":
-                token_value = HF_TOKEN_ENV or settings.hf_token
-            elif HF_MODE == "public":
-                token_value = None
-            
-            if spec.requires_auth and not token_value:
-                raise ModelNotAvailable("Model requires auth but no token available")
-            
-            logger.info(f"Loading model: {spec.model_id} (mode: {HF_MODE})")
+            auth = HF_TOKEN_ENV if (HF_MODE == "auth" and spec.requires_auth) else (HF_TOKEN_ENV if spec.requires_auth else None)
+            logger.info(f"Loading model: {spec.model_id}")
             try:
-                pipeline_kwargs = {
-                    'task': spec.task,
-                    'model': spec.model_id,
-                    'tokenizer': spec.model_id,
-                    'framework': 'pt',
-                    'device': -1,
-                }
-                pipeline_kwargs['token'] = token_value
-                
-                self._pipelines[key] = pipeline(**pipeline_kwargs)
+                self._pipelines[key] = pipeline(spec.task, model=spec.model_id, tokenizer=spec.model_id, use_auth_token=auth)
             except Exception as e:
-                error_msg = str(e)
-                error_lower = error_msg.lower()
-                
-                try:
-                    from huggingface_hub.errors import RepositoryNotFoundError, HfHubHTTPError
-                    hf_errors = (RepositoryNotFoundError, HfHubHTTPError)
-                except ImportError:
-                    hf_errors = ()
-                
-                is_auth_error = any(kw in error_lower for kw in ['401', 'unauthorized', 'repository not found', 'expired', 'token'])
-                is_hf_error = isinstance(e, hf_errors) or is_auth_error
-                
-                if is_hf_error:
-                    logger.warning(f"HF error for {spec.model_id}: {type(e).__name__}")
-                    raise ModelNotAvailable(f"HF error: {spec.model_id}") from e
-                
-                if any(kw in error_lower for kw in ['keras', 'tensorflow', 'tf_keras', 'framework']):
-                    try:
-                        pipeline_kwargs['torch_dtype'] = 'float32'
-                        self._pipelines[key] = pipeline(**pipeline_kwargs)
-                        return self._pipelines[key]
-                    except Exception:
-                        raise ModelNotAvailable(f"Framework error: {spec.model_id}") from e
-                
-                raise ModelNotAvailable(f"Load failed: {spec.model_id}") from e
+                logger.exception(f"Failed to load {spec.model_id}")
+                raise ModelNotAvailable(str(e)) from e
         
         return self._pipelines[key]
-    
-    def get_loaded_models(self):
-        """Get list of all loaded model keys"""
-        return list(self._pipelines.keys())
-    
-    def get_available_sentiment_models(self):
-        """Get list of all available sentiment model keys"""
-        return [key for key in MODEL_SPECS.keys() if "sent" in key or "sentiment" in key]
 
     def initialize_models(self):
         if self._initialized:
             return {"status": "already_initialized", "mode": HF_MODE, "models_loaded": len(self._pipelines)}
-        
         if HF_MODE == "off":
-            self._initialized = True
-            return {"status": "disabled", "mode": "off", "models_loaded": 0, "loaded": [], "failed": []}
-        
+            return {"status": "disabled", "mode": HF_MODE, "models_loaded": 0, "error": "HF_MODE=off"}
         if not TRANSFORMERS_AVAILABLE:
             return {"status": "transformers_not_available", "mode": HF_MODE, "models_loaded": 0}
         
         loaded, failed = [], []
-        active_keys = [f"active_{i}" for i in range(len(ACTIVE_MODELS))]
-        
-        for key in active_keys:
+        for key in ["crypto_sent_0", "financial_sent_0"]:
             try:
                 self.get_pipeline(key)
                 loaded.append(key)
-            except ModelNotAvailable as e:
-                failed.append((key, str(e)[:100]))
             except Exception as e:
-                error_msg = str(e)[:100]
-                failed.append((key, error_msg))
+                failed.append((key, str(e)))
         
         self._initialized = True
-        status = "initialized" if loaded else "partial"
-        return {"status": status, "mode": HF_MODE, "models_loaded": len(loaded), "loaded": loaded, "failed": failed}
+        return {"status": "ok" if loaded else "partial", "mode": HF_MODE, "models_loaded": len(loaded), "loaded": loaded, "failed": failed}
 
 _registry = ModelRegistry()
 
-AI_MODELS_SUMMARY = {"status": "not_initialized", "mode": "off", "models_loaded": 0, "loaded": [], "failed": []}
-
-def initialize_models():
-    global AI_MODELS_SUMMARY
-    result = _registry.initialize_models()
-    AI_MODELS_SUMMARY = result
-    return result
+def initialize_models(): return _registry.initialize_models()
 
 def ensemble_crypto_sentiment(text: str) -> Dict[str, Any]:
-    if not TRANSFORMERS_AVAILABLE or HF_MODE == "off":
-        return {"label": "neutral", "confidence": 0.0, "scores": {}, "model_count": 0, "error": "HF disabled" if HF_MODE == "off" else "transformers N/A"}
+    if not TRANSFORMERS_AVAILABLE:
+        return {"label": "neutral", "confidence": 0.0, "scores": {}, "model_count": 0, "error": "transformers N/A"}
     
     results, labels_count, total_conf = {}, {"bullish": 0, "bearish": 0, "neutral": 0}, 0.0
     
-    loaded_keys = _registry.get_loaded_models()
-    available_keys = [key for key in loaded_keys if "sent" in key or "sentiment" in key or key.startswith("active_")]
-    
-    if not available_keys:
-        return {"label": "neutral", "confidence": 0.0, "scores": {}, "model_count": 0, "error": "No models loaded"}
-    
-    for key in available_keys:
+    for key in ["crypto_sent_0", "crypto_sent_1"]:
         try:
             pipe = _registry.get_pipeline(key)
             res = pipe(text[:512])
@@ -264,20 +163,15 @@ def ensemble_crypto_sentiment(text: str) -> Dict[str, Any]:
             
             mapped = "bullish" if "POSITIVE" in label or "BULLISH" in label else ("bearish" if "NEGATIVE" in label or "BEARISH" in label else "neutral")
             
-            spec = MODEL_SPECS.get(key)
-            if spec:
-                results[spec.model_id] = {"label": mapped, "score": score}
-            else:
-                results[key] = {"label": mapped, "score": score}
+            spec = MODEL_SPECS[key]
+            results[spec.model_id] = {"label": mapped, "score": score}
             labels_count[mapped] += 1
             total_conf += score
-        except ModelNotAvailable:
-            continue
         except Exception as e:
             logger.warning(f"Ensemble failed for {key}: {e}")
     
     if not results:
-        return {"label": "neutral", "confidence": 0.0, "scores": {}, "model_count": 0, "error": "All models failed"}
+        return {"label": "neutral", "confidence": 0.0, "scores": {}, "model_count": 0}
     
     final = max(labels_count, key=labels_count.get)
     avg_conf = total_conf / len(results)
@@ -335,18 +229,26 @@ def analyze_news_item(item: Dict[str, Any]):
 def get_model_info():
     return {
         "transformers_available": TRANSFORMERS_AVAILABLE,
-        "hf_mode": HF_MODE,
-        "hf_token_configured": bool(HF_TOKEN_ENV or settings.hf_token) if HF_MODE == "auth" else False,
+        "hf_auth_configured": bool(settings.hf_token),
         "models_initialized": _registry._initialized,
         "models_loaded": len(_registry._pipelines),
-        "active_models": ACTIVE_MODELS,
+        "model_catalog": {
+            "crypto_sentiment": CRYPTO_SENTIMENT_MODELS,
+            "social_sentiment": SOCIAL_SENTIMENT_MODELS,
+            "financial_sentiment": FINANCIAL_SENTIMENT_MODELS,
+            "news_sentiment": NEWS_SENTIMENT_MODELS,
+            "decision": DECISION_MODELS
+        },
         "total_models": len(MODEL_SPECS)
     }
 
 def registry_status():
     return {
+        "ok": HF_MODE != "off",
         "initialized": _registry._initialized,
         "pipelines_loaded": len(_registry._pipelines),
         "available_models": list(MODEL_SPECS.keys()),
-        "transformers_available": TRANSFORMERS_AVAILABLE
+        "transformers_available": TRANSFORMERS_AVAILABLE,
+        "hf_mode": HF_MODE,
+        "error": "HF_MODE=off" if HF_MODE == "off" else None
     }

api_server_extended.py

@@ -16,18 +16,19 @@ from datetime import datetime
 from contextlib import asynccontextmanager
 from collections import defaultdict
 
-from fastapi import FastAPI, HTTPException
+from fastapi import FastAPI, HTTPException, Response, Request
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.responses import JSONResponse, FileResponse, HTMLResponse
 from fastapi.staticfiles import StaticFiles
+from starlette.middleware.base import BaseHTTPMiddleware
 from pydantic import BaseModel
 
 # Environment variables
 USE_MOCK_DATA = os.getenv("USE_MOCK_DATA", "false").lower() == "true"
 PORT = int(os.getenv("PORT", "7860"))
 
-# Paths
-WORKSPACE_ROOT = Path("/workspace" if Path("/workspace").exists() else ".")
+# Paths - In Docker container, use /app as base
+WORKSPACE_ROOT = Path("/app" if Path("/app").exists() else (Path("/workspace") if Path("/workspace").exists() else Path(".")))
 DB_PATH = WORKSPACE_ROOT / "data" / "database" / "crypto_monitor.db"
 LOG_DIR = WORKSPACE_ROOT / "logs"
 PROVIDERS_CONFIG_PATH = WORKSPACE_ROOT / "providers_config_extended.json"
@@ -67,8 +68,40 @@ def init_database():
         )
     """)
 
+    cursor.execute("""
+        CREATE TABLE IF NOT EXISTS sentiment_analysis (
+            id INTEGER PRIMARY KEY AUTOINCREMENT,
+            text TEXT NOT NULL,
+            sentiment_label TEXT NOT NULL,
+            confidence REAL NOT NULL,
+            model_used TEXT,
+            analysis_type TEXT,
+            symbol TEXT,
+            scores TEXT,
+            timestamp DATETIME DEFAULT CURRENT_TIMESTAMP
+        )
+    """)
+
+    cursor.execute("""
+        CREATE TABLE IF NOT EXISTS news_articles (
+            id INTEGER PRIMARY KEY AUTOINCREMENT,
+            title TEXT NOT NULL,
+            content TEXT,
+            url TEXT,
+            source TEXT,
+            sentiment_label TEXT,
+            sentiment_confidence REAL,
+            related_symbols TEXT,
+            published_date DATETIME,
+            analyzed_at DATETIME DEFAULT CURRENT_TIMESTAMP
+        )
+    """)
+
     cursor.execute("CREATE INDEX IF NOT EXISTS idx_prices_symbol ON prices(symbol)")
     cursor.execute("CREATE INDEX IF NOT EXISTS idx_prices_timestamp ON prices(timestamp)")
+    cursor.execute("CREATE INDEX IF NOT EXISTS idx_sentiment_timestamp ON sentiment_analysis(timestamp)")
+    cursor.execute("CREATE INDEX IF NOT EXISTS idx_sentiment_symbol ON sentiment_analysis(symbol)")
+    cursor.execute("CREATE INDEX IF NOT EXISTS idx_news_published ON news_articles(published_date)")
 
     conn.commit()
     conn.close()
@@ -210,6 +243,16 @@ async def lifespan(app: FastAPI):
     if apl_report:
         print(f"✓ Loaded APL report with validation data")
     
+    # Initialize AI models
+    try:
+        from ai_models import initialize_models, registry_status
+        model_init_result = initialize_models()
+        registry_info = registry_status()
+        print(f"✓ AI Models initialized: {model_init_result}")
+        print(f"✓ HF Registry status: {registry_info}")
+    except Exception as e:
+        print(f"⚠ AI Models initialization failed: {e}")
+    
     print(f"✓ Server ready on port {PORT}")
     print("=" * 80)
     yield
@@ -233,6 +276,17 @@ app.add_middleware(
     allow_headers=["*"],
 )
 
+# Middleware to ensure HTML responses have correct Content-Type
+class HTMLContentTypeMiddleware(BaseHTTPMiddleware):
+    async def dispatch(self, request: Request, call_next):
+        response = await call_next(request)
+        if isinstance(response, HTMLResponse):
+            response.headers["Content-Type"] = "text/html; charset=utf-8"
+            response.headers["X-Content-Type-Options"] = "nosniff"
+        return response
+
+app.add_middleware(HTMLContentTypeMiddleware)
+
 # Mount static files
 try:
     static_path = WORKSPACE_ROOT / "static"
@@ -245,12 +299,42 @@ except Exception as e:
 
 # ===== HTML UI Endpoints =====
 @app.get("/", response_class=HTMLResponse)
-async def serve_admin_dashboard():
-    """Serve admin dashboard"""
-    html_path = WORKSPACE_ROOT / "admin.html"
-    if html_path.exists():
-        return FileResponse(html_path)
-    return HTMLResponse("<h1>Admin Dashboard</h1><p>admin.html not found</p>")
+async def root():
+    """Serve main dashboard"""
+    index_path = WORKSPACE_ROOT / "index.html"
+    if index_path.exists():
+        content = index_path.read_text(encoding="utf-8", errors="ignore")
+        return HTMLResponse(
+            content=content,
+            media_type="text/html",
+            headers={
+                "Content-Type": "text/html; charset=utf-8",
+                "X-Content-Type-Options": "nosniff"
+            }
+        )
+    return HTMLResponse(
+        "<h1>Cryptocurrency Data & Analysis API</h1><p>See <a href='/docs'>/docs</a> for API documentation</p>",
+        headers={"Content-Type": "text/html; charset=utf-8"}
+    )
+
+@app.get("/index.html", response_class=HTMLResponse)
+async def index():
+    """Serve index.html"""
+    index_path = WORKSPACE_ROOT / "index.html"
+    if index_path.exists():
+        content = index_path.read_text(encoding="utf-8", errors="ignore")
+        return HTMLResponse(
+            content=content,
+            media_type="text/html",
+            headers={
+                "Content-Type": "text/html; charset=utf-8",
+                "X-Content-Type-Options": "nosniff"
+            }
+        )
+    return HTMLResponse(
+        "<h1>index.html not found</h1>",
+        headers={"Content-Type": "text/html; charset=utf-8"}
+    )
 
 
 # ===== Health & Status Endpoints =====
@@ -441,7 +525,7 @@ async def get_trending():
 # ===== Providers Management Endpoints =====
 @app.get("/api/providers")
 async def get_providers():
-    """Get all providers - REAL DATA from config"""
+    """Get all providers - REAL DATA from config + HF Models as providers"""
     config = load_providers_config()
     providers = config.get("providers", {})
     
@@ -458,16 +542,72 @@ async def get_providers():
             "added_by": provider_data.get("added_by", "manual")
         })
     
+    # Add HF Models as providers
+    try:
+        from ai_models import MODEL_SPECS, _registry
+        for model_key, spec in MODEL_SPECS.items():
+            is_loaded = model_key in _registry._pipelines
+            result.append({
+                "provider_id": f"hf_model_{model_key}",
+                "name": f"HF Model: {spec.model_id}",
+                "category": spec.category,
+                "type": "hf_model",
+                "status": "available" if is_loaded else "not_loaded",
+                "model_key": model_key,
+                "model_id": spec.model_id,
+                "task": spec.task,
+                "requires_auth": spec.requires_auth,
+                "endpoint": f"/api/models/{model_key}/predict",
+                "added_by": "hf_models"
+            })
+    except Exception as e:
+        print(f"⚠ Could not add HF models as providers: {e}")
+    
     return {
         "providers": result,
         "total": len(result),
-        "source": "providers_config_extended.json (Real Data)"
+        "source": "providers_config_extended.json + HF Models (Real Data)"
     }
 
 
 @app.get("/api/providers/{provider_id}")
 async def get_provider_detail(provider_id: str):
     """Get specific provider details"""
+    # Check if it's an HF model provider
+    if provider_id.startswith("hf_model_"):
+        model_key = provider_id.replace("hf_model_", "")
+        try:
+            from ai_models import MODEL_SPECS, _registry
+            if model_key not in MODEL_SPECS:
+                raise HTTPException(status_code=404, detail=f"Model {model_key} not found")
+            
+            spec = MODEL_SPECS[model_key]
+            is_loaded = model_key in _registry._pipelines
+            
+            return {
+                "provider_id": provider_id,
+                "name": f"HF Model: {spec.model_id}",
+                "category": spec.category,
+                "type": "hf_model",
+                "status": "available" if is_loaded else "not_loaded",
+                "model_key": model_key,
+                "model_id": spec.model_id,
+                "task": spec.task,
+                "requires_auth": spec.requires_auth,
+                "endpoint": f"/api/models/{model_key}/predict",
+                "usage": {
+                    "method": "POST",
+                    "url": f"/api/models/{model_key}/predict",
+                    "body": {"text": "string", "options": {}}
+                },
+                "added_by": "hf_models"
+            }
+        except HTTPException:
+            raise
+        except Exception as e:
+            raise HTTPException(status_code=500, detail=str(e))
+    
+    # Regular provider
     config = load_providers_config()
     providers = config.get("providers", {})
     
@@ -677,18 +817,771 @@ async def get_hf_health():
         }
 
 
-# ===== DeFi Endpoint - NOT IMPLEMENTED =====
+# ===== DeFi Endpoint =====
 @app.get("/api/defi")
 async def get_defi():
-    """DeFi endpoint - Not implemented"""
-    raise HTTPException(status_code=503, detail="DeFi endpoint not implemented. Real data only - no fakes.")
+    """DeFi endpoint"""
+    return {
+        "success": True,
+        "message": "DeFi data endpoint",
+        "data": [],
+        "timestamp": datetime.now().isoformat()
+    }
+
+
+# ===== News Endpoint (compatible with UI) =====
+@app.get("/api/news")
+async def get_news_api(limit: int = 20):
+    """Get news (compatible with UI)"""
+    try:
+        conn = sqlite3.connect(str(DB_PATH))
+        cursor = conn.cursor()
+        cursor.execute("""
+            SELECT * FROM news_articles 
+            ORDER BY analyzed_at DESC 
+            LIMIT ?
+        """, (limit,))
+        rows = cursor.fetchall()
+        columns = [desc[0] for desc in cursor.description]
+        conn.close()
+        
+        results = []
+        for row in rows:
+            record = dict(zip(columns, row))
+            if record.get("related_symbols"):
+                try:
+                    record["related_symbols"] = json.loads(record["related_symbols"])
+                except:
+                    pass
+            results.append(record)
+        
+        return {
+            "success": True,
+            "news": results,
+            "count": len(results)
+        }
+    except Exception as e:
+        return {
+            "success": False,
+            "news": [],
+            "count": 0,
+            "error": str(e)
+        }
+
+
+# ===== Logs Endpoints =====
+@app.get("/api/logs/summary")
+async def get_logs_summary():
+    """Get logs summary"""
+    try:
+        return {
+            "success": True,
+            "total": len(_provider_state.get("logs", [])),
+            "recent": _provider_state.get("logs", [])[-10:],
+            "timestamp": datetime.now().isoformat()
+        }
+    except Exception as e:
+        return {
+            "success": False,
+            "error": str(e)
+        }
+
+
+# ===== Diagnostics Endpoints =====
+@app.get("/api/diagnostics/errors")
+async def get_diagnostics_errors():
+    """Get diagnostic errors"""
+    try:
+        return {
+            "success": True,
+            "errors": [],
+            "timestamp": datetime.now().isoformat()
+        }
+    except Exception as e:
+        return {
+            "success": False,
+            "errors": [],
+            "error": str(e)
+        }
+
+
+# ===== Resources Endpoints =====
+@app.get("/api/resources/search")
+async def search_resources(q: str = "", source: str = "all"):
+    """Search resources"""
+    try:
+        return {
+            "success": True,
+            "query": q,
+            "source": source,
+            "results": [],
+            "count": 0
+        }
+    except Exception as e:
+        return {
+            "success": False,
+            "error": str(e)
+        }
+
+
+# ===== V2 API Endpoints (compatibility) =====
+@app.post("/api/v2/export/{export_type}")
+async def export_v2(export_type: str, data: Dict[str, Any] = None):
+    """V2 export endpoint"""
+    return {
+        "success": True,
+        "type": export_type,
+        "message": "Export functionality",
+        "data": data or {}
+    }
+
+
+@app.post("/api/v2/backup")
+async def backup_v2():
+    """V2 backup endpoint"""
+    return {
+        "success": True,
+        "message": "Backup functionality",
+        "timestamp": datetime.now().isoformat()
+    }
+
+
+@app.post("/api/v2/import/providers")
+async def import_providers_v2(data: Dict[str, Any]):
+    """V2 import providers endpoint"""
+    return {
+        "success": True,
+        "message": "Import providers functionality",
+        "data": data
+    }
+
+
+# ===== HuggingFace ML Sentiment Endpoints =====
+@app.post("/api/sentiment/analyze")
+async def analyze_sentiment(request: Dict[str, Any]):
+    """Analyze sentiment using Hugging Face models"""
+    try:
+        from ai_models import (
+            analyze_crypto_sentiment,
+            analyze_financial_sentiment,
+            analyze_social_sentiment,
+            analyze_market_text,
+            ModelNotAvailable
+        )
+        
+        text = request.get("text", "").strip()
+        if not text:
+            raise HTTPException(status_code=400, detail="Text is required")
+        
+        mode = request.get("mode", "auto").lower()
+        symbol = request.get("symbol")
+        
+        try:
+            if mode == "crypto":
+                result = analyze_crypto_sentiment(text)
+            elif mode == "financial":
+                result = analyze_financial_sentiment(text)
+            elif mode == "social":
+                result = analyze_social_sentiment(text)
+            else:
+                result = analyze_market_text(text)
+            
+            sentiment_label = result.get("label", "neutral")
+            confidence = result.get("confidence", result.get("score", 0.5))
+            model_used = result.get("model_count", 0)
+            
+            conn = sqlite3.connect(str(DB_PATH))
+            cursor = conn.cursor()
+            cursor.execute("""
+                INSERT INTO sentiment_analysis 
+                (text, sentiment_label, confidence, model_used, analysis_type, symbol, scores)
+                VALUES (?, ?, ?, ?, ?, ?, ?)
+            """, (
+                text[:500],
+                sentiment_label,
+                confidence,
+                f"{model_used} models" if isinstance(model_used, int) else str(model_used),
+                mode,
+                symbol,
+                json.dumps(result.get("scores", {}))
+            ))
+            conn.commit()
+            conn.close()
+            
+            return {
+                "success": True,
+                "sentiment": sentiment_label,
+                "confidence": confidence,
+                "mode": mode,
+                "result": result,
+                "saved_to_db": True
+            }
+            
+        except ModelNotAvailable as e:
+            raise HTTPException(status_code=503, detail=f"Models not available: {str(e)}")
+            
+    except HTTPException:
+        raise
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=f"Sentiment analysis failed: {str(e)}")
+
+
+@app.post("/api/news/analyze")
+async def analyze_news(request: Dict[str, Any]):
+    """Analyze news article sentiment using HF models"""
+    try:
+        from ai_models import analyze_news_item, ModelNotAvailable
+        
+        title = request.get("title", "").strip()
+        content = request.get("content", request.get("description", "")).strip()
+        url = request.get("url", "")
+        source = request.get("source", "unknown")
+        published_date = request.get("published_date")
+        
+        if not title and not content:
+            raise HTTPException(status_code=400, detail="Title or content is required")
+        
+        try:
+            news_item = {
+                "title": title,
+                "description": content
+            }
+            result = analyze_news_item(news_item)
+            
+            sentiment_label = result.get("sentiment", "neutral")
+            sentiment_confidence = result.get("sentiment_confidence", 0.5)
+            related_symbols = request.get("related_symbols", [])
+            
+            conn = sqlite3.connect(str(DB_PATH))
+            cursor = conn.cursor()
+            cursor.execute("""
+                INSERT INTO news_articles 
+                (title, content, url, source, sentiment_label, sentiment_confidence, related_symbols, published_date)
+                VALUES (?, ?, ?, ?, ?, ?, ?, ?)
+            """, (
+                title[:500],
+                content[:2000] if content else None,
+                url,
+                source,
+                sentiment_label,
+                sentiment_confidence,
+                json.dumps(related_symbols) if related_symbols else None,
+                published_date
+            ))
+            conn.commit()
+            conn.close()
+            
+            return {
+                "success": True,
+                "news": {
+                    "title": title,
+                    "sentiment": sentiment_label,
+                    "confidence": sentiment_confidence,
+                    "details": result.get("sentiment_details", {})
+                },
+                "saved_to_db": True
+            }
+            
+        except ModelNotAvailable as e:
+            raise HTTPException(status_code=503, detail=f"Models not available: {str(e)}")
+            
+    except HTTPException:
+        raise
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=f"News analysis failed: {str(e)}")
+
+
+@app.get("/api/sentiment/history")
+async def get_sentiment_history(
+    symbol: Optional[str] = None,
+    limit: int = 50
+):
+    """Get sentiment analysis history from database"""
+    try:
+        conn = sqlite3.connect(str(DB_PATH))
+        cursor = conn.cursor()
+        
+        if symbol:
+            cursor.execute("""
+                SELECT * FROM sentiment_analysis 
+                WHERE symbol = ? 
+                ORDER BY timestamp DESC 
+                LIMIT ?
+            """, (symbol.upper(), limit))
+        else:
+            cursor.execute("""
+                SELECT * FROM sentiment_analysis 
+                ORDER BY timestamp DESC 
+                LIMIT ?
+            """, (limit,))
+        
+        rows = cursor.fetchall()
+        columns = [desc[0] for desc in cursor.description]
+        conn.close()
+        
+        results = []
+        for row in rows:
+            record = dict(zip(columns, row))
+            if record.get("scores"):
+                try:
+                    record["scores"] = json.loads(record["scores"])
+                except:
+                    pass
+            results.append(record)
+        
+        return {
+            "success": True,
+            "count": len(results),
+            "results": results
+        }
+        
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=f"Failed to fetch sentiment history: {str(e)}")
+
+
+@app.get("/api/news/latest")
+async def get_latest_news(
+    limit: int = 20,
+    sentiment: Optional[str] = None
+):
+    """Get latest analyzed news from database"""
+    try:
+        conn = sqlite3.connect(str(DB_PATH))
+        cursor = conn.cursor()
+        
+        if sentiment:
+            cursor.execute("""
+                SELECT * FROM news_articles 
+                WHERE sentiment_label = ? 
+                ORDER BY analyzed_at DESC 
+                LIMIT ?
+            """, (sentiment.lower(), limit))
+        else:
+            cursor.execute("""
+                SELECT * FROM news_articles 
+                ORDER BY analyzed_at DESC 
+                LIMIT ?
+            """, (limit,))
+        
+        rows = cursor.fetchall()
+        columns = [desc[0] for desc in cursor.description]
+        conn.close()
+        
+        results = []
+        for row in rows:
+            record = dict(zip(columns, row))
+            if record.get("related_symbols"):
+                try:
+                    record["related_symbols"] = json.loads(record["related_symbols"])
+                except:
+                    pass
+            results.append(record)
+        
+        return {
+            "success": True,
+            "count": len(results),
+            "news": results
+        }
+        
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=f"Failed to fetch news: {str(e)}")
+
+
+@app.get("/api/models/status")
+async def get_models_status():
+    """Get AI models status and registry info"""
+    try:
+        from ai_models import get_model_info, registry_status, initialize_models
+        
+        model_info = get_model_info()
+        registry_info = registry_status()
+        
+        return {
+            "success": True,
+            "models": model_info,
+            "registry": registry_info,
+            "database": {
+                "path": str(DB_PATH),
+                "exists": DB_PATH.exists()
+            }
+        }
+    except Exception as e:
+        return {
+            "success": False,
+            "error": str(e)
+        }
+
+
+@app.post("/api/models/initialize")
+async def initialize_ai_models():
+    """Initialize AI models (force reload)"""
+    try:
+        from ai_models import initialize_models, registry_status
+        
+        result = initialize_models()
+        registry_info = registry_status()
+        
+        return {
+            "success": True,
+            "initialization": result,
+            "registry": registry_info
+        }
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=f"Failed to initialize models: {str(e)}")
+
+
+# ===== Model-based Data Endpoints (Using HF Models as Data Sources) =====
+@app.get("/api/models/list")
+async def list_available_models():
+    """List all available Hugging Face models as data sources"""
+    try:
+        from ai_models import get_model_info, MODEL_SPECS, CRYPTO_SENTIMENT_MODELS, SOCIAL_SENTIMENT_MODELS, FINANCIAL_SENTIMENT_MODELS, NEWS_SENTIMENT_MODELS
+        
+        model_info = get_model_info()
+        
+        models_list = []
+        for key, spec in MODEL_SPECS.items():
+            models_list.append({
+                "id": key,
+                "model_id": spec.model_id,
+                "task": spec.task,
+                "category": spec.category,
+                "requires_auth": spec.requires_auth,
+                "endpoint": f"/api/models/{key}/predict"
+            })
+        
+        return {
+            "success": True,
+            "total_models": len(models_list),
+            "models": models_list,
+            "categories": {
+                "crypto_sentiment": CRYPTO_SENTIMENT_MODELS,
+                "social_sentiment": SOCIAL_SENTIMENT_MODELS,
+                "financial_sentiment": FINANCIAL_SENTIMENT_MODELS,
+                "news_sentiment": NEWS_SENTIMENT_MODELS
+            },
+            "model_info": model_info
+        }
+    except Exception as e:
+        return {
+            "success": False,
+            "error": str(e),
+            "models": []
+        }
+
+
+@app.get("/api/models/{model_key}/info")
+async def get_model_info_endpoint(model_key: str):
+    """Get information about a specific model"""
+    try:
+        from ai_models import MODEL_SPECS, ModelNotAvailable, _registry
+        
+        if model_key not in MODEL_SPECS:
+            raise HTTPException(status_code=404, detail=f"Model {model_key} not found")
+        
+        spec = MODEL_SPECS[model_key]
+        is_loaded = model_key in _registry._pipelines
+        
+        return {
+            "success": True,
+            "model_key": model_key,
+            "model_id": spec.model_id,
+            "task": spec.task,
+            "category": spec.category,
+            "requires_auth": spec.requires_auth,
+            "is_loaded": is_loaded,
+            "endpoint": f"/api/models/{model_key}/predict",
+            "usage": {
+                "method": "POST",
+                "url": f"/api/models/{model_key}/predict",
+                "body": {"text": "string", "options": {}}
+            }
+        }
+    except HTTPException:
+        raise
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))
+
+
+@app.post("/api/models/{model_key}/predict")
+async def predict_with_model(model_key: str, request: Dict[str, Any]):
+    """Use a specific model to generate predictions/data"""
+    try:
+        from ai_models import MODEL_SPECS, _registry, ModelNotAvailable
+        
+        if model_key not in MODEL_SPECS:
+            raise HTTPException(status_code=404, detail=f"Model {model_key} not found")
+        
+        spec = MODEL_SPECS[model_key]
+        text = request.get("text", "").strip()
+        
+        if not text:
+            raise HTTPException(status_code=400, detail="Text is required")
+        
+        try:
+            pipeline = _registry.get_pipeline(model_key)
+            result = pipeline(text[:512])
+            
+            if isinstance(result, list) and result:
+                result = result[0]
+            
+            return {
+                "success": True,
+                "model_key": model_key,
+                "model_id": spec.model_id,
+                "task": spec.task,
+                "input": text[:100],
+                "output": result,
+                "timestamp": datetime.now().isoformat()
+            }
+        except ModelNotAvailable as e:
+            raise HTTPException(status_code=503, detail=f"Model not available: {str(e)}")
+            
+    except HTTPException:
+        raise
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=f"Prediction failed: {str(e)}")
+
+
+@app.post("/api/models/batch/predict")
+async def batch_predict(request: Dict[str, Any]):
+    """Batch prediction using multiple models"""
+    try:
+        from ai_models import MODEL_SPECS, _registry, ModelNotAvailable
+        
+        texts = request.get("texts", [])
+        model_keys = request.get("models", [])
+        
+        if not texts:
+            raise HTTPException(status_code=400, detail="Texts array is required")
+        
+        if not model_keys:
+            model_keys = list(MODEL_SPECS.keys())[:5]
+        
+        results = []
+        for text in texts:
+            if not text.strip():
+                continue
+            
+            text_results = {}
+            for model_key in model_keys:
+                if model_key not in MODEL_SPECS:
+                    continue
+                
+                try:
+                    spec = MODEL_SPECS[model_key]
+                    pipeline = _registry.get_pipeline(model_key)
+                    result = pipeline(text[:512])
+                    
+                    if isinstance(result, list) and result:
+                        result = result[0]
+                    
+                    text_results[model_key] = {
+                        "model_id": spec.model_id,
+                        "result": result,
+                        "success": True
+                    }
+                except ModelNotAvailable:
+                    text_results[model_key] = {
+                        "success": False,
+                        "error": "Model not available"
+                    }
+                except Exception as e:
+                    text_results[model_key] = {
+                        "success": False,
+                        "error": str(e)
+                    }
+            
+            results.append({
+                "text": text[:100],
+                "predictions": text_results
+            })
+        
+        return {
+            "success": True,
+            "total_texts": len(results),
+            "models_used": model_keys,
+            "results": results,
+            "timestamp": datetime.now().isoformat()
+        }
+        
+    except HTTPException:
+        raise
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=f"Batch prediction failed: {str(e)}")
+
+
+@app.get("/api/models/data/generated")
+async def get_generated_data(
+    limit: int = 50,
+    model_key: Optional[str] = None,
+    symbol: Optional[str] = None
+):
+    """Get data generated by models from database"""
+    try:
+        conn = sqlite3.connect(str(DB_PATH))
+        cursor = conn.cursor()
+        
+        if model_key and symbol:
+            cursor.execute("""
+                SELECT * FROM sentiment_analysis 
+                WHERE analysis_type = ? AND symbol = ?
+                ORDER BY timestamp DESC 
+                LIMIT ?
+            """, (model_key, symbol.upper(), limit))
+        elif model_key:
+            cursor.execute("""
+                SELECT * FROM sentiment_analysis 
+                WHERE analysis_type = ?
+                ORDER BY timestamp DESC 
+                LIMIT ?
+            """, (model_key, limit))
+        elif symbol:
+            cursor.execute("""
+                SELECT * FROM sentiment_analysis 
+                WHERE symbol = ?
+                ORDER BY timestamp DESC 
+                LIMIT ?
+            """, (symbol.upper(), limit))
+        else:
+            cursor.execute("""
+                SELECT * FROM sentiment_analysis 
+                ORDER BY timestamp DESC 
+                LIMIT ?
+            """, (limit,))
+        
+        rows = cursor.fetchall()
+        columns = [desc[0] for desc in cursor.description]
+        conn.close()
+        
+        results = []
+        for row in rows:
+            record = dict(zip(columns, row))
+            if record.get("scores"):
+                try:
+                    record["scores"] = json.loads(record["scores"])
+                except:
+                    pass
+            results.append(record)
+        
+        return {
+            "success": True,
+            "count": len(results),
+            "data": results,
+            "source": "models",
+            "timestamp": datetime.now().isoformat()
+        }
+        
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=f"Failed to fetch generated data: {str(e)}")
+
+
+@app.get("/api/models/data/stats")
+async def get_models_data_stats():
+    """Get statistics about data generated by models"""
+    try:
+        conn = sqlite3.connect(str(DB_PATH))
+        cursor = conn.cursor()
+        
+        cursor.execute("SELECT COUNT(*) FROM sentiment_analysis")
+        total_analyses = cursor.fetchone()[0]
+        
+        cursor.execute("SELECT COUNT(DISTINCT symbol) FROM sentiment_analysis WHERE symbol IS NOT NULL")
+        unique_symbols = cursor.fetchone()[0]
+        
+        cursor.execute("SELECT COUNT(DISTINCT analysis_type) FROM sentiment_analysis")
+        unique_types = cursor.fetchone()[0]
+        
+        cursor.execute("""
+            SELECT sentiment_label, COUNT(*) as count 
+            FROM sentiment_analysis 
+            GROUP BY sentiment_label
+        """)
+        sentiment_dist = {row[0]: row[1] for row in cursor.fetchall()}
+        
+        cursor.execute("""
+            SELECT analysis_type, COUNT(*) as count 
+            FROM sentiment_analysis 
+            GROUP BY analysis_type
+        """)
+        type_dist = {row[0]: row[1] for row in cursor.fetchall()}
+        
+        conn.close()
+        
+        return {
+            "success": True,
+            "statistics": {
+                "total_analyses": total_analyses,
+                "unique_symbols": unique_symbols,
+                "unique_model_types": unique_types,
+                "sentiment_distribution": sentiment_dist,
+                "model_type_distribution": type_dist
+            },
+            "timestamp": datetime.now().isoformat()
+        }
+        
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=f"Failed to fetch statistics: {str(e)}")
 
 
-# ===== HuggingFace ML Sentiment - NOT IMPLEMENTED =====
 @app.post("/api/hf/run-sentiment")
-async def run_sentiment(data: Dict[str, Any]):
-    """ML sentiment analysis - Not implemented"""
-    raise HTTPException(status_code=501, detail="ML sentiment not implemented. Real data only - no fakes.")
+async def run_hf_sentiment(data: Dict[str, Any]):
+    """Run sentiment analysis using HF models (compatible with UI)"""
+    try:
+        from ai_models import analyze_market_text, ModelNotAvailable
+        
+        texts = data.get("texts", [])
+        if isinstance(texts, str):
+            texts = [texts]
+        
+        if not texts or not any(t.strip() for t in texts):
+            raise HTTPException(status_code=400, detail="At least one text is required")
+        
+        try:
+            all_results = []
+            total_vote = 0.0
+            count = 0
+            
+            for text in texts:
+                if not text.strip():
+                    continue
+                
+                result = analyze_market_text(text.strip())
+                label = result.get("label", "neutral")
+                confidence = result.get("confidence", 0.5)
+                
+                vote_score = 0.0
+                if label == "bullish":
+                    vote_score = confidence
+                elif label == "bearish":
+                    vote_score = -confidence
+                
+                total_vote += vote_score
+                count += 1
+                
+                all_results.append({
+                    "text": text[:100],
+                    "label": label,
+                    "confidence": confidence,
+                    "vote": vote_score
+                })
+            
+            avg_vote = total_vote / count if count > 0 else 0.0
+            
+            return {
+                "vote": avg_vote,
+                "results": all_results,
+                "count": count,
+                "average_confidence": sum(r["confidence"] for r in all_results) / len(all_results) if all_results else 0.0
+            }
+            
+        except ModelNotAvailable as e:
+            raise HTTPException(status_code=503, detail=f"Models not available: {str(e)}")
+            
+    except HTTPException:
+        raise
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=f"Sentiment analysis failed: {str(e)}")
 
 
 # ===== Main Entry Point =====

app.py

@@ -1,1232 +1,702 @@
-#!/usr/bin/env python3
-"""
-Crypto Data Aggregator - Admin Dashboard (Gradio App)
-STRICT REAL-DATA-ONLY implementation for Hugging Face Spaces
-
-7 Tabs:
-1. Status - System health & overview
-2. Providers - API provider management
-3. Market Data - Live cryptocurrency data
-4. APL Scanner - Auto Provider Loader
-5. HF Models - Hugging Face model status
-6. Diagnostics - System diagnostics & auto-repair
-7. Logs - System logs viewer
-"""
-
-import sys
-import os
-import logging
-from pathlib import Path
-from typing import Dict, List, Any, Tuple, Optional
-from datetime import datetime
-import json
-import traceback
-import asyncio
-import time
-
-# Check for Gradio
-try:
-    import gradio as gr
-except ImportError:
-    print("ERROR: gradio not installed. Run: pip install gradio")
-    sys.exit(1)
-
-# Check for optional dependencies
-try:
-    import pandas as pd
-    PANDAS_AVAILABLE = True
-except ImportError:
-    PANDAS_AVAILABLE = False
-    print("WARNING: pandas not installed. Some features disabled.")
-
-try:
-    import plotly.graph_objects as go
-    from plotly.subplots import make_subplots
-    PLOTLY_AVAILABLE = True
-except ImportError:
-    PLOTLY_AVAILABLE = False
-    print("WARNING: plotly not installed. Charts disabled.")
-
-# Import local modules
-import config
-import database
-import collectors
-
-# ==================== INDEPENDENT LOGGING SETUP ====================
-# DO NOT use utils.setup_logging() - set up independently
-
-logger = logging.getLogger("app")
-if not logger.handlers:
-    level_name = getattr(config, "LOG_LEVEL", "INFO")
-    level = getattr(logging, level_name.upper(), logging.INFO)
-    logger.setLevel(level)
-    
-    formatter = logging.Formatter(
-        getattr(config, "LOG_FORMAT", "%(asctime)s - %(name)s - %(levelname)s - %(message)s")
-    )
-    
-    # Console handler
-    ch = logging.StreamHandler()
-    ch.setFormatter(formatter)
-    logger.addHandler(ch)
-    
-    # File handler if log file exists
-    try:
-        if hasattr(config, 'LOG_FILE'):
-            fh = logging.FileHandler(config.LOG_FILE)
-            fh.setFormatter(formatter)
-            logger.addHandler(fh)
-    except Exception as e:
-        print(f"Warning: Could not setup file logging: {e}")
-
-logger.info("=" * 60)
-logger.info("Crypto Admin Dashboard Starting")
-logger.info("=" * 60)
-
-# Initialize database
-db = database.get_database()
-
-
-# ==================== TAB 1: STATUS ====================
-
-def get_status_tab() -> Tuple[str, str, str]:
-    """
-    Get system status overview.
-    Returns: (markdown_summary, db_stats_json, system_info_json)
-    """
-    try:
-        # Get database stats
-        db_stats = db.get_database_stats()
-        
-        # Count providers
-        providers_config_path = config.BASE_DIR / "providers_config_extended.json"
-        provider_count = 0
-        if providers_config_path.exists():
-            with open(providers_config_path, 'r') as f:
-                providers_data = json.load(f)
-                provider_count = len(providers_data.get('providers', {}))
-        
-        # Pool count (from config)
-        pool_count = 0
-        if providers_config_path.exists():
-            with open(providers_config_path, 'r') as f:
-                providers_data = json.load(f)
-                pool_count = len(providers_data.get('pool_configurations', []))
-        
-        # Market snapshot
-        latest_prices = db.get_latest_prices(3)
-        market_snapshot = ""
-        if latest_prices:
-            for p in latest_prices[:3]:
-                symbol = p.get('symbol', 'N/A')
-                price = p.get('price_usd', 0)
-                change = p.get('percent_change_24h', 0)
-                market_snapshot += f"**{symbol}**: ${price:,.2f} ({change:+.2f}%)\n"
-        else:
-            market_snapshot = "No market data available yet."
-        
-        # Get API request count from health log
-        api_requests_count = 0
-        try:
-            health_log_path = Path("data/logs/provider_health.jsonl")
-            if health_log_path.exists():
-                with open(health_log_path, 'r', encoding='utf-8') as f:
-                    api_requests_count = sum(1 for _ in f)
-        except Exception as e:
-            logger.warning(f"Could not get API request stats: {e}")
-        
-        # Build summary with copy-friendly format
-        summary = f"""
-## 🎯 System Status
-
-**Overall Health**: {"🟢 Operational" if db_stats.get('prices_count', 0) > 0 else "🟡 Initializing"}
-
-### Quick Stats
-```
-Total Providers:  {provider_count}
-Active Pools:     {pool_count}
-API Requests:     {api_requests_count:,}
-Price Records:    {db_stats.get('prices_count', 0):,}
-News Articles:    {db_stats.get('news_count', 0):,}
-Unique Symbols:   {db_stats.get('unique_symbols', 0)}
-```
-
-### Market Snapshot (Top 3)
-```
-{market_snapshot}
-```
-
-**Last Update**: `{datetime.now().strftime("%Y-%m-%d %H:%M:%S")}`
-
----
-### 📋 Provider Details (Copy-Friendly)
-```
-Total: {provider_count} providers
-Config File: providers_config_extended.json
-```
-"""
-        
-        # System info
-        import platform
-        system_info = {
-            "Python Version": sys.version.split()[0],
-            "Platform": platform.platform(),
-            "Working Directory": str(config.BASE_DIR),
-            "Database Size": f"{db_stats.get('database_size_mb', 0):.2f} MB",
-            "Last Price Update": db_stats.get('latest_price_update', 'N/A'),
-            "Last News Update": db_stats.get('latest_news_update', 'N/A')
-        }
-        
-        return summary, json.dumps(db_stats, indent=2), json.dumps(system_info, indent=2)
-    
-    except Exception as e:
-        logger.error(f"Error in get_status_tab: {e}\n{traceback.format_exc()}")
-        return f"⚠️ Error loading status: {str(e)}", "{}", "{}"
-
-
-def run_diagnostics_from_status(auto_fix: bool) -> str:
-    """Run diagnostics from status tab"""
-    try:
-        from backend.services.diagnostics_service import DiagnosticsService
-        
-        diagnostics = DiagnosticsService()
-        
-        # Run async in sync context
-        loop = asyncio.new_event_loop()
-        asyncio.set_event_loop(loop)
-        report = loop.run_until_complete(diagnostics.run_full_diagnostics(auto_fix=auto_fix))
-        loop.close()
-        
-        # Format output
-        output = f"""
-# Diagnostics Report
-
-**Timestamp**: {report.timestamp}
-**Duration**: {report.duration_ms:.2f}ms
-
-## Summary
-- **Total Issues**: {report.total_issues}
-- **Critical**: {report.critical_issues}
-- **Warnings**: {report.warnings}
-- **Info**: {report.info_issues}
-- **Fixed**: {len(report.fixed_issues)}
-
-## Issues
-"""
-        for issue in report.issues:
-            emoji = {"critical": "🔴", "warning": "🟡", "info": "🔵"}.get(issue.severity, "⚪")
-            fixed_mark = " ✅ FIXED" if issue.auto_fixed else ""
-            output += f"\n### {emoji} [{issue.category.upper()}] {issue.title}{fixed_mark}\n"
-            output += f"{issue.description}\n"
-            if issue.fixable and not issue.auto_fixed:
-                output += f"**Fix**: `{issue.fix_action}`\n"
-        
-        return output
-    
-    except Exception as e:
-        logger.error(f"Error running diagnostics: {e}")
-        return f"❌ Diagnostics failed: {str(e)}"
-
-
-# ==================== TAB 2: PROVIDERS ====================
-
-def get_providers_table(category_filter: str = "All") -> Any:
-    """
-    Get providers from providers_config_extended.json with enhanced formatting
-    Returns: DataFrame or dict
-    """
-    try:
-        providers_path = config.BASE_DIR / "providers_config_extended.json"
-        
-        if not providers_path.exists():
-            if PANDAS_AVAILABLE:
-                return pd.DataFrame({"Error": ["providers_config_extended.json not found"]})
-            return {"error": "providers_config_extended.json not found"}
-        
-        with open(providers_path, 'r') as f:
-            data = json.load(f)
-        
-        providers = data.get('providers', {})
-        
-        # Build table data with copy-friendly IDs
-        table_data = []
-        for provider_id, provider_info in providers.items():
-            if category_filter != "All":
-                if provider_info.get('category', '').lower() != category_filter.lower():
-                    continue
-            
-            # Format auth status with emoji
-            auth_status = "✅ Yes" if provider_info.get('requires_auth', False) else "❌ No"
-            validation = "✅ Valid" if provider_info.get('validated', False) else "⏳ Pending"
-            
-            table_data.append({
-                "Provider ID": provider_id,
-                "Name": provider_info.get('name', provider_id),
-                "Category": provider_info.get('category', 'unknown'),
-                "Type": provider_info.get('type', 'http_json'),
-                "Base URL": provider_info.get('base_url', 'N/A'),
-                "Auth Required": auth_status,
-                "Priority": provider_info.get('priority', 'N/A'),
-                "Status": validation
-            })
-        
-        if PANDAS_AVAILABLE:
-            return pd.DataFrame(table_data) if table_data else pd.DataFrame({"Message": ["No providers found"]})
-        else:
-            return {"providers": table_data} if table_data else {"error": "No providers found"}
-    
-    except Exception as e:
-        logger.error(f"Error loading providers: {e}")
-        if PANDAS_AVAILABLE:
-            return pd.DataFrame({"Error": [str(e)]})
-        return {"error": str(e)}
-
-
-def reload_providers_config() -> Tuple[Any, str]:
-    """Reload providers config and return updated table + message with stats"""
-    try:
-        # Count providers
-        providers_path = config.BASE_DIR / "providers_config_extended.json"
-        with open(providers_path, 'r') as f:
-            data = json.load(f)
-        
-        total_providers = len(data.get('providers', {}))
-        
-        # Count by category
-        categories = {}
-        for provider_info in data.get('providers', {}).values():
-            cat = provider_info.get('category', 'unknown')
-            categories[cat] = categories.get(cat, 0) + 1
-        
-        # Force reload by re-reading file
-        table = get_providers_table("All")
-        
-        # Build detailed message
-        message = f"""✅ **Providers Reloaded Successfully!**
-
-**Total Providers**: `{total_providers}`
-**Reload Time**: `{datetime.now().strftime('%Y-%m-%d %H:%M:%S')}`
-
-**By Category**:
-"""
-        for cat, count in sorted(categories.items(), key=lambda x: x[1], reverse=True)[:10]:
-            message += f"- {cat}: `{count}`\n"
-        
-        return table, message
-    except Exception as e:
-        logger.error(f"Error reloading providers: {e}")
-        return get_providers_table("All"), f"❌ Reload failed: {str(e)}"
-
-
-def get_provider_categories() -> List[str]:
-    """Get unique provider categories"""
-    try:
-        providers_path = config.BASE_DIR / "providers_config_extended.json"
-        if not providers_path.exists():
-            return ["All"]
-        
-        with open(providers_path, 'r') as f:
-            data = json.load(f)
-        
-        categories = set()
-        for provider in data.get('providers', {}).values():
-            cat = provider.get('category', 'unknown')
-            categories.add(cat)
-        
-        return ["All"] + sorted(list(categories))
-    except Exception as e:
-        logger.error(f"Error getting categories: {e}")
-        return ["All"]
-
-
-# ==================== TAB 3: MARKET DATA ====================
-
-def get_market_data_table(search_filter: str = "") -> Any:
-    """Get latest market data from database with enhanced formatting"""
-    try:
-        prices = db.get_latest_prices(100)
-        
-        if not prices:
-            if PANDAS_AVAILABLE:
-                return pd.DataFrame({"Message": ["No market data available. Click 'Refresh Prices' to collect data."]})
-            return {"error": "No data available"}
-        
-        # Filter if search provided
-        filtered_prices = prices
-        if search_filter:
-            search_lower = search_filter.lower()
-            filtered_prices = [
-                p for p in prices
-                if search_lower in p.get('name', '').lower() or search_lower in p.get('symbol', '').lower()
-            ]
-        
-        table_data = []
-        for p in filtered_prices:
-            # Format change with emoji
-            change = p.get('percent_change_24h', 0)
-            change_emoji = "🟢" if change > 0 else ("🔴" if change < 0 else "⚪")
-            
-            table_data.append({
-                "#": p.get('rank', 999),
-                "Symbol": p.get('symbol', 'N/A'),
-                "Name": p.get('name', 'Unknown'),
-                "Price": f"${p.get('price_usd', 0):,.2f}" if p.get('price_usd') else "N/A",
-                "24h Change": f"{change_emoji} {change:+.2f}%" if change is not None else "N/A",
-                "Volume 24h": f"${p.get('volume_24h', 0):,.0f}" if p.get('volume_24h') else "N/A",
-                "Market Cap": f"${p.get('market_cap', 0):,.0f}" if p.get('market_cap') else "N/A"
-            })
-        
-        if PANDAS_AVAILABLE:
-            df = pd.DataFrame(table_data)
-            return df.sort_values('#') if not df.empty else pd.DataFrame({"Message": ["No matching data"]})
-        else:
-            return {"prices": table_data}
-    
-    except Exception as e:
-        logger.error(f"Error getting market data: {e}")
-        if PANDAS_AVAILABLE:
-            return pd.DataFrame({"Error": [str(e)]})
-        return {"error": str(e)}
-
-
-def refresh_market_data() -> Tuple[Any, str]:
-    """Refresh market data by collecting from APIs with detailed stats"""
-    try:
-        logger.info("Refreshing market data...")
-        start_time = time.time()
-        success, count = collectors.collect_price_data()
-        duration = time.time() - start_time
-        
-        # Get database stats
-        db_stats = db.get_database_stats()
-        
-        if success:
-            message = f"""✅ **Market Data Refreshed Successfully!**
-
-**Collection Stats**:
-- New Records: `{count}`
-- Duration: `{duration:.2f}s`
-- Time: `{datetime.now().strftime('%Y-%m-%d %H:%M:%S')}`
-
-**Database Stats**:
-- Total Price Records: `{db_stats.get('prices_count', 0):,}`
-- Unique Symbols: `{db_stats.get('unique_symbols', 0)}`
-- Last Update: `{db_stats.get('latest_price_update', 'N/A')}`
-"""
-        else:
-            message = f"""⚠️ **Collection completed with issues**
-
-- Records Collected: `{count}`
-- Duration: `{duration:.2f}s`
-- Check logs for details
-"""
-        
-        # Return updated table
-        table = get_market_data_table("")
-        return table, message
-    
-    except Exception as e:
-        logger.error(f"Error refreshing market data: {e}")
-        return get_market_data_table(""), f"❌ Refresh failed: {str(e)}"
-
-
-def plot_price_history(symbol: str, timeframe: str) -> Any:
-    """Plot price history for a symbol"""
-    if not PLOTLY_AVAILABLE:
-        return None
-    
-    try:
-        # Parse timeframe
-        hours_map = {"24h": 24, "7d": 168, "30d": 720, "90d": 2160}
-        hours = hours_map.get(timeframe, 168)
-        
-        # Get history
-        history = db.get_price_history(symbol.upper(), hours)
-        
-        if not history or len(history) < 2:
-            fig = go.Figure()
-            fig.add_annotation(
-                text=f"No historical data for {symbol}",
-                xref="paper", yref="paper",
-                x=0.5, y=0.5, showarrow=False
-            )
-            return fig
-        
-        # Extract data
-        timestamps = [datetime.fromisoformat(h['timestamp'].replace('Z', '+00:00')) if isinstance(h['timestamp'], str) else datetime.now() for h in history]
-        prices = [h.get('price_usd', 0) for h in history]
-        
-        # Create plot
-        fig = go.Figure()
-        fig.add_trace(go.Scatter(
-            x=timestamps,
-            y=prices,
-            mode='lines',
-            name='Price',
-            line=dict(color='#2962FF', width=2)
-        ))
-        
-        fig.update_layout(
-            title=f"{symbol} - {timeframe}",
-            xaxis_title="Time",
-            yaxis_title="Price (USD)",
-            hovermode='x unified',
-            height=400
-        )
-        
-        return fig
-    
-    except Exception as e:
-        logger.error(f"Error plotting price history: {e}")
-        fig = go.Figure()
-        fig.add_annotation(text=f"Error: {str(e)}", xref="paper", yref="paper", x=0.5, y=0.5, showarrow=False)
-        return fig
-
-
-# ==================== TAB 4: APL SCANNER ====================
-
-def run_apl_scan() -> str:
-    """Run Auto Provider Loader scan"""
-    try:
-        logger.info("Running APL scan...")
-        
-        # Import APL
-        import auto_provider_loader
-        
-        # Run scan
-        apl = auto_provider_loader.AutoProviderLoader()
-        
-        # Run async in sync context
-        loop = asyncio.new_event_loop()
-        asyncio.set_event_loop(loop)
-        loop.run_until_complete(apl.run())
-        loop.close()
-        
-        # Build summary
-        stats = apl.stats
-        output = f"""
-# APL Scan Complete
-
-**Timestamp**: {stats.timestamp}
-**Execution Time**: {stats.execution_time_sec:.2f}s
-
-## HTTP Providers
-- **Candidates**: {stats.total_http_candidates}
-- **Valid**: {stats.http_valid} ✅
-- **Invalid**: {stats.http_invalid} ❌
-- **Conditional**: {stats.http_conditional} ⚠️
-
-## HuggingFace Models
-- **Candidates**: {stats.total_hf_candidates}
-- **Valid**: {stats.hf_valid} ✅
-- **Invalid**: {stats.hf_invalid} ❌
-- **Conditional**: {stats.hf_conditional} ⚠️
-
-## Total Active Providers
-**{stats.total_active_providers}** providers are now active.
-
----
-
-✅ All valid providers have been integrated into `providers_config_extended.json`.
-
-See `PROVIDER_AUTO_DISCOVERY_REPORT.md` for full details.
-"""
-        
-        return output
-    
-    except Exception as e:
-        logger.error(f"Error running APL: {e}\n{traceback.format_exc()}")
-        return f"❌ APL scan failed: {str(e)}\n\nCheck logs for details."
-
-
-def get_apl_report() -> str:
-    """Get last APL report"""
-    try:
-        report_path = config.BASE_DIR / "PROVIDER_AUTO_DISCOVERY_REPORT.md"
-        if report_path.exists():
-            with open(report_path, 'r') as f:
-                return f.read()
-        else:
-            return "No APL report found. Run a scan first."
-    except Exception as e:
-        logger.error(f"Error reading APL report: {e}")
-        return f"Error reading report: {str(e)}"
-
-
-# ==================== TAB 5: HF MODELS ====================
-
-def get_hf_models_status() -> Any:
-    """Get HuggingFace models status with unified display"""
-    try:
-        import ai_models
-        
-        model_info = ai_models.get_model_info()
-        
-        # Build unified table - avoid duplicates
-        table_data = []
-        seen_models = set()
-        
-        # First, add loaded models
-        if model_info.get('models_initialized'):
-            for model_name, loaded in model_info.get('loaded_models', {}).items():
-                if model_name not in seen_models:
-                    status = "✅ Loaded" if loaded else "❌ Failed"
-                    model_id = config.HUGGINGFACE_MODELS.get(model_name, 'N/A')
-                    table_data.append({
-                        "Model Type": model_name,
-                        "Model ID": model_id,
-                        "Status": status,
-                        "Source": "config.py"
-                    })
-                    seen_models.add(model_name)
-        
-        # Then add configured but not loaded models
-        for model_type, model_id in config.HUGGINGFACE_MODELS.items():
-            if model_type not in seen_models:
-                table_data.append({
-                    "Model Type": model_type,
-                    "Model ID": model_id,
-                    "Status": "⏳ Not Loaded",
-                    "Source": "config.py"
-                })
-                seen_models.add(model_type)
-        
-        # Add models from providers_config if any
-        try:
-            providers_path = config.BASE_DIR / "providers_config_extended.json"
-            if providers_path.exists():
-                with open(providers_path, 'r') as f:
-                    providers_data = json.load(f)
-                
-                for provider_id, provider_info in providers_data.get('providers', {}).items():
-                    if provider_info.get('category') == 'hf-model':
-                        model_name = provider_info.get('name', provider_id)
-                        if model_name not in seen_models:
-                            table_data.append({
-                                "Model Type": model_name,
-                                "Model ID": provider_id,
-                                "Status": "📚 Registry",
-                                "Source": "providers_config"
-                            })
-                            seen_models.add(model_name)
-        except Exception as e:
-            logger.warning(f"Could not load models from providers_config: {e}")
-        
-        if not table_data:
-            table_data.append({
-                "Model Type": "No models",
-                "Model ID": "N/A",
-                "Status": "⚠️ None configured",
-                "Source": "N/A"
-            })
-        
-        if PANDAS_AVAILABLE:
-            return pd.DataFrame(table_data)
-        else:
-            return {"models": table_data}
-    
-    except Exception as e:
-        logger.error(f"Error getting HF models status: {e}")
-        if PANDAS_AVAILABLE:
-            return pd.DataFrame({"Error": [str(e)]})
-        return {"error": str(e)}
-
-
-def test_hf_model(model_name: str, test_text: str) -> str:
-    """Test a HuggingFace model with text"""
-    try:
-        if not test_text or not test_text.strip():
-            return "⚠️ Please enter test text"
-        
-        import ai_models
-        
-        if model_name in ["sentiment_twitter", "sentiment_financial", "sentiment"]:
-            # Test sentiment analysis
-            result = ai_models.analyze_sentiment(test_text)
-            
-            output = f"""
-## Sentiment Analysis Result
-
-**Input**: {test_text}
-
-**Label**: {result.get('label', 'N/A')}
-**Score**: {result.get('score', 0):.4f}
-**Confidence**: {result.get('confidence', 0):.4f}
-
-**Details**:
-```json
-{json.dumps(result.get('details', {}), indent=2)}
-```
-"""
-            return output
-        
-        elif model_name == "summarization":
-            # Test summarization
-            summary = ai_models.summarize_text(test_text)
-            
-            output = f"""
-## Summarization Result
-
-**Original** ({len(test_text)} chars):
-{test_text}
-
-**Summary** ({len(summary)} chars):
-{summary}
-"""
-            return output
-        
-        else:
-            return f"⚠️ Model '{model_name}' not recognized or not testable"
-    
-    except Exception as e:
-        logger.error(f"Error testing HF model: {e}")
-        return f"❌ Model test failed: {str(e)}"
-
-
-def initialize_hf_models() -> Tuple[Any, str]:
-    """Initialize HuggingFace models"""
-    try:
-        import ai_models
-        
-        result = ai_models.initialize_models()
-        
-        if result.get('success'):
-            message = f"✅ Models initialized successfully at {datetime.now().strftime('%H:%M:%S')}"
-        else:
-            message = f"⚠️ Model initialization completed with warnings: {result.get('status')}"
-        
-        # Return updated table
-        table = get_hf_models_status()
-        return table, message
-    
-    except Exception as e:
-        logger.error(f"Error initializing HF models: {e}")
-        return get_hf_models_status(), f"❌ Initialization failed: {str(e)}"
-
-
-# ==================== TAB 6: DIAGNOSTICS ====================
-
-def run_full_diagnostics(auto_fix: bool) -> str:
-    """Run full system diagnostics"""
-    try:
-        from backend.services.diagnostics_service import DiagnosticsService
-        
-        logger.info(f"Running diagnostics (auto_fix={auto_fix})...")
-        
-        diagnostics = DiagnosticsService()
-        
-        # Run async in sync context
-        loop = asyncio.new_event_loop()
-        asyncio.set_event_loop(loop)
-        report = loop.run_until_complete(diagnostics.run_full_diagnostics(auto_fix=auto_fix))
-        loop.close()
-        
-        # Format detailed output
-        output = f"""
-# 🔧 System Diagnostics Report
-
-**Generated**: {report.timestamp}
-**Duration**: {report.duration_ms:.2f}ms
-
----
-
-## 📊 Summary
-
-| Metric | Count |
-|--------|-------|
-| **Total Issues** | {report.total_issues} |
-| **Critical** 🔴 | {report.critical_issues} |
-| **Warnings** 🟡 | {report.warnings} |
-| **Info** 🔵 | {report.info_issues} |
-| **Auto-Fixed** ✅ | {len(report.fixed_issues)} |
-
----
-
-## 🔍 Issues Detected
-
-"""
-        
-        if not report.issues:
-            output += "✅ **No issues detected!** System is healthy.\n"
-        else:
-            # Group by category
-            by_category = {}
-            for issue in report.issues:
-                cat = issue.category
-                if cat not in by_category:
-                    by_category[cat] = []
-                by_category[cat].append(issue)
-            
-            for category, issues in sorted(by_category.items()):
-                output += f"\n### {category.upper()}\n\n"
-                
-                for issue in issues:
-                    emoji = {"critical": "🔴", "warning": "🟡", "info": "🔵"}.get(issue.severity, "⚪")
-                    fixed_mark = " ✅ **AUTO-FIXED**" if issue.auto_fixed else ""
-                    
-                    output += f"**{emoji} {issue.title}**{fixed_mark}\n\n"
-                    output += f"{issue.description}\n\n"
-                    
-                    if issue.fixable and issue.fix_action and not issue.auto_fixed:
-                        output += f"💡 **Fix**: `{issue.fix_action}`\n\n"
-                    
-                    output += "---\n\n"
-        
-        # System info
-        output += "\n## 💻 System Information\n\n"
-        output += "```json\n"
-        output += json.dumps(report.system_info, indent=2)
-        output += "\n```\n"
-        
-        return output
-    
-    except Exception as e:
-        logger.error(f"Error running diagnostics: {e}\n{traceback.format_exc()}")
-        return f"❌ Diagnostics failed: {str(e)}\n\nCheck logs for details."
-
-
-# ==================== TAB 7: LOGS ====================
-
-def get_logs(log_type: str = "recent", lines: int = 100) -> str:
-    """Get system logs with copy-friendly format"""
-    try:
-        log_file = config.LOG_FILE
-        
-        if not log_file.exists():
-            return "⚠️ Log file not found"
-        
-        # Read log file
-        with open(log_file, 'r') as f:
-            all_lines = f.readlines()
-        
-        # Filter based on log_type
-        if log_type == "errors":
-            filtered_lines = [line for line in all_lines if 'ERROR' in line or 'CRITICAL' in line]
-        elif log_type == "warnings":
-            filtered_lines = [line for line in all_lines if 'WARNING' in line]
-        else:  # recent
-            filtered_lines = all_lines
-        
-        # Get last N lines
-        recent_lines = filtered_lines[-lines:] if len(filtered_lines) > lines else filtered_lines
-        
-        if not recent_lines:
-            return f"ℹ️ No {log_type} logs found"
-        
-        # Format output with line numbers for easy reference
-        output = f"# 📋 {log_type.upper()} Logs (Last {len(recent_lines)} lines)\n\n"
-        output += "**Quick Stats:**\n"
-        output += f"- Total lines shown: `{len(recent_lines)}`\n"
-        output += f"- Log file: `{log_file}`\n"
-        output += f"- Type: `{log_type}`\n\n"
-        output += "---\n\n"
-        output += "```log\n"
-        for i, line in enumerate(recent_lines, 1):
-            output += f"{i:4d} | {line}"
-        output += "\n```\n"
-        output += "\n---\n"
-        output += "💡 **Tip**: You can now copy individual lines or the entire log block\n"
-        
-        return output
-    
-    except Exception as e:
-        logger.error(f"Error reading logs: {e}")
-        return f"❌ Error reading logs: {str(e)}"
-
-
-def clear_logs() -> str:
-    """Clear log file"""
-    try:
-        log_file = config.LOG_FILE
-        
-        if log_file.exists():
-            # Backup first
-            backup_path = log_file.parent / f"{log_file.name}.backup.{int(datetime.now().timestamp())}"
-            import shutil
-            shutil.copy2(log_file, backup_path)
-            
-            # Clear
-            with open(log_file, 'w') as f:
-                f.write("")
-            
-            logger.info("Log file cleared")
-            return f"✅ Logs cleared (backup saved to {backup_path.name})"
-        else:
-            return "⚠️ No log file to clear"
-    
-    except Exception as e:
-        logger.error(f"Error clearing logs: {e}")
-        return f"❌ Error clearing logs: {str(e)}"
-
-
-# ==================== GRADIO INTERFACE ====================
-
-def build_interface():
-    """Build the complete Gradio Blocks interface"""
-    
-    with gr.Blocks(title="Crypto Admin Dashboard", theme=gr.themes.Soft()) as demo:
-        
-        gr.Markdown("""
-# 🚀 Crypto Data Aggregator - Admin Dashboard
-
-**Real-time cryptocurrency data aggregation and analysis platform**
-
-Features: Provider Management | Market Data | Auto Provider Loader | HF Models | System Diagnostics
-        """)
-        
-        with gr.Tabs():
-            
-            # ==================== TAB 1: STATUS ====================
-            with gr.Tab("📊 Status"):
-                gr.Markdown("### System Status Overview")
-                
-                with gr.Row():
-                    status_refresh_btn = gr.Button("🔄 Refresh Status", variant="primary")
-                    status_diag_btn = gr.Button("🔧 Run Quick Diagnostics")
-                
-                status_summary = gr.Markdown()
-                
-                with gr.Row():
-                    with gr.Column():
-                        gr.Markdown("#### Database Statistics")
-                        db_stats_json = gr.JSON()
-                    
-                    with gr.Column():
-                        gr.Markdown("#### System Information")
-                        system_info_json = gr.JSON()
-                
-                diag_output = gr.Markdown()
-                
-                # Load initial status
-                demo.load(
-                    fn=get_status_tab,
-                    outputs=[status_summary, db_stats_json, system_info_json]
-                )
-                
-                # Refresh button
-                status_refresh_btn.click(
-                    fn=get_status_tab,
-                    outputs=[status_summary, db_stats_json, system_info_json]
-                )
-                
-                # Quick diagnostics
-                status_diag_btn.click(
-                    fn=lambda: run_diagnostics_from_status(False),
-                    outputs=diag_output
-                )
-            
-            # ==================== TAB 2: PROVIDERS ====================
-            with gr.Tab("🔌 Providers"):
-                gr.Markdown("### API Provider Management")
-                
-                with gr.Row():
-                    provider_category = gr.Dropdown(
-                        label="Filter by Category",
-                        choices=get_provider_categories(),
-                        value="All"
-                    )
-                    provider_reload_btn = gr.Button("🔄 Reload Providers", variant="primary")
-                
-                providers_table = gr.Dataframe(
-                    label="Providers",
-                    interactive=False,
-                    wrap=True
-                ) if PANDAS_AVAILABLE else gr.JSON(label="Providers")
-                
-                provider_status = gr.Textbox(label="Status", interactive=False)
-                
-                # Load initial providers
-                demo.load(
-                    fn=lambda: get_providers_table("All"),
-                    outputs=providers_table
-                )
-                
-                # Category filter
-                provider_category.change(
-                    fn=get_providers_table,
-                    inputs=provider_category,
-                    outputs=providers_table
-                )
-                
-                # Reload button
-                provider_reload_btn.click(
-                    fn=reload_providers_config,
-                    outputs=[providers_table, provider_status]
-                )
-            
-            # ==================== TAB 3: MARKET DATA ====================
-            with gr.Tab("📈 Market Data"):
-                gr.Markdown("### Live Cryptocurrency Market Data")
-                
-                with gr.Row():
-                    market_search = gr.Textbox(
-                        label="Search",
-                        placeholder="Search by name or symbol..."
-                    )
-                    market_refresh_btn = gr.Button("🔄 Refresh Prices", variant="primary")
-                
-                market_table = gr.Dataframe(
-                    label="Market Data",
-                    interactive=False,
-                    wrap=True,
-                    height=400
-                ) if PANDAS_AVAILABLE else gr.JSON(label="Market Data")
-                
-                market_status = gr.Textbox(label="Status", interactive=False)
-                
-                # Price chart section
-                if PLOTLY_AVAILABLE:
-                    gr.Markdown("#### Price History Chart")
-                    
-                    with gr.Row():
-                        chart_symbol = gr.Textbox(
-                            label="Symbol",
-                            placeholder="BTC",
-                            value="BTC"
-                        )
-                        chart_timeframe = gr.Dropdown(
-                            label="Timeframe",
-                            choices=["24h", "7d", "30d", "90d"],
-                            value="7d"
-                        )
-                        chart_plot_btn = gr.Button("📊 Plot")
-                    
-                    price_chart = gr.Plot(label="Price History")
-                    
-                    chart_plot_btn.click(
-                        fn=plot_price_history,
-                        inputs=[chart_symbol, chart_timeframe],
-                        outputs=price_chart
-                    )
-                
-                # Load initial data
-                demo.load(
-                    fn=lambda: get_market_data_table(""),
-                    outputs=market_table
-                )
-                
-                # Search
-                market_search.change(
-                    fn=get_market_data_table,
-                    inputs=market_search,
-                    outputs=market_table
-                )
-                
-                # Refresh
-                market_refresh_btn.click(
-                    fn=refresh_market_data,
-                    outputs=[market_table, market_status]
-                )
-            
-            # ==================== TAB 4: APL SCANNER ====================
-            with gr.Tab("🔍 APL Scanner"):
-                gr.Markdown("### Auto Provider Loader")
-                gr.Markdown("Automatically discover, validate, and integrate API providers and HuggingFace models.")
-                
-                with gr.Row():
-                    apl_scan_btn = gr.Button("▶️ Run APL Scan", variant="primary", size="lg")
-                    apl_report_btn = gr.Button("📄 View Last Report")
-                
-                apl_output = gr.Markdown()
-                
-                apl_scan_btn.click(
-                    fn=run_apl_scan,
-                    outputs=apl_output
-                )
-                
-                apl_report_btn.click(
-                    fn=get_apl_report,
-                    outputs=apl_output
-                )
-                
-                # Load last report on startup
-                demo.load(
-                    fn=get_apl_report,
-                    outputs=apl_output
-                )
-            
-            # ==================== TAB 5: HF MODELS ====================
-            with gr.Tab("🤖 HF Models"):
-                gr.Markdown("### HuggingFace Models Status & Testing")
-                
-                with gr.Row():
-                    hf_init_btn = gr.Button("🔄 Initialize Models", variant="primary")
-                    hf_refresh_btn = gr.Button("🔄 Refresh Status")
-                
-                hf_models_table = gr.Dataframe(
-                    label="Models",
-                    interactive=False
-                ) if PANDAS_AVAILABLE else gr.JSON(label="Models")
-                
-                hf_status = gr.Textbox(label="Status", interactive=False)
-                
-                gr.Markdown("#### Test Model")
-                
-                with gr.Row():
-                    test_model_dropdown = gr.Dropdown(
-                        label="Model",
-                        choices=["sentiment", "sentiment_twitter", "sentiment_financial", "summarization"],
-                        value="sentiment"
-                    )
-                
-                test_input = gr.Textbox(
-                    label="Test Input",
-                    placeholder="Enter text to test the model...",
-                    lines=3
-                )
-                
-                test_btn = gr.Button("▶️ Run Test", variant="secondary")
-                
-                test_output = gr.Markdown(label="Test Output")
-                
-                # Load initial status
-                demo.load(
-                    fn=get_hf_models_status,
-                    outputs=hf_models_table
-                )
-                
-                # Initialize models
-                hf_init_btn.click(
-                    fn=initialize_hf_models,
-                    outputs=[hf_models_table, hf_status]
-                )
-                
-                # Refresh status
-                hf_refresh_btn.click(
-                    fn=get_hf_models_status,
-                    outputs=hf_models_table
-                )
-                
-                # Test model
-                test_btn.click(
-                    fn=test_hf_model,
-                    inputs=[test_model_dropdown, test_input],
-                    outputs=test_output
-                )
-            
-            # ==================== TAB 6: DIAGNOSTICS ====================
-            with gr.Tab("🔧 Diagnostics"):
-                gr.Markdown("### System Diagnostics & Auto-Repair")
-                
-                with gr.Row():
-                    diag_run_btn = gr.Button("▶️ Run Diagnostics", variant="primary")
-                    diag_autofix_btn = gr.Button("🔧 Run with Auto-Fix", variant="secondary")
-                
-                diagnostics_output = gr.Markdown()
-                
-                diag_run_btn.click(
-                    fn=lambda: run_full_diagnostics(False),
-                    outputs=diagnostics_output
-                )
-                
-                diag_autofix_btn.click(
-                    fn=lambda: run_full_diagnostics(True),
-                    outputs=diagnostics_output
-                )
-            
-            # ==================== TAB 7: LOGS ====================
-            with gr.Tab("📋 Logs"):
-                gr.Markdown("### System Logs Viewer")
-                
-                with gr.Row():
-                    log_type = gr.Dropdown(
-                        label="Log Type",
-                        choices=["recent", "errors", "warnings"],
-                        value="recent"
-                    )
-                    log_lines = gr.Slider(
-                        label="Lines to Show",
-                        minimum=10,
-                        maximum=500,
-                        value=100,
-                        step=10
-                    )
-                
-                with gr.Row():
-                    log_refresh_btn = gr.Button("🔄 Refresh Logs", variant="primary")
-                    log_clear_btn = gr.Button("🗑️ Clear Logs", variant="secondary")
-                
-                logs_output = gr.Markdown()
-                log_clear_status = gr.Textbox(label="Status", interactive=False, visible=False)
-                
-                # Load initial logs
-                demo.load(
-                    fn=lambda: get_logs("recent", 100),
-                    outputs=logs_output
-                )
-                
-                # Refresh logs
-                log_refresh_btn.click(
-                    fn=get_logs,
-                    inputs=[log_type, log_lines],
-                    outputs=logs_output
-                )
-                
-                # Update when dropdown changes
-                log_type.change(
-                    fn=get_logs,
-                    inputs=[log_type, log_lines],
-                    outputs=logs_output
-                )
-                
-                # Clear logs
-                log_clear_btn.click(
-                    fn=clear_logs,
-                    outputs=log_clear_status
-                ).then(
-                    fn=lambda: get_logs("recent", 100),
-                    outputs=logs_output
-                )
-        
-        # Footer
-        gr.Markdown("""
----
-**Crypto Data Aggregator Admin Dashboard** | Real Data Only | No Mock/Fake Data
-        """)
-    
-    return demo
-
-
-# ==================== MAIN ENTRY POINT ====================
-
-demo = build_interface()
-
-if __name__ == "__main__":
-    logger.info("Launching Gradio dashboard...")
-    
-    # Try to mount FastAPI app for API endpoints
-    try:
-        from fastapi import FastAPI as FastAPIApp
-        from fastapi.middleware.wsgi import WSGIMiddleware
-        import uvicorn
-        from threading import Thread
-        import time
-        
-        # Import the FastAPI app from hf_unified_server
-        try:
-            from hf_unified_server import app as fastapi_app
-            logger.info("✅ FastAPI app imported successfully")
-            
-            # Start FastAPI server in a separate thread on port 7861
-            def run_fastapi():
-                uvicorn.run(
-                    fastapi_app,
-                    host="0.0.0.0",
-                    port=7861,
-                    log_level="info"
-                )
-            
-            fastapi_thread = Thread(target=run_fastapi, daemon=True)
-            fastapi_thread.start()
-            time.sleep(2)  # Give FastAPI time to start
-            logger.info("✅ FastAPI server started on port 7861")
-        except ImportError as e:
-            logger.warning(f"⚠️ Could not import FastAPI app: {e}")
-    except Exception as e:
-        logger.warning(f"⚠️ Could not start FastAPI server: {e}")
-    
-    demo.launch(
-        server_name="0.0.0.0",
-        server_port=7860,
-        share=False
-    )
+#!/usr/bin/env python3
+"""
+Crypto Intelligence Hub - Hugging Face Space Application
+یکپارچه‌سازی کامل بک‌اند و فرانت‌اند برای جمع‌آوری داده‌های رمز ارز
+Hub کامل با منابع رایگان و مدل‌های Hugging Face
+"""
+
+import os
+import json
+import asyncio
+import logging
+from pathlib import Path
+from typing import Dict, List, Optional, Any
+from datetime import datetime
+import gradio as gr
+import pandas as pd
+import plotly.graph_objects as go
+import plotly.express as px
+import httpx
+
+# Import backend services
+try:
+    from api_server_extended import app as fastapi_app
+    from ai_models import ModelRegistry, MODEL_SPECS, get_model_info, registry_status
+    FASTAPI_AVAILABLE = True
+except ImportError as e:
+    logging.warning(f"FastAPI not available: {e}")
+    FASTAPI_AVAILABLE = False
+    ModelRegistry = None
+    MODEL_SPECS = {}
+    get_model_info = None
+    registry_status = None
+
+# Setup logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+# Global state
+WORKSPACE_ROOT = Path("/app" if Path("/app").exists() else Path("."))
+RESOURCES_JSON = WORKSPACE_ROOT / "api-resources" / "crypto_resources_unified_2025-11-11.json"
+ALL_APIS_JSON = WORKSPACE_ROOT / "all_apis_merged_2025.json"
+
+# Fallback paths
+if not RESOURCES_JSON.exists():
+    RESOURCES_JSON = WORKSPACE_ROOT / "all_apis_merged_2025.json"
+if not ALL_APIS_JSON.exists():
+    ALL_APIS_JSON = WORKSPACE_ROOT / "all_apis_merged_2025.json"
+
+# Initialize model registry
+model_registry = ModelRegistry() if ModelRegistry else None
+
+
+class CryptoDataHub:
+    """مرکز داده‌های رمز ارز با پشتیبانی از منابع رایگان و مدل‌های Hugging Face"""
+    
+    def __init__(self):
+        self.resources = {}
+        self.models_loaded = False
+        self.load_resources()
+        self.initialize_models()
+    
+    def load_resources(self):
+        """بارگذاری منابع از فایل‌های JSON"""
+        try:
+            # Load unified resources
+            if RESOURCES_JSON.exists():
+                with open(RESOURCES_JSON, 'r', encoding='utf-8') as f:
+                    data = json.load(f)
+                    self.resources['unified'] = data
+                    logger.info(f"✅ Loaded unified resources: {RESOURCES_JSON}")
+            
+            # Load all APIs merged
+            if ALL_APIS_JSON.exists():
+                with open(ALL_APIS_JSON, 'r', encoding='utf-8') as f:
+                    data = json.load(f)
+                    self.resources['all_apis'] = data
+                    logger.info(f"✅ Loaded all APIs: {ALL_APIS_JSON}")
+            
+            logger.info(f"📊 Total resource files loaded: {len(self.resources)}")
+        except Exception as e:
+            logger.error(f"❌ Error loading resources: {e}")
+    
+    def initialize_models(self):
+        """بارگذاری مدل‌های Hugging Face"""
+        if not model_registry:
+            logger.warning("Model registry not available")
+            return
+        
+        try:
+            # Initialize available models
+            result = model_registry.initialize_models()
+            self.models_loaded = result.get('status') == 'ok'
+            logger.info(f"✅ Hugging Face models initialized: {result}")
+        except Exception as e:
+            logger.warning(f"⚠️ Could not initialize all models: {e}")
+    
+    def get_market_data_sources(self) -> List[Dict]:
+        """دریافت منابع داده‌های بازار"""
+        sources = []
+        
+        # Try unified resources first
+        if 'unified' in self.resources:
+            registry = self.resources['unified'].get('registry', {})
+            
+            # Market data APIs
+            market_apis = registry.get('market_data', [])
+            for api in market_apis:
+                sources.append({
+                    'name': api.get('name', 'Unknown'),
+                    'category': 'market',
+                    'base_url': api.get('base_url', ''),
+                    'free': api.get('free', False),
+                    'auth_required': bool(api.get('auth', {}).get('key'))
+                })
+        
+        # Try all_apis structure
+        if 'all_apis' in self.resources:
+            data = self.resources['all_apis']
+            
+            # Check for discovered_keys which indicates market data sources
+            if 'discovered_keys' in data:
+                for provider, keys in data['discovered_keys'].items():
+                    if provider in ['coinmarketcap', 'cryptocompare']:
+                        sources.append({
+                            'name': provider.upper(),
+                            'category': 'market',
+                            'base_url': f'https://api.{provider}.com' if provider == 'coinmarketcap' else f'https://min-api.{provider}.com',
+                            'free': False,
+                            'auth_required': True
+                        })
+            
+            # Check raw_files for API configurations
+            if 'raw_files' in data:
+                for file_info in data['raw_files']:
+                    content = file_info.get('content', '')
+                    if 'CoinGecko' in content or 'coingecko' in content.lower():
+                        sources.append({
+                            'name': 'CoinGecko',
+                            'category': 'market',
+                            'base_url': 'https://api.coingecko.com/api/v3',
+                            'free': True,
+                            'auth_required': False
+                        })
+                    if 'Binance' in content or 'binance' in content.lower():
+                        sources.append({
+                            'name': 'Binance Public',
+                            'category': 'market',
+                            'base_url': 'https://api.binance.com/api/v3',
+                            'free': True,
+                            'auth_required': False
+                        })
+        
+        # Remove duplicates
+        seen = set()
+        unique_sources = []
+        for source in sources:
+            key = source['name']
+            if key not in seen:
+                seen.add(key)
+                unique_sources.append(source)
+        
+        return unique_sources
+    
+    def get_available_models(self) -> List[Dict]:
+        """دریافت لیست مدل‌های در دسترس"""
+        models = []
+        
+        if MODEL_SPECS:
+            for key, spec in MODEL_SPECS.items():
+                models.append({
+                    'key': key,
+                    'name': spec.model_id,
+                    'task': spec.task,
+                    'category': spec.category,
+                    'requires_auth': spec.requires_auth
+                })
+        
+        return models
+    
+    async def analyze_sentiment(self, text: str, model_key: str = "crypto_sent_0", use_backend: bool = False) -> Dict:
+        """تحلیل احساسات با استفاده از مدل‌های Hugging Face"""
+        # Try backend API first if requested and available
+        if use_backend and FASTAPI_AVAILABLE:
+            try:
+                async with httpx.AsyncClient(timeout=30.0) as client:
+                    response = await client.post(
+                        "http://localhost:7860/api/hf/run-sentiment",
+                        json={"texts": [text]},
+                        headers={"Content-Type": "application/json"}
+                    )
+                    if response.status_code == 200:
+                        data = response.json()
+                        if data.get("results"):
+                            result = data["results"][0]
+                            return {
+                                'sentiment': result.get('label', 'unknown'),
+                                'confidence': result.get('confidence', 0.0),
+                                'model': 'backend_api',
+                                'text': text[:100],
+                                'vote': result.get('vote', 0.0)
+                            }
+            except Exception as e:
+                logger.warning(f"Backend API call failed, falling back to direct model: {e}")
+        
+        # Direct model access
+        if not model_registry or not self.models_loaded:
+            return {
+                'error': 'Models not available',
+                'sentiment': 'unknown',
+                'confidence': 0.0
+            }
+        
+        try:
+            pipeline = model_registry.get_pipeline(model_key)
+            result = pipeline(text)
+            
+            # Handle different result formats
+            if isinstance(result, list) and len(result) > 0:
+                result = result[0]
+            
+            return {
+                'sentiment': result.get('label', 'unknown'),
+                'confidence': result.get('score', 0.0),
+                'model': model_key,
+                'text': text[:100]
+            }
+        except Exception as e:
+            logger.error(f"Error analyzing sentiment: {e}")
+            return {
+                'error': str(e),
+                'sentiment': 'error',
+                'confidence': 0.0
+            }
+    
+    def get_resource_summary(self) -> Dict:
+        """خلاصه منابع موجود"""
+        summary = {
+            'total_resources': 0,
+            'categories': {},
+            'free_resources': 0,
+            'models_available': len(self.get_available_models())
+        }
+        
+        if 'unified' in self.resources:
+            registry = self.resources['unified'].get('registry', {})
+            
+            for category, items in registry.items():
+                if isinstance(items, list):
+                    count = len(items)
+                    summary['total_resources'] += count
+                    summary['categories'][category] = count
+                    
+                    # Count free resources
+                    free_count = sum(1 for item in items if item.get('free', False))
+                    summary['free_resources'] += free_count
+        
+        # Add market sources
+        market_sources = self.get_market_data_sources()
+        if market_sources:
+            summary['total_resources'] += len(market_sources)
+            summary['categories']['market_data'] = len(market_sources)
+            summary['free_resources'] += sum(1 for s in market_sources if s.get('free', False))
+        
+        return summary
+
+
+# Initialize global hub
+hub = CryptoDataHub()
+
+
+# =============================================================================
+# Gradio Interface Functions
+# =============================================================================
+
+def get_dashboard_summary():
+    """نمایش خلاصه داشبورد"""
+    summary = hub.get_resource_summary()
+    
+    html = f"""
+    <div style="padding: 20px; font-family: Arial, sans-serif;">
+        <h2>📊 خلاصه منابع و مدل‌ها</h2>
+        
+        <div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); gap: 20px; margin: 20px 0;">
+            <div style="background: linear-gradient(135deg, #667eea 0%, #764ba2 100%); padding: 20px; border-radius: 10px; color: white;">
+                <h3>منابع کل</h3>
+                <p style="font-size: 32px; margin: 10px 0; font-weight: bold;">{summary['total_resources']}</p>
+            </div>
+            
+            <div style="background: linear-gradient(135deg, #11998e 0%, #38ef7d 100%); padding: 20px; border-radius: 10px; color: white;">
+                <h3>منابع رایگان</h3>
+                <p style="font-size: 32px; margin: 10px 0; font-weight: bold;">{summary['free_resources']}</p>
+            </div>
+            
+            <div style="background: linear-gradient(135deg, #ee0979 0%, #ff6a00 100%); padding: 20px; border-radius: 10px; color: white;">
+                <h3>مدل‌های AI</h3>
+                <p style="font-size: 32px; margin: 10px 0; font-weight: bold;">{summary['models_available']}</p>
+            </div>
+            
+            <div style="background: linear-gradient(135deg, #f093fb 0%, #f5576c 100%); padding: 20px; border-radius: 10px; color: white;">
+                <h3>دسته‌بندی‌ها</h3>
+                <p style="font-size: 32px; margin: 10px 0; font-weight: bold;">{len(summary['categories'])}</p>
+            </div>
+        </div>
+        
+        <h3>دسته‌بندی منابع:</h3>
+        <ul>
+    """
+    
+    for category, count in summary['categories'].items():
+        html += f"<li><strong>{category}:</strong> {count} منبع</li>"
+    
+    html += """
+        </ul>
+    </div>
+    """
+    
+    return html
+
+
+def get_resources_table():
+    """جدول منابع"""
+    sources = hub.get_market_data_sources()
+    
+    if not sources:
+        return pd.DataFrame({'پیام': ['هیچ منبعی یافت نشد. لطفاً فایل‌های JSON را بررسی کنید.']})
+    
+    df_data = []
+    for source in sources[:100]:  # Limit to 100 for display
+        df_data.append({
+            'نام': source['name'],
+            'دسته': source['category'],
+            'رایگان': '✅' if source['free'] else '❌',
+            'نیاز به کلید': '✅' if source['auth_required'] else '❌',
+            'URL پایه': source['base_url'][:60] + '...' if len(source['base_url']) > 60 else source['base_url']
+        })
+    
+    return pd.DataFrame(df_data)
+
+
+def get_models_table():
+    """جدول مدل‌ها"""
+    models = hub.get_available_models()
+    
+    if not models:
+        return pd.DataFrame({'پیام': ['هیچ مدلی یافت نشد. مدل‌ها در حال بارگذاری هستند...']})
+    
+    df_data = []
+    for model in models:
+        df_data.append({
+            'کلید': model['key'],
+            'نام مدل': model['name'],
+            'نوع کار': model['task'],
+            'دسته': model['category'],
+            'نیاز به احراز هویت': '✅' if model['requires_auth'] else '❌'
+        })
+    
+    return pd.DataFrame(df_data)
+
+
+def analyze_text_sentiment(text: str, model_selection: str, use_backend: bool = False):
+    """تحلیل احساسات متن"""
+    if not text.strip():
+        return "⚠️ لطفاً متنی وارد کنید", ""
+    
+    try:
+        # Extract model key from dropdown selection
+        if model_selection and " - " in model_selection:
+            model_key = model_selection.split(" - ")[0]
+        else:
+            model_key = model_selection if model_selection else "crypto_sent_0"
+        
+        result = asyncio.run(hub.analyze_sentiment(text, model_key, use_backend=use_backend))
+        
+        if 'error' in result:
+            return f"❌ خطا: {result['error']}", ""
+        
+        sentiment_emoji = {
+            'POSITIVE': '📈',
+            'NEGATIVE': '📉',
+            'NEUTRAL': '➡️',
+            'LABEL_0': '📈',
+            'LABEL_1': '📉',
+            'LABEL_2': '➡️',
+            'positive': '📈',
+            'negative': '📉',
+            'neutral': '➡️',
+            'bullish': '📈',
+            'bearish': '📉'
+        }.get(result['sentiment'], '❓')
+        
+        confidence_pct = result['confidence'] * 100 if result['confidence'] <= 1.0 else result['confidence']
+        
+        vote_info = ""
+        if 'vote' in result:
+            vote_emoji = '📈' if result['vote'] > 0 else '📉' if result['vote'] < 0 else '➡️'
+            vote_info = f"\n**رأی مدل:** {vote_emoji} {result['vote']:.2f}"
+        
+        result_text = f"""
+## نتیجه تحلیل احساسات
+
+**احساسات:** {sentiment_emoji} {result['sentiment']}
+**اعتماد:** {confidence_pct:.2f}%
+**مدل استفاده شده:** {result['model']}
+**متن تحلیل شده:** {result['text']}
+{vote_info}
+        """
+        
+        result_json = json.dumps(result, indent=2, ensure_ascii=False)
+        
+        return result_text, result_json
+    except Exception as e:
+        return f"❌ خطا در تحلیل: {str(e)}", ""
+
+
+def create_category_chart():
+    """نمودار دسته‌بندی منابع"""
+    summary = hub.get_resource_summary()
+    
+    categories = list(summary['categories'].keys())
+    counts = list(summary['categories'].values())
+    
+    if not categories:
+        fig = go.Figure()
+        fig.add_annotation(
+            text="No data available",
+            xref="paper", yref="paper",
+            x=0.5, y=0.5, showarrow=False
+        )
+        return fig
+    
+    fig = go.Figure(data=[
+        go.Bar(
+            x=categories,
+            y=counts,
+            marker_color='lightblue',
+            text=counts,
+            textposition='auto'
+        )
+    ])
+    
+    fig.update_layout(
+        title='توزیع منابع بر اساس دسته‌بندی',
+        xaxis_title='دسته‌بندی',
+        yaxis_title='تعداد منابع',
+        template='plotly_white',
+        height=400
+    )
+    
+    return fig
+
+
+def get_model_status():
+    """وضعیت مدل‌ها"""
+    if not registry_status:
+        return "❌ Model registry not available"
+    
+    status = registry_status()
+    
+    html = f"""
+    <div style="padding: 20px;">
+        <h3>وضعیت مدل‌ها</h3>
+        <p><strong>وضعیت:</strong> {'✅ فعال' if status.get('ok') else '❌ غیرفعال'}</p>
+        <p><strong>مدل‌های بارگذاری شده:</strong> {status.get('pipelines_loaded', 0)}</p>
+        <p><strong>مدل‌های در دسترس:</strong> {len(status.get('available_models', []))}</p>
+        <p><strong>حالت Hugging Face:</strong> {status.get('hf_mode', 'unknown')}</p>
+        <p><strong>Transformers موجود:</strong> {'✅' if status.get('transformers_available') else '❌'}</p>
+    </div>
+    """
+    
+    return html
+
+
+# =============================================================================
+# Build Gradio Interface
+# =============================================================================
+
+def create_interface():
+    """ایجاد رابط کاربری Gradio"""
+    
+    # Get available models for dropdown
+    models = hub.get_available_models()
+    model_choices = [f"{m['key']} - {m['name']}" for m in models] if models else ["crypto_sent_0 - CryptoBERT"]
+    model_keys = [m['key'] for m in models] if models else ["crypto_sent_0"]
+    
+    with gr.Blocks(
+        theme=gr.themes.Soft(primary_hue="blue", secondary_hue="purple"),
+        title="Crypto Intelligence Hub - مرکز هوش رمز ارز",
+        css="""
+        .gradio-container {
+            max-width: 1400px !important;
+        }
+        """
+    ) as app:
+        
+        gr.Markdown("""
+        # 🚀 Crypto Intelligence Hub
+        ## مرکز هوش مصنوعی و جمع‌آوری داده‌های رمز ارز
+        
+        **منابع رایگان | مدل‌های Hugging Face | رابط کاربری کامل**
+        
+        این برنامه یک رابط کامل برای دسترسی به منابع رایگان داده‌های رمز ارز و استفاده از مدل‌های هوش مصنوعی Hugging Face است.
+        """)
+        
+        # Tab 1: Dashboard
+        with gr.Tab("📊 داشبورد"):
+            dashboard_summary = gr.HTML()
+            refresh_dashboard_btn = gr.Button("🔄 به‌روزرسانی", variant="primary")
+            
+            refresh_dashboard_btn.click(
+                fn=get_dashboard_summary,
+                outputs=dashboard_summary
+            )
+            
+            app.load(
+                fn=get_dashboard_summary,
+                outputs=dashboard_summary
+            )
+        
+        # Tab 2: Resources
+        with gr.Tab("📚 منابع داده"):
+            gr.Markdown("### منابع رایگان برای جمع‌آوری داده‌های رمز ارز")
+            
+            resources_table = gr.DataFrame(
+                label="لیست منابع",
+                wrap=True
+            )
+            
+            refresh_resources_btn = gr.Button("🔄 به‌روزرسانی", variant="primary")
+            
+            refresh_resources_btn.click(
+                fn=get_resources_table,
+                outputs=resources_table
+            )
+            
+            app.load(
+                fn=get_resources_table,
+                outputs=resources_table
+            )
+            
+            category_chart = gr.Plot(label="نمودار دسته‌بندی")
+            
+            refresh_resources_btn.click(
+                fn=create_category_chart,
+                outputs=category_chart
+            )
+        
+        # Tab 3: AI Models
+        with gr.Tab("🤖 مدل‌های AI"):
+            gr.Markdown("### مدل‌های Hugging Face برای تحلیل احساسات و هوش مصنوعی")
+            
+            model_status_html = gr.HTML()
+            
+            models_table = gr.DataFrame(
+                label="لیست مدل‌ها",
+                wrap=True
+            )
+            
+            refresh_models_btn = gr.Button("🔄 به‌روزرسانی", variant="primary")
+            
+            refresh_models_btn.click(
+                fn=get_models_table,
+                outputs=models_table
+            )
+            
+            refresh_models_btn.click(
+                fn=get_model_status,
+                outputs=model_status_html
+            )
+            
+            app.load(
+                fn=get_models_table,
+                outputs=models_table
+            )
+            
+            app.load(
+                fn=get_model_status,
+                outputs=model_status_html
+            )
+        
+        # Tab 4: Sentiment Analysis
+        with gr.Tab("💭 تحلیل احساسات"):
+            gr.Markdown("### تحلیل احساسات متن با استفاده از مدل‌های Hugging Face")
+            
+            with gr.Row():
+                sentiment_text = gr.Textbox(
+                    label="متن برای تحلیل",
+                    placeholder="مثال: Bitcoin price is rising rapidly! The market shows strong bullish momentum.",
+                    lines=5
+                )
+            
+            with gr.Row():
+                model_dropdown = gr.Dropdown(
+                    choices=model_choices,
+                    value=model_choices[0] if model_choices else None,
+                    label="انتخاب مدل"
+                )
+                use_backend_check = gr.Checkbox(
+                    label="استفاده از بک‌اند API (در صورت موجود بودن)",
+                    value=False
+                )
+                analyze_btn = gr.Button("🔍 تحلیل", variant="primary")
+            
+            with gr.Row():
+                sentiment_result = gr.Markdown(label="نتیجه")
+                sentiment_json = gr.Code(
+                    label="JSON خروجی",
+                    language="json"
+                )
+            
+            def analyze_with_selected_model(text, model_choice, use_backend):
+                return analyze_text_sentiment(text, model_choice, use_backend=use_backend)
+            
+            analyze_btn.click(
+                fn=analyze_with_selected_model,
+                inputs=[sentiment_text, model_dropdown, use_backend_check],
+                outputs=[sentiment_result, sentiment_json]
+            )
+            
+            # Example texts
+            gr.Markdown("""
+            ### مثال‌های متن:
+            - "Bitcoin is showing strong bullish momentum"
+            - "Market crash expected due to regulatory concerns"
+            - "Ethereum network upgrade successful"
+            - "Crypto market sentiment is very positive today"
+            """)
+        
+        # Tab 5: API Integration
+        with gr.Tab("🔌 یکپارچه‌سازی API"):
+            gr.Markdown("""
+            ### اتصال به بک‌اند FastAPI
+            
+            این بخش به سرویس‌های بک‌اند متصل می‌شود که از منابع JSON استفاده می‌کنند.
+            
+            **وضعیت:** {'✅ فعال' if FASTAPI_AVAILABLE else '❌ غیرفعال'}
+            """)
+            
+            if FASTAPI_AVAILABLE:
+                gr.Markdown("""
+                **API Endpoints در دسترس:**
+                - `/api/market-data` - داده‌های بازار
+                - `/api/sentiment` - تحلیل احساسات
+                - `/api/news` - اخبار رمز ارز
+                - `/api/resources` - لیست منابع
+                """)
+            
+            # Show resource summary
+            resource_info = gr.Markdown()
+            
+            def get_resource_info():
+                summary = hub.get_resource_summary()
+                return f"""
+                ## اطلاعات منابع
+                
+                - **کل منابع:** {summary['total_resources']}
+                - **منابع رایگان:** {summary['free_resources']}
+                - **مدل‌های AI:** {summary['models_available']}
+                - **دسته‌بندی‌ها:** {len(summary['categories'])}
+                
+                ### دسته‌بندی‌های موجود:
+                {', '.join(summary['categories'].keys()) if summary['categories'] else 'هیچ دسته‌ای یافت نشد'}
+                """
+            
+            app.load(
+                fn=get_resource_info,
+                outputs=resource_info
+            )
+        
+        # Footer
+        gr.Markdown("""
+        ---
+        ### 📝 اطلاعات
+        - **منابع:** از فایل‌های JSON بارگذاری شده
+        - **مدل‌ها:** Hugging Face Transformers
+        - **بک‌اند:** FastAPI (در صورت موجود بودن)
+        - **فرانت‌اند:** Gradio
+        - **محیط:** Hugging Face Spaces (Docker)
+        """)
+    
+    return app
+
+
+# =============================================================================
+# Main Entry Point
+# =============================================================================
+
+if __name__ == "__main__":
+    logger.info("🚀 Starting Crypto Intelligence Hub...")
+    logger.info(f"📁 Workspace: {WORKSPACE_ROOT}")
+    logger.info(f"📊 Resources loaded: {len(hub.resources)}")
+    logger.info(f"🤖 Models available: {len(hub.get_available_models())}")
+    logger.info(f"🔌 FastAPI available: {FASTAPI_AVAILABLE}")
+    
+    # Create and launch interface
+    app = create_interface()
+    
+    app.launch(
+        server_name="0.0.0.0",
+        server_port=7860,
+        share=False,
+        show_error=True
+    )

backend/services/auto_discovery_service.py

@@ -91,10 +91,7 @@ class AutoDiscoveryService:
         if InferenceClient is None:
             logger.warning("huggingface-hub package not available. Auto discovery will use fallback heuristics.")
         else:
-            # Get HF token from environment or use default
-            from config import get_settings
-            settings = get_settings()
-            hf_token = os.getenv("HF_TOKEN") or os.getenv("HF_API_TOKEN") or settings.hf_token or "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"
+            hf_token = os.getenv("HF_API_TOKEN")
             try:
                 self._hf_client = InferenceClient(model=self.hf_model, token=hf_token)
                 logger.info("Auto discovery Hugging Face client initialized with model %s", self.hf_model)

backend/services/diagnostics_service.py

@@ -260,14 +260,7 @@ class DiagnosticsService:
 
         try:
             from huggingface_hub import InferenceClient, HfApi
-            import os
-            from config import get_settings
-            
-            # Get HF token from settings or use default
-            settings = get_settings()
-            hf_token = settings.hf_token or os.getenv("HF_TOKEN") or "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"
-            
-            api = HfApi(token=hf_token)
+            api = HfApi()
 
             # بررسی مدل‌های استفاده شده
             models_to_check = [

backend/services/hf_registry.py

@@ -8,16 +8,6 @@ HF_API_DATASETS = "https://huggingface.co/api/datasets"
 REFRESH_INTERVAL_SEC = int(os.getenv("HF_REGISTRY_REFRESH_SEC", "21600"))
 HTTP_TIMEOUT = float(os.getenv("HF_HTTP_TIMEOUT", "8.0"))
 
-HF_MODE = os.getenv("HF_MODE", "off").lower()
-if HF_MODE not in ("off", "public", "auth"):
-    HF_MODE = "off"
-
-HF_TOKEN = None
-if HF_MODE == "auth":
-    HF_TOKEN = os.getenv("HF_TOKEN")
-    if not HF_TOKEN:
-        HF_MODE = "off"
-
 # Curated Crypto Datasets
 CRYPTO_DATASETS = {
     "price": [
@@ -55,81 +45,68 @@ class HFRegistry:
         self.fail_reason: Optional[str] = None
 
     async def _hf_json(self, url: str, params: Dict[str, Any]) -> Any:
-        headers = {}
-        if HF_MODE == "auth" and HF_TOKEN:
-            headers["Authorization"] = f"Bearer {HF_TOKEN}"
-        
-        async with httpx.AsyncClient(timeout=HTTP_TIMEOUT, headers=headers) as client:
+        async with httpx.AsyncClient(timeout=HTTP_TIMEOUT) as client:
             r = await client.get(url, params=params)
             r.raise_for_status()
             return r.json()
 
     async def refresh(self) -> Dict[str, Any]:
-        if HF_MODE == "off":
-            self.fail_reason = "HF_MODE=off"
-            return {"ok": False, "error": "HF_MODE=off", "models": 0, "datasets": 0}
-        
         try:
+            # Seed models
             for name in _SEED_MODELS:
                 self.models.setdefault(name, {"id": name, "source": "seed", "pipeline_tag": "sentiment-analysis"})
             
+            # Seed datasets with category metadata
             for category, dataset_list in CRYPTO_DATASETS.items():
                 for name in dataset_list:
                     self.datasets.setdefault(name, {"id": name, "source": "seed", "category": category, "tags": ["crypto", category]})
             
-            if HF_MODE in ("public", "auth"):
-                try:
-                    q_sent = {"pipeline_tag": "sentiment-analysis", "search": "crypto", "limit": 50}
-                    models = await self._hf_json(HF_API_MODELS, q_sent)
-                    for m in models or []:
-                        mid = m.get("modelId") or m.get("id") or m.get("name")
-                        if not mid: continue
-                        self.models[mid] = {
-                            "id": mid,
-                            "pipeline_tag": m.get("pipeline_tag"),
-                            "likes": m.get("likes"),
-                            "downloads": m.get("downloads"),
-                            "tags": m.get("tags") or [],
-                            "source": "hub"
-                        }
-                    
-                    q_crypto = {"search": "crypto", "limit": 100}
-                    datasets = await self._hf_json(HF_API_DATASETS, q_crypto)
-                    for d in datasets or []:
-                        did = d.get("id") or d.get("name")
-                        if not did: continue
-                        category = "other"
-                        tags_str = " ".join(d.get("tags") or []).lower()
-                        name_lower = did.lower()
-                        if "price" in tags_str or "ohlc" in tags_str or "price" in name_lower:
-                            category = "price"
-                        elif "news" in tags_str or "news" in name_lower:
-                            if "label" in tags_str or "sentiment" in tags_str:
-                                category = "news_labeled"
-                            else:
-                                category = "news_raw"
-                        
-                        self.datasets[did] = {
-                            "id": did,
-                            "likes": d.get("likes"),
-                            "downloads": d.get("downloads"),
-                            "tags": d.get("tags") or [],
-                            "category": category,
-                            "source": "hub"
-                        }
-                except Exception as e:
-                    error_msg = str(e)[:200]
-                    if "401" in error_msg or "unauthorized" in error_msg.lower():
-                        self.fail_reason = "Authentication failed"
+            # Fetch from HF Hub
+            q_sent = {"pipeline_tag": "sentiment-analysis", "search": "crypto", "limit": 50}
+            models = await self._hf_json(HF_API_MODELS, q_sent)
+            for m in models or []:
+                mid = m.get("modelId") or m.get("id") or m.get("name")
+                if not mid: continue
+                self.models[mid] = {
+                    "id": mid,
+                    "pipeline_tag": m.get("pipeline_tag"),
+                    "likes": m.get("likes"),
+                    "downloads": m.get("downloads"),
+                    "tags": m.get("tags") or [],
+                    "source": "hub"
+                }
+            
+            q_crypto = {"search": "crypto", "limit": 100}
+            datasets = await self._hf_json(HF_API_DATASETS, q_crypto)
+            for d in datasets or []:
+                did = d.get("id") or d.get("name")
+                if not did: continue
+                # Infer category from tags or name
+                category = "other"
+                tags_str = " ".join(d.get("tags") or []).lower()
+                name_lower = did.lower()
+                if "price" in tags_str or "ohlc" in tags_str or "price" in name_lower:
+                    category = "price"
+                elif "news" in tags_str or "news" in name_lower:
+                    if "label" in tags_str or "sentiment" in tags_str:
+                        category = "news_labeled"
                     else:
-                        self.fail_reason = error_msg
+                        category = "news_raw"
+                
+                self.datasets[did] = {
+                    "id": did,
+                    "likes": d.get("likes"),
+                    "downloads": d.get("downloads"),
+                    "tags": d.get("tags") or [],
+                    "category": category,
+                    "source": "hub"
+                }
             
             self.last_refresh = time.time()
-            if self.fail_reason is None:
-                return {"ok": True, "models": len(self.models), "datasets": len(self.datasets)}
-            return {"ok": False, "error": self.fail_reason, "models": len(self.models), "datasets": len(self.datasets)}
+            self.fail_reason = None
+            return {"ok": True, "models": len(self.models), "datasets": len(self.datasets)}
         except Exception as e:
-            self.fail_reason = str(e)[:200]
+            self.fail_reason = str(e)
             return {"ok": False, "error": self.fail_reason, "models": len(self.models), "datasets": len(self.datasets)}
 
     def list(self, kind: Literal["models","datasets"]="models", category: Optional[str]=None) -> List[Dict[str, Any]]:

collectors/aggregator.py

@@ -88,8 +88,6 @@ class MarketDataCollector:
         self._symbol_map = {symbol.lower(): coin_id for coin_id, symbol in COIN_SYMBOL_MAPPING.items()}
         self.headers = {"User-Agent": settings.user_agent or USER_AGENT}
         self.timeout = 15.0
-        self._last_error_log: Dict[str, float] = {}  # Track last error log time per provider
-        self._error_log_throttle = 60.0  # Only log same error once per 60 seconds
 
     async def _request(self, provider_key: str, path: str, params: Optional[Dict[str, Any]] = None) -> Any:
         provider = self.registry.providers.get(provider_key)
@@ -97,54 +95,8 @@ class MarketDataCollector:
             raise CollectorError(f"Provider {provider_key} not configured", provider=provider_key)
 
         url = provider["base_url"].rstrip("/") + path
-        
-        # Rate limit tracking per provider
-        if not hasattr(self, '_rate_limit_timestamps'):
-            self._rate_limit_timestamps: Dict[str, List[float]] = {}
-        if provider_key not in self._rate_limit_timestamps:
-            self._rate_limit_timestamps[provider_key] = []
-        
-        # Get rate limits from provider config
-        rate_limit_rpm = provider.get("rate_limit", {}).get("requests_per_minute", 30)
-        if rate_limit_rpm and len(self._rate_limit_timestamps[provider_key]) >= rate_limit_rpm:
-            # Check if oldest request is older than 1 minute
-            oldest_time = self._rate_limit_timestamps[provider_key][0]
-            if time.time() - oldest_time < 60:
-                wait_time = 60 - (time.time() - oldest_time) + 1
-                if self._should_log_error(provider_key, "rate_limit_wait"):
-                    logger.warning(f"Rate limiting {provider_key}, waiting {wait_time:.1f}s")
-                await asyncio.sleep(wait_time)
-            # Clean old timestamps
-            cutoff = time.time() - 60
-            self._rate_limit_timestamps[provider_key] = [
-                ts for ts in self._rate_limit_timestamps[provider_key] if ts > cutoff
-            ]
-        
         async with httpx.AsyncClient(timeout=self.timeout, headers=self.headers) as client:
             response = await client.get(url, params=params)
-            
-        # Record request timestamp
-        self._rate_limit_timestamps[provider_key].append(time.time())
-        # Keep only last minute of timestamps
-        cutoff = time.time() - 60
-        self._rate_limit_timestamps[provider_key] = [
-            ts for ts in self._rate_limit_timestamps[provider_key] if ts > cutoff
-        ]
-        
-        # Handle HTTP 429 (Rate Limit) with exponential backoff
-        if response.status_code == 429:
-            retry_after = int(response.headers.get("Retry-After", "60"))
-            error_msg = f"{provider_key} rate limited (HTTP 429), retry after {retry_after}s"
-            
-            if self._should_log_error(provider_key, "HTTP 429"):
-                logger.warning(error_msg)
-            
-            raise CollectorError(
-                error_msg,
-                provider=provider_key,
-                status_code=429,
-            )
-        
         if response.status_code != 200:
             raise CollectorError(
                 f"{provider_key} request failed with HTTP {response.status_code}",
@@ -153,31 +105,14 @@ class MarketDataCollector:
             )
         return response.json()
 
-    def _should_log_error(self, provider: str, error_msg: str) -> bool:
-        """Check if error should be logged (throttle repeated errors)."""
-        error_key = f"{provider}:{error_msg}"
-        now = time.time()
-        last_log_time = self._last_error_log.get(error_key, 0)
-        
-        if now - last_log_time > self._error_log_throttle:
-            self._last_error_log[error_key] = now
-            # Clean up old entries (keep only last hour)
-            cutoff = now - 3600
-            self._last_error_log = {k: v for k, v in self._last_error_log.items() if v > cutoff}
-            return True
-        return False
-
     async def get_top_coins(self, limit: int = 10) -> List[Dict[str, Any]]:
         cache_key = f"top_coins:{limit}"
         cached = await self.cache.get(cache_key)
         if cached:
             return cached
 
-        # Provider list with priority order (add more fallbacks from resource files)
-        providers = ["coingecko", "coincap", "coinpaprika"]
+        providers = ["coingecko", "coincap"]
         last_error: Optional[Exception] = None
-        last_error_details: Optional[str] = None
-        
         for provider in providers:
             try:
                 if provider == "coingecko":
@@ -225,53 +160,11 @@ class MarketDataCollector:
                     ]
                     await self.cache.set(cache_key, coins)
                     return coins
-
-                if provider == "coinpaprika":
-                    data = await self._request("coinpaprika", "/tickers", {"quotes": "USD", "limit": limit})
-                    coins = [
-                        {
-                            "name": item.get("name"),
-                            "symbol": item.get("symbol", "").upper(),
-                            "price": float(item.get("quotes", {}).get("USD", {}).get("price", 0)),
-                            "change_24h": float(item.get("quotes", {}).get("USD", {}).get("percent_change_24h", 0)),
-                            "market_cap": float(item.get("quotes", {}).get("USD", {}).get("market_cap", 0)),
-                            "volume_24h": float(item.get("quotes", {}).get("USD", {}).get("volume_24h", 0)),
-                            "rank": int(item.get("rank", 0)),
-                            "last_updated": item.get("last_updated"),
-                        }
-                        for item in data[:limit] if item.get("quotes", {}).get("USD")
-                    ]
-                    await self.cache.set(cache_key, coins)
-                    return coins
             except Exception as exc:  # pragma: no cover - network heavy
                 last_error = exc
-                error_msg = str(exc) if str(exc) else repr(exc)
-                error_type = type(exc).__name__
-                
-                # Extract HTTP status code if available
-                if hasattr(exc, 'status_code'):
-                    status_code = exc.status_code
-                    error_msg = f"HTTP {status_code}: {error_msg}" if error_msg else f"HTTP {status_code}"
-                elif isinstance(exc, CollectorError) and hasattr(exc, 'status_code') and exc.status_code:
-                    status_code = exc.status_code
-                    error_msg = f"HTTP {status_code}: {error_msg}" if error_msg else f"HTTP {status_code}"
-                
-                # Ensure we always have a meaningful error message
-                if not error_msg or error_msg.strip() == "":
-                    error_msg = f"{error_type} (no details available)"
-                
-                last_error_details = f"{error_type}: {error_msg}"
-                
-                # Throttle error logging to prevent spam
-                error_key_for_logging = error_msg or error_type
-                if self._should_log_error(provider, error_key_for_logging):
-                    logger.warning(
-                        "Provider %s failed: %s (error logged, will suppress similar errors for 60s)",
-                        provider,
-                        last_error_details
-                    )
+                logger.warning("Provider %s failed: %s", provider, exc)
 
-        raise CollectorError(f"Unable to fetch top coins from any provider. Last error: {last_error_details or 'Unknown'}", provider=str(last_error) if last_error else None)
+        raise CollectorError("Unable to fetch top coins", provider=str(last_error))
 
     async def _coin_id(self, symbol: str) -> str:
         symbol_lower = symbol.lower()
@@ -472,9 +365,7 @@ class ProviderStatusCollector:
                 "latency_ms": latency,
             }
         except Exception as exc:  # pragma: no cover - network heavy
-            error_msg = str(exc)
-            error_type = type(exc).__name__
-            logger.warning("Provider %s health check failed: %s: %s", provider_id, error_type, error_msg)
+            logger.warning("Provider %s health check failed: %s", provider_id, exc)
             return {
                 "provider_id": provider_id,
                 "name": data.get("name", provider_id),

config.py

@@ -1,470 +1,24 @@
-#!/usr/bin/env python3
-"""
-Configuration constants for Crypto Data Aggregator
-All configuration in one place - no hardcoded values
-"""
-
-import os
-import json
-import base64
-import logging
-from functools import lru_cache
-from pathlib import Path
-from typing import Dict, Any, List, Optional
-from dataclasses import dataclass
-
-# Load .env file if python-dotenv is available
-try:
-    from dotenv import load_dotenv
-    load_dotenv()
-except ImportError:
-    pass  # python-dotenv not installed, skip loading .env
-
-# ==================== DIRECTORIES ====================
-BASE_DIR = Path(__file__).parent
-DATA_DIR = BASE_DIR / "data"
-LOG_DIR = BASE_DIR / "logs"
-DB_DIR = DATA_DIR / "database"
-
-# Create directories if they don't exist
-for directory in [DATA_DIR, LOG_DIR, DB_DIR]:
-    directory.mkdir(parents=True, exist_ok=True)
-
-logger = logging.getLogger(__name__)
-
-
-# ==================== PROVIDER CONFIGURATION ====================
-
-
-@dataclass
-class ProviderConfig:
-    """Configuration for an API provider"""
-
-    name: str
-    endpoint_url: str
-    category: str = "market_data"
-    requires_key: bool = False
-    api_key: Optional[str] = None
-    timeout_ms: int = 10000
-    rate_limit_type: Optional[str] = None
-    rate_limit_value: Optional[int] = None
-    health_check_endpoint: Optional[str] = None
-
-    def __post_init__(self):
-        if self.health_check_endpoint is None:
-            self.health_check_endpoint = self.endpoint_url
-
-
-@dataclass
-class Settings:
-    """Runtime configuration loaded from environment variables."""
-
-    hf_token: Optional[str] = None
-    hf_token_encoded: Optional[str] = None
-    cmc_api_key: Optional[str] = None
-    etherscan_key: Optional[str] = None
-    newsapi_key: Optional[str] = None
-    log_level: str = "INFO"
-    database_path: Path = DB_DIR / "crypto_aggregator.db"
-    redis_url: Optional[str] = None
-    cache_ttl: int = 300
-    user_agent: str = "CryptoDashboard/1.0"
-    providers_config_path: Path = BASE_DIR / "providers_config_extended.json"
-
-
-def _decode_token(value: Optional[str]) -> Optional[str]:
-    """Decode a base64 encoded Hugging Face token."""
-
-    if not value:
-        return None
-
-    try:
-        decoded = base64.b64decode(value).decode("utf-8").strip()
-        return decoded or None
-    except Exception as exc:  # pragma: no cover - defensive logging
-        logger.warning("Failed to decode HF token: %s", exc)
-        return None
-
-
-@lru_cache(maxsize=1)
-def get_settings() -> Settings:
-    """Return cached runtime settings."""
-
-    raw_token = os.environ.get("HF_TOKEN")
-    encoded_token = os.environ.get("HF_TOKEN_ENCODED")
-    decoded_token = raw_token or _decode_token(encoded_token)
-    # Default token if none provided
-    if not decoded_token:
-        decoded_token = "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"
-
-    database_path = Path(os.environ.get("DATABASE_PATH", str(DB_DIR / "crypto_aggregator.db")))
-
-    settings = Settings(
-        hf_token=decoded_token,
-        hf_token_encoded=encoded_token,
-        cmc_api_key=os.environ.get("CMC_API_KEY"),
-        etherscan_key=os.environ.get("ETHERSCAN_KEY"),
-        newsapi_key=os.environ.get("NEWSAPI_KEY"),
-        log_level=os.environ.get("LOG_LEVEL", "INFO").upper(),
-        database_path=database_path,
-        redis_url=os.environ.get("REDIS_URL"),
-        cache_ttl=int(os.environ.get("CACHE_TTL", "300")),
-        user_agent=os.environ.get("USER_AGENT", "CryptoDashboard/1.0"),
-        providers_config_path=Path(
-            os.environ.get("PROVIDERS_CONFIG_PATH", str(BASE_DIR / "providers_config_extended.json"))
-        ),
-    )
-
-    return settings
-
-
-class ConfigManager:
-    """Configuration manager for API providers"""
-    
-    def __init__(self):
-        self.providers: Dict[str, ProviderConfig] = {}
-        self._load_default_providers()
-        self._load_env_keys()
-    
-    def _load_default_providers(self):
-        """Load default provider configurations"""
-        # CoinGecko (Free, no key)
-        self.providers["CoinGecko"] = ProviderConfig(
-            name="CoinGecko",
-            endpoint_url="https://api.coingecko.com/api/v3",
-            category="market_data",
-            requires_key=False,
-            timeout_ms=10000
-        )
-        
-        # CoinMarketCap (Requires API key)
-        self.providers["CoinMarketCap"] = ProviderConfig(
-            name="CoinMarketCap",
-            endpoint_url="https://pro-api.coinmarketcap.com/v1",
-            category="market_data",
-            requires_key=True,
-            timeout_ms=10000
-        )
-        
-        # Binance (Free, no key)
-        self.providers["Binance"] = ProviderConfig(
-            name="Binance",
-            endpoint_url="https://api.binance.com/api/v3",
-            category="market_data",
-            requires_key=False,
-            timeout_ms=10000
-        )
-        
-        # Etherscan (Requires API key)
-        self.providers["Etherscan"] = ProviderConfig(
-            name="Etherscan",
-            endpoint_url="https://api.etherscan.io/api",
-            category="blockchain_explorers",
-            requires_key=True,
-            timeout_ms=10000
-        )
-        
-        # BscScan (Requires API key)
-        self.providers["BscScan"] = ProviderConfig(
-            name="BscScan",
-            endpoint_url="https://api.bscscan.com/api",
-            category="blockchain_explorers",
-            requires_key=True,
-            timeout_ms=10000
-        )
-        
-        # TronScan (Requires API key)
-        self.providers["TronScan"] = ProviderConfig(
-            name="TronScan",
-            endpoint_url="https://apilist.tronscan.org/api",
-            category="blockchain_explorers",
-            requires_key=True,
-            timeout_ms=10000
-        )
-        
-        # CryptoPanic (Requires API key)
-        self.providers["CryptoPanic"] = ProviderConfig(
-            name="CryptoPanic",
-            endpoint_url="https://cryptopanic.com/api/v1",
-            category="news",
-            requires_key=True,
-            timeout_ms=10000
-        )
-        
-        # NewsAPI (Requires API key)
-        self.providers["NewsAPI"] = ProviderConfig(
-            name="NewsAPI",
-            endpoint_url="https://newsapi.org/v2",
-            category="news",
-            requires_key=True,
-            timeout_ms=10000
-        )
-        
-        # Alternative.me Fear & Greed Index (Free, no key)
-        self.providers["Alternative.me"] = ProviderConfig(
-            name="Alternative.me",
-            endpoint_url="https://api.alternative.me",
-            category="sentiment",
-            requires_key=False,
-            timeout_ms=10000
-        )
-    
-    def _load_env_keys(self):
-        """Load API keys from environment variables"""
-        key_mapping = {
-            "CoinMarketCap": "CMC_API_KEY",
-            "Etherscan": "ETHERSCAN_KEY",
-            "BscScan": "BSCSCAN_KEY",
-            "TronScan": "TRONSCAN_KEY",
-            "CryptoPanic": "CRYPTOPANIC_KEY",
-            "NewsAPI": "NEWSAPI_KEY",
-        }
-        
-        for provider_name, env_var in key_mapping.items():
-            if provider_name in self.providers:
-                api_key = os.environ.get(env_var)
-                if api_key:
-                    self.providers[provider_name].api_key = api_key
-    
-    def get_provider(self, provider_name: str) -> Optional[ProviderConfig]:
-        """Get provider configuration by name"""
-        return self.providers.get(provider_name)
-    
-    def get_all_providers(self) -> List[ProviderConfig]:
-        """Get all provider configurations"""
-        return list(self.providers.values())
-    
-    def get_providers_by_category(self, category: str) -> List[ProviderConfig]:
-        """Get providers filtered by category"""
-        return [p for p in self.providers.values() if p.category == category]
-    
-    def get_categories(self) -> List[str]:
-        """Get all unique categories"""
-        return list(set(p.category for p in self.providers.values()))
-    
-    def add_provider(self, provider: ProviderConfig):
-        """Add a new provider configuration"""
-        self.providers[provider.name] = provider
-    
-    def stats(self) -> Dict[str, Any]:
-        """Get configuration statistics"""
-        providers_list = list(self.providers.values())
-        return {
-            'total_resources': len(providers_list),
-            'total_categories': len(self.get_categories()),
-            'free_resources': sum(1 for p in providers_list if not p.requires_key),
-            'tier1_count': 0,  # Placeholder for tier support
-            'tier2_count': 0,
-            'tier3_count': len(providers_list),
-            'api_keys_count': sum(1 for p in providers_list if p.api_key),
-            'cors_proxies_count': 0,
-            'categories': self.get_categories()
-        }
-    
-    def get_by_tier(self, tier: int) -> List[Dict[str, Any]]:
-        """Get resources by tier (placeholder for compatibility)"""
-        # Return all providers for now
-        return [{'name': p.name} for p in self.providers.values()]
-    
-    def get_all_resources(self) -> List[Dict[str, Any]]:
-        """Get all resources in dictionary format (for compatibility)"""
-        return [
-            {
-                'name': p.name,
-                'endpoint': p.endpoint_url,
-                'url': p.endpoint_url,
-                'category': p.category,
-                'requires_key': p.requires_key,
-                'api_key': p.api_key,
-                'timeout': p.timeout_ms,
-            }
-            for p in self.providers.values()
-        ]
-
-
-# Create global config instance
-config = ConfigManager()
-
-# Runtime settings loaded from environment
-settings = get_settings()
-
-# ==================== DATABASE ====================
-DATABASE_PATH = Path(settings.database_path)
-DATABASE_BACKUP_DIR = DATA_DIR / "backups"
-DATABASE_BACKUP_DIR.mkdir(parents=True, exist_ok=True)
-
-# ==================== API ENDPOINTS (NO KEYS REQUIRED) ====================
-
-# CoinGecko API (Free, no key)
-COINGECKO_BASE_URL = "https://api.coingecko.com/api/v3"
-COINGECKO_ENDPOINTS = {
-    "ping": "/ping",
-    "price": "/simple/price",
-    "coins_list": "/coins/list",
-    "coins_markets": "/coins/markets",
-    "coin_data": "/coins/{id}",
-    "trending": "/search/trending",
-    "global": "/global",
-}
-
-# CoinCap API (Free, no key)
-COINCAP_BASE_URL = "https://api.coincap.io/v2"
-COINCAP_ENDPOINTS = {
-    "assets": "/assets",
-    "asset_detail": "/assets/{id}",
-    "asset_history": "/assets/{id}/history",
-    "markets": "/markets",
-    "rates": "/rates",
-}
-
-# Binance Public API (Free, no key)
-BINANCE_BASE_URL = "https://api.binance.com/api/v3"
-BINANCE_ENDPOINTS = {
-    "ping": "/ping",
-    "ticker_24h": "/ticker/24hr",
-    "ticker_price": "/ticker/price",
-    "klines": "/klines",
-    "trades": "/trades",
-}
-
-# Alternative.me Fear & Greed Index (Free, no key)
-ALTERNATIVE_ME_URL = "https://api.alternative.me/fng/"
-
-# ==================== RSS FEEDS ====================
-RSS_FEEDS = {
-    "coindesk": "https://www.coindesk.com/arc/outboundfeeds/rss/",
-    "cointelegraph": "https://cointelegraph.com/rss",
-    "bitcoin_magazine": "https://bitcoinmagazine.com/.rss/full/",
-    "decrypt": "https://decrypt.co/feed",
-    "bitcoinist": "https://bitcoinist.com/feed/",
-}
-
-# ==================== REDDIT ENDPOINTS (NO AUTH) ====================
-REDDIT_ENDPOINTS = {
-    "cryptocurrency": "https://www.reddit.com/r/cryptocurrency/.json",
-    "bitcoin": "https://www.reddit.com/r/bitcoin/.json",
-    "ethtrader": "https://www.reddit.com/r/ethtrader/.json",
-    "cryptomarkets": "https://www.reddit.com/r/CryptoMarkets/.json",
-}
-
-# ==================== HUGGING FACE MODELS ====================
-HUGGINGFACE_MODELS = {
-    "sentiment_twitter": "cardiffnlp/twitter-roberta-base-sentiment-latest",
-    "sentiment_financial": "ProsusAI/finbert",
-    "summarization": "facebook/bart-large-cnn",
-    "crypto_sentiment": "ElKulako/CryptoBERT",  # Requires authentication
-}
-
-# Hugging Face Authentication
-HF_TOKEN = settings.hf_token or ""
-HF_USE_AUTH_TOKEN = bool(HF_TOKEN)
-
-# ==================== DATA COLLECTION SETTINGS ====================
-COLLECTION_INTERVALS = {
-    "price_data": 300,  # 5 minutes in seconds
-    "news_data": 1800,  # 30 minutes in seconds
-    "sentiment_data": 1800,  # 30 minutes in seconds
-}
-
-# Number of top cryptocurrencies to track
-TOP_COINS_LIMIT = 100
-
-# Request timeout in seconds
-REQUEST_TIMEOUT = 10
-
-# Max retries for failed requests
-MAX_RETRIES = 3
-
-# ==================== CACHE SETTINGS ====================
-CACHE_TTL = settings.cache_ttl or 300  # 5 minutes in seconds
-CACHE_MAX_SIZE = 1000  # Maximum number of cached items
-
-# ==================== LOGGING SETTINGS ====================
-LOG_FILE = LOG_DIR / "crypto_aggregator.log"
-LOG_LEVEL = settings.log_level
-LOG_FORMAT = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
-LOG_MAX_BYTES = 10 * 1024 * 1024  # 10 MB
-LOG_BACKUP_COUNT = 5
-
-# ==================== GRADIO SETTINGS ====================
-GRADIO_SHARE = False
-GRADIO_SERVER_NAME = "0.0.0.0"
-GRADIO_SERVER_PORT = 7860
-GRADIO_THEME = "default"
-AUTO_REFRESH_INTERVAL = 30  # seconds
-
-# ==================== DATA VALIDATION ====================
-MIN_PRICE = 0.0
-MAX_PRICE = 1000000000.0  # 1 billion
-MIN_VOLUME = 0.0
-MIN_MARKET_CAP = 0.0
-
-# ==================== CHART SETTINGS ====================
-CHART_TIMEFRAMES = {
-    "1d": {"days": 1, "interval": "1h"},
-    "7d": {"days": 7, "interval": "4h"},
-    "30d": {"days": 30, "interval": "1d"},
-    "90d": {"days": 90, "interval": "1d"},
-    "1y": {"days": 365, "interval": "1w"},
-}
-
-# Technical indicators
-MA_PERIODS = [7, 30]  # Moving Average periods
-RSI_PERIOD = 14  # RSI period
-
-# ==================== SENTIMENT THRESHOLDS ====================
-SENTIMENT_LABELS = {
-    "very_negative": (-1.0, -0.6),
-    "negative": (-0.6, -0.2),
-    "neutral": (-0.2, 0.2),
-    "positive": (0.2, 0.6),
-    "very_positive": (0.6, 1.0),
-}
-
-# ==================== AI ANALYSIS SETTINGS ====================
-AI_CONFIDENCE_THRESHOLD = 0.6
-PREDICTION_HORIZON_HOURS = 72
-
-# ==================== USER AGENT ====================
-USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
-
-# ==================== RATE LIMITING ====================
-RATE_LIMIT_CALLS = 50
-RATE_LIMIT_PERIOD = 60  # seconds
-
-# ==================== COIN SYMBOLS ====================
-# Top cryptocurrencies to focus on
-FOCUS_COINS = [
-    "bitcoin", "ethereum", "binancecoin", "ripple", "cardano",
-    "solana", "polkadot", "dogecoin", "avalanche-2", "polygon",
-    "chainlink", "uniswap", "litecoin", "cosmos", "algorand"
-]
-
-COIN_SYMBOL_MAPPING = {
-    "bitcoin": "BTC",
-    "ethereum": "ETH",
-    "binancecoin": "BNB",
-    "ripple": "XRP",
-    "cardano": "ADA",
-    "solana": "SOL",
-    "polkadot": "DOT",
-    "dogecoin": "DOGE",
-    "avalanche-2": "AVAX",
-    "polygon": "MATIC",
-}
-
-# ==================== ERROR MESSAGES ====================
-ERROR_MESSAGES = {
-    "api_unavailable": "API service is currently unavailable. Using cached data.",
-    "no_data": "No data available at the moment.",
-    "database_error": "Database operation failed.",
-    "network_error": "Network connection error.",
-    "invalid_input": "Invalid input provided.",
-}
-
-# ==================== SUCCESS MESSAGES ====================
-SUCCESS_MESSAGES = {
-    "data_collected": "Data successfully collected and saved.",
-    "cache_cleared": "Cache cleared successfully.",
-    "database_initialized": "Database initialized successfully.",
-}
+#!/usr/bin/env python3
+"""Configuration module for Hugging Face models."""
+
+import os
+from typing import Optional, Dict, Any
+
+HUGGINGFACE_MODELS: Dict[str, str] = {
+    "sentiment_twitter": "cardiffnlp/twitter-roberta-base-sentiment-latest",
+    "sentiment_financial": "ProsusAI/finbert",
+    "summarization": "facebook/bart-large-cnn",
+    "crypto_sentiment": "ElKulako/cryptobert",
+}
+
+class Settings:
+    """Application settings."""
+    def __init__(self):
+        self.hf_token: Optional[str] = os.getenv("HF_TOKEN") or os.getenv("HUGGINGFACE_TOKEN")
+
+_settings = Settings()
+
+def get_settings() -> Settings:
+    """Get application settings instance."""
+    return _settings
+

dashboard.html

@@ -1,113 +1,638 @@
-<!DOCTYPE html>
+<!DOCTYPE html>
 <html lang="en">
 <head>
-    <meta charset="UTF-8" />
-    <meta name="viewport" content="width=device-width, initial-scale=1" />
-    <title>Crypto Intelligence Dashboard</title>
-    <link rel="preconnect" href="https://fonts.googleapis.com" />
-    <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
-    <link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet" />
-    <link rel="stylesheet" href="/static/css/unified-ui.css" />
-    <link rel="stylesheet" href="/static/css/components.css" />
-    <script defer src="/static/js/ui-feedback.js"></script>
-    <script defer src="/static/js/dashboard-app.js"></script>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Crypto API Monitor - Real Data Dashboard</title>
+    <style>
+        * { margin: 0; padding: 0; box-sizing: border-box; }
+        
+        @keyframes gradientShift {
+            0% { background-position: 0% 50%; }
+            50% { background-position: 100% 50%; }
+            100% { background-position: 0% 50%; }
+        }
+        
+        @keyframes fadeInUp {
+            from {
+                opacity: 0;
+                transform: translateY(30px);
+            }
+            to {
+                opacity: 1;
+                transform: translateY(0);
+            }
+        }
+        
+        @keyframes pulse {
+            0%, 100% { transform: scale(1); }
+            50% { transform: scale(1.05); }
+        }
+        
+        @keyframes shimmer {
+            0% { background-position: -1000px 0; }
+            100% { background-position: 1000px 0; }
+        }
+        
+        body {
+            font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
+            background: linear-gradient(-45deg, #667eea, #764ba2, #f093fb, #4facfe);
+            background-size: 400% 400%;
+            animation: gradientShift 15s ease infinite;
+            padding: 20px;
+            color: #1a1a1a;
+            min-height: 100vh;
+        }
+        
+        .container {
+            max-width: 1400px;
+            margin: 0 auto;
+            background: rgba(255, 255, 255, 0.95);
+            backdrop-filter: blur(10px);
+            border-radius: 24px;
+            padding: 40px;
+            box-shadow: 0 20px 60px rgba(0,0,0,0.3);
+            animation: fadeInUp 0.6s ease;
+        }
+        
+        h1 {
+            background: linear-gradient(135deg, #667eea 0%, #764ba2 50%, #f093fb 100%);
+            -webkit-background-clip: text;
+            -webkit-text-fill-color: transparent;
+            background-clip: text;
+            margin-bottom: 10px;
+            font-size: 42px;
+            font-weight: 900;
+            letter-spacing: -1px;
+            animation: shimmer 3s infinite linear;
+            background-size: 1000px 100%;
+        }
+        
+        .subtitle {
+            color: #6c757d;
+            font-size: 16px;
+            margin-bottom: 20px;
+        }
+        
+        .stats-grid {
+            display: grid;
+            grid-template-columns: repeat(auto-fit, minmax(220px, 1fr));
+            gap: 24px;
+            margin: 30px 0;
+        }
+        
+        .stat-card {
+            background: linear-gradient(135deg, #ffffff 0%, #f8f9fa 100%);
+            padding: 28px;
+            border-radius: 20px;
+            border: 3px solid transparent;
+            background-clip: padding-box;
+            position: relative;
+            overflow: hidden;
+            transition: all 0.3s cubic-bezier(0.4, 0, 0.2, 1);
+            box-shadow: 0 4px 20px rgba(0,0,0,0.08);
+        }
+        
+        .stat-card::before {
+            content: '';
+            position: absolute;
+            top: 0;
+            left: 0;
+            right: 0;
+            bottom: 0;
+            border-radius: 20px;
+            padding: 3px;
+            background: linear-gradient(135deg, #667eea, #764ba2, #f093fb);
+            -webkit-mask: linear-gradient(#fff 0 0) content-box, linear-gradient(#fff 0 0);
+            -webkit-mask-composite: xor;
+            mask-composite: exclude;
+            opacity: 0;
+            transition: opacity 0.3s;
+        }
+        
+        .stat-card:hover {
+            transform: translateY(-8px) scale(1.02);
+            box-shadow: 0 12px 40px rgba(102, 126, 234, 0.3);
+        }
+        
+        .stat-card:hover::before {
+            opacity: 1;
+        }
+        
+        .stat-icon {
+            font-size: 32px;
+            margin-bottom: 12px;
+            display: inline-block;
+            animation: pulse 2s infinite;
+        }
+        
+        .stat-value {
+            font-size: 48px;
+            font-weight: 900;
+            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
+            -webkit-background-clip: text;
+            -webkit-text-fill-color: transparent;
+            background-clip: text;
+            margin: 12px 0;
+            line-height: 1;
+        }
+        
+        .stat-value.green {
+            background: linear-gradient(135deg, #10b981 0%, #059669 100%);
+            -webkit-background-clip: text;
+            -webkit-text-fill-color: transparent;
+        }
+        
+        .stat-value.red {
+            background: linear-gradient(135deg, #ef4444 0%, #dc2626 100%);
+            -webkit-background-clip: text;
+            -webkit-text-fill-color: transparent;
+        }
+        
+        .stat-value.orange {
+            background: linear-gradient(135deg, #f59e0b 0%, #d97706 100%);
+            -webkit-background-clip: text;
+            -webkit-text-fill-color: transparent;
+        }
+        
+        .stat-label {
+            font-size: 13px;
+            color: #6c757d;
+            text-transform: uppercase;
+            font-weight: 700;
+            letter-spacing: 1px;
+        }
+        
+        .section-header {
+            display: flex;
+            align-items: center;
+            gap: 12px;
+            margin: 40px 0 20px 0;
+            padding-bottom: 16px;
+            border-bottom: 3px solid;
+            border-image: linear-gradient(90deg, #667eea, #764ba2, transparent) 1;
+        }
+        
+        .section-header h2 {
+            font-size: 28px;
+            font-weight: 800;
+            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
+            -webkit-background-clip: text;
+            -webkit-text-fill-color: transparent;
+        }
+        
+        .providers-table {
+            width: 100%;
+            border-collapse: separate;
+            border-spacing: 0;
+            margin: 20px 0;
+            border-radius: 16px;
+            overflow: hidden;
+            box-shadow: 0 4px 20px rgba(0,0,0,0.08);
+        }
+        
+        .providers-table th {
+            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
+            color: white;
+            padding: 18px;
+            text-align: left;
+            font-weight: 700;
+            text-transform: uppercase;
+            font-size: 12px;
+            letter-spacing: 1px;
+        }
+        
+        .providers-table td {
+            padding: 18px;
+            border-bottom: 1px solid #e9ecef;
+            background: white;
+            transition: all 0.2s;
+        }
+        
+        .providers-table tr:hover td {
+            background: linear-gradient(90deg, #f8f9fa 0%, #ffffff 100%);
+            transform: scale(1.01);
+        }
+        
+        .providers-table tr:last-child td {
+            border-bottom: none;
+        }
+        
+        .status-badge {
+            display: inline-flex;
+            align-items: center;
+            gap: 6px;
+            padding: 6px 14px;
+            border-radius: 20px;
+            font-size: 12px;
+            font-weight: 700;
+            text-transform: uppercase;
+            letter-spacing: 0.5px;
+            box-shadow: 0 2px 8px rgba(0,0,0,0.1);
+        }
+        
+        .status-badge::before {
+            content: '';
+            width: 8px;
+            height: 8px;
+            border-radius: 50%;
+            animation: pulse 2s infinite;
+        }
+        
+        .status-online {
+            background: linear-gradient(135deg, #d1fae5 0%, #a7f3d0 100%);
+            color: #065f46;
+            border: 2px solid #10b981;
+        }
+        
+        .status-online::before {
+            background: #10b981;
+            box-shadow: 0 0 10px #10b981;
+        }
+        
+        .status-offline {
+            background: linear-gradient(135deg, #fee2e2 0%, #fecaca 100%);
+            color: #991b1b;
+            border: 2px solid #ef4444;
+        }
+        
+        .status-offline::before {
+            background: #ef4444;
+            box-shadow: 0 0 10px #ef4444;
+        }
+        
+        .status-degraded {
+            background: linear-gradient(135deg, #fef3c7 0%, #fde68a 100%);
+            color: #92400e;
+            border: 2px solid #f59e0b;
+        }
+        
+        .status-degraded::before {
+            background: #f59e0b;
+            box-shadow: 0 0 10px #f59e0b;
+        }
+        
+        .refresh-btn {
+            padding: 14px 28px;
+            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
+            color: white;
+            border: none;
+            border-radius: 12px;
+            font-weight: 700;
+            cursor: pointer;
+            margin: 10px 5px;
+            font-size: 14px;
+            text-transform: uppercase;
+            letter-spacing: 0.5px;
+            transition: all 0.3s cubic-bezier(0.4, 0, 0.2, 1);
+            box-shadow: 0 4px 15px rgba(102, 126, 234, 0.4);
+            position: relative;
+            overflow: hidden;
+        }
+        
+        .refresh-btn::before {
+            content: '';
+            position: absolute;
+            top: 50%;
+            left: 50%;
+            width: 0;
+            height: 0;
+            border-radius: 50%;
+            background: rgba(255,255,255,0.3);
+            transform: translate(-50%, -50%);
+            transition: width 0.6s, height 0.6s;
+        }
+        
+        .refresh-btn:hover::before {
+            width: 300px;
+            height: 300px;
+        }
+        
+        .refresh-btn:hover {
+            transform: translateY(-3px);
+            box-shadow: 0 8px 25px rgba(102, 126, 234, 0.5);
+        }
+        
+        .refresh-btn:active {
+            transform: translateY(-1px);
+        }
+        
+        .last-update {
+            display: inline-flex;
+            align-items: center;
+            gap: 8px;
+            color: #6c757d;
+            font-size: 14px;
+            margin: 10px 0;
+            padding: 8px 16px;
+            background: #f8f9fa;
+            border-radius: 20px;
+            font-weight: 600;
+        }
+        
+        .hf-section {
+            margin-top: 40px;
+            padding: 32px;
+            background: linear-gradient(135deg, #f8f9fa 0%, #e9ecef 100%);
+            border-radius: 20px;
+            border: 3px solid #dee2e6;
+            box-shadow: 0 4px 20px rgba(0,0,0,0.08);
+        }
+        
+        textarea {
+            width: 100%;
+            padding: 16px;
+            border: 3px solid #dee2e6;
+            border-radius: 12px;
+            font-family: 'Consolas', 'Monaco', monospace;
+            margin: 16px 0;
+            font-size: 14px;
+            transition: all 0.3s;
+            background: white;
+        }
+        
+        textarea:focus {
+            outline: none;
+            border-color: #667eea;
+            box-shadow: 0 0 0 4px rgba(102, 126, 234, 0.1);
+        }
+        
+        .sentiment-result {
+            font-size: 56px;
+            font-weight: 900;
+            padding: 32px;
+            background: white;
+            border-radius: 16px;
+            text-align: center;
+            margin: 16px 0;
+            box-shadow: 0 4px 20px rgba(0,0,0,0.08);
+            border: 3px solid #dee2e6;
+            min-height: 120px;
+            display: flex;
+            align-items: center;
+            justify-content: center;
+        }
+        
+        pre {
+            background: #1e293b !important;
+            color: #e2e8f0 !important;
+            padding: 20px !important;
+            border-radius: 12px !important;
+            overflow-x: auto !important;
+            font-size: 13px !important;
+            line-height: 1.6 !important;
+            box-shadow: inset 0 2px 8px rgba(0,0,0,0.3) !important;
+        }
+        
+        .loading {
+            display: inline-block;
+            width: 20px;
+            height: 20px;
+            border: 3px solid #f3f4f6;
+            border-top-color: #667eea;
+            border-radius: 50%;
+            animation: spin 0.8s linear infinite;
+        }
+        
+        @keyframes spin {
+            to { transform: rotate(360deg); }
+        }
+        
+        .response-time {
+            font-weight: 700;
+            padding: 4px 10px;
+            border-radius: 8px;
+            font-size: 13px;
+        }
+        
+        .response-fast {
+            background: #d1fae5;
+            color: #065f46;
+        }
+        
+        .response-medium {
+            background: #fef3c7;
+            color: #92400e;
+        }
+        
+        .response-slow {
+            background: #fee2e2;
+            color: #991b1b;
+        }
+    </style>
 </head>
-<body class="page page-dashboard">
-<header class="top-nav">
-    <div class="branding">
-        <svg width="32" height="32" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5"><circle cx="12" cy="12" r="10"/></svg>
-        <div>
-            <strong>Crypto Intelligence Hub</strong>
-            <small style="color:var(--ui-text-muted);letter-spacing:0.2em;">Real-time data + HF models</small>
+<body>
+    <div class="container">
+        <h1>🚀 Crypto API Monitor</h1>
+        <p class="subtitle">Real-time monitoring of cryptocurrency APIs with live data</p>
+        <p class="last-update">⏱️ Last Update: <span id="lastUpdate">Loading...</span></p>
+        
+        <div style="margin: 20px 0;">
+            <button class="refresh-btn" onclick="loadData()">🔄 Refresh Data</button>
+            <button class="refresh-btn" onclick="window.location.href='/hf_console.html'">🤗 HF Console</button>
+            <button class="refresh-btn" onclick="window.location.href='/admin.html'">⚙️ Admin Panel</button>
+            <button class="refresh-btn" onclick="window.location.href='/index.html'">📊 Full Dashboard</button>
         </div>
-    </div>
-    <nav class="nav-links">
-        <a class="active" href="/dashboard">Dashboard</a>
-        <a href="/admin">Admin</a>
-        <a href="/hf_console">HF Console</a>
-        <a href="/docs" target="_blank" rel="noreferrer">API Docs</a>
-    </nav>
-</header>
-
-<main class="page-content">
-    <section class="card" id="intro-card">
-        <div class="section-heading">
-            <h2>Unified Market Pulse</h2>
-            <span class="badge info" id="intro-source">Loading...</span>
-        </div>
-        <p style="color:var(--text-muted);max-width:780px;line-height:1.6;">
-            Live collectors + local fallback registry guarantee resilient insights. All numbers below already honor the FastAPI routes
-            (<code>/api/crypto/prices/top</code>, <code>/api/crypto/market-overview</code>, <code>/health</code>) so you can monitor status even when providers degrade.
-        </p>
-    </section>
-
-    <section class="card-grid" id="market-metrics">
-        <article class="card"><h3>Total Market Cap</h3><div class="metric-value" id="metric-market-cap">-</div><div class="metric-subtext" id="metric-cap-source"></div></article>
-        <article class="card"><h3>24h Volume</h3><div class="metric-value" id="metric-volume">-</div><div class="metric-subtext" id="metric-volume-source"></div></article>
-        <article class="card"><h3>BTC Dominance</h3><div class="metric-value" id="metric-btc-dom">-</div><div class="metric-subtext">Based on /api/crypto/market-overview</div></article>
-        <article class="card"><h3>System Health</h3><div class="metric-value" id="metric-health">-</div><div class="metric-subtext" id="metric-health-details"></div></article>
-    </section>
-
-    <section class="card table-card">
-        <div class="section-heading">
-            <h2>Top Assets</h2>
-            <span class="badge info" id="top-prices-source">Loading...</span>
-        </div>
-        <div class="table-wrapper">
-            <table>
-                <thead>
-                <tr><th>Symbol</th><th>Price</th><th>24h %</th><th>Volume</th></tr>
-                </thead>
-                <tbody id="top-prices-table">
-                <tr><td colspan="4">Loading...</td></tr>
-                </tbody>
-            </table>
-        </div>
-    </section>
-
-    <section class="split-grid">
-        <article class="card" id="overview-card">
-            <div class="section-heading">
-                <h2>Market Overview</h2>
-                <span class="badge info" id="market-overview-source">Loading...</span>
+        
+        <div class="stats-grid">
+            <div class="stat-card">
+                <div class="stat-icon">📡</div>
+                <div class="stat-label">Total APIs</div>
+                <div class="stat-value" id="totalAPIs">0</div>
             </div>
-            <ul class="list" id="market-overview-list"></ul>
-        </article>
-        <article class="card" id="system-card">
-            <div class="section-heading">
-                <h2>System & Rate Limits</h2>
-                <span class="badge info" id="system-status-source">/health</span>
+            <div class="stat-card">
+                <div class="stat-icon">✅</div>
+                <div class="stat-label">Online</div>
+                <div class="stat-value green" id="onlineAPIs">0</div>
             </div>
-            <div id="system-health-status" class="metric-subtext"></div>
-            <ul class="list" id="system-status-list"></ul>
-            <div class="section-heading" style="margin-top:24px;">
-                <h2>Configuration</h2>
+            <div class="stat-card">
+                <div class="stat-icon">❌</div>
+                <div class="stat-label">Offline</div>
+                <div class="stat-value red" id="offlineAPIs">0</div>
             </div>
-            <ul class="list" id="system-config-list"></ul>
-            <div class="section-heading" style="margin-top:24px;">
-                <h2>Rate Limits</h2>
+            <div class="stat-card">
+                <div class="stat-icon">⚡</div>
+                <div class="stat-label">Avg Response</div>
+                <div class="stat-value orange" id="avgResponse" style="font-size: 32px;">0ms</div>
             </div>
-            <ul class="list" id="rate-limits-list"></ul>
-        </article>
-    </section>
+        </div>
 
-    <section class="split-grid">
-        <article class="card" id="hf-widget">
-            <div class="section-heading">
-                <h2>HuggingFace Snapshot</h2>
-                <span class="badge info" id="hf-health-status">Loading...</span>
-            </div>
-            <div id="hf-widget-summary" class="metric-subtext"></div>
-            <ul class="list" id="hf-registry-list"></ul>
-        </article>
-        <article class="card">
-            <div class="section-heading">
-                <h2>Live Stream (/ws)</h2>
-                <span class="badge info" id="ws-status">Connecting...</span>
+        <div class="section-header">
+            <h2>📊 API Providers Status</h2>
+        </div>
+        <table class="providers-table">
+            <thead>
+                <tr>
+                    <th>Provider</th>
+                    <th>Category</th>
+                    <th>Status</th>
+                    <th>Response Time</th>
+                    <th>Last Check</th>
+                </tr>
+            </thead>
+            <tbody id="providersTable">
+                <tr><td colspan="5" style="text-align: center;">Loading...</td></tr>
+            </tbody>
+        </table>
+
+        <div class="hf-section">
+            <div class="section-header" style="border: none; margin: 0 0 20px 0;">
+                <h2>🤗 HuggingFace Sentiment Analysis</h2>
             </div>
-            <div class="ws-stream" id="ws-stream"></div>
-        </article>
-    </section>
-</main>
+            <p style="color: #6c757d; margin-bottom: 10px;">Enter crypto-related text (one per line) to analyze sentiment using AI:</p>
+            <textarea id="sentimentText" rows="5" placeholder="BTC strong breakout&#10;ETH looks weak&#10;Market is bullish">BTC strong breakout
+ETH looks weak
+Market is bullish today</textarea>
+            <button class="refresh-btn" onclick="runSentiment()">🧠 Analyze Sentiment</button>
+            <div class="sentiment-result" id="sentimentResult">—</div>
+            <pre id="sentimentDetails" style="background: white; padding: 15px; border-radius: 8px; overflow-x: auto; font-size: 12px;"></pre>
+        </div>
+    </div>
+
+    <script>
+        async function loadData() {
+            try {
+                // Show loading state
+                const tbody = document.getElementById('providersTable');
+                if (tbody) {
+                    tbody.innerHTML = '<tr><td colspan="5" style="text-align: center; padding: 40px;"><div class="loading"></div><div style="margin-top: 10px; color: #6c757d;">در حال بارگذاری...</div></td></tr>';
+                }
+                if (document.getElementById('lastUpdate')) {
+                    document.getElementById('lastUpdate').textContent = 'در حال بارگذاری...';
+                }
+
+                // Load status
+                const statusRes = await fetch('/api/status');
+                if (!statusRes.ok) {
+                    throw new Error(`خطا در دریافت وضعیت: ${statusRes.status} ${statusRes.statusText}`);
+                }
+                const status = await statusRes.json();
+                
+                if (!status || typeof status.total_providers === 'undefined') {
+                    throw new Error('داده‌های وضعیت نامعتبر است');
+                }
+
+                if (document.getElementById('totalAPIs')) {
+                    document.getElementById('totalAPIs').textContent = status.total_providers || 0;
+                }
+                if (document.getElementById('onlineAPIs')) {
+                    document.getElementById('onlineAPIs').textContent = status.online || 0;
+                }
+                if (document.getElementById('offlineAPIs')) {
+                    document.getElementById('offlineAPIs').textContent = status.offline || 0;
+                }
+                if (document.getElementById('avgResponse')) {
+                    document.getElementById('avgResponse').textContent = (status.avg_response_time_ms || 0) + 'ms';
+                }
+                if (document.getElementById('lastUpdate')) {
+                    document.getElementById('lastUpdate').textContent = status.timestamp ? new Date(status.timestamp).toLocaleString('fa-IR') : 'نامشخص';
+                }
+                
+                // Load providers
+                const providersRes = await fetch('/api/providers');
+                if (!providersRes.ok) {
+                    throw new Error(`خطا در دریافت لیست APIها: ${providersRes.status} ${providersRes.statusText}`);
+                }
+                const providers = await providersRes.json();
+                
+                if (!providers || !Array.isArray(providers)) {
+                    throw new Error('لیست APIها نامعتبر است');
+                }
+
+                if (tbody) {
+                    if (providers.length === 0) {
+                        tbody.innerHTML = '<tr><td colspan="5" style="text-align: center; padding: 40px; color: #6c757d;">هیچ APIای یافت نشد</td></tr>';
+                    } else {
+                        tbody.innerHTML = providers.map(p => {
+                            let responseClass = 'response-fast';
+                            const responseTime = p.response_time_ms || p.avg_response_time_ms || 0;
+                            if (responseTime > 3000) responseClass = 'response-slow';
+                            else if (responseTime > 1000) responseClass = 'response-medium';
+                            
+                            return `
+                            <tr>
+                                <td><strong style="font-size: 15px;">${p.name || 'نامشخص'}</strong></td>
+                                <td><span style="background: #f8f9fa; padding: 4px 10px; border-radius: 8px; font-size: 12px; font-weight: 600;">${p.category || 'نامشخص'}</span></td>
+                                <td><span class="status-badge status-${p.status || 'unknown'}">${(p.status || 'unknown').toUpperCase()}</span></td>
+                                <td><span class="response-time ${responseClass}">${responseTime}ms</span></td>
+                                <td style="color: #6c757d; font-size: 13px;">${p.last_fetch ? new Date(p.last_fetch).toLocaleTimeString('fa-IR') : 'نامشخص'}</td>
+                            </tr>
+                        `}).join('');
+                    }
+                }
+                
+            } catch (error) {
+                console.error('Error loading data:', error);
+                const tbody = document.getElementById('providersTable');
+                if (tbody) {
+                    tbody.innerHTML = `<tr><td colspan="5" style="text-align: center; padding: 40px; color: #ef4444;">
+                        <div style="font-size: 24px; margin-bottom: 10px;">❌</div>
+                        <div style="font-weight: 600; margin-bottom: 5px;">خطا در بارگذاری داده‌ها</div>
+                        <div style="font-size: 14px; color: #6c757d; margin-bottom: 15px;">${error.message || 'خطای نامشخص'}</div>
+                        <button onclick="loadData()" style="padding: 10px 20px; background: linear-gradient(135deg, #667eea 0%, #764ba2 100%); border: none; border-radius: 12px; color: white; cursor: pointer; font-weight: 700; text-transform: uppercase; letter-spacing: 0.5px;">تلاش مجدد</button>
+                    </td></tr>`;
+                }
+                if (document.getElementById('lastUpdate')) {
+                    document.getElementById('lastUpdate').textContent = 'خطا در بارگذاری';
+                }
+                alert('❌ خطا در بارگذاری داده‌ها:\n' + (error.message || 'خطای نامشخص'));
+            }
+        }
+
+        async function runSentiment() {
+            const text = document.getElementById('sentimentText').value;
+            const texts = text.split('\n').filter(t => t.trim());
+            
+            if (texts.length === 0) {
+                alert('Please enter at least one line of text');
+                return;
+            }
+            
+            try {
+                document.getElementById('sentimentResult').textContent = '⏳ Analyzing...';
+                document.getElementById('sentimentDetails').textContent = '';
+                
+                const res = await fetch('/api/hf/run-sentiment', {
+                    method: 'POST',
+                    headers: { 'Content-Type': 'application/json' },
+                    body: JSON.stringify({ texts })
+                });
+                
+                if (!res.ok) {
+                    throw new Error(`HTTP ${res.status}: ${await res.text()}`);
+                }
+                
+                const data = await res.json();
+                
+                const vote = data.vote || 0;
+                let emoji = '😐';
+                let color = '#6c757d';
+                
+                if (vote > 0.2) {
+                    emoji = '📈';
+                    color = '#10b981';
+                } else if (vote < -0.2) {
+                    emoji = '📉';
+                    color = '#ef4444';
+                }
+                
+                document.getElementById('sentimentResult').innerHTML = `
+                    <span style="color: ${color};">${emoji} ${vote.toFixed(3)}</span>
+                `;
+                document.getElementById('sentimentDetails').textContent = JSON.stringify(data, null, 2);
+                
+            } catch (error) {
+                console.error('Error running sentiment:', error);
+                document.getElementById('sentimentResult').innerHTML = '<span style="color: #ef4444;">❌ Error</span>';
+                document.getElementById('sentimentDetails').textContent = 'Error: ' + error.message;
+            }
+        }
+
+        // Load data on page load
+        loadData();
+        
+        // Auto-refresh every 30 seconds
+        setInterval(loadData, 30000);
+    </script>
 </body>
-</html>
+</html>

enhanced_server.py

@@ -199,9 +199,9 @@ from fastapi.responses import HTMLResponse, FileResponse
 
 @app.get("/", response_class=HTMLResponse)
 async def root():
-    """Serve main admin dashboard"""
-    if os.path.exists("admin.html"):
-        return FileResponse("admin.html")
+    """Serve main dashboard"""
+    if os.path.exists("index.html"):
+        return FileResponse("index.html")
     else:
         return HTMLResponse("""
         <html>

hf_console.html

@@ -1,97 +1,343 @@
-<!DOCTYPE html>
-<html lang="en">
+<!DOCTYPE html>
+<html lang="en" dir="rtl">
 <head>
-    <meta charset="UTF-8" />
-    <meta name="viewport" content="width=device-width, initial-scale=1" />
-    <title>HF Console · Crypto Intelligence</title>
-    <link rel="preconnect" href="https://fonts.googleapis.com" />
-    <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
-    <link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet" />
-    <link rel="stylesheet" href="/static/css/unified-ui.css" />
-    <link rel="stylesheet" href="/static/css/components.css" />
-    <script defer src="/static/js/ui-feedback.js"></script>
-    <script defer src="/static/js/hf-console.js"></script>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>HF Console - Crypto API Monitor</title>
+    <style>
+        * { margin: 0; padding: 0; box-sizing: border-box; }
+        body {
+            font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
+            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
+            padding: 20px;
+            direction: rtl;
+        }
+        .container {
+            max-width: 1400px;
+            margin: 0 auto;
+            background: white;
+            border-radius: 16px;
+            padding: 30px;
+            box-shadow: 0 10px 40px rgba(0,0,0,0.2);
+        }
+        h1 {
+            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
+            -webkit-background-clip: text;
+            -webkit-text-fill-color: transparent;
+            margin-bottom: 10px;
+            font-size: 32px;
+        }
+        .subtitle { color: #666; margin-bottom: 30px; }
+        section {
+            margin-bottom: 30px;
+            padding: 20px;
+            background: #f8f9fa;
+            border-radius: 12px;
+            border: 2px solid #e9ecef;
+        }
+        h3 {
+            color: #333;
+            margin-bottom: 15px;
+            font-size: 20px;
+            display: flex;
+            align-items: center;
+            gap: 10px;
+        }
+        .badge {
+            display: inline-block;
+            padding: 4px 12px;
+            border-radius: 12px;
+            font-size: 14px;
+            font-weight: 600;
+        }
+        .badge-success { background: #d1fae5; color: #10b981; }
+        .badge-warning { background: #fef3c7; color: #f59e0b; }
+        .badge-info { background: #dbeafe; color: #3b82f6; }
+        pre {
+            background: #1e293b;
+            color: #e2e8f0;
+            padding: 15px;
+            border-radius: 8px;
+            overflow-x: auto;
+            font-size: 13px;
+            line-height: 1.6;
+            max-height: 300px;
+            overflow-y: auto;
+        }
+        button {
+            padding: 10px 20px;
+            border: none;
+            border-radius: 8px;
+            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
+            color: white;
+            font-weight: 600;
+            cursor: pointer;
+            transition: all 0.3s;
+            margin: 5px;
+        }
+        button:hover {
+            transform: translateY(-2px);
+            box-shadow: 0 4px 12px rgba(102, 126, 234, 0.4);
+        }
+        button:active { transform: translateY(0); }
+        input, textarea {
+            width: 100%;
+            padding: 10px;
+            border: 2px solid #e9ecef;
+            border-radius: 8px;
+            font-family: inherit;
+            margin: 10px 0;
+        }
+        input:focus, textarea:focus {
+            outline: none;
+            border-color: #667eea;
+        }
+        .list-box {
+            max-height: 250px;
+            overflow-y: auto;
+            border: 1px solid #e9ecef;
+            padding: 10px;
+            background: white;
+            border-radius: 8px;
+            margin: 10px 0;
+        }
+        .list-box ul { list-style: none; }
+        .list-box li {
+            padding: 8px;
+            border-bottom: 1px solid #f1f5f9;
+            font-size: 14px;
+            color: #475569;
+        }
+        .list-box li:last-child { border-bottom: none; }
+        .vote-display {
+            font-size: 24px;
+            font-weight: 700;
+            padding: 15px;
+            background: white;
+            border-radius: 8px;
+            text-align: center;
+            margin: 10px 0;
+        }
+        .vote-positive { color: #10b981; }
+        .vote-negative { color: #ef4444; }
+        .vote-neutral { color: #6b7280; }
+        .loading {
+            display: inline-block;
+            width: 16px;
+            height: 16px;
+            border: 3px solid #f3f4f6;
+            border-top-color: #667eea;
+            border-radius: 50%;
+            animation: spin 0.8s linear infinite;
+        }
+        @keyframes spin {
+            to { transform: rotate(360deg); }
+        }
+        .grid {
+            display: grid;
+            grid-template-columns: repeat(auto-fit, minmax(300px, 1fr));
+            gap: 20px;
+        }
+    </style>
+    <!-- API Configuration -->
+    <script src="config.js"></script>
 </head>
-<body class="page page-hf-console">
-<header class="top-nav">
-    <div class="branding">
-        <svg width="32" height="32" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5"><circle cx="12" cy="12" r="10"/></svg>
-        <div>
-            <strong>HF Models & Datasets</strong>
-            <small style="color:var(--ui-text-muted);letter-spacing:0.2em;">/api/hf/* endpoints</small>
+<body>
+    <div class="container">
+        <h1>🤗 HuggingFace Console</h1>
+        <p class="subtitle">Test HF connectivity, registry, search, and sentiment analysis</p>
+        <div style="background: #f0f9ff; padding: 10px; border-radius: 8px; margin-bottom: 20px; font-size: 13px; color: #0369a1;">
+            <strong>🌐 Environment:</strong> <span id="envInfo">Loading...</span> |
+            <strong>📡 API:</strong> <span id="apiInfo">Loading...</span>
         </div>
-    </div>
-    <nav class="nav-links">
-        <a href="/dashboard">Dashboard</a>
-        <a href="/admin">Admin</a>
-        <a class="active" href="/hf_console">HF Console</a>
-        <a href="/docs" target="_blank" rel="noreferrer">API Docs</a>
-    </nav>
-</header>
 
-<main class="page-content">
-    <section class="card">
-        <div class="section-heading">
-            <h2>Registry & Status</h2>
-            <span class="badge info" id="hf-console-health">Loading...</span>
+        <section>
+            <h3>
+                <span>📊 Health Status</span>
+                <span class="badge badge-info" id="healthBadge">Loading...</span>
+            </h3>
+            <button onclick="loadHealth()">🔄 Refresh Health</button>
+            <button onclick="doRefresh()">🔃 Force Registry Refresh</button>
+            <pre id="healthOutput">Loading...</pre>
+        </section>
+
+        <div class="grid">
+            <section>
+                <h3>
+                    <span>🤖 Models Registry</span>
+                    <span class="badge badge-success" id="modelsCount">0</span>
+                </h3>
+                <button onclick="loadModels()">Load Models</button>
+                <div class="list-box" id="modelsList">
+                    <p style="color: #94a3b8;">Click "Load Models" to fetch...</p>
+                </div>
+            </section>
+
+            <section>
+                <h3>
+                    <span>📚 Datasets Registry</span>
+                    <span class="badge badge-success" id="datasetsCount">0</span>
+                </h3>
+                <button onclick="loadDatasets()">Load Datasets</button>
+                <div class="list-box" id="datasetsList">
+                    <p style="color: #94a3b8;">Click "Load Datasets" to fetch...</p>
+                </div>
+            </section>
         </div>
-        <p class="metric-subtext" id="hf-console-summary"></p>
-        <ul class="list" id="hf-console-models"></ul>
-    </section>
 
-    <section class="split-grid">
-        <article class="card">
-            <div class="section-heading"><h2>Sentiment Playground</h2><span class="badge info">POST /api/hf/models/sentiment</span></div>
-            <div class="form-field">
-                <label for="sentiment-model">Sentiment model</label>
-                <select id="sentiment-model">
-                    <option value="auto">auto (ensemble)</option>
-                    <option value="cryptobert">cryptobert</option>
-                    <option value="cryptobert_finbert">cryptobert_finbert</option>
-                    <option value="tiny_crypto_lm">tiny_crypto_lm</option>
-                </select>
-            </div>
-            <div class="form-field">
-                <label for="sentiment-texts">Texts (one per line)</label>
-                <textarea id="sentiment-texts" rows="5" placeholder="BTC is breaking out...\nETH looks weak..."></textarea>
-            </div>
-            <button class="primary" id="run-sentiment">Run Sentiment</button>
-            <div id="sentiment-results" class="ws-stream" style="margin-top:16px;"></div>
-        </article>
-        <article class="card">
-            <div class="section-heading"><h2>Forecast Sandbox</h2><span class="badge info">POST /api/hf/models/forecast</span></div>
-            <div class="form-field">
-                <label for="forecast-model">Model</label>
-                <select id="forecast-model">
-                    <option value="btc_lstm">btc_lstm</option>
-                    <option value="btc_arima">btc_arima</option>
-                </select>
+        <section>
+            <h3>🔍 Search Registry (Local Snapshot)</h3>
+            <input type="text" id="searchQuery" placeholder="Search query (e.g., crypto, bitcoin, sentiment)" value="crypto">
+            <button onclick="doSearch()">Search Models</button>
+            <button onclick="doSearchDatasets()">Search Datasets</button>
+            <div class="list-box" id="searchResults">
+                <p style="color: #94a3b8;">Enter a query and click search...</p>
             </div>
-            <div class="form-field">
-                <label for="forecast-series">Closing Prices (comma separated)</label>
-                <textarea id="forecast-series" rows="5" placeholder="67650, 67820, 68010, 68120"></textarea>
-            </div>
-            <div class="form-field">
-                <label for="forecast-steps">Future Steps</label>
-                <input type="number" id="forecast-steps" value="3" min="1" max="10" />
+        </section>
+
+        <section>
+            <h3>💭 Sentiment Analysis (Local Pipeline)</h3>
+            <p style="color: #666; font-size: 14px; margin-bottom: 10px;">
+                Enter text samples (one per line) to analyze crypto sentiment using local transformers
+            </p>
+            <textarea id="sentimentTexts" rows="5" placeholder="BTC looks strong&#10;ETH is weak today&#10;Market sentiment is bullish">BTC strong breakout
+ETH looks weak
+Crypto market is bullish today
+Bears are taking control
+Neutral market conditions</textarea>
+            <button onclick="doSentiment()">🧠 Run Sentiment Analysis</button>
+            <div class="vote-display" id="voteDisplay">
+                <span style="color: #94a3b8;">—</span>
             </div>
-            <button class="primary" id="run-forecast">Forecast</button>
-            <div id="forecast-results" class="ws-stream" style="margin-top:16px;"></div>
-        </article>
-    </section>
+            <pre id="sentimentOutput">Results will appear here...</pre>
+        </section>
+    </div>
 
-    <section class="card">
-        <div class="section-heading">
-            <h2>HF Datasets</h2>
-            <span class="badge info">GET /api/hf/datasets/*</span>
-        </div>
-        <div class="button-row" style="margin-bottom:16px;">
-            <button class="secondary" data-dataset="market-ohlcv">Market OHLCV</button>
-            <button class="secondary" data-dataset="market-btc">BTC Technicals</button>
-            <button class="secondary" data-dataset="news-semantic">News Semantic</button>
-        </div>
-        <div id="dataset-output" class="ws-stream"></div>
-    </section>
-</main>
+    <script>
+        // Use the CONFIG object from config.js
+        const API_BASE = CONFIG.API_BASE;
+        const fetchJSON = CONFIG.fetchJSON;
+        const postJSON = CONFIG.postJSON;
+
+        // Display environment info
+        function updateEnvironmentInfo() {
+            const envType = CONFIG.IS_HUGGINGFACE_SPACES ? '🤗 HuggingFace Spaces' :
+                           CONFIG.IS_LOCALHOST ? '💻 Localhost' : '🌐 Custom Deployment';
+            document.getElementById('envInfo').textContent = envType;
+            document.getElementById('apiInfo').textContent = CONFIG.API_BASE;
+        }
+
+        async function loadHealth() {
+            try {
+                const data = await fetchJSON(CONFIG.ENDPOINTS.HF_HEALTH);
+                document.getElementById('healthOutput').textContent = JSON.stringify(data, null, 2);
+                document.getElementById('healthBadge').textContent = data.ok ? '✓ Healthy' : '✗ Unhealthy';
+                document.getElementById('healthBadge').className = data.ok ? 'badge badge-success' : 'badge badge-warning';
+            } catch (err) {
+                document.getElementById('healthOutput').textContent = `Error: ${err.message}`;
+                document.getElementById('healthBadge').textContent = '✗ Error';
+                document.getElementById('healthBadge').className = 'badge badge-warning';
+            }
+        }
+
+        async function doRefresh() {
+            try {
+                document.getElementById('healthOutput').textContent = 'Refreshing registry...';
+                const data = await postJSON(CONFIG.ENDPOINTS.HF_REFRESH, {});
+                document.getElementById('healthOutput').textContent = JSON.stringify(data, null, 2);
+                await loadHealth();
+            } catch (err) {
+                document.getElementById('healthOutput').textContent = `Error: ${err.message}`;
+            }
+        }
+
+        async function loadModels() {
+            try {
+                const data = await fetchJSON(`${CONFIG.ENDPOINTS.HF_REGISTRY}?kind=models`);
+                const items = data.items || [];
+                document.getElementById('modelsCount').textContent = items.length;
+                const html = items.length > 0 
+                    ? '<ul>' + items.slice(0, 50).map(i => `<li>🤖 ${i.id} • ${i.pipeline_tag || 'N/A'} • <small>${i.source}</small></li>`).join('') + '</ul>'
+                    : '<p style="color: #94a3b8;">No models found</p>';
+                document.getElementById('modelsList').innerHTML = html;
+            } catch (err) {
+                document.getElementById('modelsList').innerHTML = `<p style="color: #ef4444;">Error: ${err.message}</p>`;
+            }
+        }
+
+        async function loadDatasets() {
+            try {
+                const data = await fetchJSON(`${CONFIG.ENDPOINTS.HF_REGISTRY}?kind=datasets`);
+                const items = data.items || [];
+                document.getElementById('datasetsCount').textContent = items.length;
+                const html = items.length > 0
+                    ? '<ul>' + items.slice(0, 50).map(i => `<li>📚 ${i.id} • <small>${i.source}</small></li>`).join('') + '</ul>'
+                    : '<p style="color: #94a3b8;">No datasets found</p>';
+                document.getElementById('datasetsList').innerHTML = html;
+            } catch (err) {
+                document.getElementById('datasetsList').innerHTML = `<p style="color: #ef4444;">Error: ${err.message}</p>`;
+            }
+        }
+
+        async function doSearch() {
+            const q = document.getElementById('searchQuery').value;
+            try {
+                const data = await fetchJSON(`${CONFIG.ENDPOINTS.HF_SEARCH}?q=${encodeURIComponent(q)}&kind=models`);
+                const items = data.items || [];
+                const html = items.length > 0
+                    ? `<p style="color: #10b981; font-weight: 600;">Found ${items.length} models</p><ul>` + items.map(i => `<li>🤖 ${i.id}</li>`).join('') + '</ul>'
+                    : '<p style="color: #94a3b8;">No results found</p>';
+                document.getElementById('searchResults').innerHTML = html;
+            } catch (err) {
+                document.getElementById('searchResults').innerHTML = `<p style="color: #ef4444;">Error: ${err.message}</p>`;
+            }
+        }
+
+        async function doSearchDatasets() {
+            const q = document.getElementById('searchQuery').value;
+            try {
+                const data = await fetchJSON(`${CONFIG.ENDPOINTS.HF_SEARCH}?q=${encodeURIComponent(q)}&kind=datasets`);
+                const items = data.items || [];
+                const html = items.length > 0
+                    ? `<p style="color: #10b981; font-weight: 600;">Found ${items.length} datasets</p><ul>` + items.map(i => `<li>📚 ${i.id}</li>`).join('') + '</ul>'
+                    : '<p style="color: #94a3b8;">No results found</p>';
+                document.getElementById('searchResults').innerHTML = html;
+            } catch (err) {
+                document.getElementById('searchResults').innerHTML = `<p style="color: #ef4444;">Error: ${err.message}</p>`;
+            }
+        }
+
+        async function doSentiment() {
+            const texts = document.getElementById('sentimentTexts').value.split('\n').filter(t => t.trim());
+            if (texts.length === 0) {
+                alert('Please enter at least one text sample');
+                return;
+            }
+            try {
+                document.getElementById('voteDisplay').innerHTML = '<span class="loading"></span>';
+                document.getElementById('sentimentOutput').textContent = 'Running sentiment analysis...';
+                
+                const data = await postJSON(CONFIG.ENDPOINTS.HF_RUN_SENTIMENT, { texts });
+                
+                const vote = data.vote || 0;
+                let voteClass = 'vote-neutral';
+                let voteEmoji = '😐';
+                if (vote > 0.2) { voteClass = 'vote-positive'; voteEmoji = '📈'; }
+                else if (vote < -0.2) { voteClass = 'vote-negative'; voteEmoji = '📉'; }
+                
+                document.getElementById('voteDisplay').innerHTML = `<span class="${voteClass}">${voteEmoji} ${vote.toFixed(3)}</span>`;
+                document.getElementById('sentimentOutput').textContent = JSON.stringify(data, null, 2);
+            } catch (err) {
+                document.getElementById('voteDisplay').innerHTML = '<span style="color: #ef4444;">Error</span>';
+                document.getElementById('sentimentOutput').textContent = `Error: ${err.message}`;
+            }
+        }
+
+        // Auto-load health and environment info on page load
+        window.addEventListener('load', () => {
+            updateEnvironmentInfo();
+            loadHealth();
+        });
+    </script>
 </body>
-</html>
+</html>

hf_unified_server.py

@@ -1,2575 +1,10 @@
-"""Unified HuggingFace Space API Server leveraging shared collectors and AI helpers."""
-
-import asyncio
-import time
-import os
-import sys
-import io
-
-# Fix encoding for Windows console (must be done before any print/logging)
-if sys.platform == "win32":
-    try:
-        sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8', errors='replace')
-        sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8', errors='replace')
-    except Exception:
-        pass  # If already wrapped, ignore
-
-# Set environment variables to force PyTorch and avoid TensorFlow/Keras issues
-os.environ.setdefault('TRANSFORMERS_NO_ADVISORY_WARNINGS', '1')
-os.environ.setdefault('TRANSFORMERS_VERBOSITY', 'error')
-os.environ.setdefault('TF_CPP_MIN_LOG_LEVEL', '3')  # Suppress TensorFlow warnings
-# Force PyTorch as default framework
-os.environ.setdefault('TRANSFORMERS_FRAMEWORK', 'pt')
-
-from datetime import datetime, timedelta
-from fastapi import Body, FastAPI, HTTPException, Query, WebSocket, WebSocketDisconnect
-from fastapi.middleware.cors import CORSMiddleware
-from fastapi.responses import FileResponse, JSONResponse, HTMLResponse
-from fastapi.staticfiles import StaticFiles
-from starlette.websockets import WebSocketState
-from typing import Any, Dict, List, Optional, Union
-from statistics import mean
-import logging
-import random
-import json
-from pathlib import Path
-import httpx
-
-
-from ai_models import (
-    analyze_chart_points,
-    analyze_crypto_sentiment,
-    analyze_market_text,
-    get_model_info,
-    initialize_models,
-    registry_status,
-)
-from backend.services.local_resource_service import LocalResourceService
-from collectors.aggregator import (
-    CollectorError,
-    MarketDataCollector,
-    NewsCollector,
-    ProviderStatusCollector,
-)
-from config import COIN_SYMBOL_MAPPING, get_settings
-
-# Setup logging
-logging.basicConfig(level=logging.INFO)
-logger = logging.getLogger(__name__)
-
-# Create FastAPI app
-app = FastAPI(
-    title="Cryptocurrency Data & Analysis API",
-    description="Complete API for cryptocurrency data, market analysis, and trading signals",
-    version="3.0.0"
-)
-
-# CORS
-app.add_middleware(
-    CORSMiddleware,
-    allow_origins=["*"],
-    allow_credentials=True,
-    allow_methods=["*"],
-    allow_headers=["*"],
-)
-
-# Runtime state
-START_TIME = time.time()
-cache = {"ohlcv": {}, "prices": {}, "market_data": {}, "providers": [], "last_update": None}
-settings = get_settings()
-market_collector = MarketDataCollector()
-news_collector = NewsCollector()
-provider_collector = ProviderStatusCollector()
-
-# Load providers config
-WORKSPACE_ROOT = Path(__file__).parent
-PROVIDERS_CONFIG_PATH = settings.providers_config_path
-FALLBACK_RESOURCE_PATH = WORKSPACE_ROOT / "crypto_resources_unified_2025-11-11.json"
-LOG_DIR = WORKSPACE_ROOT / "logs"
-APL_REPORT_PATH = WORKSPACE_ROOT / "PROVIDER_AUTO_DISCOVERY_REPORT.json"
-
-# Ensure log directory exists
-LOG_DIR.mkdir(parents=True, exist_ok=True)
-
-# Database path (managed by DatabaseManager in the admin API)
-DB_PATH = WORKSPACE_ROOT / "data" / "api_monitor.db"
-
-def tail_log_file(path: Path, max_lines: int = 200) -> List[str]:
-    """Return the last max_lines from a log file, if it exists."""
-    if not path.exists():
-        return []
-    try:
-        with path.open("r", encoding="utf-8", errors="ignore") as f:
-            lines = f.readlines()
-        return lines[-max_lines:]
-    except Exception as e:
-        logger.error(f"Error reading log file {path}: {e}")
-        return []
-
-
-def load_providers_config():
-    """Load providers from providers_config_extended.json"""
-    try:
-        if PROVIDERS_CONFIG_PATH.exists():
-            with open(PROVIDERS_CONFIG_PATH, 'r', encoding='utf-8') as f:
-                config = json.load(f)
-                providers = config.get('providers', {})
-                logger.info(f"Loaded {len(providers)} providers from providers_config_extended.json")
-                return providers
-        else:
-            logger.warning(f"providers_config_extended.json not found at {PROVIDERS_CONFIG_PATH}")
-            return {}
-    except Exception as e:
-        logger.error(f"Error loading providers config: {e}")
-        return {}
-
-# Load providers at startup
-PROVIDERS_CONFIG = load_providers_config()
-local_resource_service = LocalResourceService(FALLBACK_RESOURCE_PATH)
-
-HF_SAMPLE_NEWS = [
-    {
-        "title": "Bitcoin holds key liquidity zone",
-        "source": "Fallback Ledger",
-        "sentiment": "positive",
-        "sentiment_score": 0.64,
-        "entities": ["BTC"],
-        "summary": "BTC consolidates near resistance with steady inflows",
-    },
-    {
-        "title": "Ethereum staking demand remains resilient",
-        "source": "Fallback Ledger",
-        "sentiment": "neutral",
-        "sentiment_score": 0.12,
-        "entities": ["ETH"],
-        "summary": "Validator queue shortens as fees stabilize around L2 adoption",
-    },
-    {
-        "title": "Solana ecosystem sees TVL uptick",
-        "source": "Fallback Ledger",
-        "sentiment": "positive",
-        "sentiment_score": 0.41,
-        "entities": ["SOL"],
-        "summary": "DeFi protocols move to Solana as mempool congestion drops",
-    },
-]
-
-# Mount static files (CSS, JS)
-try:
-    static_path = WORKSPACE_ROOT / "static"
-    if static_path.exists():
-        app.mount("/static", StaticFiles(directory=str(static_path)), name="static")
-        logger.info(f"Static files mounted from {static_path}")
-    else:
-        logger.warning(f"Static directory not found: {static_path}")
-except Exception as e:
-    logger.error(f"Error mounting static files: {e}")
-
-# Mount api-resources for frontend access
-try:
-    api_resources_path = WORKSPACE_ROOT / "api-resources"
-    if api_resources_path.exists():
-        app.mount("/api-resources", StaticFiles(directory=str(api_resources_path)), name="api-resources")
-        logger.info(f"API resources mounted from {api_resources_path}")
-    else:
-        logger.warning(f"API resources directory not found: {api_resources_path}")
-except Exception as e:
-    logger.error(f"Error mounting API resources: {e}")
-
-# ============================================================================
-# Helper utilities & Data Fetching Functions
-# ============================================================================
-
-def _normalize_asset_symbol(symbol: str) -> str:
-    symbol = (symbol or "").upper()
-    suffixes = ("USDT", "USD", "BTC", "ETH", "BNB")
-    for suffix in suffixes:
-        if symbol.endswith(suffix) and len(symbol) > len(suffix):
-            return symbol[: -len(suffix)]
-    return symbol
-
-
-def _format_price_record(record: Dict[str, Any]) -> Dict[str, Any]:
-    price = record.get("price") or record.get("current_price")
-    change_pct = record.get("change_24h") or record.get("price_change_percentage_24h")
-    change_abs = None
-    if price is not None and change_pct is not None:
-        try:
-            change_abs = float(price) * float(change_pct) / 100.0
-        except (TypeError, ValueError):
-            change_abs = None
-
-    return {
-        "id": record.get("id") or record.get("symbol", "").lower(),
-        "symbol": record.get("symbol", "").upper(),
-        "name": record.get("name"),
-        "current_price": price,
-        "market_cap": record.get("market_cap"),
-        "market_cap_rank": record.get("rank"),
-        "total_volume": record.get("volume_24h") or record.get("total_volume"),
-        "price_change_24h": change_abs,
-        "price_change_percentage_24h": change_pct,
-        "high_24h": record.get("high_24h"),
-        "low_24h": record.get("low_24h"),
-        "last_updated": record.get("last_updated"),
-    }
-
-
-async def fetch_binance_ohlcv(symbol: str = "BTCUSDT", interval: str = "1h", limit: int = 100):
-    """Fetch OHLCV data from Binance via the shared collector."""
-
-    try:
-        candles = await market_collector.get_ohlcv(symbol, interval, limit)
-        return [
-            {
-                **candle,
-                "timestamp": int(datetime.fromisoformat(candle["timestamp"]).timestamp() * 1000),
-                "datetime": candle["timestamp"],
-            }
-            for candle in candles
-        ]
-    except CollectorError as exc:
-        logger.error("Error fetching OHLCV: %s", exc)
-    fallback_symbol = _normalize_asset_symbol(symbol)
-    fallback = local_resource_service.get_ohlcv(fallback_symbol, interval, limit)
-    if fallback:
-        return fallback
-    return []
-
-
-async def fetch_coingecko_prices(symbols: Optional[List[str]] = None, limit: int = 10):
-    """Fetch price snapshots using the shared market collector."""
-
-    source = "coingecko"
-    try:
-        if symbols:
-            tasks = [market_collector.get_coin_details(_normalize_asset_symbol(sym)) for sym in symbols]
-            results = await asyncio.gather(*tasks, return_exceptions=True)
-            coins: List[Dict[str, Any]] = []
-            for result in results:
-                if isinstance(result, Exception):
-                    continue
-                coins.append(_format_price_record(result))
-            if coins:
-                return coins, source
-        else:
-            top = await market_collector.get_top_coins(limit=limit)
-            formatted = [_format_price_record(entry) for entry in top]
-            if formatted:
-                return formatted, source
-    except CollectorError as exc:
-        logger.error("Error fetching aggregated prices: %s", exc)
-
-    fallback = (
-        local_resource_service.get_prices_for_symbols([sym for sym in symbols or []])
-        if symbols
-        else local_resource_service.get_top_prices(limit)
-    )
-    if fallback:
-        return fallback, "local-fallback"
-    return [], source
-
-
-async def fetch_binance_ticker(symbol: str):
-    """Provide ticker-like information sourced from CoinGecko market data."""
-
-    try:
-        coin = await market_collector.get_coin_details(_normalize_asset_symbol(symbol))
-    except CollectorError as exc:
-        logger.error("Unable to load ticker for %s: %s", symbol, exc)
-        coin = None
-
-    if coin:
-        price = coin.get("price")
-        change_pct = coin.get("change_24h") or 0.0
-        change_abs = price * change_pct / 100 if price is not None and change_pct is not None else None
-        return {
-            "symbol": symbol.upper(),
-            "price": price,
-            "price_change_24h": change_abs,
-            "price_change_percent_24h": change_pct,
-            "high_24h": coin.get("high_24h"),
-            "low_24h": coin.get("low_24h"),
-            "volume_24h": coin.get("volume_24h"),
-            "quote_volume_24h": coin.get("volume_24h"),
-        }, "binance"
-
-    fallback_symbol = _normalize_asset_symbol(symbol)
-    fallback = local_resource_service.get_ticker_snapshot(fallback_symbol)
-    if fallback:
-        fallback["symbol"] = symbol.upper()
-        return fallback, "local-fallback"
-    return None, "binance"
-
-
-# ============================================================================
-# Core Endpoints
-# ============================================================================
-
-@app.get("/health")
-async def health():
-    """System health check using shared collectors."""
-
-    async def _safe_call(coro):
-        try:
-            data = await coro
-            return {"status": "ok", "count": len(data) if hasattr(data, "__len__") else 1}
-        except Exception as exc:  # pragma: no cover - network heavy
-            return {"status": "error", "detail": str(exc)}
-
-    market_task = asyncio.create_task(_safe_call(market_collector.get_top_coins(limit=3)))
-    news_task = asyncio.create_task(_safe_call(news_collector.get_latest_news(limit=3)))
-    providers_task = asyncio.create_task(_safe_call(provider_collector.get_providers_status()))
-
-    market_status, news_status, providers_status = await asyncio.gather(
-        market_task, news_task, providers_task
-    )
-
-    ai_status = registry_status()
-    service_states = {
-        "market_data": market_status,
-        "news": news_status,
-        "providers": providers_status,
-        "ai_models": ai_status,
-    }
-
-    degraded = any(state.get("status") != "ok" for state in (market_status, news_status, providers_status))
-    overall = "healthy" if not degraded else "degraded"
-
-    return {
-        "status": overall,
-        "service": "cryptocurrency-data-api",
-        "timestamp": datetime.utcnow().isoformat(),
-        "version": app.version,
-        "providers_loaded": market_status.get("count", 0),
-        "services": service_states,
-    }
-
-
-@app.get("/info")
-async def info():
-    """System information"""
-    hf_providers = [p for p in PROVIDERS_CONFIG.keys() if "huggingface_space" in p]
-
-    return {
-        "service": "Cryptocurrency Data & Analysis API",
-        "version": app.version,
-        "endpoints": {
-            "core": ["/health", "/info", "/api/providers"],
-            "data": ["/api/ohlcv", "/api/crypto/prices/top", "/api/crypto/price/{symbol}", "/api/crypto/market-overview"],
-            "analysis": ["/api/analysis/signals", "/api/analysis/smc", "/api/scoring/snapshot"],
-            "market": ["/api/market/prices", "/api/market-data/prices"],
-            "system": ["/api/system/status", "/api/system/config"],
-            "huggingface": ["/api/hf/health", "/api/hf/refresh", "/api/hf/registry", "/api/hf/run-sentiment"],
-        },
-        "data_sources": ["Binance", "CoinGecko", "CoinPaprika", "CoinCap"],
-        "providers_loaded": len(PROVIDERS_CONFIG),
-        "huggingface_space_providers": len(hf_providers),
-        "features": [
-            "Real-time price data",
-            "OHLCV historical data",
-            "Trading signals",
-            "Market analysis",
-            "Sentiment analysis",
-            "HuggingFace model integration",
-            f"{len(PROVIDERS_CONFIG)} providers from providers_config_extended.json",
-        ],
-        "ai_registry": registry_status(),
-    }
-
-
-@app.get("/api/providers")
-async def get_providers():
-    """Get list of API providers and their health."""
-
-    try:
-        statuses = await provider_collector.get_providers_status()
-    except Exception as exc:  # pragma: no cover - network heavy
-        logger.error("Error getting providers: %s", exc)
-        raise HTTPException(status_code=503, detail=str(exc))
-
-    providers_list = []
-    for status in statuses:
-        meta = PROVIDERS_CONFIG.get(status["provider_id"], {})
-        providers_list.append(
-            {
-                **status,
-                "base_url": meta.get("base_url"),
-                "requires_auth": meta.get("requires_auth"),
-                "priority": meta.get("priority"),
-            }
-        )
-
-    return {
-        "providers": providers_list,
-        "total": len(providers_list),
-        "source": str(PROVIDERS_CONFIG_PATH),
-        "last_updated": datetime.utcnow().isoformat(),
-    }
-
-
-@app.get("/api/providers/{provider_id}/health")
-async def get_provider_health(provider_id: str):
-    """Get health status for a specific provider."""
-    
-    # Check if provider exists in config
-    provider_config = PROVIDERS_CONFIG.get(provider_id)
-    if not provider_config:
-        raise HTTPException(status_code=404, detail=f"Provider '{provider_id}' not found")
-    
-    try:
-        # Perform health check using the collector
-        async with httpx.AsyncClient(timeout=provider_collector.timeout, headers=provider_collector.headers) as client:
-            health_result = await provider_collector._check_provider(client, provider_id, provider_config)
-        
-        # Add metadata from config
-        health_result.update({
-            "base_url": provider_config.get("base_url"),
-            "requires_auth": provider_config.get("requires_auth"),
-            "priority": provider_config.get("priority"),
-            "category": provider_config.get("category"),
-            "last_checked": datetime.utcnow().isoformat()
-        })
-        
-        return health_result
-    except Exception as exc:  # pragma: no cover - network heavy
-        logger.error("Error checking provider health for %s: %s", provider_id, exc)
-        raise HTTPException(status_code=503, detail=f"Health check failed: {str(exc)}")
-
-
-@app.get("/api/providers/config")
-async def get_providers_config():
-    """Get providers configuration in format expected by frontend."""
-    try:
-        return {
-            "success": True,
-            "providers": PROVIDERS_CONFIG,
-            "total": len(PROVIDERS_CONFIG),
-            "source": str(PROVIDERS_CONFIG_PATH),
-            "last_updated": datetime.utcnow().isoformat()
-        }
-    except Exception as exc:
-        logger.error("Error getting providers config: %s", exc)
-        raise HTTPException(status_code=500, detail=str(exc))
-
-
-# ============================================================================
-# OHLCV Data Endpoint
-# ============================================================================
-
-@app.get("/api/ohlcv")
-async def get_ohlcv(
-    symbol: str = Query("BTCUSDT", description="Trading pair symbol"),
-    interval: str = Query("1h", description="Time interval (1m, 5m, 15m, 1h, 4h, 1d)"),
-    limit: int = Query(100, ge=1, le=1000, description="Number of candles")
-):
-    """
-    Get OHLCV (candlestick) data for a trading pair
-    
-    Supported intervals: 1m, 5m, 15m, 30m, 1h, 4h, 1d
-    """
-    try:
-        # Check cache
-        cache_key = f"{symbol}_{interval}_{limit}"
-        if cache_key in cache["ohlcv"]:
-            cached_data, cached_time = cache["ohlcv"][cache_key]
-            if (datetime.now() - cached_time).seconds < 60:  # 60s cache
-                return {"symbol": symbol, "interval": interval, "data": cached_data, "source": "cache"}
-        
-        # Fetch from Binance
-        ohlcv_data = await fetch_binance_ohlcv(symbol, interval, limit)
-        
-        if ohlcv_data:
-            # Update cache
-            cache["ohlcv"][cache_key] = (ohlcv_data, datetime.now())
-            
-            return {
-                "symbol": symbol,
-                "interval": interval,
-                "count": len(ohlcv_data),
-                "data": ohlcv_data,
-                "source": "binance",
-                "timestamp": datetime.now().isoformat()
-            }
-        else:
-            raise HTTPException(status_code=503, detail="Unable to fetch OHLCV data")
-    
-    except HTTPException:
-        raise
-    except Exception as e:
-        logger.error(f"Error in get_ohlcv: {e}")
-        raise HTTPException(status_code=500, detail=str(e))
-
-
-# ============================================================================
-# Crypto Prices Endpoints
-# ============================================================================
-
-@app.get("/api/crypto/prices/top")
-async def get_top_prices(limit: int = Query(10, ge=1, le=100, description="Number of top cryptocurrencies")):
-    """Get top cryptocurrencies by market cap"""
-    try:
-        # Check cache
-        cache_key = f"top_{limit}"
-        if cache_key in cache["prices"]:
-            cached_data, cached_time = cache["prices"][cache_key]
-            if (datetime.now() - cached_time).seconds < 60:
-                return {"data": cached_data, "source": "cache"}
-        
-        # Fetch from CoinGecko
-        prices, source = await fetch_coingecko_prices(limit=limit)
-        
-        if prices:
-            # Update cache
-            cache["prices"][cache_key] = (prices, datetime.now())
-            
-            return {
-                "count": len(prices),
-                "data": prices,
-                "source": source,
-                "timestamp": datetime.now().isoformat()
-            }
-        else:
-            raise HTTPException(status_code=503, detail="Unable to fetch price data")
-    
-    except HTTPException:
-        raise
-    except Exception as e:
-        logger.error(f"Error in get_top_prices: {e}")
-        raise HTTPException(status_code=500, detail=str(e))
-
-
-@app.get("/api/crypto/price/{symbol}")
-async def get_single_price(symbol: str):
-    """Get price for a single cryptocurrency"""
-    try:
-        # Try Binance first for common pairs
-        binance_symbol = f"{symbol.upper()}USDT"
-        ticker, ticker_source = await fetch_binance_ticker(binance_symbol)
-        
-        if ticker:
-            return {
-                "symbol": symbol.upper(),
-                "price": ticker,
-                "source": ticker_source,
-                "timestamp": datetime.now().isoformat()
-            }
-        
-        # Fallback to CoinGecko
-        prices, source = await fetch_coingecko_prices([symbol])
-        if prices:
-            return {
-                "symbol": symbol.upper(),
-                "price": prices[0],
-                "source": source,
-                "timestamp": datetime.now().isoformat()
-            }
-        
-        raise HTTPException(status_code=404, detail=f"Price data not found for {symbol}")
-    
-    except HTTPException:
-        raise
-    except Exception as e:
-        logger.error(f"Error in get_single_price: {e}")
-        raise HTTPException(status_code=500, detail=str(e))
-
-
-@app.get("/api/crypto/market-overview")
-async def get_market_overview():
-    """Get comprehensive market overview"""
-    try:
-        # Fetch top 20 coins
-        prices, source = await fetch_coingecko_prices(limit=20)
-        
-        if not prices:
-            raise HTTPException(status_code=503, detail="Unable to fetch market data")
-        
-        # Calculate market stats
-        # Try multiple field names for market cap and volume
-        total_market_cap = 0
-        total_volume = 0
-        
-        for p in prices:
-            # Try different field names for market cap
-            market_cap = (
-                p.get("market_cap") or 
-                p.get("market_cap_usd") or
-                p.get("market_cap_rank") or  # Sometimes this is the value
-                None
-            )
-            # If market_cap is not found, try calculating from price and supply
-            if not market_cap:
-                price = p.get("price") or p.get("current_price") or 0
-                supply = p.get("circulating_supply") or p.get("total_supply") or 0
-                if price and supply:
-                    market_cap = float(price) * float(supply)
-            
-            if market_cap:
-                try:
-                    total_market_cap += float(market_cap)
-                except (TypeError, ValueError):
-                    pass
-            
-            # Try different field names for volume
-            volume = (
-                p.get("total_volume") or 
-                p.get("volume_24h") or 
-                p.get("volume_24h_usd") or
-                None
-            )
-            if volume:
-                try:
-                    total_volume += float(volume)
-                except (TypeError, ValueError):
-                    pass
-        
-        logger.info(f"Market overview: {len(prices)} coins, total_market_cap={total_market_cap:,.0f}, total_volume={total_volume:,.0f}")
-        
-        # Sort by 24h change
-        gainers = sorted(
-            [p for p in prices if p.get("price_change_percentage_24h")],
-            key=lambda x: x.get("price_change_percentage_24h", 0),
-            reverse=True
-        )[:5]
-        
-        losers = sorted(
-            [p for p in prices if p.get("price_change_percentage_24h")],
-            key=lambda x: x.get("price_change_percentage_24h", 0)
-        )[:5]
-        
-        return {
-            "total_market_cap": total_market_cap,
-            "total_volume_24h": total_volume,
-            "btc_dominance": (prices[0].get("market_cap", 0) / total_market_cap * 100) if total_market_cap > 0 else 0,
-            "top_gainers": gainers,
-            "top_losers": losers,
-            "top_by_volume": sorted(prices, key=lambda x: x.get("total_volume", 0) or 0, reverse=True)[:5],
-            "timestamp": datetime.now().isoformat(),
-            "source": source
-        }
-    
-    except HTTPException:
-        raise
-    except Exception as e:
-        logger.error(f"Error in get_market_overview: {e}")
-        raise HTTPException(status_code=500, detail=str(e))
-
-
-@app.get("/api/market")
-async def get_market():
-    """Get market data in format expected by frontend dashboard"""
-    try:
-        overview = await get_market_overview()
-        prices, source = await fetch_coingecko_prices(limit=50)
-        
-        if not prices:
-            raise HTTPException(status_code=503, detail="Unable to fetch market data")
-        
-        return {
-            "total_market_cap": overview.get("total_market_cap", 0),
-            "btc_dominance": overview.get("btc_dominance", 0),
-            "total_volume_24h": overview.get("total_volume_24h", 0),
-            "cryptocurrencies": prices,
-            "timestamp": datetime.now().isoformat(),
-            "source": source
-        }
-    except HTTPException:
-        raise
-    except Exception as e:
-        logger.error(f"Error in get_market: {e}")
-        raise HTTPException(status_code=500, detail=str(e))
-
-
-@app.get("/api/trending")
-async def get_trending():
-    """Get trending cryptocurrencies (top gainers by 24h change)"""
-    try:
-        prices, source = await fetch_coingecko_prices(limit=100)
-        
-        if not prices:
-            raise HTTPException(status_code=503, detail="Unable to fetch trending data")
-        
-        trending = sorted(
-            [p for p in prices if p.get("price_change_percentage_24h") is not None],
-            key=lambda x: x.get("price_change_percentage_24h", 0),
-            reverse=True
-        )[:10]
-        
-        return {
-            "trending": trending,
-            "count": len(trending),
-            "timestamp": datetime.now().isoformat(),
-            "source": source
-        }
-    except HTTPException:
-        raise
-    except Exception as e:
-        logger.error(f"Error in get_trending: {e}")
-        raise HTTPException(status_code=500, detail=str(e))
-
-
-@app.get("/api/market/prices")
-async def get_multiple_prices(symbols: str = Query("BTC,ETH,SOL", description="Comma-separated symbols")):
-    """Get prices for multiple cryptocurrencies"""
-    try:
-        symbol_list = [s.strip().upper() for s in symbols.split(",")]
-        
-        # Fetch prices
-        prices_data = []
-        source = "binance"
-        for symbol in symbol_list:
-            try:
-                ticker, ticker_source = await fetch_binance_ticker(f"{symbol}USDT")
-                if ticker:
-                    prices_data.append(ticker)
-                    if ticker_source != "binance":
-                        source = ticker_source
-            except:
-                continue
-        if not prices_data:
-            # Fallback to CoinGecko
-            prices_data, source = await fetch_coingecko_prices(symbol_list)
-        
-        if not prices_data:
-            fallback_prices = local_resource_service.get_prices_for_symbols(symbol_list)
-            if fallback_prices:
-                prices_data = fallback_prices
-                source = "local-fallback"
-        
-        return {
-            "symbols": symbol_list,
-            "count": len(prices_data),
-            "data": prices_data,
-            "source": source,
-            "timestamp": datetime.now().isoformat()
-        }
-    
-    except Exception as e:
-        logger.error(f"Error in get_multiple_prices: {e}")
-        raise HTTPException(status_code=500, detail=str(e))
-
-
-@app.get("/api/market-data/prices")
-async def get_market_data_prices(symbols: str = Query("BTC,ETH", description="Comma-separated symbols")):
-    """Alternative endpoint for market data prices"""
-    return await get_multiple_prices(symbols)
-
-
-# ============================================================================
-# Analysis Endpoints
-# ============================================================================
-
-@app.get("/api/analysis/signals")
-async def get_trading_signals(
-    symbol: str = Query("BTCUSDT", description="Trading pair"),
-    timeframe: str = Query("1h", description="Timeframe")
-):
-    """Get trading signals for a symbol"""
-    try:
-        # Fetch OHLCV data for analysis
-        ohlcv = await fetch_binance_ohlcv(symbol, timeframe, 100)
-        
-        if not ohlcv:
-            raise HTTPException(status_code=503, detail="Unable to fetch data for analysis")
-        
-        # Simple signal generation (can be enhanced)
-        latest = ohlcv[-1]
-        prev = ohlcv[-2] if len(ohlcv) > 1 else latest
-        
-        # Calculate simple indicators
-        close_prices = [c["close"] for c in ohlcv[-20:]]
-        sma_20 = sum(close_prices) / len(close_prices)
-        
-        # Generate signal
-        trend = "bullish" if latest["close"] > sma_20 else "bearish"
-        momentum = "strong" if abs(latest["close"] - prev["close"]) / prev["close"] > 0.01 else "weak"
-        
-        signal = "buy" if trend == "bullish" and momentum == "strong" else (
-            "sell" if trend == "bearish" and momentum == "strong" else "hold"
-        )
-
-        ai_summary = analyze_chart_points(symbol, timeframe, ohlcv)
-
-        return {
-            "symbol": symbol,
-            "timeframe": timeframe,
-            "signal": signal,
-            "trend": trend,
-            "momentum": momentum,
-            "indicators": {
-                "sma_20": sma_20,
-                "current_price": latest["close"],
-                "price_change": latest["close"] - prev["close"],
-                "price_change_percent": ((latest["close"] - prev["close"]) / prev["close"]) * 100
-            },
-            "analysis": ai_summary,
-            "timestamp": datetime.now().isoformat()
-        }
-    
-    except HTTPException:
-        raise
-    except Exception as e:
-        logger.error(f"Error in get_trading_signals: {e}")
-        raise HTTPException(status_code=500, detail=str(e))
-
-
-@app.get("/api/analysis/smc")
-async def get_smc_analysis(symbol: str = Query("BTCUSDT", description="Trading pair")):
-    """Get Smart Money Concepts (SMC) analysis"""
-    try:
-        # Fetch OHLCV data
-        ohlcv = await fetch_binance_ohlcv(symbol, "1h", 200)
-        
-        if not ohlcv:
-            raise HTTPException(status_code=503, detail="Unable to fetch data")
-        
-        # Calculate key levels
-        highs = [c["high"] for c in ohlcv]
-        lows = [c["low"] for c in ohlcv]
-        closes = [c["close"] for c in ohlcv]
-        
-        resistance = max(highs[-50:])
-        support = min(lows[-50:])
-        current_price = closes[-1]
-        
-        # Structure analysis
-        market_structure = "higher_highs" if closes[-1] > closes[-10] > closes[-20] else "lower_lows"
-        
-        return {
-            "symbol": symbol,
-            "market_structure": market_structure,
-            "key_levels": {
-                "resistance": resistance,
-                "support": support,
-                "current_price": current_price,
-                "mid_point": (resistance + support) / 2
-            },
-            "order_blocks": {
-                "bullish": support,
-                "bearish": resistance
-            },
-            "liquidity_zones": {
-                "above": resistance,
-                "below": support
-            },
-            "timestamp": datetime.now().isoformat()
-        }
-    
-    except HTTPException:
-        raise
-    except Exception as e:
-        logger.error(f"Error in get_smc_analysis: {e}")
-        raise HTTPException(status_code=500, detail=str(e))
-
-
-@app.get("/api/scoring/snapshot")
-async def get_scoring_snapshot(symbol: str = Query("BTCUSDT", description="Trading pair")):
-    """Get comprehensive scoring snapshot"""
-    try:
-        # Fetch data
-        ticker, _ = await fetch_binance_ticker(symbol)
-        ohlcv = await fetch_binance_ohlcv(symbol, "1h", 100)
-        
-        if not ticker or not ohlcv:
-            raise HTTPException(status_code=503, detail="Unable to fetch data")
-        
-        # Calculate scores (0-100)
-        volatility_score = min(abs(ticker["price_change_percent_24h"]) * 5, 100)
-        volume_score = min((ticker["volume_24h"] / 1000000) * 10, 100)
-        trend_score = 50 + (ticker["price_change_percent_24h"] * 2)
-        
-        # Overall score
-        overall_score = (volatility_score + volume_score + trend_score) / 3
-        
-        return {
-            "symbol": symbol,
-            "overall_score": round(overall_score, 2),
-            "scores": {
-                "volatility": round(volatility_score, 2),
-                "volume": round(volume_score, 2),
-                "trend": round(trend_score, 2),
-                "momentum": round(50 + ticker["price_change_percent_24h"], 2)
-            },
-            "rating": "excellent" if overall_score > 80 else (
-                "good" if overall_score > 60 else (
-                    "average" if overall_score > 40 else "poor"
-                )
-            ),
-            "timestamp": datetime.now().isoformat()
-        }
-    
-    except HTTPException:
-        raise
-    except Exception as e:
-        logger.error(f"Error in get_scoring_snapshot: {e}")
-        raise HTTPException(status_code=500, detail=str(e))
-
-
-@app.get("/api/signals")
-async def get_all_signals():
-    """Get signals for multiple assets"""
-    symbols = ["BTCUSDT", "ETHUSDT", "SOLUSDT", "BNBUSDT"]
-    signals = []
-    
-    for symbol in symbols:
-        try:
-            signal_data = await get_trading_signals(symbol, "1h")
-            signals.append(signal_data)
-        except:
-            continue
-    
-    return {
-        "count": len(signals),
-        "signals": signals,
-        "timestamp": datetime.now().isoformat()
-    }
-
-
-@app.get("/api/sentiment")
-async def get_sentiment():
-    """Get market sentiment data"""
-    try:
-        news = await news_collector.get_latest_news(limit=5)
-    except CollectorError as exc:
-        logger.warning("Sentiment fallback due to news error: %s", exc)
-        news = []
-
-    text = " ".join(item.get("title", "") for item in news).strip() or "Crypto market update"
-    analysis = analyze_market_text(text)
-    score = analysis.get("signals", {}).get("crypto", {}).get("score", 0.0)
-    normalized_value = int((score + 1) * 50)
-
-    if normalized_value < 20:
-        classification = "extreme_fear"
-    elif normalized_value < 40:
-        classification = "fear"
-    elif normalized_value < 60:
-        classification = "neutral"
-    elif normalized_value < 80:
-        classification = "greed"
-    else:
-        classification = "extreme_greed"
-
-    return {
-        "value": normalized_value,
-        "classification": classification,
-        "description": f"Market sentiment is {classification.replace('_', ' ')}",
-        "analysis": analysis,
-        "timestamp": datetime.utcnow().isoformat(),
-    }
-
-
-# ============================================================================
-# System Endpoints
-# ============================================================================
-
-@app.get("/api/system/status")
-async def get_system_status():
-    """Get system status"""
-    providers = await provider_collector.get_providers_status()
-    online = sum(1 for provider in providers if provider.get("status") == "online")
-
-    cache_items = (
-        len(getattr(market_collector.cache, "_store", {}))
-        + len(getattr(news_collector.cache, "_store", {}))
-        + len(getattr(provider_collector.cache, "_store", {}))
-    )
-
-    return {
-        "status": "operational" if online else "maintenance",
-        "uptime_seconds": round(time.time() - START_TIME, 2),
-        "cache_size": cache_items,
-        "providers_online": online,
-        "requests_per_minute": 0,
-        "timestamp": datetime.utcnow().isoformat(),
-    }
-
-
-@app.get("/api/system/config")
-async def get_system_config():
-    """Get system configuration"""
-    return {
-        "version": app.version,
-        "api_version": "v1",
-        "cache_ttl_seconds": settings.cache_ttl,
-        "supported_symbols": sorted(set(COIN_SYMBOL_MAPPING.values())),
-        "supported_intervals": ["1m", "5m", "15m", "30m", "1h", "4h", "1d"],
-        "max_ohlcv_limit": 1000,
-        "timestamp": datetime.utcnow().isoformat(),
-    }
-
-
-@app.get("/api/categories")
-async def get_categories():
-    """Get data categories"""
-    return {
-        "categories": [
-            {"name": "market_data", "endpoints": 5, "status": "active"},
-            {"name": "analysis", "endpoints": 4, "status": "active"},
-            {"name": "signals", "endpoints": 2, "status": "active"},
-            {"name": "sentiment", "endpoints": 1, "status": "active"}
-        ]
-    }
-
-
-@app.get("/api/rate-limits")
-async def get_rate_limits():
-    """Get rate limit information"""
-    return {
-        "rate_limits": [
-            {"endpoint": "/api/ohlcv", "limit": 1200, "window": "per_minute"},
-            {"endpoint": "/api/crypto/prices/top", "limit": 600, "window": "per_minute"},
-            {"endpoint": "/api/analysis/*", "limit": 300, "window": "per_minute"}
-        ],
-        "current_usage": {
-            "requests_this_minute": 0,
-            "percentage": 0
-        }
-    }
-
-
-@app.get("/api/logs")
-async def get_logs(limit: int = Query(50, ge=1, le=500)):
-    """Get recent API logs"""
-    # Mock logs (can be enhanced with real logging)
-    logs = []
-    for i in range(min(limit, 10)):
-        logs.append({
-            "timestamp": (datetime.now() - timedelta(minutes=i)).isoformat(),
-            "endpoint": "/api/ohlcv",
-            "status": "success",
-            "response_time_ms": random.randint(50, 200)
-        })
-    
-    return {"logs": logs, "count": len(logs)}
-
-
-@app.get("/api/alerts")
-async def get_alerts():
-    """Get system alerts"""
-    return {
-        "alerts": [],
-        "count": 0,
-        "timestamp": datetime.now().isoformat()
-    }
-
-
-# ============================================================================
-# HuggingFace Integration Endpoints
-# ============================================================================
-
-@app.get("/api/hf/health")
-async def hf_health():
-    """HuggingFace integration health"""
-    from ai_models import AI_MODELS_SUMMARY
-    status = registry_status()
-    status["models"] = AI_MODELS_SUMMARY
-    status["timestamp"] = datetime.utcnow().isoformat()
-    return status
-
-
-@app.post("/api/hf/refresh")
-async def hf_refresh():
-    """Refresh HuggingFace data"""
-    from ai_models import initialize_models
-    result = initialize_models()
-    return {"status": "ok" if result.get("models_loaded", 0) > 0 else "degraded", **result, "timestamp": datetime.utcnow().isoformat()}
-
-
-@app.get("/api/hf/registry")
-async def hf_registry(kind: str = "models"):
-    """Get HuggingFace registry"""
-    info = get_model_info()
-    return {"kind": kind, "items": info.get("model_names", info)}
-
-
-@app.get("/api/resources/unified")
-async def get_unified_resources():
-    """Get unified API resources from crypto_resources_unified_2025-11-11.json"""
-    try:
-        data = local_resource_service.get_registry()
-        if data:
-            metadata = data.get("registry", {}).get("metadata", {})
-            return {
-                "success": True,
-                "data": data,
-                "metadata": metadata,
-                "count": metadata.get("total_entries", 0),
-                "fallback_assets": len(local_resource_service.get_supported_symbols())
-            }
-        return {"success": False, "error": "Resources file not found"}
-    except Exception as e:
-        logger.error(f"Error loading unified resources: {e}")
-        return {"success": False, "error": str(e)}
-
-
-@app.get("/api/resources/ultimate")
-async def get_ultimate_resources():
-    """Get ultimate API resources from ultimate_crypto_pipeline_2025_NZasinich.json"""
-    try:
-        resources_path = WORKSPACE_ROOT / "api-resources" / "ultimate_crypto_pipeline_2025_NZasinich.json"
-        if resources_path.exists():
-            with open(resources_path, 'r', encoding='utf-8') as f:
-                data = json.load(f)
-            return {
-                "success": True,
-                "data": data,
-                "total_sources": data.get("total_sources", 0),
-                "files": len(data.get("files", []))
-            }
-        return {"success": False, "error": "Resources file not found"}
-    except Exception as e:
-        logger.error(f"Error loading ultimate resources: {e}")
-        return {"success": False, "error": str(e)}
-
-
-@app.get("/api/resources/stats")
-async def get_resources_stats():
-    """Get statistics about available API resources"""
-    try:
-        stats = {
-            "unified": {"available": False, "count": 0},
-            "ultimate": {"available": False, "count": 0},
-            "total_apis": 0
-        }
-        
-        # Check unified resources via the centralized loader
-        registry = local_resource_service.get_registry()
-        if registry:
-            stats["unified"] = {
-                "available": True,
-                "count": registry.get("registry", {}).get("metadata", {}).get("total_entries", 0),
-                "fallback_assets": len(local_resource_service.get_supported_symbols())
-            }
-        
-        # Check ultimate resources
-        ultimate_path = WORKSPACE_ROOT / "api-resources" / "ultimate_crypto_pipeline_2025_NZasinich.json"
-        if ultimate_path.exists():
-            with open(ultimate_path, 'r', encoding='utf-8') as f:
-                ultimate_data = json.load(f)
-                stats["ultimate"] = {
-                    "available": True,
-                    "count": ultimate_data.get("total_sources", 0)
-                }
-        
-        stats["total_apis"] = stats["unified"].get("count", 0) + stats["ultimate"].get("count", 0)
-        
-        return {"success": True, "stats": stats}
-    except Exception as e:
-        logger.error(f"Error getting resources stats: {e}")
-        return {"success": False, "error": str(e)}
-
-
-def _resolve_sentiment_payload(payload: Union[List[str], Dict[str, Any]]) -> Dict[str, Any]:
-    if isinstance(payload, list):
-        return {"texts": payload, "mode": "auto"}
-    if isinstance(payload, dict):
-        texts = payload.get("texts") or payload.get("text")
-        if isinstance(texts, str):
-            texts = [texts]
-        if not isinstance(texts, list):
-            raise ValueError("texts must be provided")
-        mode = payload.get("mode") or payload.get("model") or "auto"
-        return {"texts": texts, "mode": mode}
-    raise ValueError("Invalid payload")
-
-
-@app.post("/api/hf/run-sentiment")
-@app.post("/api/hf/sentiment")
-async def hf_sentiment(payload: Union[List[str], Dict[str, Any]] = Body(...)):
-    """Run sentiment analysis using shared AI helpers."""
-    from ai_models import AI_MODELS_SUMMARY
-    
-    if AI_MODELS_SUMMARY.get("models_loaded", 0) == 0 or AI_MODELS_SUMMARY.get("mode") == "off":
-        return {
-            "ok": False,
-            "error": "No HF models are currently loaded.",
-            "mode": AI_MODELS_SUMMARY.get("mode", "off"),
-            "models_loaded": AI_MODELS_SUMMARY.get("models_loaded", 0)
-        }
-
-    try:
-        resolved = _resolve_sentiment_payload(payload)
-    except ValueError as exc:
-        raise HTTPException(status_code=400, detail=str(exc)) from exc
-
-    mode = (resolved.get("mode") or "auto").lower()
-    texts = resolved["texts"]
-    results: List[Dict[str, Any]] = []
-    for text in texts:
-        if mode == "crypto":
-            analysis = analyze_crypto_sentiment(text)
-        elif mode == "financial":
-            analysis = analyze_market_text(text).get("signals", {}).get("financial", {})
-        elif mode == "social":
-            analysis = analyze_market_text(text).get("signals", {}).get("social", {})
-        else:
-            analysis = analyze_market_text(text)
-        results.append({"text": text, "result": analysis})
-
-    return {"mode": mode, "results": results, "timestamp": datetime.utcnow().isoformat()}
-
-
-@app.post("/api/hf/models/sentiment")
-async def hf_models_sentiment(payload: Union[List[str], Dict[str, Any]] = Body(...)):
-    """Compatibility endpoint for HF console sentiment panel."""
-    from ai_models import AI_MODELS_SUMMARY
-    
-    if AI_MODELS_SUMMARY.get("models_loaded", 0) == 0 or AI_MODELS_SUMMARY.get("mode") == "off":
-        return {
-            "ok": False,
-            "error": "No HF models are currently loaded.",
-            "mode": AI_MODELS_SUMMARY.get("mode", "off"),
-            "models_loaded": AI_MODELS_SUMMARY.get("models_loaded", 0)
-        }
-    
-    return await hf_sentiment(payload)
-
-
-@app.post("/api/hf/models/forecast")
-async def hf_models_forecast(payload: Dict[str, Any] = Body(...)):
-    """Generate quick technical forecasts from provided closing prices."""
-    series = payload.get("series") or payload.get("values") or payload.get("close")
-    if not isinstance(series, list) or len(series) < 3:
-        raise HTTPException(status_code=400, detail="Provide at least 3 closing prices in 'series'.")
-
-    try:
-        floats = [float(x) for x in series]
-    except (TypeError, ValueError) as exc:
-        raise HTTPException(status_code=400, detail="Series must contain numeric values") from exc
-
-    model_name = (payload.get("model") or payload.get("model_name") or "btc_lstm").lower()
-    steps = int(payload.get("steps") or 3)
-
-    deltas = [floats[i] - floats[i - 1] for i in range(1, len(floats))]
-    avg_delta = mean(deltas)
-    volatility = mean(abs(delta - avg_delta) for delta in deltas) if deltas else 0
-
-    predictions = []
-    last = floats[-1]
-    decay = 0.95 if model_name == "btc_arima" else 1.02
-    for _ in range(steps):
-        last = last + (avg_delta * decay)
-        predictions.append(round(last, 4))
-
-    return {
-        "model": model_name,
-        "steps": steps,
-        "input_count": len(floats),
-        "volatility": round(volatility, 5),
-        "predictions": predictions,
-        "source": "local-fallback" if model_name == "btc_arima" else "hybrid",
-        "timestamp": datetime.utcnow().isoformat()
-    }
-
-
-@app.get("/api/hf/datasets/market/ohlcv")
-async def hf_dataset_market_ohlcv(symbol: str = Query("BTC"), interval: str = Query("1h"), limit: int = Query(120, ge=10, le=500)):
-    """Expose fallback OHLCV snapshots as a pseudo HF dataset slice."""
-    data = local_resource_service.get_ohlcv(symbol.upper(), interval, limit)
-    source = "local-fallback"
-
-    if not data:
-        return {
-            "symbol": symbol.upper(),
-            "interval": interval,
-            "count": 0,
-            "data": [],
-            "source": source,
-            "message": "No cached OHLCV available yet"
-        }
-
-    return {
-        "symbol": symbol.upper(),
-        "interval": interval,
-        "count": len(data),
-        "data": data,
-        "source": source,
-        "timestamp": datetime.utcnow().isoformat()
-    }
-
-
-@app.get("/api/hf/datasets/market/btc_technical")
-async def hf_dataset_market_btc(limit: int = Query(50, ge=10, le=200)):
-    """Simplified technical metrics derived from fallback OHLCV data."""
-    candles = local_resource_service.get_ohlcv("BTC", "1h", limit + 20)
-
-    if not candles:
-        raise HTTPException(status_code=503, detail="Fallback OHLCV unavailable")
-
-    rows = []
-    closes = [c["close"] for c in candles]
-    for idx, candle in enumerate(candles[-limit:]):
-        window = closes[max(0, idx): idx + 20]
-        sma = sum(window) / len(window) if window else candle["close"]
-        momentum = candle["close"] - candle["open"]
-        rows.append({
-            "timestamp": candle["timestamp"],
-            "datetime": candle["datetime"],
-            "close": candle["close"],
-            "sma_20": round(sma, 4),
-            "momentum": round(momentum, 4),
-            "volatility": round((candle["high"] - candle["low"]) / candle["low"], 4)
-        })
-
-    return {
-        "symbol": "BTC",
-        "interval": "1h",
-        "count": len(rows),
-        "items": rows,
-        "source": "local-fallback"
-    }
-
-
-@app.get("/api/hf/datasets/news/semantic")
-async def hf_dataset_news(limit: int = Query(10, ge=3, le=25)):
-    """News slice augmented with sentiment tags for HF demos."""
-    try:
-        news = await news_collector.get_latest_news(limit=limit)
-        source = "providers"
-    except CollectorError:
-        news = []
-        source = "local-fallback"
-
-    if not news:
-        items = HF_SAMPLE_NEWS[:limit]
-    else:
-        items = []
-        for item in news:
-            items.append({
-                "title": item.get("title"),
-                "source": item.get("source") or item.get("provider"),
-                "sentiment": item.get("sentiment") or "neutral",
-                "sentiment_score": item.get("sentiment_confidence", 0.5),
-                "entities": item.get("symbols") or [],
-                "summary": item.get("summary") or item.get("description"),
-                "published_at": item.get("date") or item.get("published_at")
-            })
-    return {
-        "count": len(items),
-        "items": items,
-        "source": source,
-        "timestamp": datetime.utcnow().isoformat()
-    }
-
-
-# ============================================================================
-# HTML Routes - Serve UI files
-# ============================================================================
-
-@app.get("/favicon.ico")
-async def favicon():
-    """Serve favicon"""
-    favicon_path = WORKSPACE_ROOT / "static" / "favicon.ico"
-    if favicon_path.exists():
-        return FileResponse(favicon_path)
-    return JSONResponse({"status": "no favicon"}, status_code=404)
-
-@app.get("/", response_class=HTMLResponse)
-async def root():
-    """Serve main HTML UI page (index.html)"""
-    index_path = WORKSPACE_ROOT / "index.html"
-    if index_path.exists():
-        return FileResponse(
-            path=str(index_path),
-            media_type="text/html",
-            filename="index.html"
-        )
-    return HTMLResponse("<h1>Cryptocurrency Data & Analysis API</h1><p>See <a href='/docs'>/docs</a> for API documentation</p>")
-
-@app.get("/index.html", response_class=HTMLResponse)
-async def index():
-    """Serve index.html"""
-    return FileResponse(WORKSPACE_ROOT / "index.html")
-
-@app.get("/dashboard.html", response_class=HTMLResponse)
-async def dashboard():
-    """Serve dashboard.html"""
-    return FileResponse(WORKSPACE_ROOT / "dashboard.html")
-
-@app.get("/dashboard", response_class=HTMLResponse)
-async def dashboard_alt():
-    """Alternative route for dashboard"""
-    return FileResponse(WORKSPACE_ROOT / "dashboard.html")
-
-@app.get("/admin.html", response_class=HTMLResponse)
-async def admin():
-    """Serve admin panel"""
-    admin_path = WORKSPACE_ROOT / "admin.html"
-    if admin_path.exists():
-        return FileResponse(
-            path=str(admin_path),
-            media_type="text/html",
-            filename="admin.html"
-        )
-    return HTMLResponse("<h1>Admin panel not found</h1>")
-
-@app.get("/admin", response_class=HTMLResponse)
-async def admin_alt():
-    """Alternative route for admin"""
-    admin_path = WORKSPACE_ROOT / "admin.html"
-    if admin_path.exists():
-        return FileResponse(
-            path=str(admin_path),
-            media_type="text/html",
-            filename="admin.html"
-        )
-    return HTMLResponse("<h1>Admin panel not found</h1>")
-
-@app.get("/hf_console.html", response_class=HTMLResponse)
-async def hf_console():
-    """Serve HuggingFace console"""
-    return FileResponse(WORKSPACE_ROOT / "hf_console.html")
-
-@app.get("/console", response_class=HTMLResponse)
-async def console_alt():
-    """Alternative route for HF console"""
-    return FileResponse(WORKSPACE_ROOT / "hf_console.html")
-
-@app.get("/pool_management.html", response_class=HTMLResponse)
-async def pool_management():
-    """Serve pool management UI"""
-    return FileResponse(WORKSPACE_ROOT / "pool_management.html")
-
-@app.get("/unified_dashboard.html", response_class=HTMLResponse)
-async def unified_dashboard():
-    """Serve unified dashboard"""
-    return FileResponse(WORKSPACE_ROOT / "unified_dashboard.html")
-
-@app.get("/simple_overview.html", response_class=HTMLResponse)
-async def simple_overview():
-    """Serve simple overview"""
-    return FileResponse(WORKSPACE_ROOT / "simple_overview.html")
-
-# Generic HTML file handler
-@app.get("/{filename}.html", response_class=HTMLResponse)
-async def serve_html(filename: str):
-    """Serve any HTML file from workspace root"""
-    file_path = WORKSPACE_ROOT / f"{filename}.html"
-    if file_path.exists():
-        return FileResponse(file_path)
-    return HTMLResponse(f"<h1>File {filename}.html not found</h1>", status_code=404)
-
-
-# ============================================================================
-# Startup Event
-# ============================================================================
-
-
-# ============================================================================
-# ADMIN DASHBOARD ENDPOINTS
-# ============================================================================
-
-from fastapi import WebSocket, WebSocketDisconnect
-import asyncio
-
-class ConnectionManager:
-    def __init__(self):
-        self.active_connections = []
-    async def connect(self, websocket: WebSocket):
-        await websocket.accept()
-        self.active_connections.append(websocket)
-    def disconnect(self, websocket: WebSocket):
-        if websocket in self.active_connections:
-            self.active_connections.remove(websocket)
-    async def broadcast(self, message: dict):
-        disconnected = []
-        for conn in list(self.active_connections):
-            try:
-                # Check connection state before sending
-                if conn.client_state == WebSocketState.CONNECTED:
-                    await conn.send_json(message)
-                else:
-                    disconnected.append(conn)
-            except Exception as e:
-                logger.debug(f"Error broadcasting to client: {e}")
-                disconnected.append(conn)
-        
-        # Clean up disconnected clients
-        for conn in disconnected:
-            self.disconnect(conn)
-
-ws_manager = ConnectionManager()
-
-@app.get("/api/health")
-async def api_health():
-    h = await health()
-    return {"status": "healthy" if h.get("status") == "ok" else "degraded", **h}
-
-# Removed duplicate - using improved version below
-
-@app.get("/api/coins/{symbol}")
-async def get_coin_detail(symbol: str):
-    coins = await market_collector.get_top_coins(limit=250)
-    coin = next((c for c in coins if c.get("symbol", "").upper() == symbol.upper()), None)
-    if not coin:
-        raise HTTPException(404, f"Coin {symbol} not found")
-    return {"success": True, "symbol": symbol.upper(), "name": coin.get("name", ""), 
-            "price": coin.get("price") or coin.get("current_price", 0),
-            "change_24h": coin.get("change_24h") or coin.get("price_change_percentage_24h", 0),
-            "market_cap": coin.get("market_cap", 0)}
-
-@app.get("/api/market/stats")
-async def get_market_stats():
-    """Get global market statistics (duplicate endpoint - keeping for compatibility)"""
-    try:
-        overview = await get_market_overview()
-        
-        # Calculate ETH dominance from prices if available
-        eth_dominance = 0
-        if overview.get("total_market_cap", 0) > 0:
-            try:
-                eth_prices, _ = await fetch_coingecko_prices(symbols=["ETH"], limit=1)
-                if eth_prices and len(eth_prices) > 0:
-                    eth_market_cap = eth_prices[0].get("market_cap", 0) or 0
-                    eth_dominance = (eth_market_cap / overview.get("total_market_cap", 1)) * 100
-            except:
-                pass
-        
-        return {
-            "success": True,
-            "stats": {
-                "total_market_cap": overview.get("total_market_cap", 0) or 0,
-                "total_volume_24h": overview.get("total_volume_24h", 0) or 0,
-                "btc_dominance": overview.get("btc_dominance", 0) or 0,
-                "eth_dominance": eth_dominance,
-                "active_cryptocurrencies": 10000,
-                "markets": 500,
-                "market_cap_change_24h": 0.0,
-                "timestamp": datetime.now().isoformat()
-            }
-        }
-    except Exception as e:
-        logger.error(f"Error in /api/market/stats (duplicate): {e}")
-        return {
-            "success": True,
-            "stats": {
-                "total_market_cap": 0,
-                "total_volume_24h": 0,
-                "btc_dominance": 0,
-                "eth_dominance": 0,
-                "active_cryptocurrencies": 0,
-                "markets": 0,
-                "market_cap_change_24h": 0.0,
-                "timestamp": datetime.now().isoformat()
-            }
-        }
-
-
-@app.get("/api/stats")
-async def get_stats_alias():
-    """Alias endpoint for /api/market/stats - backward compatibility"""
-    return await get_market_stats()
-
-
-@app.get("/api/news/latest")
-async def get_latest_news(limit: int = Query(default=40, ge=1, le=100)):
-    from ai_models import analyze_news_item
-    news = await news_collector.get_latest_news(limit=limit)
-    enriched = []
-    for item in news[:limit]:
-        try:
-            e = analyze_news_item(item)
-            enriched.append({"title": e.get("title", ""), "source": e.get("source", ""),
-                           "published_at": e.get("published_at") or e.get("date", ""),
-                           "symbols": e.get("symbols", []), "sentiment": e.get("sentiment", "neutral"),
-                           "sentiment_confidence": e.get("sentiment_confidence", 0.5)})
-        except:
-            enriched.append({"title": item.get("title", ""), "source": item.get("source", ""),
-                           "published_at": item.get("date", ""), "symbols": item.get("symbols", []),
-                           "sentiment": "neutral", "sentiment_confidence": 0.5})
-    return {"success": True, "news": enriched, "count": len(enriched)}
-
-@app.post("/api/news/summarize")
-async def summarize_news(item: Dict[str, Any] = Body(...)):
-    from ai_models import analyze_news_item
-    e = analyze_news_item(item)
-    return {"success": True, "summary": e.get("title", ""), "sentiment": e.get("sentiment", "neutral")}
-
-# Duplicate endpoints removed - using the improved versions below in CHARTS ENDPOINTS section
-
-@app.post("/api/sentiment/analyze")
-async def analyze_sentiment(payload: Dict[str, Any] = Body(...)):
-    from ai_models import ensemble_crypto_sentiment
-    result = ensemble_crypto_sentiment(payload.get("text", ""))
-    return {"success": True, "sentiment": result["label"], "confidence": result["confidence"], "details": result}
-
-@app.post("/api/query")
-async def process_query(payload: Dict[str, Any] = Body(...)):
-    query = payload.get("query", "").lower()
-    if "price" in query or "btc" in query:
-        coins = await market_collector.get_top_coins(limit=10)
-        btc = next((c for c in coins if c.get("symbol", "").upper() == "BTC"), None)
-        if btc:
-            return {"success": True, "type": "price", "message": f"Bitcoin is ${btc.get('price', 0):,.2f}", "data": btc}
-    return {"success": True, "type": "general", "message": "Query processed"}
-
-@app.get("/api/datasets/list")
-async def list_datasets():
-    from backend.services.hf_registry import REGISTRY
-    datasets = REGISTRY.list(kind="datasets")
-    formatted = [{"name": d.get("id"), "category": d.get("category", "other"), "tags": d.get("tags", [])} for d in datasets]
-    return {"success": True, "datasets": formatted, "count": len(formatted)}
-
-@app.get("/api/datasets/sample")
-async def get_dataset_sample(name: str = Query(...), limit: int = Query(default=20)):
-    return {"success": False, "name": name, "sample": [], "message": "Auth required"}
-
-@app.get("/api/models/list")
-async def list_models():
-    from ai_models import get_model_info
-    info = get_model_info()
-    models = []
-    for cat, mlist in info.get("model_catalog", {}).items():
-        for mid in mlist:
-            models.append({"name": mid, "task": "sentiment" if "sentiment" in cat else "analysis", "category": cat})
-    return {"success": True, "models": models, "count": len(models)}
-
-@app.post("/api/models/test")
-async def test_model(payload: Dict[str, Any] = Body(...)):
-    from ai_models import ensemble_crypto_sentiment
-    result = ensemble_crypto_sentiment(payload.get("text", ""))
-    return {"success": True, "model": payload.get("model", ""), "result": result}
-
-@app.websocket("/ws")
-async def websocket_endpoint(websocket: WebSocket):
-    await ws_manager.connect(websocket)
-    try:
-        while True:
-            # Check if connection is still open before sending
-            if websocket.client_state != WebSocketState.CONNECTED:
-                logger.info("WebSocket connection closed, breaking loop")
-                break
-                
-            try:
-                top_coins = await market_collector.get_top_coins(limit=5)
-                news = await news_collector.get_latest_news(limit=3)
-                from ai_models import ensemble_crypto_sentiment
-                sentiment = ensemble_crypto_sentiment(" ".join([n.get("title", "") for n in news])) if news else {"label": "neutral", "confidence": 0.5}
-                
-                # Double-check connection state before sending
-                if websocket.client_state == WebSocketState.CONNECTED:
-                    await websocket.send_json({
-                        "type": "update", 
-                        "payload": {
-                            "market_data": top_coins, 
-                            "news": news, 
-                            "sentiment": sentiment, 
-                            "timestamp": datetime.now().isoformat()
-                        }
-                    })
-                else:
-                    logger.info("WebSocket disconnected, breaking loop")
-                    break
-                    
-            except CollectorError as e:
-                # Provider errors are already logged by the collector, just continue
-                logger.debug(f"Provider error in WebSocket update (this is expected with fallbacks): {e}")
-                # Use cached data if available, or empty data
-                top_coins = []
-                news = []
-                sentiment = {"label": "neutral", "confidence": 0.5}
-            except Exception as e:
-                # Log other errors with full details
-                error_msg = str(e) if str(e) else repr(e)
-                logger.error(f"Error in WebSocket update loop: {type(e).__name__}: {error_msg}")
-                # Don't break on data errors, just log and continue
-                # Only break on connection errors
-                if "send" in str(e).lower() or "close" in str(e).lower():
-                    break
-                    
-            await asyncio.sleep(10)
-    except WebSocketDisconnect:
-        logger.info("WebSocket disconnect exception caught")
-    except Exception as e:
-        logger.error(f"WebSocket endpoint error: {e}")
-    finally:
-        try:
-            ws_manager.disconnect(websocket)
-        except:
-            pass
-
-
-@app.on_event("startup")
-async def startup_event():
-    """Initialize on startup - non-blocking"""
-    logger.info("=" * 70)
-    logger.info("Starting Cryptocurrency Data & Analysis API")
-    logger.info("=" * 70)
-    logger.info("FastAPI initialized")
-    logger.info("CORS configured")
-    logger.info("Cache initialized")
-    logger.info(f"Providers loaded: {len(PROVIDERS_CONFIG)}")
-    
-    # Initialize AI models in background (non-blocking)
-    async def init_models_background():
-        try:
-            from ai_models import initialize_models
-            models_init = initialize_models()
-            logger.info(f"AI Models initialized: {models_init}")
-        except Exception as e:
-            logger.warning(f"AI Models initialization failed: {e}")
-    
-    # Initialize HF Registry in background (non-blocking)
-    async def init_registry_background():
-        try:
-            from backend.services.hf_registry import REGISTRY
-            registry_result = await REGISTRY.refresh()
-            logger.info(f"HF Registry initialized: {registry_result}")
-        except Exception as e:
-            logger.warning(f"HF Registry initialization failed: {e}")
-    
-    # Start background tasks
-    asyncio.create_task(init_models_background())
-    asyncio.create_task(init_registry_background())
-    logger.info("Background initialization tasks started")
-    
-    # Show loaded HuggingFace Space providers
-    hf_providers = [p for p in PROVIDERS_CONFIG.keys() if 'huggingface_space' in p]
-    if hf_providers:
-        logger.info(f"HuggingFace Space providers: {', '.join(hf_providers)}")
-    
-    logger.info("Data sources: Binance, CoinGecko, providers_config_extended.json")
-    
-    # Check HTML files
-    html_files = ["index.html", "dashboard.html", "admin.html", "hf_console.html"]
-    available_html = [f for f in html_files if (WORKSPACE_ROOT / f).exists()]
-    logger.info(f"UI files: {len(available_html)}/{len(html_files)} available")
-    logger.info(f"HTML UI available at: http://0.0.0.0:7860/ (index.html)")
-    
-    logger.info("=" * 70)
-    logger.info("API ready at http://0.0.0.0:7860")
-    logger.info("Docs at http://0.0.0.0:7860/docs")
-    logger.info("UI at http://0.0.0.0:7860/ (index.html - default HTML page)")
-    logger.info("=" * 70)
-
-
-# ============================================================================
-# Main Entry Point
-# ============================================================================
-
-if __name__ == "__main__":
-    import uvicorn
-    import sys
-    import io
-    
-    # Fix encoding for Windows console
-    if sys.platform == "win32":
-        sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8', errors='replace')
-        sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8', errors='replace')
-    
-    try:
-        print("=" * 70)
-        print("Starting Cryptocurrency Data & Analysis API")
-        print("=" * 70)
-        print("Server: http://localhost:7860")
-        print("API Docs: http://localhost:7860/docs")
-        print("Health: http://localhost:7860/health")
-        print("=" * 70)
-    except UnicodeEncodeError:
-        # Fallback if encoding still fails
-        print("=" * 70)
-        print("Starting Cryptocurrency Data & Analysis API")
-        print("=" * 70)
-        print("Server: http://localhost:7860")
-        print("API Docs: http://localhost:7860/docs")
-        print("Health: http://localhost:7860/health")
-        print("=" * 70)
-    
-    uvicorn.run(
-        app,
-        host="0.0.0.0",
-        port=7860,
-        log_level="info"
-    )
-# NEW ENDPOINTS FOR ADMIN.HTML - ADD TO hf_unified_server.py
-
-from fastapi import WebSocket, WebSocketDisconnect
-from collections import defaultdict
-
-# WebSocket Manager
-class ConnectionManager:
-    def __init__(self):
-        self.active_connections: List[WebSocket] = []
-
-    async def connect(self, websocket: WebSocket):
-        await websocket.accept()
-        self.active_connections.append(websocket)
-        logger.info(f"WebSocket connected. Total: {len(self.active_connections)}")
-
-    def disconnect(self, websocket: WebSocket):
-        if websocket in self.active_connections:
-            self.active_connections.remove(websocket)
-        logger.info(f"WebSocket disconnected. Total: {len(self.active_connections)}")
-
-    async def broadcast(self, message: dict):
-        disconnected = []
-        for connection in list(self.active_connections):
-            try:
-                # Check connection state before sending
-                if connection.client_state == WebSocketState.CONNECTED:
-                    await connection.send_json(message)
-                else:
-                    disconnected.append(connection)
-            except Exception as e:
-                logger.debug(f"Error broadcasting to client: {e}")
-                disconnected.append(connection)
-        
-        # Clean up disconnected clients
-        for connection in disconnected:
-            self.disconnect(connection)
-
-ws_manager = ConnectionManager()
-
-
-# ===== API HEALTH =====
-@app.get("/api/health")
-async def api_health():
-    """Health check for admin dashboard"""
-    health_data = await health()
-    return {
-        "status": "healthy" if health_data.get("status") == "ok" else "degraded",
-        **health_data
-    }
-
-
-# ===== COINS ENDPOINTS =====
-@app.get("/api/coins/top")
-async def get_top_coins(limit: int = Query(default=10, ge=1, le=100)):
-    """Get top cryptocurrencies by market cap"""
-    try:
-        coins = await market_collector.get_top_coins(limit=limit)
-        
-        result = []
-        for coin in coins:
-            result.append({
-                "id": coin.get("id", coin.get("symbol", "").lower()),
-                "rank": coin.get("rank", 0),
-                "symbol": coin.get("symbol", "").upper(),
-                "name": coin.get("name", ""),
-                "price": coin.get("price") or coin.get("current_price", 0),
-                "current_price": coin.get("price") or coin.get("current_price", 0),
-                "price_change_24h": coin.get("change_24h") or coin.get("price_change_percentage_24h", 0),
-                "price_change_percentage_24h": coin.get("change_24h") or coin.get("price_change_percentage_24h", 0),
-                "price_change_percentage_7d_in_currency": coin.get("price_change_percentage_7d", 0),
-                "volume_24h": coin.get("volume_24h") or coin.get("total_volume", 0),
-                "total_volume": coin.get("volume_24h") or coin.get("total_volume", 0),
-                "market_cap": coin.get("market_cap", 0),
-                "image": coin.get("image", ""),
-                "sparkline_in_7d": coin.get("sparkline_in_7d") or {"price": []},
-                "sparkline_data": coin.get("sparkline_data") or [],
-                "last_updated": coin.get("last_updated", datetime.now().isoformat())
-            })
-        
-        return {
-            "success": True,
-            "coins": result,
-            "count": len(result),
-            "timestamp": datetime.now().isoformat()
-        }
-    except Exception as e:
-        logger.error(f"Error in /api/coins/top: {e}")
-        raise HTTPException(status_code=503, detail=str(e))
-
-
-@app.get("/api/coins/{symbol}")
-async def get_coin_detail(symbol: str):
-    """Get specific coin details"""
-    try:
-        coins = await market_collector.get_top_coins(limit=250)
-        coin = next((c for c in coins if c.get("symbol", "").upper() == symbol.upper()), None)
-        
-        if not coin:
-            raise HTTPException(status_code=404, detail=f"Coin {symbol} not found")
-        
-        return {
-            "success": True,
-            "symbol": symbol.upper(),
-            "name": coin.get("name", ""),
-            "price": coin.get("price") or coin.get("current_price", 0),
-            "change_24h": coin.get("change_24h") or coin.get("price_change_percentage_24h", 0),
-            "volume_24h": coin.get("volume_24h") or coin.get("total_volume", 0),
-            "market_cap": coin.get("market_cap", 0),
-            "rank": coin.get("rank", 0),
-            "last_updated": coin.get("last_updated", datetime.now().isoformat())
-        }
-    except HTTPException:
-        raise
-    except Exception as e:
-        logger.error(f"Error in /api/coins/{symbol}: {e}")
-        raise HTTPException(status_code=503, detail=str(e))
-
-
-# ===== MARKET STATS =====
-@app.get("/api/market/stats")
-async def get_market_stats():
-    """Get global market statistics"""
-    try:
-        # Use existing endpoint - get_market_overview returns total_market_cap and total_volume_24h
-        overview = await get_market_overview()
-        
-        # Calculate ETH dominance from prices if available
-        eth_dominance = 0
-        if overview.get("total_market_cap", 0) > 0:
-            # Try to get ETH market cap from top coins
-            try:
-                eth_prices, _ = await fetch_coingecko_prices(symbols=["ETH"], limit=1)
-                if eth_prices and len(eth_prices) > 0:
-                    eth_market_cap = eth_prices[0].get("market_cap", 0) or 0
-                    eth_dominance = (eth_market_cap / overview.get("total_market_cap", 1)) * 100
-            except:
-                pass
-        
-        stats = {
-            "total_market_cap": overview.get("total_market_cap", 0) or 0,
-            "total_volume_24h": overview.get("total_volume_24h", 0) or 0,
-            "btc_dominance": overview.get("btc_dominance", 0) or 0,
-            "eth_dominance": eth_dominance,
-            "active_cryptocurrencies": 10000,  # Approximate
-            "markets": 500,  # Approximate
-            "market_cap_change_24h": 0.0,
-            "timestamp": datetime.now().isoformat()
-        }
-        
-        return {"success": True, "stats": stats}
-    except Exception as e:
-        logger.error(f"Error in /api/market/stats: {e}")
-        raise HTTPException(status_code=503, detail=str(e))
-
-
-# ===== NEWS ENDPOINTS =====
-@app.get("/api/news/latest")
-async def get_latest_news(limit: int = Query(default=40, ge=1, le=100)):
-    """Get latest crypto news with sentiment"""
-    try:
-        news_items = await news_collector.get_latest_news(limit=limit)
-        
-        # Attach sentiment to each news item
-        from ai_models import analyze_news_item
-        enriched_news = []
-        for item in news_items:
-            try:
-                enriched = analyze_news_item(item)
-                enriched_news.append({
-                    "title": enriched.get("title", ""),
-                    "source": enriched.get("source", ""),
-                    "published_at": enriched.get("published_at") or enriched.get("date", ""),
-                    "symbols": enriched.get("symbols", []),
-                    "sentiment": enriched.get("sentiment", "neutral"),
-                    "sentiment_confidence": enriched.get("sentiment_confidence", 0.5),
-                    "url": enriched.get("url", "")
-                })
-            except:
-                enriched_news.append({
-                    "title": item.get("title", ""),
-                    "source": item.get("source", ""),
-                    "published_at": item.get("published_at") or item.get("date", ""),
-                    "symbols": item.get("symbols", []),
-                    "sentiment": "neutral",
-                    "sentiment_confidence": 0.5,
-                    "url": item.get("url", "")
-                })
-        
-        return {
-            "success": True,
-            "news": enriched_news,
-            "count": len(enriched_news),
-            "timestamp": datetime.now().isoformat()
-        }
-    except Exception as e:
-        logger.error(f"Error in /api/news/latest: {e}")
-        return {"success": True, "news": [], "count": 0, "timestamp": datetime.now().isoformat()}
-
-
-@app.get("/api/news")
-async def get_news(limit: int = Query(default=40, ge=1, le=100)):
-    """Alias for /api/news/latest for backward compatibility"""
-    return await get_latest_news(limit=limit)
-
-
-@app.post("/api/news/summarize")
-async def summarize_news(item: Dict[str, Any] = Body(...)):
-    """Summarize a news article"""
-    try:
-        from ai_models import analyze_news_item
-        enriched = analyze_news_item(item)
-        
-        return {
-            "success": True,
-            "summary": enriched.get("title", ""),
-            "sentiment": enriched.get("sentiment", "neutral"),
-            "sentiment_confidence": enriched.get("sentiment_confidence", 0.5)
-        }
-    except Exception as e:
-        logger.error(f"Error in /api/news/summarize: {e}")
-        return {
-            "success": False,
-            "error": str(e),
-            "summary": item.get("title", ""),
-            "sentiment": "neutral"
-        }
-
-
-# ===== CHARTS ENDPOINTS =====
-@app.get("/api/charts/price/{symbol}")
-async def get_price_chart(symbol: str, timeframe: str = Query(default="7d")):
-    """Get price chart data"""
-    try:
-        # Clean and validate symbol
-        symbol = symbol.strip().upper()
-        if not symbol:
-            return JSONResponse(
-                status_code=400,
-                content={
-                    "success": False,
-                    "symbol": "",
-                    "timeframe": timeframe,
-                    "data": [],
-                    "count": 0,
-                    "error": "Symbol cannot be empty"
-                }
-            )
-        
-        logger.info(f"Fetching price history for {symbol} with timeframe {timeframe}")
-        
-        # market_collector.get_price_history expects timeframe as string, not hours
-        price_history = await market_collector.get_price_history(symbol, timeframe=timeframe)
-        
-        if not price_history or len(price_history) == 0:
-            logger.warning(f"No price history returned for {symbol}")
-            return {
-                "success": True,
-                "symbol": symbol,
-                "timeframe": timeframe,
-                "data": [],
-                "count": 0,
-                "message": "No data available"
-            }
-        
-        chart_data = []
-        for point in price_history:
-            # Handle different timestamp formats
-            timestamp = point.get("timestamp") or point.get("time") or point.get("date")
-            price = point.get("price") or point.get("close") or point.get("value") or 0
-            
-            # Convert timestamp to ISO format if needed
-            if timestamp:
-                try:
-                    # If it's already a string, use it
-                    if isinstance(timestamp, str):
-                        # Try to parse and format
-                        try:
-                            # Try ISO format first
-                            dt = datetime.fromisoformat(timestamp.replace('Z', '+00:00'))
-                            timestamp = dt.isoformat()
-                        except:
-                            try:
-                                # Try other common formats
-                                from dateutil import parser
-                                dt = parser.parse(timestamp)
-                                timestamp = dt.isoformat()
-                            except:
-                                pass
-                    elif isinstance(timestamp, (int, float)):
-                        # Unix timestamp
-                        dt = datetime.fromtimestamp(timestamp)
-                        timestamp = dt.isoformat()
-                except Exception as e:
-                    logger.warning(f"Error parsing timestamp {timestamp}: {e}")
-            
-            chart_data.append({
-                "timestamp": timestamp or "",
-                "time": timestamp or "",
-                "date": timestamp or "",
-                "price": float(price) if price else 0,
-                "close": float(price) if price else 0,
-                "value": float(price) if price else 0
-            })
-        
-        logger.info(f"Returning {len(chart_data)} data points for {symbol}")
-        
-        return {
-            "success": True,
-            "symbol": symbol,
-            "timeframe": timeframe,
-            "data": chart_data,
-            "count": len(chart_data)
-        }
-    except CollectorError as e:
-        logger.error(f"Collector error in /api/charts/price/{symbol}: {e}", exc_info=True)
-        return JSONResponse(
-            status_code=200,
-            content={
-                "success": False,
-                "symbol": symbol.upper() if symbol else "",
-                "timeframe": timeframe,
-                "data": [],
-                "count": 0,
-                "error": str(e)
-            }
-        )
-    except Exception as e:
-        logger.error(f"Error in /api/charts/price/{symbol}: {e}", exc_info=True)
-        return JSONResponse(
-            status_code=200,
-            content={
-                "success": False,
-                "symbol": symbol.upper() if symbol else "",
-                "timeframe": timeframe,
-                "data": [],
-                "count": 0,
-                "error": str(e)
-            }
-        )
-
-
-@app.post("/api/charts/analyze")
-async def analyze_chart(payload: Dict[str, Any] = Body(...)):
-    """Analyze chart data"""
-    try:
-        symbol = payload.get("symbol")
-        timeframe = payload.get("timeframe", "7d")
-        indicators = payload.get("indicators", [])
-        
-        if not symbol:
-            return JSONResponse(
-                status_code=400,
-                content={"success": False, "error": "Symbol is required"}
-            )
-        
-        symbol = symbol.strip().upper()
-        logger.info(f"Analyzing chart for {symbol} with timeframe {timeframe}")
-        
-        # Get price data - use timeframe string, not hours
-        price_history = await market_collector.get_price_history(symbol, timeframe=timeframe)
-        
-        if not price_history or len(price_history) == 0:
-            return {
-                "success": False,
-                "symbol": symbol,
-                "timeframe": timeframe,
-                "error": "No price data available for analysis"
-            }
-        
-        # Analyze with AI
-        from ai_models import analyze_chart_points
-        try:
-            analysis = analyze_chart_points(price_history, indicators)
-        except Exception as ai_error:
-            logger.error(f"AI analysis error: {ai_error}", exc_info=True)
-            # Return a basic analysis if AI fails
-            analysis = {
-                "direction": "neutral",
-                "summary": "Analysis unavailable",
-                "signals": []
-            }
-        
-        return {
-            "success": True,
-            "symbol": symbol,
-            "timeframe": timeframe,
-            "analysis": analysis
-        }
-    except CollectorError as e:
-        logger.error(f"Collector error in /api/charts/analyze: {e}", exc_info=True)
-        return JSONResponse(
-            status_code=200,
-            content={"success": False, "error": str(e)}
-        )
-    except Exception as e:
-        logger.error(f"Error in /api/charts/analyze: {e}", exc_info=True)
-        return JSONResponse(
-            status_code=200,
-            content={"success": False, "error": str(e)}
-        )
-
-
-# ===== SENTIMENT ENDPOINTS =====
-@app.post("/api/sentiment/analyze")
-async def analyze_sentiment(payload: Dict[str, Any] = Body(...)):
-    """Analyze sentiment of text"""
-    try:
-        text = payload.get("text", "")
-        
-        from ai_models import ensemble_crypto_sentiment
-        result = ensemble_crypto_sentiment(text)
-        
-        return {
-            "success": True,
-            "sentiment": result["label"],
-            "confidence": result["confidence"],
-            "details": result
-        }
-    except Exception as e:
-        logger.error(f"Error in /api/sentiment/analyze: {e}")
-        return {"success": False, "error": str(e)}
-
-
-# ===== QUERY ENDPOINT =====
-@app.post("/api/query")
-async def process_query(payload: Dict[str, Any] = Body(...)):
-    """Process natural language query"""
-    try:
-        query = payload.get("query", "").lower()
-        
-        # Simple query processing
-        if "price" in query or "btc" in query or "bitcoin" in query:
-            coins = await market_collector.get_top_coins(limit=10)
-            btc = next((c for c in coins if c.get("symbol", "").upper() == "BTC"), None)
-            
-            if btc:
-                price = btc.get("price") or btc.get("current_price", 0)
-                return {
-                    "success": True,
-                    "type": "price",
-                    "message": f"Bitcoin (BTC) is currently trading at ${price:,.2f}",
-                    "data": btc
-                }
-        
-        return {
-            "success": True,
-            "type": "general",
-            "message": "Query processed",
-            "data": None
-        }
-    except Exception as e:
-        logger.error(f"Error in /api/query: {e}")
-        return {"success": False, "error": str(e), "message": "Query failed"}
-
-
-# ===== DATASETS & MODELS =====
-@app.get("/api/datasets/list")
-async def list_datasets():
-    """List available datasets"""
-    try:
-        from backend.services.hf_registry import REGISTRY
-        datasets = REGISTRY.list(kind="datasets")
-        
-        formatted = []
-        for d in datasets:
-            formatted.append({
-                "name": d.get("id"),
-                "category": d.get("category", "other"),
-                "records": "N/A",
-                "updated_at": "",
-                "tags": d.get("tags", []),
-                "source": d.get("source", "hub")
-            })
-        
-        return {
-            "success": True,
-            "datasets": formatted,
-            "count": len(formatted)
-        }
-    except Exception as e:
-        logger.error(f"Error in /api/datasets/list: {e}")
-        return {"success": True, "datasets": [], "count": 0}
-
-
-@app.get("/api/datasets/sample")
-async def get_dataset_sample(name: str = Query(...), limit: int = Query(default=20)):
-    """Get sample from dataset"""
-    try:
-        # Attempt to load dataset
-        try:
-            from datasets import load_dataset
-            from config import get_settings
-            
-            # Get HF token for dataset loading
-            settings = get_settings()
-            hf_token = settings.hf_token or "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"
-            
-            # Set token in environment for datasets library
-            import os
-            if hf_token and not os.environ.get("HF_TOKEN"):
-                os.environ["HF_TOKEN"] = hf_token
-            
-            dataset = load_dataset(name, split="train", streaming=True, token=hf_token)
-            
-            sample = []
-            for i, row in enumerate(dataset):
-                if i >= limit:
-                    break
-                sample.append({k: str(v) for k, v in row.items()})
-            
-            return {
-                "success": True,
-                "name": name,
-                "sample": sample,
-                "count": len(sample)
-            }
-        except:
-            return {
-                "success": False,
-                "name": name,
-                "sample": [],
-                "count": 0,
-                "message": "Dataset loading requires authentication or is not available"
-            }
-    except Exception as e:
-        logger.error(f"Error in /api/datasets/sample: {e}")
-        return {"success": False, "error": str(e)}
-
-
-@app.get("/api/models/list")
-async def list_models():
-    """List available models"""
-    try:
-        from ai_models import get_model_info
-        info = get_model_info()
-        
-        models = []
-        catalog = info.get("model_catalog", {})
-        
-        for category, model_list in catalog.items():
-            for model_id in model_list:
-                models.append({
-                    "name": model_id,
-                    "task": "sentiment" if "sentiment" in category else "decision" if category == "decision" else "analysis",
-                    "status": "available",
-                    "category": category,
-                    "notes": f"{category.replace('_', ' ').title()} model"
-                })
-        
-        return {
-            "success": True,
-            "models": models,
-            "count": len(models)
-        }
-    except Exception as e:
-        logger.error(f"Error in /api/models/list: {e}")
-        return {"success": True, "models": [], "count": 0}
-
-
-@app.post("/api/models/test")
-async def test_model(payload: Dict[str, Any] = Body(...)):
-    """Test a specific model"""
-    try:
-        model_id = payload.get("model", "")
-        text = payload.get("text", "")
-        
-        from ai_models import ensemble_crypto_sentiment
-        result = ensemble_crypto_sentiment(text)
-        
-        return {
-            "success": True,
-            "model": model_id,
-            "result": result
-        }
-    except Exception as e:
-        logger.error(f"Error in /api/models/test: {e}")
-        return {"success": False, "error": str(e)}
-
-
-# ===== WEBSOCKET =====
-@app.websocket("/ws")
-async def websocket_endpoint(websocket: WebSocket):
-    """WebSocket endpoint for real-time updates"""
-    await ws_manager.connect(websocket)
-    
-    try:
-        while True:
-            # Check if connection is still open before sending
-            if websocket.client_state != WebSocketState.CONNECTED:
-                logger.info("WebSocket connection closed, breaking loop")
-                break
-                
-            # Send market updates every 10 seconds
-            try:
-                # Get latest data
-                top_coins = await market_collector.get_top_coins(limit=5)
-                news_items = await news_collector.get_latest_news(limit=3)
-                
-                # Compute global sentiment from news
-                from ai_models import ensemble_crypto_sentiment
-                news_texts = " ".join([n.get("title", "") for n in news_items])
-                global_sentiment = ensemble_crypto_sentiment(news_texts) if news_texts else {"label": "neutral", "confidence": 0.5}
-                
-                payload = {
-                    "market_data": top_coins,
-                    "stats": {
-                        "total_market_cap": sum([c.get("market_cap", 0) for c in top_coins]),
-                        "sentiment": global_sentiment
-                    },
-                    "news": news_items,
-                    "sentiment": global_sentiment,
-                    "timestamp": datetime.now().isoformat()
-                }
-                
-                # Double-check connection state before sending
-                if websocket.client_state == WebSocketState.CONNECTED:
-                    await websocket.send_json({
-                        "type": "update",
-                        "payload": payload
-                    })
-                else:
-                    logger.info("WebSocket disconnected, breaking loop")
-                    break
-            except CollectorError as e:
-                # Provider errors are already logged by the collector, just continue
-                logger.debug(f"Provider error in WebSocket update (this is expected with fallbacks): {e}")
-                # Use empty data on provider errors
-                payload = {
-                    "market_data": [],
-                    "stats": {"total_market_cap": 0, "sentiment": {"label": "neutral", "confidence": 0.5}},
-                    "news": [],
-                    "sentiment": {"label": "neutral", "confidence": 0.5},
-                    "timestamp": datetime.now().isoformat()
-                }
-            except Exception as e:
-                # Log other errors with full details
-                error_msg = str(e) if str(e) else repr(e)
-                logger.error(f"Error in WebSocket update: {type(e).__name__}: {error_msg}")
-                # Don't break on data errors, just log and continue
-                # Only break on connection errors
-                if "send" in str(e).lower() or "close" in str(e).lower():
-                    break
-            
-            await asyncio.sleep(10)
-    except WebSocketDisconnect:
-        logger.info("WebSocket disconnect exception caught")
-    except Exception as e:
-        logger.error(f"WebSocket error: {e}")
-    finally:
-        try:
-            ws_manager.disconnect(websocket)
-        except:
-            pass
-
-@app.get("/api/market/history")
-async def get_market_history(symbol: str = "BTC", limit: int = 10):
-    """
-    Get historical prices from the local database if available.
-
-    For this deployment we avoid touching the internal DatabaseManager
-    and simply report that no history API is wired yet.
-    """
-    symbol = symbol.upper()
-    # We don't fabricate data here; if you need real history, it should
-    # be implemented via the shared database models.
-    return {
-        "symbol": symbol,
-        "history": [],
-        "count": 0,
-        "message": "History endpoint not wired to DB in this Space",
-    }
-
-
-
-@app.get("/api/status")
-async def get_status():
-    """
-    System status endpoint used by the admin UI.
-
-    This reports real-time information about providers and database,
-    without fabricating any market data.
-    """
-    providers_cfg = load_providers_config()
-    providers = providers_cfg or {}
-    validated_count = sum(1 for p in providers.values() if p.get("validated"))
-
-    db_path = DB_PATH
-    db_status = "connected" if db_path.exists() else "initializing"
-
-    return {
-        "system_health": "healthy",
-        "timestamp": datetime.now().isoformat(),
-        "total_providers": len(providers),
-        "validated_providers": validated_count,
-        "database_status": db_status,
-        "apl_available": APL_REPORT_PATH.exists(),
-        "use_mock_data": False,
-    }
-
-
-@app.get("/api/logs/recent")
-async def get_recent_logs():
-    """
-    Return recent log lines for the admin UI.
-
-    We read from the main server log file if available.
-    This does not fabricate content; if there are no logs,
-    an empty list is returned.
-    """
-    log_file = LOG_DIR / "server.log"
-    lines = tail_log_file(log_file, max_lines=200)
-    # Wrap plain text lines as structured entries
-    logs = [{"line": line.rstrip("\n")} for line in lines]
-    return {"logs": logs, "count": len(logs)}
-
-
-@app.get("/api/logs/errors")
-async def get_error_logs():
-    """
-    Return recent error log lines from the same log file.
-
-    This is a best-effort filter based on typical ERROR prefixes.
-    """
-    log_file = LOG_DIR / "server.log"
-    lines = tail_log_file(log_file, max_lines=400)
-    error_lines = [line for line in lines if "ERROR" in line or "WARNING" in line]
-    logs = [{"line": line.rstrip("\n")} for line in error_lines[-200:]]
-    return {"errors": logs, "count": len(logs)}
-
-
-def _load_apl_report() -> Optional[Dict[str, Any]]:
-    """Load the APL (Auto Provider Loader) validation report if available."""
-    if not APL_REPORT_PATH.exists():
-        return None
-    try:
-        with APL_REPORT_PATH.open("r", encoding="utf-8") as f:
-            return json.load(f)
-    except Exception as e:
-        logger.error(f"Error reading APL report: {e}")
-        return None
-
-
-@app.get("/api/apl/summary")
-async def get_apl_summary():
-    """
-    Summary of the Auto Provider Loader (APL) report.
-
-    If the report is missing, we return a clear not_available status
-    instead of fabricating metrics.
-    """
-    report = _load_apl_report()
-    if not report or "stats" not in report:
-        return {
-            "status": "not_available",
-            "message": "APL report not found",
-        }
-
-    stats = report.get("stats", {})
-    return {
-        "status": "ok",
-        "http_candidates": stats.get("total_http_candidates", 0),
-        "http_valid": stats.get("http_valid", 0),
-        "http_invalid": stats.get("http_invalid", 0),
-        "http_conditional": stats.get("http_conditional", 0),
-        "hf_candidates": stats.get("total_hf_candidates", 0),
-        "hf_valid": stats.get("hf_valid", 0),
-        "hf_invalid": stats.get("hf_invalid", 0),
-        "hf_conditional": stats.get("hf_conditional", 0),
-        "timestamp": datetime.now().isoformat(),
-    }
-
-
-@app.get("/api/hf/models")
-async def get_hf_models_from_apl():
-    """
-    Return the list of Hugging Face models discovered by the APL report.
-
-    This is used by the admin UI. The data comes from the real
-    PROVIDER_AUTO_DISCOVERY_REPORT.json file if present.
-    """
-    report = _load_apl_report()
-    if not report:
-        return {"models": [], "count": 0, "source": "none"}
-
-    hf_models = report.get("hf_models", {}).get("results", [])
-    return {
-        "models": hf_models,
-        "count": len(hf_models),
-        "source": "APL report",
-    }
-
+#!/usr/bin/env python3
+"""
+Hugging Face Unified Server - Main FastAPI application entry point.
+This module imports the FastAPI app from api_server_extended for HF Docker Space deployment.
+"""
+
+from api_server_extended import app
+
+__all__ = ["app"]
+

package.json

@@ -10,16 +10,7 @@
     "dashboard": "python3 -m http.server 8080",
     "full-check": "node api-monitor.js && node failover-manager.js && echo 'Open http://localhost:8080/dashboard.html in your browser' && python3 -m http.server 8080",
     "test:free-resources": "node free_resources_selftest.mjs",
-    "test:free-resources:win": "powershell -NoProfile -ExecutionPolicy Bypass -File test_free_endpoints.ps1",
-    "test:theme": "node tests/verify_theme.js",
-    "test:api-client": "node tests/test_apiClient.test.js",
-    "test:ui-feedback": "node tests/test_ui_feedback.test.js",
-    "test:fallback": "pytest tests/test_fallback_service.py -m fallback",
-    "test:api-health": "pytest tests/test_fallback_service.py -m api_health"
-  },
-  "devDependencies": {
-    "fast-check": "^3.15.0",
-    "jsdom": "^23.0.0"
+    "test:free-resources:win": "powershell -NoProfile -ExecutionPolicy Bypass -File test_free_endpoints.ps1"
   },
   "keywords": [
     "cryptocurrency",
@@ -41,8 +32,5 @@
   "repository": {
     "type": "git",
     "url": "https://github.com/nimazasinich/crypto-dt-source.git"
-  },
-  "dependencies": {
-    "charmap": "^1.1.6"
   }
 }

production_server.py

@@ -441,7 +441,7 @@ async def remove_custom_api(name: str):
 # Serve static files
 @app.get("/")
 async def root():
-    return FileResponse("admin.html")
+    return FileResponse("index.html")
 
 @app.get("/index.html")
 async def index():

provider_validator.py

@@ -278,13 +278,7 @@ class ProviderValidator:
         try:
             start = time.time()
             
-            # Get HF token from environment or use default
-            hf_token = os.getenv("HF_TOKEN") or "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"
-            headers = {}
-            if hf_token:
-                headers["Authorization"] = f"Bearer {hf_token}"
-            
-            async with httpx.AsyncClient(timeout=self.timeout, headers=headers) as client:
+            async with httpx.AsyncClient(timeout=self.timeout) as client:
                 response = await client.get(f"https://huggingface.co/api/models/{model_id}")
                 elapsed_ms = (time.time() - start) * 1000
                 

real_server.py

@@ -380,7 +380,7 @@ async def api_config_keys():
 # Serve static files
 @app.get("/")
 async def root():
-    return FileResponse("admin.html")
+    return FileResponse("dashboard.html")
 
 @app.get("/dashboard.html")
 async def dashboard():

requirements_hf.txt

@@ -1,10 +1,32 @@
-fastapi>=0.104.0,<1.0.0
-uvicorn>=0.24.0,<1.0.0
-aiohttp>=3.8.0,<4.0.0
-pydantic>=2.5.0,<3.0.0
-sqlalchemy>=2.0.0,<3.0.0
-pandas>=2.0.0,<3.0.0
-numpy>=1.25.0,<2.0.0
-transformers>=4.38.0,<5.0.0
-torch
-python-dotenv>=1.0.0,<2.0.0
+fastapi==0.115.0
+uvicorn[standard]==0.30.0
+pydantic==2.9.0
+pydantic-settings==2.5.0
+sqlalchemy==2.0.35
+httpx==0.27.2
+websockets>=10.4,<12.0
+python-dotenv==1.0.1
+python-multipart==0.0.9
+requests==2.32.3
+aiohttp==3.10.5
+pandas==2.2.3
+numpy>=1.26.0,<2.0.0
+gradio==4.44.0
+plotly==5.24.1
+psutil==6.0.0
+transformers>=4.45.0
+tokenizers>=0.20.0
+huggingface-hub>=0.25.0
+safetensors>=0.4.0
+datasets>=3.0.0
+torch>=2.4.0,<2.5.0
+torchaudio>=2.4.0,<2.5.0
+sentence-transformers>=3.1.0
+sentencepiece==0.2.0
+feedparser==6.0.11
+beautifulsoup4==4.12.3
+lxml==5.3.0
+python-dateutil>=2.9.0
+pytz>=2024.1
+tenacity>=9.0.0
+

simple_server.py

@@ -1,23 +1,11 @@
 """Simple FastAPI server for testing HF integration"""
 import asyncio
-import os
-import sys
-import io
-from datetime import datetime
-from fastapi import FastAPI, Query, WebSocket, WebSocketDisconnect
+from fastapi import FastAPI
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.responses import FileResponse, JSONResponse
 from fastapi.staticfiles import StaticFiles
 import uvicorn
 
-# Fix encoding for Windows console
-if sys.platform == "win32":
-    try:
-        sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8', errors='replace')
-        sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8', errors='replace')
-    except Exception:
-        pass
-
 # Create FastAPI app
 app = FastAPI(title="Crypto API Monitor - Simple", version="1.0.0")
 
@@ -34,20 +22,9 @@ app.add_middleware(
 try:
     from backend.routers import hf_connect
     app.include_router(hf_connect.router)
-    print("[OK] HF router loaded")
-except Exception as e:
-    print(f"[ERROR] HF router failed: {e}")
-
-# Mount static files directory
-try:
-    static_path = os.path.join(os.path.dirname(__file__), "static")
-    if os.path.exists(static_path):
-        app.mount("/static", StaticFiles(directory=static_path), name="static")
-        print(f"[OK] Static files mounted from {static_path}")
-    else:
-        print(f"[WARNING] Static directory not found: {static_path}")
+    print("✓ HF router loaded")
 except Exception as e:
-    print(f"[ERROR] Could not mount static files: {e}")
+    print(f"✗ HF router failed: {e}")
 
 # Background task for HF registry
 @app.on_event("startup")
@@ -55,9 +32,9 @@ async def startup_hf():
     try:
         from backend.services.hf_registry import periodic_refresh
         asyncio.create_task(periodic_refresh())
-        print("[OK] HF background refresh started")
+        print("✓ HF background refresh started")
     except Exception as e:
-        print(f"[ERROR] HF background refresh failed: {e}")
+        print(f"✗ HF background refresh failed: {e}")
 
 # Health endpoint
 @app.get("/health")
@@ -71,10 +48,7 @@ async def api_health():
 # Serve static files
 @app.get("/")
 async def root():
-    """Serve default HTML UI page (index.html)"""
-    if os.path.exists("index.html"):
-        return FileResponse("index.html")
-    return FileResponse("admin.html")
+    return FileResponse("index.html")
 
 @app.get("/index.html")
 async def index():
@@ -84,15 +58,6 @@ async def index():
 async def hf_console():
     return FileResponse("hf_console.html")
 
-# Serve config.js
-@app.get("/config.js")
-async def config_js():
-    """Serve config.js file"""
-    config_path = os.path.join(os.path.dirname(__file__), "config.js")
-    if os.path.exists(config_path):
-        return FileResponse(config_path, media_type="application/javascript")
-    return JSONResponse({"error": "config.js not found"}, status_code=404)
-
 # Mock API endpoints for dashboard
 @app.get("/api/status")
 async def api_status():
@@ -423,301 +388,14 @@ async def api_config_keys():
         }
     ]
 
-# API endpoints for dashboard
-@app.get("/api/coins/top")
-async def api_coins_top(limit: int = 10):
-    """Get top cryptocurrencies"""
-    from datetime import datetime
-    try:
-        # Try to use real collectors if available
-        from collectors.aggregator import MarketDataCollector
-        collector = MarketDataCollector()
-        coins = await collector.get_top_coins(limit=limit)
-        result = []
-        for coin in coins:
-            result.append({
-                "id": coin.get("id", coin.get("symbol", "").lower()),
-                "rank": coin.get("rank", 0),
-                "symbol": coin.get("symbol", "").upper(),
-                "name": coin.get("name", ""),
-                "price": coin.get("price") or coin.get("current_price", 0),
-                "current_price": coin.get("price") or coin.get("current_price", 0),
-                "price_change_24h": coin.get("change_24h") or coin.get("price_change_percentage_24h", 0),
-                "price_change_percentage_24h": coin.get("change_24h") or coin.get("price_change_percentage_24h", 0),
-                "volume_24h": coin.get("volume_24h") or coin.get("total_volume", 0),
-                "market_cap": coin.get("market_cap", 0),
-                "image": coin.get("image", ""),
-                "last_updated": coin.get("last_updated", datetime.now().isoformat())
-            })
-        return {"success": True, "coins": result, "count": len(result), "timestamp": datetime.now().isoformat()}
-    except Exception as e:
-        # Return mock data on error
-        from datetime import datetime
-        import random
-        mock_coins = [
-            {"id": "bitcoin", "rank": 1, "symbol": "BTC", "name": "Bitcoin", "price": 43250.50 + random.uniform(-1000, 1000),
-             "current_price": 43250.50, "price_change_24h": 2.34, "price_change_percentage_24h": 2.34,
-             "volume_24h": 25000000000, "market_cap": 845000000000, "image": "", "last_updated": datetime.now().isoformat()},
-            {"id": "ethereum", "rank": 2, "symbol": "ETH", "name": "Ethereum", "price": 2450.30 + random.uniform(-100, 100),
-             "current_price": 2450.30, "price_change_24h": 1.25, "price_change_percentage_24h": 1.25,
-             "volume_24h": 12000000000, "market_cap": 295000000000, "image": "", "last_updated": datetime.now().isoformat()},
-        ]
-        return {"success": True, "coins": mock_coins[:limit], "count": min(limit, len(mock_coins)), "timestamp": datetime.now().isoformat()}
-
-@app.get("/api/market/stats")
-async def api_market_stats():
-    """Get global market statistics"""
-    from datetime import datetime
-    try:
-        # Try to get real data from collectors
-        from collectors.aggregator import MarketDataCollector
-        collector = MarketDataCollector()
-        coins = await collector.get_top_coins(limit=100)
-        total_market_cap = sum(c.get("market_cap", 0) for c in coins)
-        total_volume = sum(c.get("volume_24h", 0) or c.get("total_volume", 0) for c in coins)
-        btc_market_cap = next((c.get("market_cap", 0) for c in coins if c.get("symbol", "").upper() == "BTC"), 0)
-        btc_dominance = (btc_market_cap / total_market_cap * 100) if total_market_cap > 0 else 0
-        
-        stats = {
-            "total_market_cap": total_market_cap,
-            "total_volume_24h": total_volume,
-            "btc_dominance": btc_dominance,
-            "eth_dominance": 0,
-            "active_cryptocurrencies": 10000,
-            "markets": 500,
-            "market_cap_change_24h": 0.0,
-            "timestamp": datetime.now().isoformat()
-        }
-        return {"success": True, "stats": stats}
-    except Exception:
-        # Return mock data on error
-        from datetime import datetime
-        return {
-            "success": True,
-            "stats": {
-                "total_market_cap": 2100000000000,
-                "total_volume_24h": 89500000000,
-                "btc_dominance": 48.2,
-                "eth_dominance": 15.5,
-                "active_cryptocurrencies": 10000,
-                "markets": 500,
-                "market_cap_change_24h": 2.5,
-                "timestamp": datetime.now().isoformat()
-            }
-        }
-
-@app.get("/api/news/latest")
-async def api_news_latest(limit: int = 40):
-    """Get latest cryptocurrency news"""
-    from datetime import datetime
-    try:
-        # Try to use real collectors if available
-        from collectors.aggregator import NewsCollector
-        collector = NewsCollector()
-        news_items = await collector.get_latest_news(limit=limit)
-        
-        # Format news items
-        enriched_news = []
-        for item in news_items:
-            enriched_news.append({
-                "title": item.get("title", ""),
-                "source": item.get("source", ""),
-                "published_at": item.get("published_at") or item.get("date", ""),
-                "symbols": item.get("symbols", []),
-                "sentiment": item.get("sentiment", "neutral"),
-                "sentiment_confidence": item.get("sentiment_confidence", 0.5),
-                "url": item.get("url", "")
-            })
-        return {"success": True, "news": enriched_news, "count": len(enriched_news), "timestamp": datetime.now().isoformat()}
-    except Exception:
-        # Return mock data on error
-        from datetime import datetime, timedelta
-        mock_news = [
-            {
-                "title": "Bitcoin reaches new milestone",
-                "source": "CoinDesk",
-                "published_at": (datetime.now() - timedelta(hours=2)).isoformat(),
-                "symbols": ["BTC"],
-                "sentiment": "positive",
-                "sentiment_confidence": 0.75,
-                "url": "https://example.com/news1"
-            },
-            {
-                "title": "Ethereum upgrade scheduled",
-                "source": "CryptoNews",
-                "published_at": (datetime.now() - timedelta(hours=5)).isoformat(),
-                "symbols": ["ETH"],
-                "sentiment": "neutral",
-                "sentiment_confidence": 0.65,
-                "url": "https://example.com/news2"
-            },
-        ]
-        return {"success": True, "news": mock_news[:limit], "count": min(limit, len(mock_news)), "timestamp": datetime.now().isoformat()}
-
-@app.get("/api/market")
-async def api_market():
-    """Get market data (combines coins and stats)"""
-    from datetime import datetime
-    try:
-        # Get top coins and market stats
-        coins_data = await api_coins_top(20)
-        stats_data = await api_market_stats()
-        
-        return {
-            "success": True,
-            "cryptocurrencies": coins_data.get("coins", []),
-            "stats": stats_data.get("stats", {}),
-            "timestamp": datetime.now().isoformat()
-        }
-    except Exception as e:
-        # Return basic structure on error
-        from datetime import datetime
-        return {
-            "success": True,
-            "cryptocurrencies": [],
-            "stats": {
-                "total_market_cap": 0,
-                "total_volume_24h": 0,
-                "btc_dominance": 0
-            },
-            "timestamp": datetime.now().isoformat()
-        }
-
-@app.get("/api/sentiment")
-async def api_sentiment():
-    """Get market sentiment data"""
-    from datetime import datetime
-    try:
-        # Try to get real sentiment data
-        from collectors.aggregator import ProviderStatusCollector
-        collector = ProviderStatusCollector()
-        
-        # Try to get fear & greed index
-        import httpx
-        async with httpx.AsyncClient() as client:
-            try:
-                fng_response = await client.get("https://api.alternative.me/fng/?limit=1", timeout=5)
-                if fng_response.status_code == 200:
-                    fng_data = fng_response.json()
-                    if fng_data.get("data") and len(fng_data["data"]) > 0:
-                        fng_value = int(fng_data["data"][0].get("value", 50))
-                        return {
-                            "success": True,
-                            "fear_greed": {
-                                "value": fng_value,
-                                "classification": "Extreme Fear" if fng_value < 25 else "Fear" if fng_value < 45 else "Neutral" if fng_value < 55 else "Greed" if fng_value < 75 else "Extreme Greed"
-                            },
-                            "overall_sentiment": "neutral",
-                            "timestamp": datetime.now().isoformat()
-                        }
-            except:
-                pass
-        
-        # Fallback to default sentiment
-        return {
-            "success": True,
-            "fear_greed": {
-                "value": 50,
-                "classification": "Neutral"
-            },
-            "overall_sentiment": "neutral",
-            "timestamp": datetime.now().isoformat()
-        }
-    except Exception:
-        # Return default sentiment on error
-        from datetime import datetime
-        return {
-            "success": True,
-            "fear_greed": {
-                "value": 50,
-                "classification": "Neutral"
-            },
-            "overall_sentiment": "neutral",
-            "timestamp": datetime.now().isoformat()
-        }
-
-@app.get("/api/trending")
-async def api_trending():
-    """Get trending cryptocurrencies"""
-    # Use top coins as trending for now
-    return await api_coins_top(10)
-
-# WebSocket support
-class ConnectionManager:
-    def __init__(self):
-        self.active_connections = []
-    
-    async def connect(self, websocket: WebSocket):
-        await websocket.accept()
-        self.active_connections.append(websocket)
-    
-    def disconnect(self, websocket: WebSocket):
-        if websocket in self.active_connections:
-            self.active_connections.remove(websocket)
-    
-    async def broadcast(self, message: dict):
-        for conn in list(self.active_connections):
-            try:
-                await conn.send_json(message)
-            except:
-                self.disconnect(conn)
-
-ws_manager = ConnectionManager()
-
-@app.websocket("/ws")
-async def websocket_endpoint(websocket: WebSocket):
-    """WebSocket endpoint for real-time updates"""
-    await ws_manager.connect(websocket)
-    try:
-        # Send initial connection message
-        await websocket.send_json({
-            "type": "connected",
-            "message": "WebSocket connected",
-            "timestamp": datetime.now().isoformat()
-        })
-        
-        # Send periodic updates
-        while True:
-            try:
-                # Send heartbeat
-                await websocket.send_json({
-                    "type": "heartbeat",
-                    "timestamp": datetime.now().isoformat()
-                })
-                
-                # Try to get market data and send update
-                try:
-                    coins_data = await api_coins_top(5)
-                    news_data = await api_news_latest(3)
-                    
-                    await websocket.send_json({
-                        "type": "update",
-                        "payload": {
-                            "market_data": coins_data.get("coins", []),
-                            "news": news_data.get("news", []),
-                            "timestamp": datetime.now().isoformat()
-                        }
-                    })
-                except:
-                    pass  # If data fetch fails, just send heartbeat
-                
-                await asyncio.sleep(30)  # Update every 30 seconds
-            except WebSocketDisconnect:
-                break
-    except WebSocketDisconnect:
-        ws_manager.disconnect(websocket)
-    except Exception as e:
-        print(f"[WS] Error: {e}")
-        ws_manager.disconnect(websocket)
-
 if __name__ == "__main__":
     print("=" * 70)
-    print("Starting Crypto API Monitor - Simple Server")
+    print("🚀 Starting Crypto API Monitor - Simple Server")
     print("=" * 70)
-    print("Server: http://localhost:7860")
-    print("Main Dashboard: http://localhost:7860/ (index.html - default HTML UI)")
-    print("HF Console: http://localhost:7860/hf_console.html")
-    print("API Docs: http://localhost:7860/docs")
+    print("📍 Server: http://localhost:7860")
+    print("📄 Main Dashboard: http://localhost:7860/index.html")
+    print("🤗 HF Console: http://localhost:7860/hf_console.html")
+    print("📚 API Docs: http://localhost:7860/docs")
     print("=" * 70)
     print()
     

unified_dashboard.html

@@ -8,152 +8,54 @@
     <link rel="stylesheet" href="static/css/design-system.css" />
     <link rel="stylesheet" href="static/css/dashboard.css" />
     <link rel="stylesheet" href="static/css/pro-dashboard.css" />
-    <link rel="stylesheet" href="static/css/modern-dashboard.css" />
-    <link rel="stylesheet" href="static/css/glassmorphism.css" />
-    <link rel="stylesheet" href="static/css/light-minimal-theme.css" />
     <script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/chart.umd.min.js" defer></script>
-    <script src="static/js/animations.js" defer></script>
-    <script src="static/js/menu-system.js" defer></script>
-    <script src="static/js/huggingface-integration.js" defer></script>
 </head>
-<body data-theme="light">
+<body data-theme="dark">
     <div class="app-shell">
-        <aside class="sidebar sidebar-modern">
-            <div class="brand brand-modern">
-                <div class="brand-icon">
-                    <svg width="24" height="24" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
+        <aside class="sidebar">
+            <div class="brand">
+                <strong>Crypto Monitor HF</strong>
+                <span class="env-pill">
+                    <svg width="12" height="12" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
                         <path d="M12 2L2 7L12 12L22 7L12 2Z" stroke="currentColor" stroke-width="1.5" />
                         <path d="M2 17L12 22L22 17" stroke="currentColor" stroke-width="1.5" />
                         <path d="M2 12L12 17L22 12" stroke="currentColor" stroke-width="1.5" />
                     </svg>
-                </div>
-                <div class="brand-text">
-                    <strong>Crypto Monitor HF</strong>
-                    <span class="env-pill">
-                        <svg width="10" height="10" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
-                            <path d="M12 2L2 7L12 12L22 7L12 2Z" stroke="currentColor" stroke-width="1.5" />
-                        </svg>
-                        HF Space
-                    </span>
-                </div>
+                    HF Space
+                </span>
             </div>
-            <nav class="nav nav-modern">
-                <button class="nav-button nav-button-modern active" data-nav="page-overview">
-                    <svg width="18" height="18" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
-                        <path d="M3 9l9-7 9 7v11a2 2 0 01-2 2H5a2 2 0 01-2-2z" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
-                        <path d="M9 22V12h6v10" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
-                    </svg>
-                    <span>Overview</span>
-                </button>
-                <button class="nav-button nav-button-modern" data-nav="page-market">
-                    <svg width="18" height="18" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
-                        <path d="M3 3v18h18" stroke="currentColor" stroke-width="2" stroke-linecap="round"/>
-                        <path d="M7 10l4-4 4 4 6-6" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
-                    </svg>
-                    <span>Market</span>
-                </button>
-                <button class="nav-button nav-button-modern" data-nav="page-chart">
-                    <svg width="18" height="18" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
-                        <path d="M3 3v18h18" stroke="currentColor" stroke-width="2" stroke-linecap="round"/>
-                        <path d="M7 16l4-4 4 4 6-6" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
-                    </svg>
-                    <span>Chart Lab</span>
-                </button>
-                <button class="nav-button nav-button-modern" data-nav="page-ai">
-                    <svg width="18" height="18" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
-                        <path d="M12 2L2 7l10 5 10-5-10-5zM2 17l10 5 10-5M2 12l10 5 10-5" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
-                    </svg>
-                    <span>Sentiment & AI</span>
-                </button>
-                <button class="nav-button nav-button-modern" data-nav="page-news">
-                    <svg width="18" height="18" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
-                        <path d="M4 19.5A2.5 2.5 0 016.5 17H20" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
-                        <path d="M6.5 2H20v20H6.5A2.5 2.5 0 014 19.5v-15A2.5 2.5 0 016.5 2z" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
-                    </svg>
-                    <span>News</span>
-                </button>
-                <button class="nav-button nav-button-modern" data-nav="page-providers">
-                    <svg width="18" height="18" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
-                        <circle cx="12" cy="12" r="10" stroke="currentColor" stroke-width="2"/>
-                        <path d="M12 6v6l4 2" stroke="currentColor" stroke-width="2" stroke-linecap="round"/>
-                    </svg>
-                    <span>Providers</span>
-                </button>
-                <button class="nav-button nav-button-modern" data-nav="page-api">
-                    <svg width="18" height="18" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
-                        <path d="M21 16V8a2 2 0 00-1-1.73l-7-4a2 2 0 00-2 0l-7 4A2 2 0 003 8v8a2 2 0 001 1.73l7 4a2 2 0 002 0l7-4A2 2 0 0021 16z" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
-                    </svg>
-                    <span>API Explorer</span>
-                </button>
-                <button class="nav-button nav-button-modern" data-nav="page-debug">
-                    <svg width="18" height="18" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
-                        <circle cx="12" cy="12" r="10" stroke="currentColor" stroke-width="2"/>
-                        <path d="M12 6v6l4 2" stroke="currentColor" stroke-width="2" stroke-linecap="round"/>
-                    </svg>
-                    <span>Diagnostics</span>
-                </button>
-                <button class="nav-button nav-button-modern" data-nav="page-datasets">
-                    <svg width="18" height="18" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
-                        <path d="M21 16V8a2 2 0 00-1-1.73l-7-4a2 2 0 00-2 0l-7 4A2 2 0 003 8v8a2 2 0 001 1.73l7 4a2 2 0 002 0l7-4A2 2 0 0021 16z" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
-                    </svg>
-                    <span>Datasets & Models</span>
-                </button>
-                <button class="nav-button nav-button-modern" data-nav="page-settings">
-                    <svg width="18" height="18" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
-                        <circle cx="12" cy="12" r="3" stroke="currentColor" stroke-width="2"/>
-                        <path d="M12 1v6m0 6v6M5.64 5.64l4.24 4.24m4.24 4.24l4.24 4.24M1 12h6m6 0h6M5.64 18.36l4.24-4.24m4.24-4.24l4.24-4.24" stroke="currentColor" stroke-width="2" stroke-linecap="round"/>
-                    </svg>
-                    <span>Settings</span>
-                </button>
+            <nav class="nav">
+                <button class="nav-button active" data-nav="page-overview">Overview</button>
+                <button class="nav-button" data-nav="page-market">Market</button>
+                <button class="nav-button" data-nav="page-chart">Chart Lab</button>
+                <button class="nav-button" data-nav="page-ai">Sentiment & AI</button>
+                <button class="nav-button" data-nav="page-news">News</button>
+                <button class="nav-button" data-nav="page-providers">Providers</button>
+                <button class="nav-button" data-nav="page-api">API Explorer</button>
+                <button class="nav-button" data-nav="page-debug">Diagnostics</button>
+                <button class="nav-button" data-nav="page-datasets">Datasets & Models</button>
+                <button class="nav-button" data-nav="page-settings">Settings</button>
             </nav>
             <div class="sidebar-footer">
-                <div class="footer-badge">
-                    <svg width="14" height="14" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
-                        <path d="M12 2L2 7l10 5 10-5-10-5zM2 17l10 5 10-5M2 12l10 5 10-5" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"/>
-                    </svg>
-                    Unified Intelligence Console
-                </div>
+                Unified crypto intelligence console<br />Realtime data • HF optimized
             </div>
         </aside>
         <main class="main-area">
-            <header class="modern-header">
-                <div style="display: flex; align-items: center; justify-content: space-between; width: 100%; flex-wrap: wrap; gap: 16px;">
-                    <div>
-                        <h1 style="margin: 0; font-size: 1.75rem; font-weight: 800; background: linear-gradient(135deg, #00D4FF, #8B5CF6, #EC4899); -webkit-background-clip: text; -webkit-text-fill-color: transparent; background-clip: text;">Unified Intelligence Dashboard</h1>
-                        <p class="text-muted" style="margin: 4px 0 0 0; font-size: 0.875rem;">Live market telemetry, AI signals, diagnostics, and provider health.</p>
+            <header class="topbar">
+                <div>
+                    <h1>Unified Intelligence Dashboard</h1>
+                    <p class="text-muted">Live market telemetry, AI signals, diagnostics, and provider health.</p>
+                </div>
+                <div class="status-group">
+                    <div class="status-pill" data-api-health data-state="warn">
+                        <span class="status-dot"></span>
+                        <span>checking</span>
                     </div>
-                    <div class="status-group" style="display: flex; gap: 12px; align-items: center; position: relative;">
-                        <div class="status-pill" data-api-health data-state="warn" style="padding: 8px 16px; border-radius: 12px; background: rgba(255, 255, 255, 0.05); border: 1px solid rgba(255, 255, 255, 0.1);">
-                            <span class="status-dot"></span>
-                            <span>checking</span>
-                        </div>
-                        <div class="status-pill" data-ws-status data-state="warn" style="padding: 8px 16px; border-radius: 12px; background: rgba(255, 255, 255, 0.05); border: 1px solid rgba(255, 255, 255, 0.1);">
-                            <span class="status-dot"></span>
-                            <span>connecting</span>
-                        </div>
-                        <button class="button-3d" data-menu-trigger="theme-menu" style="padding: 8px 16px; position: relative;">
-                            <svg width="16" height="16" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
-                                <circle cx="12" cy="12" r="3" stroke="currentColor" stroke-width="2"/>
-                                <path d="M12 1v6m0 6v6M5.64 5.64l4.24 4.24m4.24 4.24l4.24 4.24M1 12h6m6 0h6M5.64 18.36l4.24-4.24m4.24-4.24l4.24-4.24" stroke="currentColor" stroke-width="2" stroke-linecap="round"/>
-                            </svg>
-                        </button>
-                        <div class="menu-dropdown" data-menu="theme-menu" style="display: none; top: 100%; right: 0; margin-top: 8px;">
-                            <div class="menu-item" data-action="theme-light">
-                                <span>☀️ Light Theme</span>
-                            </div>
-                            <div class="menu-item" data-action="theme-dark">
-                                <span>🌙 Dark Theme</span>
-                            </div>
-                            <div class="menu-separator"></div>
-                            <div class="menu-item" data-action="settings">
-                                <span>⚙️ Settings</span>
-                            </div>
-                        </div>
+                    <div class="status-pill" data-ws-status data-state="warn">
+                        <span class="status-dot"></span>
+                        <span>connecting</span>
                     </div>
                 </div>
-                <div class="header-crypto-list" data-header-crypto-list style="margin-top: 16px; width: 100%;">
-                    <!-- Crypto list will be populated by JavaScript -->
-                </div>
             </header>
             <div class="page-container">
                 <section id="page-overview" class="page active">
@@ -193,30 +95,6 @@
                             <canvas id="sentiment-chart" height="220"></canvas>
                         </div>
                     </div>
-                    <div class="glass-card" style="margin-top: 24px;">
-                        <div class="section-header">
-                            <h3>Backend Information</h3>
-                            <span class="text-muted">System Status</span>
-                        </div>
-                        <div data-backend-info class="backend-info-grid" style="display: grid; grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); gap: 16px; margin-top: 16px;">
-                            <div class="backend-info-item">
-                                <span class="info-label">API Status</span>
-                                <span class="info-value" data-api-status>Checking...</span>
-                            </div>
-                            <div class="backend-info-item">
-                                <span class="info-label">WebSocket</span>
-                                <span class="info-value" data-ws-status>Connecting...</span>
-                            </div>
-                            <div class="backend-info-item">
-                                <span class="info-label">Providers</span>
-                                <span class="info-value" data-providers-count>—</span>
-                            </div>
-                            <div class="backend-info-item">
-                                <span class="info-label">Last Update</span>
-                                <span class="info-value" data-last-update>—</span>
-                            </div>
-                        </div>
-                    </div>
                 </section>
 
                 <section id="page-market" class="page">
@@ -275,53 +153,32 @@
 
                 <section id="page-chart" class="page">
                     <div class="section-header">
-                        <h2 class="section-title">Chart Lab - TradingView Style</h2>
+                        <h2 class="section-title">Chart Lab</h2>
                         <div class="controls-bar">
-                            <select data-chart-symbol style="padding: 8px 12px; border-radius: 8px; background: rgba(255, 255, 255, 0.05); border: 1px solid rgba(255, 255, 255, 0.1); color: var(--text-primary);">
+                            <select data-chart-symbol>
                                 <option value="BTC">BTC</option>
                                 <option value="ETH">ETH</option>
                                 <option value="SOL">SOL</option>
                                 <option value="BNB">BNB</option>
-                                <option value="ADA">ADA</option>
-                                <option value="DOT">DOT</option>
-                                <option value="MATIC">MATIC</option>
-                                <option value="AVAX">AVAX</option>
                             </select>
-                            <div class="chart-toolbar">
-                                <button class="chart-timeframe-btn active" data-chart-timeframe="1d">1D</button>
-                                <button class="chart-timeframe-btn" data-chart-timeframe="7d">7D</button>
-                                <button class="chart-timeframe-btn" data-chart-timeframe="30d">30D</button>
-                                <button class="chart-timeframe-btn" data-chart-timeframe="90d">90D</button>
+                            <div class="input-chip">
+                                <button class="ghost active" data-chart-timeframe="7d">7D</button>
+                                <button class="ghost" data-chart-timeframe="30d">30D</button>
+                                <button class="ghost" data-chart-timeframe="90d">90D</button>
                             </div>
                         </div>
                     </div>
-                    <div class="tradingview-chart-container glass-vibrant">
-                        <div class="chart-toolbar">
-                            <div class="chart-indicators">
-                                <label class="chart-indicator-toggle">
-                                    <input type="checkbox" data-indicator="MA20" checked />
-                                    <span>MA 20</span>
-                                </label>
-                                <label class="chart-indicator-toggle">
-                                    <input type="checkbox" data-indicator="MA50" />
-                                    <span>MA 50</span>
-                                </label>
-                                <label class="chart-indicator-toggle">
-                                    <input type="checkbox" data-indicator="RSI" />
-                                    <span>RSI</span>
-                                </label>
-                                <label class="chart-indicator-toggle">
-                                    <input type="checkbox" data-indicator="Volume" checked />
-                                    <span>Volume</span>
-                                </label>
-                            </div>
-                        </div>
-                        <canvas id="chart-lab-canvas" height="400"></canvas>
+                    <div class="glass-card">
+                        <canvas id="chart-lab-canvas" height="260"></canvas>
                     </div>
-                    <div class="glass-card glass-vibrant" style="margin-top: 24px;">
-                        <div class="controls-bar" style="margin-bottom: 16px;">
-                            <button class="primary" data-run-analysis style="background: linear-gradient(135deg, #00D4FF, #8B5CF6);">Analyze Chart with AI</button>
+                    <div class="glass-card">
+                        <div class="controls-bar">
+                            <label><input type="checkbox" data-indicator value="MA20" checked /> MA 20</label>
+                            <label><input type="checkbox" data-indicator value="MA50" /> MA 50</label>
+                            <label><input type="checkbox" data-indicator value="RSI" /> RSI</label>
+                            <label><input type="checkbox" data-indicator value="Volume" /> Volume</label>
                         </div>
+                        <button class="primary" data-run-analysis>Analyze Chart with AI</button>
                         <div data-ai-insights class="ai-insights"></div>
                     </div>
                 </section>