| # π Cryptocurrency API Resource Monitor | |
| **Comprehensive cryptocurrency market intelligence API resource management system** | |
| Monitor and manage all API resources from blockchain explorers, market data providers, RPC nodes, news feeds, and more. Track online status, validate endpoints, categorize by domain, and maintain availability metrics across all cryptocurrency data sources. | |
| ## π Table of Contents | |
| - [Features](#-features) | |
| - [Monitored Resources](#-monitored-resources) | |
| - [Quick Start](#-quick-start) | |
| - [Usage](#-usage) | |
| - [Architecture](#-architecture) | |
| - [API Categories](#-api-categories) | |
| - [Status Classification](#-status-classification) | |
| - [Alert Conditions](#-alert-conditions) | |
| - [Failover Management](#-failover-management) | |
| - [Dashboard](#-dashboard) | |
| - [Configuration](#-configuration) | |
| ## β¨ Features | |
| ### Core Monitoring | |
| - β **Real-time health checks** for 50+ cryptocurrency APIs | |
| - β **Response time tracking** with millisecond precision | |
| - β **Success/failure rate monitoring** per provider | |
| - β **Automatic status classification** (ONLINE/DEGRADED/SLOW/UNSTABLE/OFFLINE) | |
| - β **SSL certificate validation** and expiration tracking | |
| - β **Rate limit detection** (429, 403 responses) | |
| ### Redundancy & Failover | |
| - β **Automatic failover chain building** for each data type | |
| - β **Multi-tier resource prioritization** (TIER-1 critical, TIER-2 high, TIER-3 medium, TIER-4 low) | |
| - β **Single Point of Failure (SPOF) detection** | |
| - β **Backup provider recommendations** | |
| - β **Cross-provider data validation** | |
| ### Alerting & Reporting | |
| - β **Critical alert system** for TIER-1 API failures | |
| - β **Performance degradation warnings** | |
| - β **JSON export reports** for integration | |
| - β **Historical uptime statistics** | |
| - β **Real-time web dashboard** with auto-refresh | |
| ### Security & Privacy | |
| - β **API key masking** in all outputs (first/last 4 chars only) | |
| - β **Secure credential storage** from registry | |
| - β **Rate limit compliance** with configurable delays | |
| - β **CORS proxy support** for browser compatibility | |
| ## π Monitored Resources | |
| ### Blockchain Explorers | |
| - **Etherscan** (2 keys): Ethereum blockchain data, transactions, smart contracts | |
| - **BscScan** (1 key): BSC blockchain explorer, BEP-20 tokens | |
| - **TronScan** (1 key): Tron network explorer, TRC-20 tokens | |
| ### Market Data Providers | |
| - **CoinGecko**: Real-time prices, market caps, trending coins (FREE) | |
| - **CoinMarketCap** (2 keys): Professional market data | |
| - **CryptoCompare** (1 key): OHLCV data, historical snapshots | |
| - **CoinPaprika**: Comprehensive market information | |
| - **CoinCap**: Asset pricing and exchange rates | |
| ### RPC Nodes | |
| **Ethereum:** Ankr, PublicNode, Cloudflare, LlamaNodes | |
| **BSC:** Official BSC, Ankr, PublicNode | |
| **Polygon:** Official, Ankr | |
| **Tron:** TronGrid, TronStack | |
| ### News & Sentiment | |
| - **CryptoPanic**: Aggregated news with sentiment scores | |
| - **NewsAPI** (1 key): General crypto news | |
| - **Alternative.me**: Fear & Greed Index | |
| - **Reddit**: r/cryptocurrency JSON feeds | |
| ### Additional Resources | |
| - **Whale Tracking**: WhaleAlert API | |
| - **CORS Proxies**: AllOrigins, CORS.SH, Corsfix, ThingProxy | |
| - **On-Chain Analytics**: The Graph, Blockchair | |
| **Total: 50+ monitored endpoints across 7 categories** | |
| ## π Quick Start | |
| ### Prerequisites | |
| - Node.js 14.0.0 or higher | |
| - Python 3.x (for dashboard server) | |
| ### Installation | |
| ```bash | |
| # Clone the repository | |
| git clone https://github.com/nimazasinich/crypto-dt-source.git | |
| cd crypto-dt-source | |
| # No dependencies to install - uses Node.js built-in modules! | |
| ``` | |
| ### Run Your First Health Check | |
| ```bash | |
| # Run a complete health check | |
| node api-monitor.js | |
| # This will: | |
| # - Load API keys from all_apis_merged_2025.json | |
| # - Check all 50+ endpoints | |
| # - Generate api-monitor-report.json | |
| # - Display status report in terminal | |
| ``` | |
| ### View the Dashboard | |
| # Start the web server | |
| npm run dashboard | |
| # Open in browser: | |
| # http://localhost:8080/dashboard.html | |
| ``` | |
| --- | |
| ## π Usage | |
| ### 1. Single Health Check | |
| ```bash | |
| node api-monitor.js | |
| ``` | |
| **Output:** | |
| ``` | |
| β Registry loaded successfully | |
| Found 7 API key categories | |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β CRYPTOCURRENCY API RESOURCE MONITOR - Health Check β | |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| Checking blockchainExplorers... | |
| Checking marketData... | |
| Checking newsAndSentiment... | |
| Checking rpcNodes... | |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β RESOURCE STATUS REPORT β | |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| π BLOCKCHAINEXPLORERS | |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β Etherscan-1 ONLINE 245ms [TIER-1] | |
| β Etherscan-2 ONLINE 312ms [TIER-1] | |
| β BscScan ONLINE 189ms [TIER-1] | |
| β TronScan ONLINE 567ms [TIER-2] | |
| π MARKETDATA | |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β CoinGecko ONLINE 142ms [TIER-1] | |
| β CoinGecko-Price ONLINE 156ms [TIER-1] | |
| β CoinMarketCap-1 DEGRADED 2340ms [TIER-1] | |
| β CoinMarketCap-2 ONLINE 487ms [TIER-1] | |
| β CryptoCompare ONLINE 298ms [TIER-2] | |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β SUMMARY β | |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| Total Resources: 52 | |
| Online: 48 (92.3%) | |
| Degraded: 3 (5.8%) | |
| Offline: 1 (1.9%) | |
| Overall Health: 92.3% | |
| β Report exported to api-monitor-report.json | |
| ``` | |
| ### 2. Continuous Monitoring | |
| ```bash | |
| node api-monitor.js --continuous | |
| ``` | |
| Runs health checks every 5 minutes and continuously updates the report. | |
| ### 3. Failover Analysis | |
| ```bash | |
| node failover-manager.js | |
| ``` | |
| **Output:** | |
| ``` | |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β FAILOVER CHAIN BUILDER β | |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| π ETHEREUMPRICE Failover Chain: | |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| π― [PRIMARY] CoinGecko ONLINE 142ms [TIER-1] | |
| β [BACKUP] CoinMarketCap-2 ONLINE 487ms [TIER-1] | |
| β [BACKUP-2] CryptoCompare ONLINE 298ms [TIER-2] | |
| β [BACKUP-3] CoinPaprika ONLINE 534ms [TIER-2] | |
| π ETHEREUMEXPLORER Failover Chain: | |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| π― [PRIMARY] Etherscan-1 ONLINE 245ms [TIER-1] | |
| β [BACKUP] Etherscan-2 ONLINE 312ms [TIER-1] | |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β SINGLE POINT OF FAILURE ANALYSIS β | |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| π‘ [MEDIUM] rpcPolygon: Only two resources available | |
| π [HIGH] sentiment: Only one resource available (SPOF) | |
| β Failover configuration exported to failover-config.json | |
| ``` | |
| ### 4. Launch Complete Dashboard | |
| ```bash | |
| npm run full-check | |
| ``` | |
| Runs monitor β failover analysis β starts web dashboard | |
| --- | |
| ## ποΈ Architecture | |
| ``` | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β API REGISTRY JSON β | |
| β (all_apis_merged_2025.json) β | |
| β - Discovered keys (masked) β | |
| β - Raw API configurations β | |
| ββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββ | |
| β | |
| βΌ | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β CRYPTO API MONITOR β | |
| β (api-monitor.js) β | |
| β β | |
| β βββββββββββββββββββββββββββββββββββββββββββ β | |
| β β Resource Loader β β | |
| β β - Parse registry β β | |
| β β - Extract API keys β β | |
| β β - Build endpoint URLs β β | |
| β βββββββββββββββββββββββββββββββββββββββββββ β | |
| β β β | |
| β βββββββββββββββββββββββββββββββββββββββββββ β | |
| β β Health Check Engine β β | |
| β β - HTTP/HTTPS requests β β | |
| β β - Response time measurement β β | |
| β β - Status code validation β β | |
| β β - RPC endpoint testing β β | |
| β βββββββββββββββββββββββββββββββββββββββββββ β | |
| β β β | |
| β βββββββββββββββββββββββββββββββββββββββββββ β | |
| β β Status Classifier β β | |
| β β - Success rate calculation β β | |
| β β - Response time averaging β β | |
| β β - ONLINE/DEGRADED/OFFLINE β β | |
| β βββββββββββββββββββββββββββββββββββββββββββ β | |
| β β β | |
| β βββββββββββββββββββββββββββββββββββββββββββ β | |
| β β Alert System β β | |
| β β - TIER-1 failure detection β β | |
| β β - Performance warnings β β | |
| β β - Critical notifications β β | |
| β βββββββββββββββββββββββββββββββββββββββββββ β | |
| ββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββ | |
| β | |
| βΌ | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β MONITORING REPORT JSON β | |
| β (api-monitor-report.json) β | |
| β - Summary statistics β | |
| β - Per-resource status β | |
| β - Historical data β | |
| β - Active alerts β | |
| ββββββββββ¬βββββββββββββββββββββββββββββββ¬ββββββββββββββββββ | |
| β β | |
| βΌ βΌ | |
| βββββββββββββββββββββββ ββββββββββββββββββββββββββββββββ | |
| β FAILOVER MANAGER β β WEB DASHBOARD β | |
| β (failover-manager) β β (dashboard.html) β | |
| β β β β | |
| β - Build chains β β - Real-time visualization β | |
| β - SPOF detection β β - Auto-refresh β | |
| β - Redundancy report β β - Alert display β | |
| β - Export config β β - Health metrics β | |
| βββββββββββββββββββββββ ββββββββββββββββββββββββββββββββ | |
| ``` | |
| --- | |
| ## π API Categories | |
| ### 1. Blockchain Explorers | |
| **Purpose:** Query blockchain data, transactions, balances, smart contracts | |
| **Resources:** | |
| - Etherscan (Ethereum) - 2 keys | |
| - BscScan (BSC) - 1 key | |
| - TronScan (Tron) - 1 key | |
| **Use Cases:** | |
| - Get wallet balances | |
| - Track transactions | |
| - Monitor token transfers | |
| - Query smart contracts | |
| - Get gas prices | |
| ### 2. Market Data | |
| **Purpose:** Real-time cryptocurrency prices, market caps, volume | |
| **Resources:** | |
| - CoinGecko (FREE, no key required) β | |
| - CoinMarketCap - 2 keys | |
| - CryptoCompare - 1 key | |
| - CoinPaprika (FREE) | |
| - CoinCap (FREE) | |
| **Use Cases:** | |
| - Live price feeds | |
| - Historical OHLCV data | |
| - Market cap rankings | |
| - Trading volume | |
| - Trending coins | |
| ### 3. RPC Nodes | |
| **Purpose:** Direct blockchain interaction via JSON-RPC | |
| **Resources:** | |
| - **Ethereum:** Ankr, PublicNode, Cloudflare, LlamaNodes | |
| - **BSC:** Official, Ankr, PublicNode | |
| - **Polygon:** Official, Ankr | |
| - **Tron:** TronGrid, TronStack | |
| **Use Cases:** | |
| - Send transactions | |
| - Read smart contracts | |
| - Get block data | |
| - Subscribe to events | |
| - Query state | |
| ### 4. News & Sentiment | |
| **Purpose:** Crypto news aggregation and market sentiment | |
| **Resources:** | |
| - CryptoPanic (FREE) | |
| - Alternative.me Fear & Greed Index (FREE) | |
| - NewsAPI - 1 key | |
| - Reddit r/cryptocurrency (FREE) | |
| **Use Cases:** | |
| - News feed aggregation | |
| - Sentiment analysis | |
| - Fear & Greed tracking | |
| - Social signals | |
| ### 5. Whale Tracking | |
| **Purpose:** Monitor large cryptocurrency transactions | |
| **Resources:** | |
| - WhaleAlert API | |
| **Use Cases:** | |
| - Track whale movements | |
| - Exchange flow monitoring | |
| - Large transaction alerts | |
| ### 6. CORS Proxies | |
| **Purpose:** Bypass CORS restrictions in browser applications | |
| **Resources:** | |
| - AllOrigins (unlimited) | |
| - CORS.SH (fast) | |
| - Corsfix (60 req/min) | |
| - ThingProxy (10 req/sec) | |
| **Use Cases:** | |
| - Browser-based API calls | |
| - Frontend applications | |
| - CORS workarounds | |
| --- | |
| ## π Status Classification | |
| The monitor automatically classifies each API into one of five states: | |
| | Status | Success Rate | Response Time | Description | | |
| |--------|--------------|---------------|-------------| | |
| | π’ **ONLINE** | β₯95% | <2 seconds | Fully operational, optimal performance | | |
| | π‘ **DEGRADED** | 80-95% | 2-5 seconds | Functional but slower than normal | | |
| | π **SLOW** | 70-80% | 5-10 seconds | Significant performance issues | | |
| | π΄ **UNSTABLE** | 50-70% | Any | Frequent failures, unreliable | | |
| | β« **OFFLINE** | <50% | Any | Not responding or completely down | | |
| **Classification Logic:** | |
| - Based on last 10 health checks | |
| - Success rate = successful responses / total attempts | |
| - Response time = average of successful requests only | |
| --- | |
| ## β οΈ Alert Conditions | |
| The system triggers alerts for: | |
| ### Critical Alerts | |
| - β TIER-1 API offline (Etherscan, CoinGecko, Infura, Alchemy) | |
| - β All providers in a category offline | |
| - β Zero available resources for essential data type | |
| ### Warning Alerts | |
| - β οΈ Response time >5 seconds sustained for 15 minutes | |
| - β οΈ Success rate dropped below 80% | |
| - β οΈ Single Point of Failure (only 1 provider available) | |
| - β οΈ Rate limit reached (>80% consumed) | |
| ### Info Alerts | |
| - βΉοΈ API key approaching expiration | |
| - βΉοΈ SSL certificate expires within 7 days | |
| - βΉοΈ New resource added to registry | |
| --- | |
| ## π Failover Management | |
| ### Automatic Failover Chains | |
| The system builds intelligent failover chains for each data type: | |
| ```javascript | |
| // Example: Ethereum Price Failover Chain | |
| const failoverConfig = require('./failover-config.json'); | |
| async function getEthereumPrice() { | |
| const chain = failoverConfig.chains.ethereumPrice; | |
| for (const resource of chain) { | |
| try { | |
| // Try primary first (CoinGecko) | |
| const response = await fetch(resource.url + '/api/v3/simple/price?ids=ethereum&vs_currencies=usd'); | |
| const data = await response.json(); | |
| return data.ethereum.usd; | |
| } catch (error) { | |
| console.log(`${resource.name} failed, trying next in chain...`); | |
| continue; | |
| } | |
| } | |
| throw new Error('All resources in failover chain failed'); | |
| } | |
| ``` | |
| ### Priority Tiers | |
| **TIER-1 (CRITICAL):** Etherscan, BscScan, CoinGecko, Infura, Alchemy | |
| **TIER-2 (HIGH):** CoinMarketCap, CryptoCompare, TronScan, NewsAPI | |
| **TIER-3 (MEDIUM):** Alternative.me, Reddit, CORS proxies, public RPCs | |
| **TIER-4 (LOW):** Experimental APIs, community nodes, backup sources | |
| Failover chains prioritize lower tier numbers first. | |
| --- | |
| ## π¨ Dashboard | |
| ### Features | |
| - **Real-time monitoring** with auto-refresh every 5 minutes | |
| - **Visual health indicators** with color-coded status | |
| - **Category breakdown** showing all resources by type | |
| - **Alert notifications** prominently displayed | |
| - **Health bar** showing overall system status | |
| - **Response times** for each endpoint | |
| - **Tier badges** showing resource priority | |
| ### Screenshots | |
| **Summary Cards:** | |
| ``` | |
| βββββββββββββββββββ βββββββββββββββββββ βββββββββββββββββββ βββββββββββββββββββ | |
| β Total Resources β β Online β β Degraded β β Offline β | |
| β 52 β β 48 (92.3%) β β 3 (5.8%) β β 1 (1.9%) β | |
| βββββββββββββββββββ βββββββββββββββββββ βββββββββββββββββββ βββββββββββββββββββ | |
| ``` | |
| **Resource List:** | |
| ``` | |
| π BLOCKCHAIN EXPLORERS | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β Etherscan-1 [TIER-1] ONLINE 245ms | |
| β Etherscan-2 [TIER-1] ONLINE 312ms | |
| β BscScan [TIER-1] ONLINE 189ms | |
| ``` | |
| ### Access | |
| ```bash | |
| npm run dashboard | |
| # Open: http://localhost:8080/dashboard.html | |
| ``` | |
| --- | |
| ## βοΈ Configuration | |
| ### Monitor Configuration | |
| Edit `api-monitor.js`: | |
| ```javascript | |
| const CONFIG = { | |
| REGISTRY_FILE: './all_apis_merged_2025.json', | |
| CHECK_INTERVAL: 5 * 60 * 1000, // 5 minutes | |
| TIMEOUT: 10000, // 10 seconds | |
| MAX_RETRIES: 3, | |
| RETRY_DELAY: 2000, | |
| THRESHOLDS: { | |
| ONLINE: { responseTime: 2000, successRate: 0.95 }, | |
| DEGRADED: { responseTime: 5000, successRate: 0.80 }, | |
| SLOW: { responseTime: 10000, successRate: 0.70 }, | |
| UNSTABLE: { responseTime: Infinity, successRate: 0.50 } | |
| } | |
| }; | |
| ``` | |
| ### Adding New Resources | |
| Edit the `API_REGISTRY` object in `api-monitor.js`: | |
| ```javascript | |
| marketData: { | |
| // ... existing resources ... | |
| newProvider: [ | |
| { | |
| name: 'MyNewAPI', | |
| url: 'https://api.example.com', | |
| testEndpoint: '/health', | |
| requiresKey: false, | |
| tier: 3 | |
| } | |
| ] | |
| } | |
| ``` | |
| --- | |
| ## π Security Notes | |
| - β API keys are **never logged** in full (masked to first/last 4 chars) | |
| - β Registry file should be kept **secure** and not committed to public repos | |
| - β Use **environment variables** for production deployments | |
| - β Rate limits are **automatically respected** with delays | |
| - β SSL/TLS is used for all external API calls | |
| --- | |
| ## π Output Files | |
| | File | Purpose | Format | | |
| |------|---------|--------| | |
| | `api-monitor-report.json` | Complete health check results | JSON | | |
| | `failover-config.json` | Failover chain configuration | JSON | | |
| ### api-monitor-report.json Structure | |
| ```json | |
| { | |
| "timestamp": "2025-11-10T22:30:00.000Z", | |
| "summary": { | |
| "totalResources": 52, | |
| "onlineResources": 48, | |
| "degradedResources": 3, | |
| "offlineResources": 1 | |
| }, | |
| "categories": { | |
| "blockchainExplorers": [...], | |
| "marketData": [...], | |
| "rpcNodes": [...] | |
| }, | |
| "alerts": [ | |
| { | |
| "severity": "CRITICAL", | |
| "message": "TIER-1 API offline: Etherscan-1", | |
| "timestamp": "2025-11-10T22:28:15.000Z" | |
| } | |
| ], | |
| "history": { | |
| "CoinGecko": [ | |
| { | |
| "success": true, | |
| "responseTime": 142, | |
| "timestamp": "2025-11-10T22:30:00.000Z" | |
| } | |
| ] | |
| } | |
| } | |
| ``` | |
| --- | |
| ## π οΈ Troubleshooting | |
| ### "Failed to load registry" | |
| **Cause:** `all_apis_merged_2025.json` not found | |
| **Solution:** Ensure the file exists in the same directory | |
| ### "Request timeout" errors | |
| **Cause:** API endpoint is slow or down | |
| **Solution:** Normal behavior, will be classified as SLOW/OFFLINE | |
| ### "CORS error" in dashboard | |
| **Cause:** Report JSON not accessible | |
| **Solution:** Run `npm run dashboard` to start local server | |
| ### Rate limit errors (429) | |
| **Cause:** Too many requests to API | |
| **Solution:** Increase `CHECK_INTERVAL` or reduce resource list | |
| --- | |
| ## π License | |
| MIT License - see LICENSE file for details | |
| --- | |
| ## π€ Contributing | |
| Contributions welcome! To add new API resources: | |
| 1. Update `API_REGISTRY` in `api-monitor.js` | |
| 2. Add test endpoint | |
| 3. Classify into appropriate tier | |
| 4. Update this README | |
| --- | |
| ## π Support | |
| For issues or questions: | |
| - Open an issue on GitHub | |
| - Check the troubleshooting section | |
| - Review configuration opt | |
| **Built with β€οΈ for the cryptocurrency community** | |
| *Monitor smarter, not harder | |
| # Crypto Resource Aggregator | |
| A centralized API aggregator for cryptocurrency resources hosted on Hugging Face Spaces. | |
| ## Overview | |
| This aggregator consolidates multiple cryptocurrency data sources including: | |
| - **Block Explorers**: Etherscan, BscScan, TronScan | |
| - **Market Data**: CoinGecko, CoinMarketCap, CryptoCompare | |
| - **RPC Endpoints**: Ethereum, BSC, Tron, Polygon | |
| - **News APIs**: Crypto news and sentiment analysis | |
| - **Whale Tracking**: Large transaction monitoring | |
| - **On-chain Analytics**: Blockchain data analysis | |
| ## Features | |
| ### β Real-Time Monitoring | |
| - Continuous health checks for all resources | |
| - Automatic status updates (online/offline) | |
| - Response time tracking | |
| - Consecutive failure counting | |
| ### π History Tracking | |
| - Complete query history with timestamps | |
| - Resource usage statistics | |
| - Success/failure rates | |
| - Average response times | |
| ### π No Mock Data | |
| - All responses return real data from actual APIs | |
| - Error status returned when resources are unavailable | |
| - Transparent error messaging | |
| ### π Fallback Support | |
| - Automatic fallback to alternative resources | |
| - Multiple API keys for rate limit management | |
| - CORS proxy support for browser access | |
| ## API Endpoints | |
| ### Resource Management | |
| #### `GET /` | |
| Root endpoint with API information and available endpoints. | |
| #### `GET /resources` | |
| List all available resource categories and their counts. | |
| **Response:** | |
| ```json | |
| { | |
| "total_categories": 7, | |
| "resources": { | |
| "block_explorers": ["etherscan", "bscscan", "tronscan"], | |
| "market_data": ["coingecko", "coinmarketcap"], | |
| "rpc_endpoints": [...], | |
| ... | |
| }, | |
| "timestamp": "2025-11-10T..." | |
| } | |
| ``` | |
| #### `GET /resources/{category}` | |
| Get all resources in a specific category. | |
| **Example:** `/resources/market_data` | |
| ### Query Resources | |
| #### `POST /query` | |
| Query a specific resource with parameters. | |
| **Request Body:** | |
| ```json | |
| { | |
| "resource_type": "market_data", | |
| "resource_name": "coingecko", | |
| "endpoint": "/simple/price", | |
| "params": { | |
| "ids": "bitcoin,ethereum", | |
| "vs_currencies": "usd" | |
| } | |
| } | |
| ``` | |
| **Response:** | |
| ```json | |
| { | |
| "success": true, | |
| "resource_type": "market_data", | |
| "resource_name": "coingecko", | |
| "data": { | |
| "bitcoin": {"usd": 45000}, | |
| "ethereum": {"usd": 3000} | |
| }, | |
| "response_time": 0.234, | |
| "timestamp": "2025-11-10T..." | |
| } | |
| ``` | |
| ### Status Monitoring | |
| #### `GET /status` | |
| Get real-time status of all resources. | |
| **Response:** | |
| ```json | |
| { | |
| "total_resources": 15, | |
| "online": 13, | |
| "offline": 2, | |
| "resources": [ | |
| { | |
| "resource": "block_explorers.etherscan", | |
| "status": "online", | |
| "response_time": 0.123, | |
| "error": null, | |
| "timestamp": "2025-11-10T..." | |
| }, | |
| ... | |
| ], | |
| "timestamp": "2025-11-10T..." | |
| } | |
| ``` | |
| #### `GET /status/{category}/{name}` | |
| Check status of a specific resource. | |
| **Example:** `/status/market_data/coingecko` | |
| ### History & Analytics | |
| #### `GET /history` | |
| Get query history (default: last 100 queries). | |
| **Query Parameters:** | |
| - `limit` (optional): Number of records to return (default: 100) | |
| - `resource_type` (optional): Filter by resource type | |
| **Response:** | |
| ```json | |
| { | |
| "count": 100, | |
| "history": [ | |
| { | |
| "id": 1, | |
| "timestamp": "2025-11-10T10:30:00", | |
| "resource_type": "market_data", | |
| "resource_name": "coingecko", | |
| "endpoint": "https://api.coingecko.com/...", | |
| "status": "success", | |
| "response_time": 0.234, | |
| "error_message": null | |
| }, | |
| ... | |
| ] | |
| } | |
| ``` | |
| #### `GET /history/stats` | |
| Get aggregated statistics from query history. | |
| **Response:** | |
| ```json | |
| { | |
| "total_queries": 1523, | |
| "successful_queries": 1487, | |
| "success_rate": 97.6, | |
| "most_queried_resources": [ | |
| {"resource": "coingecko", "count": 456}, | |
| {"resource": "etherscan", "count": 234} | |
| ], | |
| "average_response_time": 0.345, | |
| "timestamp": "2025-11-10T..." | |
| } | |
| ``` | |
| #### `GET /health` | |
| System health check endpoint. | |
| ## Usage Examples | |
| ### JavaScript/TypeScript | |
| ```javascript | |
| // Get Bitcoin price from CoinGecko | |
| const response = await fetch('https://your-space.hf.space/query', { | |
| method: 'POST', | |
| headers: { | |
| 'Content-Type': 'application/json' | |
| }, | |
| body: JSON.stringify({ | |
| resource_type: 'market_data', | |
| resource_name: 'coingecko', | |
| endpoint: '/simple/price', | |
| params: { | |
| ids: 'bitcoin', | |
| vs_currencies: 'usd' | |
| } | |
| }) | |
| }); | |
| const data = await response.json(); | |
| console.log('BTC Price:', data.data.bitcoin.usd); | |
| // Check Ethereum balance | |
| const balanceResponse = await fetch('https://your-space.hf.space/query', { | |
| method: 'POST', | |
| headers: { | |
| 'Content-Type': 'application/json' | |
| }, | |
| body: JSON.stringify({ | |
| resource_type: 'block_explorers', | |
| resource_name: 'etherscan', | |
| endpoint: '', | |
| params: { | |
| module: 'account', | |
| action: 'balance', | |
| address: '0x742d35Cc6634C0532925a3b844Bc9e7595f0bEb', | |
| tag: 'latest' | |
| } | |
| }) | |
| }); | |
| const balanceData = await balanceResponse.json(); | |
| console.log('ETH Balance:', balanceData.data.result / 1e18); | |
| ``` | |
| ### Python | |
| ```python | |
| import requests | |
| # Query CoinGecko for multiple coins | |
| response = requests.post('https://your-space.hf.space/query', json={ | |
| 'resource_type': 'market_data', | |
| 'resource_name': 'coingecko', | |
| 'endpoint': '/simple/price', | |
| 'params': { | |
| 'ids': 'bitcoin,ethereum,tron', | |
| 'vs_currencies': 'usd,eur' | |
| } | |
| }) | |
| data = response.json() | |
| if data['success']: | |
| print('Prices:', data['data']) | |
| else: | |
| print('Error:', data['error']) | |
| # Get resource status | |
| status = requests.get('https://your-space.hf.space/status') | |
| print(f"Resources online: {status.json()['online']}/{status.json()['total_resources']}") | |
| ``` | |
| ### cURL | |
| ```bash | |
| # List all resources | |
| curl https://your-space.hf.space/resources | |
| # Query a resource | |
| curl -X POST https://your-space.hf.space/query \ | |
| -H "Content-Type: application/json" \ | |
| -d '{ | |
| "resource_type": "market_data", | |
| "resource_name": "coingecko", | |
| "endpoint": "/simple/price", | |
| "params": { | |
| "ids": "bitcoin", | |
| "vs_currencies": "usd" | |
| } | |
| }' | |
| # Get status | |
| curl https://your-space.hf.space/status | |
| # Get history | |
| curl https://your-space.hf.space/history?limit=50 | |
| ``` | |
| ## Resource Categories | |
| ### Block Explorers | |
| - **Etherscan**: Ethereum blockchain explorer with API key | |
| - **BscScan**: BSC blockchain explorer with API key | |
| - **TronScan**: Tron blockchain explorer with API key | |
| ### Market Data | |
| - **CoinGecko**: Free, no API key required | |
| - **CoinMarketCap**: Requires API key, 333 calls/day free tier | |
| - **CryptoCompare**: 100K calls/month free tier | |
| ### RPC Endpoints | |
| - Ethereum (Infura, Alchemy, Ankr) | |
| - Binance Smart Chain | |
| - Tron | |
| - Polygon | |
| ## Database Schema | |
| ### query_history | |
| Tracks all API queries made through the aggregator. | |
| ```sql | |
| CREATE TABLE query_history ( | |
| id INTEGER PRIMARY KEY AUTOINCREMENT, | |
| timestamp DATETIME DEFAULT CURRENT_TIMESTAMP, | |
| resource_type TEXT NOT NULL, | |
| resource_name TEXT NOT NULL, | |
| endpoint TEXT NOT NULL, | |
| status TEXT NOT NULL, | |
| response_time REAL, | |
| error_message TEXT | |
| ); | |
| ``` | |
| ### resource_status | |
| Tracks the health status of each resource. | |
| ```sql | |
| CREATE TABLE resource_status ( | |
| id INTEGER PRIMARY KEY AUTOINCREMENT, | |
| resource_name TEXT NOT NULL UNIQUE, | |
| last_check DATETIME DEFAULT CURRENT_TIMESTAMP, | |
| status TEXT NOT NULL, | |
| consecutive_failures INTEGER DEFAULT 0, | |
| last_success DATETIME, | |
| last_error TEXT | |
| ); | |
| ``` | |
| ## Error Handling | |
| The aggregator returns structured error responses: | |
| ```json | |
| { | |
| "success": false, | |
| "resource_type": "market_data", | |
| "resource_name": "coinmarketcap", | |
| "error": "HTTP 429 - Rate limit exceeded", | |
| "response_time": 0.156, | |
| "timestamp": "2025-11-10T..." | |
| } | |
| ``` | |
| ## Deployment on Hugging Face | |
| 1. Create a new Space on Hugging Face | |
| 2. Select "Gradio" as the SDK (we'll use FastAPI which is compatible) | |
| 3. Upload the following files: | |
| - `app.py` | |
| - `requirements.txt` | |
| - `all_apis_merged_2025.json` | |
| - `README.md` | |
| 4. The Space will automatically deploy | |
| ## Local Development | |
| ```bash | |
| # Install dependencies | |
| pip install -r requirements.txt | |
| # Run the application | |
| python app.py | |
| # Access the API | |
| # Documentation: http://localhost:7860/docs | |
| # API: http://localhost:7860 | |
| ``` | |
| ## Integration with Your Main App | |
| ```javascript | |
| // Create a client wrapper | |
| class CryptoAggregator { | |
| constructor(baseUrl = 'https://your-space.hf.space') { | |
| this.baseUrl = baseUrl; | |
| } | |
| async query(resourceType, resourceName, endpoint = '', params = {}) { | |
| const response = await fetch(`${this.baseUrl}/query`, { | |
| method: 'POST', | |
| headers: { 'Content-Type': 'application/json' }, | |
| body: JSON.stringify({ | |
| resource_type: resourceType, | |
| resource_name: resourceName, | |
| endpoint: endpoint, | |
| params: params | |
| }) | |
| }); | |
| return await response.json(); | |
| } | |
| async getStatus() { | |
| const response = await fetch(`${this.baseUrl}/status`); | |
| return await response.json(); | |
| } | |
| async getHistory(limit = 100) { | |
| const response = await fetch(`${this.baseUrl}/history?limit=${limit}`); | |
| return await response.json(); | |
| } | |
| } | |
| // Usage | |
| const aggregator = new CryptoAggregator(); | |
| // Get Bitcoin price | |
| const price = await aggregator.query('market_data', 'coingecko', '/simple/price', { | |
| ids: 'bitcoin', | |
| vs_currencies: 'usd' | |
| }); | |
| // Check system status | |
| const status = await aggregator.getStatus(); | |
| console.log(`${status.online}/${status.total_resources} resources online`); | |
| ``` | |
| ## Monitoring & Maintenance | |
| - Check `/status` regularly to ensure resources are online | |
| - Monitor `/history/stats` for usage patterns and success rates | |
| - Review consecutive failures in the database | |
| - Update API keys when needed | |
| ## License | |
| This aggregator is built for educational and development purposes. | |
| API keys should be kept secure and rate limits respected. | |
| ## Support | |
| For issues or questions: | |
| 1. Check the `/health` endpoint | |
| 2. Review `/history` for error patterns | |
| 3. Verify resource status with `/status` | |
| 4. Check individual resource documentation | |
| --- | |
| Built with FastAPI and deployed on Hugging Face Spaces | |