Improve documentation

app.py

@@ -196,6 +196,14 @@ if check_password():
                 "documentations/lte_capacity_docs.py",
                 title="📘LTE Capacity Documentation",
             ),
+            st.Page(
+                "documentations/trafic_analysis_doc.py",
+                title="📘Trafic Analysis Documentation",
+            ),
+            st.Page(
+                "documentations/anomaly_detection_doc.py",
+                title="📘Anomaly Detection Documentation",
+            ),
         ],
     }
 

apps/clustering.py

@@ -137,8 +137,12 @@ st.title("Automatic Site Clustering App")
 # Add description
 st.write(
     """This app allows you to cluster sites based on their latitude and longitude.
-    **Please choose a file containing the latitude and longitude region and site code columns.**
-                      """
+    
+    Please choose a file containing :
+    - latitude and longitude columns
+    - region column
+    - site code column
+    """
 )
 
 # Download Sample file

documentations/anomaly_detection_doc.py

@@ -0,0 +1,125 @@
+import streamlit as st
+
+st.markdown(
+    """
+# KPI Anomaly Detection Documentation
+
+## Overview
+The KPI Anomaly Detection application is designed to automatically identify and analyze anomalies in Key Performance Indicators (KPIs) using change point detection algorithms. It helps in identifying significant changes in KPI trends that may indicate network issues or other important events.
+
+## Features
+
+### 1. Data Processing
+- Supports both CSV and Excel file formats
+- Automatic date parsing and data cleaning
+- Handles missing values appropriately
+- Processes multiple KPIs in a single run
+
+### 2. Anomaly Detection
+- Utilizes the PELT (Pruned Exact Linear Time) algorithm for change point detection
+- Configurable penalty parameter to control sensitivity
+- Identifies both sudden and gradual changes in KPI trends
+- Filters out insignificant changes based on mean value differences
+
+### 3. Visualization
+- Interactive time series plots with Plotly
+- Visual indicators for detected change points
+- Displays initial and final mean values for comparison
+- Responsive design for different screen sizes
+
+### 4. Reporting
+- Export detected anomalies to Excel
+- Separate sheets for each KPI
+- Includes all relevant data points and change point indicators
+
+## Input Requirements
+
+### Required File Format
+- **CSV or Excel** file containing time series KPI data
+- First 5 columns should be (in order):
+  1. Date/Time
+  2. Controller ID
+  3. BTS ID
+  4. Cell ID
+  5. DN (Directory Number)
+- Remaining columns should contain KPI values
+
+### Data Requirements
+- At least 30 data points per cell for reliable detection
+- Consistent time intervals between measurements
+- Numeric values for KPI columns
+
+## Usage
+
+### 1. Upload Data
+- Click "Upload KPI file" and select your CSV or Excel file
+- The application will automatically detect the file format
+
+### 2. Configure Detection
+- Adjust the "Penalty" parameter to control sensitivity:
+  - Lower values = More sensitive (more change points detected)
+  - Higher values = Less sensitive (only major changes detected)
+- Default value of 2.5 works well for most cases
+
+### 3. Review Results
+- The application will display a list of KPIs with detected anomalies
+- Select a KPI and cell to view detailed analysis
+- The plot shows:
+  - KPI values over time (blue line)
+  - Detected change points (red markers)
+  - Initial mean (gray dotted line)
+  - Final mean (black dashed line)
+
+### 4. Export Results
+- Click "Generate Excel file with anomalies" to export all detected anomalies
+- Each KPI is saved in a separate sheet
+- The Excel file includes all data points with change point indicators
+
+## Technical Details
+
+### Algorithm
+- Uses the PELT (Pruned Exact Linear Time) algorithm from the `ruptures` library
+- Model: RBF (Radial Basis Function) kernel for detecting changes in mean
+- Automatic pruning of similar change points
+
+### Performance Considerations
+- Processing time depends on:
+  - Number of cells in the dataset
+  - Number of KPIs
+  - Length of the time series
+- Large datasets may take several minutes to process
+- Results are cached for better performance when adjusting parameters
+
+## Troubleshooting
+
+### Common Issues
+1. **No anomalies detected**
+   - Try reducing the penalty value
+   - Check if the data contains enough variation
+   - Ensure there are at least 30 data points per cell
+
+2. **Too many false positives**
+   - Increase the penalty value
+   - Check data for noise or outliers
+   - Consider pre-processing the data
+
+3. **File format errors**
+   - Ensure the file is not open in another program
+   - Check that the file is not corrupted
+   - Verify the column structure matches requirements
+
+## Best Practices
+1. Start with the default penalty value (2.5) and adjust as needed
+2. For large datasets, consider processing in smaller chunks
+3. Review detected anomalies in context with network events or changes
+4. Regularly update the application to get the latest improvements
+
+## Dependencies
+- Python 3.7+
+- pandas
+- numpy
+- plotly
+- ruptures
+- streamlit
+"""
+)

documentations/trafic_analysis_doc.py

@@ -0,0 +1,118 @@
+import streamlit as st
+
+st.markdown(
+    """
+
+# Traffic Analysis Application Documentation
+
+## Overview
+The Traffic Analysis application is a Streamlit-based tool designed to analyze and visualize 2G, 3G, and LTE network traffic data. It provides insights into voice and data traffic patterns, performs comparative analysis between different time periods, and generates visual reports.
+
+## Features
+
+### 1. Multi-Technology Support
+- **2G Traffic Analysis**
+  - Voice traffic analysis
+  - Data traffic analysis
+  - PS (Packet Switched) traffic metrics
+
+- **3G Traffic Analysis**
+  - CS (Circuit Switched) voice traffic
+  - PS data traffic
+  - Combined traffic metrics
+
+- **LTE Traffic Analysis**
+  - UL/DL data traffic
+  - Combined traffic volume
+
+### 2. Comparative Analysis
+- Pre/Post period comparison
+- Traffic trend analysis
+- Monthly traffic aggregation
+- Traffic growth/decline metrics
+
+### 3. Visualization
+- Interactive maps showing traffic distribution
+- Top N sites analysis
+- Traffic trend charts
+- Comparative bar charts
+
+## Input Requirements
+
+### Required Files
+1. **2G Traffic Report**
+   - Required columns: `BCF name`, `PERIOD_START_TIME`, `TRAFFIC_PS DL`, `PS_UL_Load`
+   - Format: CSV or ZIP containing CSV
+
+2. **3G Traffic Report**
+   - Required columns: `WBTS name`, `PERIOD_START_TIME`, `Total CS traffic - Erl`, `Total_Data_Traffic`
+   - Format: CSV or ZIP containing CSV
+
+3. **LTE Traffic Report**
+   - Required columns: `LNBTS name`, `PERIOD_START_TIME`, `4G/LTE DL Traffic Volume (GBytes)`, `4G/LTE UL Traffic Volume (GBytes)`
+   - Format: CSV or ZIP containing CSV
+
+## Usage
+
+### 1. Data Upload
+- Upload the three required reports using the file uploaders
+- Supported formats: CSV or ZIP (containing CSV)
+
+### 2. Date Range Selection
+- **Pre-period**: Select the baseline period for comparison
+- **Post-period**: Select the period to compare against the baseline
+- **Last period**: Select the most recent period for current analysis
+
+### 3. Analysis Configuration
+- Set the number of top traffic sites to display
+- The application will automatically process and analyze the data
+
+### 4. Results
+- **Summary Analysis**: Overview of traffic metrics
+- **Top Sites**: Lists and charts of highest traffic sites
+- **Maps**: Geographical visualization of traffic distribution
+- **Monthly Trends**: Traffic patterns over time
+
+## Technical Implementation
+
+### Key Functions
+
+1. **Data Processing**
+   - preprocess_2g(): Processes 2G traffic data
+   - preprocess_3g(): Processes 3G traffic data
+   - preprocess_lte(): Processes LTE traffic data
+
+2. **Analysis Functions**
+   - merge_and_compare(): Combines and compares traffic data across technologies
+   - monthly_data_analysis(): Aggregates data by month for trend analysis
+
+3. **Visualization**
+   - Interactive maps using Plotly
+   - Bar charts for top sites
+   - Data tables with sortable columns
+
+## Dependencies
+- Python 3.7+
+- pandas
+- plotly
+- streamlit
+- numpy
+
+## Output
+The application generates:
+1. Summary tables of traffic metrics
+2. Interactive visualizations
+3. Exportable reports in Excel format
+
+## Best Practices
+1. Ensure input files follow the required format
+2. Select appropriate date ranges for meaningful comparison
+3. Review top traffic sites for network optimization opportunities
+4. Use the monthly analysis to identify long-term trends
+
+## Troubleshooting
+- If data doesn't load, check file formats and required columns
+- Ensure date ranges are properly selected
+- Verify that uploaded files contain valid data
+"""
+)