naicasc/all-MiniLM-L6-v2-finetuned-it-project-docs-qa-10000-5ep

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2 on the it_project_docs dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • it_project_docs

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("gabrielegabellone/all-mini-itprojectdocs-10000-5-epoch")
# Run inference
queries = [
    "How is configuration managed in CoreOps, and what is the order of precedence?",
]
documents = [
    '## Configuration\n\nCoreOps is highly configurable, allowing you to tailor its behavior to your specific environment. Configuration can be managed through a central `coreops.yaml` file, environment variables, or command-line flags. The order of precedence is: command-line flags > environment variables > `coreops.yaml` > default values.',
    "## About PylonGrid\n\nPylonGrid aims to simplify the complexity of managing large-scale distributed computing environments. It acts as a central nervous system for your compute resources, abstracting away the underlying infrastructure details and providing a unified interface for submitting, monitoring, and managing computational jobs. Whether you're running scientific simulations, data processing pipelines, or microservices, PylonGrid ensures your workloads are executed on the most suitable resources, maximizing throughput and minimizing operational overhead.\n\nKey principles guiding PylonGrid's development:\n*   **Efficiency:** Optimize resource utilization through intelligent scheduling algorithms.\n*   **Reliability:** Ensure job completion even in the face of node failures or network partitions.\n*   **Flexibility:** Support diverse workloads and integrate with various infrastructure types (on-prem, cloud, hybrid).\n*   **Scalability:** Designed to manage thousands of nodes and millions of jobs.\n*   **Usability:** Provide intuitive CLI and API for developers and operators.",
    '## Getting Started: Configuration\n\nOmniTrail\'s configuration is primarily managed through environment variables for Docker Compose and values files for Helm deployments. Key configuration aspects include data sources, storage, and authentication.\n\n### Environment Variables (Docker Compose)\n\nEdit the `deploy/docker-compose.yml` file or create a `.env` file in the root directory. Common variables:\n\n*   `OT_STORAGE_RETENTION_DAYS`: Number of days to retain data (e.g., `30`). Default is `7`.\n*   `OT_API_PORT`: Port for the API service (e.g., `8081`). Default is `8081`.\n*   `OT_WEB_PORT`: Port for the Web UI service (e.g., `8080`). Default is `8080`.\n*   `OT_AUTH_ENABLED`: Set to `true` to enable user authentication. Default is `false`.\n*   `OT_AUTH_JWT_SECRET`: Secret key for JWT token generation (required if `OT_AUTH_ENABLED=true`).\n\nExample `.env`:\n```dotenv\nOT_STORAGE_RETENTION_DAYS=90\nOT_AUTH_ENABLED=true\nOT_AUTH_JWT_SECRET=your_super_secret_jwt_key_here\n```\n\n### Helm Values (Kubernetes)\n\nFor Kubernetes, configuration is managed via the `values.yaml` file or by passing `--set` flags during `helm install` or `helm upgrade`.\n\nCreate a custom `my-values.yaml` file:\n```yaml\nomnitrail-core:\n  storage:\n    retentionDays: 90\nomnitrail-web:\n  service:\n    type: LoadBalancer\nomnitrail-api:\n  auth:\n    enabled: true\n    jwtSecret: "your_super_secret_jwt_key_here"\n```\n\nThen install with:\n```bash\nhelm install omnitrail omnitrail/omnitrail -n omnitrail -f my-values.yaml\n```\n\n### Data Source Configuration\n\nOmniTrail agents and SDKs are configured separately to point to the OmniTrail ingestion endpoint. For example, an `omnitrail-agent` configuration might look like:\n\n```yaml\n# /etc/omnitrail-agent/config.yaml\ningestion_url: "http://omnitrail-api.omnitrail.svc.cluster.local:8081/api/v1/ingest"\nmetrics:\n  enabled: true\n  collect_interval: 15s\nlogs:\n  enabled: true\n  paths:\n    - /var/log/*.log\n```\nRefer to the [Configuration Guide](https://docs.omnitrail.io/configuration) for a complete list of configurable parameters.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 384] [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[ 0.8637, -0.0863,  0.1323]])

Evaluation

Metrics

Triplet

• Dataset: itprojectdocs-dev
• Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.9946

Training Details

Training Dataset

it_project_docs

• Dataset: it_project_docs at 1ac9a47
• Size: 8,019 training samples
• Columns: question and answer
• Approximate statistics based on the first 1000 samples:

  	question	answer
  type	string	string
  details	min: 9 tokens mean: 18.71 tokens max: 45 tokens	min: 17 tokens mean: 202.1 tokens max: 256 tokens

• Samples:

  question	answer
  What database is recommended for the AetherMap project?	### Prerequisites To run AetherMap, you will need: * Operating System: Linux (Ubuntu 20.04+, CentOS 7+, RHEL 8+), macOS. Windows is supported via WSL2. * Docker & Docker Compose: Recommended for containerized deployment (version 20.10+ and 1.29+ respectively). * Go: Version 1.20+ (if building from source). * Node.js & npm/yarn: Version 18+ (if developing the web UI or building from source). * Database: PostgreSQL 12+ (for persistent storage of configuration, historical data, and discovered topology). A local SQLite database can be used for quick testing. * Minimum Hardware: 4 vCPUs, 8GB RAM, 50GB SSD storage (for small deployments). Larger deployments will require more resources.
  What container runtime is necessary for the CodeRun project to function?	## Installation CodeRun is designed for deployment in Linux environments, leveraging containerization for its core functionality. It can be deployed as a standalone service or integrated into existing infrastructure. This section outlines the various methods to get CodeRun up and running. Regardless of the installation method, CodeRun requires a compatible container runtime (like Docker) to be installed and properly configured on the host system. It's recommended to run CodeRun on a dedicated server or VM for optimal performance and security isolation.
  How do I configure CoreForge for a new project using the coreforge.yaml file?	## Configuration CoreForge uses a coreforge.yaml file (or coreforge.json) in your project directory for global settings and provider configurations. Provider-specific credentials are typically managed via environment variables or cloud provider CLI configurations. Example coreforge.yaml: yaml<br># coreforge.yaml<br><br>project_name: "MyWebAppInfrastructure"<br>state_backend:<br> type: "s3" # or "local", "gcs", "azureblob"<br> config:<br> bucket: "coreforge-state-bucket-123"<br> region: "us-east-1"<br> key_prefix: "my-web-app/"<br><br>providers:<br> aws:<br> region: "us-east-1"<br> # AWS credentials typically from environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)<br> kubernetes:<br> kubeconfig_path: "~/.kube/config"<br> context: "my-cluster-context"<br><br>plugins_dir: "./plugins"<br>log_level: "info"<br> * project_name: A unique identifier for your infrastructure project. * state_backend: Defines where CoreForge stores its state file. Options include local (for developme...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

it_project_docs

• Dataset: it_project_docs at 1ac9a47
• Size: 2,005 evaluation samples
• Columns: question and answer
• Approximate statistics based on the first 1000 samples:

  	question	answer
  type	string	string
  details	min: 10 tokens mean: 18.9 tokens max: 54 tokens	min: 17 tokens mean: 204.79 tokens max: 256 tokens

• Samples:

  question	answer
  How can I benchmark the performance of optimized code against original code using the LoopNest project?	## Benchmarking & Performance Analysis LoopNest includes built-in capabilities to benchmark the performance of original versus optimized code, providing empirical evidence of its effectiveness. It also generates detailed reports to help users understand the performance impact of transformations. ### Running Benchmarks To run benchmarks, you typically need to provide a test harness or a main function that calls the kernel you wish to benchmark. LoopNest can then compile and execute both versions. bash<br>loopnest benchmark my_kernel.c --output my_kernel_optimized.c --target cpu --iterations 100 --warmup 10 --report benchmark_results.json<br> Explanation of Arguments: * benchmark: The command to run performance comparisons. * --iterations 100: Number of times to run the kernel for measurement. * --warmup 10: Number of initial runs to warm up caches and stabilize CPU frequencies. * --report benchmark_results.json: Generates a JSON report containing detailed timin...
  What kind of cloud provider credentials and database are needed before installing NestHawk?	### Prerequisites Before installing NestHawk, ensure you have the following: * Go (1.18+): Required if building from source. * Docker (20.10+): Required if using Docker. * Cloud Provider Credentials: Appropriate IAM roles or service accounts with read-only access to your cloud resources for discovery. Refer to our Cloud Provider Setup Guide for detailed instructions on configuring permissions. * PostgreSQL (12+): A database instance for storing NestHawk's operational data and analysis results.
  How can a developer set the default cloud provider for LambdaShell using the ls config command?	## Configuration LambdaShell uses a global configuration file located at ~/.lambdashell/config.json (or %USERPROFILE%\.lambdashell\config.json on Windows) to store default settings and cloud provider credentials. This file is automatically created upon your first use of ls config or ls init. ### Key Configuration Options * Cloud Provider Credentials: Configure profiles for AWS, Azure, and GCP. LambdaShell leverages the credentials configured in your respective cloud provider CLIs (e.g., AWS profiles, Azure service principals, GCP project IDs). * Default Region/Location: Set a default region or location for your deployments to avoid specifying it with every command. * Log Retention Policies: Define default log retention periods for functions. * Cost Thresholds: Set alerts or warnings for estimated function costs. ### Example config.json ```json { "defaultProvider": "aws", "aws": { "defaultRegion": "us-east-1", "profile": "default" }, ...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 256
• per_device_eval_batch_size: 256
• num_train_epochs: 5
• warmup_ratio: 0.1
• fp16: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 256
• per_device_eval_batch_size: 256
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss	itprojectdocs-dev_cosine_accuracy
-1	-1	-	-	0.9339
1.0	32	0.3561	0.1620	0.9894
2.0	64	0.1284	0.1083	0.9943
3.0	96	0.0923	0.0994	0.9941
4.0	128	0.0705	0.0919	0.9943
5.0	160	0.0675	0.0903	0.9946
-1	-1	-	-	0.9946

Framework Versions

• Python: 3.12.11
• Sentence Transformers: 5.1.0
• Transformers: 4.55.4
• PyTorch: 2.8.0+cu126
• Accelerate: 1.10.1
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}