enridaga/checkthat_task4b__ea31198ade00_test_titles_abstracts_gtr-t5-base_fine_tune_TrueK10

SentenceTransformer based on sentence-transformers/gtr-t5-base

This is a sentence-transformers model finetuned from sentence-transformers/gtr-t5-base. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/gtr-t5-base
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: T5EncoderModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (3): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("enridaga/checkthat_task4b__ea31198ade00_test_titles_abstracts_gtr-t5-base_fine_tune_TrueK10")
# Run inference
sentences = [
    'Today, we are officially released! this study investigates how covid-19, race/ethnicity & #aces are linked. communities that report high rates of childhood trauma also report high rates of covid-19. they are also likely to have a higher % of minority individuals.',
    'The Unintended Consequence of Novel Coronavirus (COVID-19) Pandemic on Racial Inequities Associated With Adverse Childhood Experiences (ACEs): Findings From a Population-Based Study A rising concern is the COVID-19 pandemic effect on adverse childhood experiences (ACEs) due to increased parental stress and social/physical isolation. These pandemic effects are likely to be higher in already marginalized communities. The objective of this ecological study was to examine the relationship between COVID-19 cases and deaths, race/ethnicity, and the estimated number of adults with ACEs using data from South Carolina (SC). COVID-19 reported cases and death data were obtained from the SC Department of Health and Environmental Control. ACE data was used from the 2014–2016 SC Behavioral Risk Factor Surveillance System. Census data were used to obtain county population data. To measure the relationship between these variables, the Spearman rank-order correlation test was used because the data distribution was non-normal. There was a moderate relationship between the estimated number of adults with one or more ACEs and deaths (ρ = 0.89) and race/ethnicity-specific COVID-19 case counts by county (Black: ρ = 0.76; =White: ρ = 0.96; Hispanic: ρ = 0.89). Further, the Spearman correlation test showed the strongest relationship between COVID-19 deaths and race-ethnicity-specific county populations was with the Black adult population (ρ = 0.90). Given the known link between existing health inequities and exposure to COVID-19, these results demonstrate that the current pandemic could have unintended consequences on the well-being of children and caregivers. Response efforts should consider promoting protective factors for children and families and advocating for equitable policies and systems that serve children.',
    'Effect of early treatment with fluvoxamine on risk of emergency care and hospitalisation among patients with COVID-19: the TOGETHER randomised, platform clinical trial BACKGROUND: Recent evidence indicates a potential therapeutic role of fluvoxamine for COVID-19. In the TOGETHER trial for acutely symptomatic patients with COVID-19, we aimed to assess the efficacy of fluvoxamine versus placebo in preventing hospitalisation defined as either retention in a COVID-19 emergency setting or transfer to a tertiary hospital due to COVID-19. METHODS: This placebo-controlled, randomised, adaptive platform trial done among high-risk symptomatic Brazilian adults confirmed positive for SARS-CoV-2 included eligible patients from 11 clinical sites in Brazil with a known risk factor for progression to severe disease. Patients were randomly assigned (1:1) to either fluvoxamine (100 mg twice daily for 10 days) or placebo (or other treatment groups not reported here). The trial team, site staff, and patients were masked to treatment allocation. Our primary outcome was a composite endpoint of hospitalisation defined as either retention in a COVID-19 emergency setting or transfer to tertiary hospital due to COVID-19 up to 28 days post-random assignment on the basis of intention to treat. Modified intention to treat explored patients receiving at least 24 h of treatment before a primary outcome event and per-protocol analysis explored patients with a high level adherence (>80%). We used a Bayesian analytic framework to establish the effects along with probability of success of intervention compared with placebo. The trial is registered at ClinicalTrials.gov (NCT04727424) and is ongoing. FINDINGS: The study team screened 9803 potential participants for this trial. The trial was initiated on June 2, 2020, with the current protocol reporting randomisation to fluvoxamine from Jan 20 to Aug 5, 2021, when the trial arms were stopped for superiority. 741 patients were allocated to fluvoxamine and 756 to placebo. The average age of participants was 50 years (range 18–102 years); 58% were female. The proportion of patients observed in a COVID-19 emergency setting for more than 6 h or transferred to a teritary hospital due to COVID-19 was lower for the fluvoxamine group compared with placebo (79 [11%] of 741 vs 119 [16%] of 756); relative risk [RR] 0·68; 95% Bayesian credible interval [95% BCI]: 0·52–0·88), with a probability of superiority of 99·8% surpassing the prespecified superiority threshold of 97·6% (risk difference 5·0%). Of the composite primary outcome events, 87% were hospitalisations. Findings for the primary outcome were similar for the modified intention-to-treat analysis (RR 0·69, 95% BCI 0·53–0·90) and larger in the per-protocol analysis (RR 0·34, 95% BCI, 0·21–0·54). There were 17 deaths in the fluvoxamine group and 25 deaths in the placebo group in the primary intention-to-treat analysis (odds ratio [OR] 0·68, 95% CI: 0·36–1·27). There was one death in the fluvoxamine group and 12 in the placebo group for the per-protocol population (OR 0·09; 95% CI 0·01–0·47). We found no significant differences in number of treatment emergent adverse events among patients in the fluvoxamine and placebo groups. INTERPRETATION: Treatment with fluvoxamine (100 mg twice daily for 10 days) among high-risk outpatients with early diagnosed COVID-19 reduced the need for hospitalisation defined as retention in a COVID-19 emergency setting or transfer to a tertiary hospital. FUNDING: FastGrants and The Rainwater Charitable Foundation. TRANSLATION: For the Portuguese translation of the abstract see Supplementary Materials section.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

• Size: 12,842 training samples
• Columns: sentence_0, sentence_1, and sentence_2
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	sentence_2
  type	string	string	string
  details	min: 9 tokens mean: 50.17 tokens max: 132 tokens	min: 21 tokens mean: 375.84 tokens max: 512 tokens	min: 23 tokens mean: 378.99 tokens max: 512 tokens

• Samples:

  sentence_0	sentence_1	sentence_2
  Atmospheric shifts and ozone growth in Mexico City during 2020 lockdown period #covid	Atmospheric Changes and Ozone Increase in Mexico City during 2020 Lockdown Period Atmospheric pollutant (NO 2 , SO 2 , CO, O 3 and PM 2.5 ) variations during the COVID-19 pandemic (during 2020) have been studied from Mexico City in Central America. Meteorological factors (i.e) rainfall, temperature along with relative humidity played an important role in increasing the photochemical reaction for the formation of O 3 and PM 2.5 . Concentration pattern of O 3 and PM 2.5 were higher in all the stations in spite of the reduced primary pollutants. However, higher level of O 3 and PM 2.5 during the lockdown period in 2020 is mainly due to the air-mass exchange which happened through the broader channel in the north (Tenango del Aire Pass) and in the southeast (Cuautla-Cuernavaca valley). The higher values of particulate matter are compensated by domestic heating (“ Quédate en Casa ”/ Stay at Home ), whereas the increase of O 3 is supported by the higher solar radiation and household activiti...	Identification of novel bat coronaviruses sheds light on the evolutionary origins of SARS-CoV-2 and related viruses Despite the discovery of animal coronaviruses related to SARS-CoV-2, the evolutionary origins of this virus are elusive. We describe a meta-transcriptomic study of 411 bat samples collected from a small geographical region in Yunnan province, China, between May 2019 and November 2020. We identified 24 full-length coronavirus genomes, including four novel SARS-CoV-2 related and three SARS-CoV related viruses. Rhinolophus pusillus virus RpYN06 was the closest relative of SARS-CoV-2 in most of the genome, although it possessed a more divergent spike gene. The other three SARS-CoV-2 related coronaviruses carried a genetically distinct spike gene that could weakly bind to the hACE2 receptor in vitro. Ecological modeling predicted the co-existence of up to 23 Rhinolophus bat species, with the largest contiguous hotspots extending from South Laos and Vietnam to southern China. O...
  new studies from and colleagues. bme patients with covid-19 were 10 years younger, but ethnicity did not independently impact outcomes.	Impact of ethnicity on outcome of severe COVID-19 infection. Data from an ethnically diverse UK tertiary centre During the current COVID-19 pandemic, anecdotal reports suggest that BAME background patients may be disproportionately affected compared to White but few objective data are available. We took advantage of near real-time hospital data access and analysis pipelines to look at the impact of ethnicity in 437 consecutive patients admitted during March to Kings College Hospital NHS Trust in London. Our key findings are firstly that BAME patients are significantly younger and have different co-morbidity profiles than White individuals. Secondly, there is no significant effect of ethnicity itself on severe outcomes (death or ITU admission) within 14-days of symptom onset, with adjustment for age, sex, comorbidities.	Cough aerosol in healthy participants: fundamental knowledge to optimize droplet-spread infectious respiratory disease management BACKGROUND: The Influenza A H1N1 virus can be transmitted via direct, indirect, and airborne route to non-infected subjects when an infected patient coughs, which expels a number of different sized droplets to the surrounding environment as an aerosol. The objective of the current study was to characterize the human cough aerosol pattern with the aim of developing a standard human cough bioaerosol model for Influenza Pandemic control. METHOD: 45 healthy non-smokers participated in the open bench study by giving their best effort cough. A laser diffraction system was used to obtain accurate, time-dependent, quantitative measurements of the size and number of droplets expelled by the cough aerosol. RESULTS: Voluntary coughs generated droplets ranging from 0.1 - 900 microns in size. Droplets of less than one-micron size represent 97% of the total number of meas...
  a new chinese preprint tested various strains of coronavirus and found that -- shock! -- the d614g strain that has now taken over the globe is indeed more infectious.	The impact of mutations in SARS-CoV-2 spike on viral infectivity and antigenicity Summary The spike protein of SARS-CoV-2 has been undergoing mutations and is highly glycosylated. It is critically important to investigate the biological significance of these mutations. Here we investigated 80 variants and 26 glycosylation site modifications for the infectivity and reactivity to a panel of neutralizing antibodies and sera from convalescent patients. D614G, along with several variants containing both D614G and another amino acid change, were significantly more infectious. Most variants with amino acid change at receptor binding domain were less infectious but variants including A475V, L452R, V483A and F490L became resistant to some neutralizing antibodies. Moreover, the majority of glycosylation deletions were less infectious whilst deletion of both N331 and N343 glycosylation drastically reduced infectivity, revealing the importance of glycosylation for viral infectivity. Interestingly,...	Preliminary Estimate of Excess Mortality During the COVID-19 Outbreak - New York City, March 11-May 2, 2020. SARS-CoV-2, the virus that causes coronavirus disease 2019 (COVID-19), was first identified in December 2019 in Wuhan, China, and has since spread worldwide. On March 11, 2020, the World Health Organization declared COVID-19 a pandemic (1). That same day, the first confirmed COVID-19-associated fatality occurred in New York City (NYC). To identify confirmed COVID-19-associated deaths, defined as those occurring in persons with laboratory-confirmed SARS-CoV-2 infection, on March 13, 2020, the New York City Department of Health and Mental Hygiene (DOHMH) initiated a daily match between all deaths reported to the DOHMH electronic vital registry system (eVital) (2) and laboratory-confirmed cases of COVID-19. Deaths for which COVID-19, SARS-CoV-2, or an equivalent term is listed on the death certificate as an immediate, underlying, or contributing cause of death, but that do not have...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 8
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	Training Loss
0.3113	500	0.5501
0.6227	1000	0.2908
0.9340	1500	0.284
1.2453	2000	0.2207
1.5567	2500	0.2075
1.8680	3000	0.1969
2.1793	3500	0.1885
2.4907	4000	0.1704
2.8020	4500	0.1858

Framework Versions

• Python: 3.11.12
• Sentence Transformers: 3.4.1
• Transformers: 4.51.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.6.0
• Datasets: 3.6.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}