yasserrmd/GeoScholar-QA-1.2B

GeoScholar-QA

📖 Overview

GeoScholar-QA is a large language model fine-tuned for academic question answering in the field of geoscience. It is built upon the Liquid AI LFM2 base model and has been trained using the Unsloth framework with the GeoGPT-QA dataset. [4] This model's primary strength lies in its ability to explain concepts and theories within the earth sciences. It is not designed to provide consistently accurate statistics or citations.

---

📚 Dataset: GeoGPT-QA

• Dataset name: GeoGPT-QA
• Publisher / Project: GeoGPT-Research-Project
• Size: Approximately 41,400 rows in the train split. [4]
• Format: The dataset is tabular (originally CSV, automatically converted to Parquet) and includes fields such as question, answer, title, authors, doi, journal, volume, pages, and license. [4]
• Language: English [4]
• License: CC-BY 4.0 — You are permitted to share and adapt the dataset, but you must provide attribution and indicate if any changes were made. [4]

---

🔧 Model Details & Training

• Base model: Liquid AI LFM2 is a hybrid model designed for on-device deployment, offering a balance of quality, speed, and memory efficiency. [8]
• Fine-tuning framework: Unsloth is a framework designed to speed up and optimize the fine-tuning of large language models, making it more accessible on limited hardware. [10, 11]
• Training data: GeoGPT-QA [4]
• Objective: The model was trained through Supervised Fine-Tuning (SFT) using question-answer pairs. The focus of this training was on geoscience theory, conceptual knowledge, and explanations. [19]
• Effective batch size: A low per-device batch size was utilized along with gradient accumulation to prevent out-of-memory (OOM) errors during training.
• Training progress: (Example) Approximately 3000 steps, covering about 58% of the dataset. Please adjust this based on your final training run.

---

🧑‍🏫 Intended Use

GeoScholar-QA is intended for the following applications:

• Providing academic explanations in various fields of geoscience, such as plate tectonics, hydrology, and geomorphology.
• Serving as a teaching and learning aid for students and educators.
• Enhancing conceptual and theoretical understanding of earth science principles.

This model is not intended for:

• High-risk or decision-making tasks that require precise numerical data or statistics.
• Reliance on the generated citations or study results without independent verification.

---

⚠️ Limitations

• The model may generate inaccurate numbers, study names, datasets, or locations.
• Answers in applied or technical contexts may be overgeneralized or vague.
• It should not be used as a substitute for verification by a domain expert, especially in research or policy-making settings.

---

✅ Example Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "yasserrmd/GeoScholar-QA-1.2B"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")

messages = [
    {"role": "user", "content": "How do plate tectonics explain the formation of volcanoes along subduction zones?"}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to("cuda")

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    temperature=0.3,
    repetition_penalty=1.05
)

print(tokenizer.decode(outputs, skip_special_tokens=True))

---

📝 License & Attribution

• This model is trained using the GeoGPT-QA dataset, which is licensed under CC-BY 4.0. [4]
• You must give appropriate credit to the GeoGPT Research Project and include a link to the dataset.
• If you adapt or build upon this model, you must indicate that changes have been made.

---

🔭 Tags / Metadata for Hugging Face Model Card

• Model type: Text generation / QA
• Domain: Geoscience, Earth Sciences
• Base model: Liquid AI LFM2
• Training method: SFT (Supervised Fine-Tuning)
• License: CC-BY 4.0