Base Model: unsloth/Llama-3.2-3B-Instruct
Fine-tuning Dataset: 0xZee/UniversalScienceKownledge-finetome-top-20k
Model Type: Causal Language Model
License: Apache 2.0
Finetuning Technique: The model was likely fine-tuned using techniques such as LoRA or QLoRA, which are efficient for smaller datasets.

Intended Use

This model is designed for tasks requiring scientific understanding, question answering, and knowledge retrieval in scientific contexts. It can be used for:

Assisting researchers with scientific queries
Educational tools in science domains
Automated scientific knowledge discovery and summarization

Training Procedure

The model was fine-tuned on the UniversalScienceKownledge-finetome-top-20k dataset. Specific hyperparameters used in training are not provided, but typical fine-tuning approaches for LLMs include:

Learning rate: Often around 2e-5 for Mistral-based models
LoRA rank: Likely between 512 and 768
Batch size: Small, possibly between 1 and 4

Limitations

Performance is dependent on the quality and scope of the training data.
May not generalize well to scientific domains outside of its training data.
Potential for generating plausible-sounding but incorrect information (hallucinations).

Bias, Risks, and Limitations

As with all language models, this model may reflect biases present in its training data. Users should be aware of potential biases in scientific literature and exercise caution when using the model for sensitive applications.

Evaluation Results

Quantitative evaluation results are not provided. Users are encouraged to evaluate the model's performance on their specific tasks and domains of interest.

How to Use

You can use this model with the Transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "0xZee/llama3-2-3b-finetuned-ScienceKnowledge"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = "Explain the concept of quantum entanglement."
input_ids = tokenizer(prompt, return_tensors="pt").input_ids
output = model.generate(input_ids, max_new_tokens=100)
print(tokenizer.decode(output, skip_special_tokens=True))

0xZee/llama3-2-3b-finetuned-ScienceKnowledge

model-name: llama3-2-3b-finetuned-ScienceKnowledge

base_model: unsloth/Llama-3.2-3B-Instruct

datasets:

llama3-2-3b-finetuned-ScienceKnowledge

Model Details

Intended Use

Training Procedure

Limitations

Bias, Risks, and Limitations

Evaluation Results

How to Use

0xZee/
llama3-2-3b-finetuned-ScienceKnowledge

model-name: `llama3-2-3b-finetuned-ScienceKnowledge`

base_model: `unsloth/Llama-3.2-3B-Instruct`