A LoRA fine-tuned version of the meta-llama/Llama-3.1-70B-Instruct model on the Alpaca dataset, optimized using PEFT and accelerated on Intel Gaudi3 HPU hardware.

📝 Model Summary

This model is a fine-tuned variant of LLaMA 3.1 70B Instruct, trained on the Alpaca dataset using Parameter-Efficient Fine-Tuning (PEFT) via LoRA. The goal of this fine-tuning was to improve instruction-following performance on lightweight resources, leveraging Intel’s Gaudi3 HPU for efficient training.

📄 Model Details

Base Model: meta-llama/Llama-3.1-70B-Instruct
Fine-tuned Model: pranjalsingh/alpaca-Llama-3.1-70B-Instruct-chat
Fine-tuned By: Pranjal Singh Thakur
Dataset: Stanford Alpaca dataset
PEFT Library: PEFT v0.12.0
Fine-tuning Technique: LoRA
Epochs: 2
Training Hardware: 1 Node with 8× Intel Gaudi3 HPUs
Language(s): English
License: Same as base model (LLaMA 3)
Credit: Intel for providing Gaudi3 HPU infrastructure

🚀 Usage

Direct Use

Use the model as an instruction-following chatbot or in downstream applications requiring LLM completion with lightweight deployment using LoRA adapters.

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-70B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-70B-Instruct")

model = PeftModel.from_pretrained(base_model, "pranjalsingh/alpaca-Llama-3.1-70B-Instruct-chat")

inputs = tokenizer("### Instruction: Explain quantum computing in simple terms.", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

📊 Evaluation Results

Metric	Value
Eval Accuracy	73.27%
Eval Loss	1.02
Perplexity	2.79
Evaluation Runtime	20.97s
Samples Evaluated	101
Samples/Sec	4.82
Max Memory Used (GB)	126.2
Total Available Memory	126.54 GB
Memory Allocated (GB)	41.06

🛠 Training Configuration

Epochs: 2
Precision: Likely mixed precision (bf16/fp16 on Gaudi3)
Hardware: Intel Gaudi3 HPU (8 cards, 1 node)
Frameworks: PEFT, Hugging Face Transformers
Batching & Tokenization: Not explicitly provided

📦 Model Sources

Repository: Hugging Face Model Card
Dataset: Stanford Alpaca
Base Model: meta-llama/Llama-3.1-70B-Instruct

⚠️ Limitations & Risks

Not suitable for multilingual tasks (trained only on English data).
May reflect biases present in the Alpaca dataset.
Not recommended for sensitive or safety-critical applications.
Fine-tuning was conducted for instruction tasks — may not generalize to other domains.

♻️ Environmental Impact

Parameter	Value
Compute Platform	Intel Gaudi3
Cards Used	8× HPU
Training Duration	~2 Epochs
Region	[More info needed]
Emission Estimate	[Use MLCO2 to calculate]

👨‍💻 Author & Acknowledgment

Author: Pranjal Singh Thakur
Credit: Intel (for compute resources using Gaudi3 HPU)

🔖 Citation

Coming soon.

pranjalsingh/alpaca-Llama-3.1-70B-Instruct-chat

🧠 Model Card: pranjalsingh/alpaca-Llama-3.1-70B-Instruct-chat