You can deploy this model and start earning money today!

DATA-ADAPT/
offline-embedding

meta-llama/Llama-2-7b-hf

adapter

Views

No views yet

offline-embedding

Offline training with embedding-based retrieved data (2.5% of full dataset).

Note: This checkpoint is from a single random seed (seed=3) and a specific training step (step 1040). Results may vary across seeds.

Details

Key	Value
Base model	`meta-llama/Llama-2-7b-hf`
Task	MMLU
Data selection	Embedding Retrieval
Data ratio	2.5%
Online	False
LoRA rank	128
LoRA alpha	512
Target modules	q_proj, k_proj, v_proj, o_proj
Seed	3
Checkpoint step	1040

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf")
model = PeftModel.from_pretrained(base_model, "DATA-ADAPT/offline-embedding")
tokenizer = AutoTokenizer.from_pretrained("DATA-ADAPT/offline-embedding")