You can deploy this model and start earning money today!

PiLabZJU/
AlignSurvey-Llama-3.1-8B

Name: AlignSurvey-Llama-3.1-8B
Availability: InStock

meta-llama/Llama-3.1-8B-Instruct

adapter

llama3.1

Views

No views yet

AlignSurvey Llama-3.1-8B Stage-I LoRA

This repository contains the Stage-I LoRA adapter for meta-llama/Llama-3.1-8B-Instruct. It was trained for one epoch on the AlignSurvey Social Foundation Corpus before any task-specific Stage-II fine-tuning.

Important base-model information

The original training run loaded the base model from <local-model-root>/Meta-Llama-3.1-8B-Instruct. The exact Hugging Face commit was not recorded in the training artifacts and must be confirmed from that server-side base-model directory before release. Do not claim a commit until it has been verified.

Tokenizer: use the tokenizer from the same base-model revision.

LLaMA-Factory template: llama3.

Access to the base model requires acceptance of Meta's Llama 3.1 license on Hugging Face.

LoRA configuration

PEFT type: LoRA
Rank: 8
Alpha: 16
Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, and down_proj

Stage-I training

Learning rate: 1e-4
Scheduler: cosine
Warmup ratio: 0.1
Seed: 42

Loading

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = "meta-llama/Llama-3.1-8B-Instruct"
adapter_dir = "path/to/adapter"

model = AutoModelForCausalLM.from_pretrained(base_model, torch_dtype="auto")
model = PeftModel.from_pretrained(model, adapter_dir)
tokenizer = AutoTokenizer.from_pretrained(base_model)

To create a merged model, call model.merge_and_unload() and then save_pretrained(). Distribution and use remain subject to the Meta Llama 3.1, dataset, and original survey-source licenses.

PiLabZJU/AlignSurvey-Llama-3.1-8B