You can deploy this model and start earning money today!

lhpku20010120/
llama3.1-8b-k12kgraph

Name: llama3.1-8b-k12kgraph
Availability: InStock

Views

No views yet

Llama3.1-8B-K12KGraph

Built with Llama.

Llama3.1-8B-K12KGraph is a full-parameter supervised fine-tuning (SFT) model based on meta-llama/Llama-3.1-8B. It was trained on the K12-Train data associated with the K12-KGraph project for educational question answering.

This repository contains a standalone Transformers checkpoint. The original Llama-3.1-8B weights do not need to be downloaded separately.

Model details

Architecture: LlamaForCausalLM
Parameters: approximately 8B
Fine-tuning method: full-parameter SFT
Training framework: LLaMA-Factory
Context length used for preprocessing: 32,768 tokens
Weight dtype: bfloat16
Languages: Chinese and English
License: Llama 3.1 Community License

Intended use

The model is intended for research on K-12 educational question answering, knowledge-intensive reasoning, and evaluation of models trained with knowledge-graph-derived educational data.

Usage

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "lhpku20010120/llama3.1-8b-k12kgraph"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": "Explain the difference between the light-dependent and light-independent reactions in photosynthesis.",
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    outputs = model.generate(
        inputs,
        max_new_tokens=512,
        do_sample=False,
    )

answer = tokenizer.decode(
    outputs[0, inputs.shape[-1]:],
    skip_special_tokens=True,
)
print(answer)

The checkpoint uses the included chat_template.jinja, which follows the Llama 3 chat format with role headers and end-of-turn tokens.

Training

The following information was recovered from the exported trainer metadata:

Setting	Value
Base model	meta-llama/Llama-3.1-8B
Epochs	3
Per-device batch size	1
Gradient accumulation	4
Number of devices	8
Effective global batch size	32
Learning rate	5e-6
Scheduler	cosine
Warmup ratio	0.1
Optimizer	AdamW
Seed	42
Transformers	4.56.2
PyTorch	2.5.1+cu121
Datasets	3.2.0
Tokenizers	0.22.2

Limitations

The model may reproduce errors and biases present in the base model or fine-tuning data.
Generated answers may be incorrect, incomplete, or hallucinated. The model should not be treated as an authoritative source for high-stakes educational decisions.
Its strongest expected domain is K-12 educational content; performance may degrade outside that domain.

Data and copyright

Use of this model is subject to the Llama 3.1 Community License and the Llama 3.1 Acceptable Use Policy. Users are also responsible for complying with the licenses and terms associated with the training data, benchmark data, and generated content. The release of model weights does not grant rights to reproduce any third-party textbook or benchmark material.

Acknowledgements

Built with Llama. This model is based on Llama 3.1 8B and was trained with LLaMA-Factory. Please cite the Llama 3.1 and K12-KGraph projects when using this checkpoint in research.