This repository contains LoRA-finetuned and merged weights based on openPangu-Embedded-7B-V1.1. The LoRA adapters were merged into the base model to produce full weights suitable for standard inference.

Base Model

Base model: FreedomIntelligence/openPangu-Embedded-7B-V1.1
License: OPENPANGU Model License Agreement v1.0 (see LICENSE)

Training Data

Private dataset (not released).

Training Procedure

Finetuning: LoRA using LLaMA-Factory.
Export: merged full weights with llamafactory-cli export.

Example (paths are placeholders):

bash

llamafactory-cli export \
  --model_name_or_path <base_model_dir> \
  --adapter_name_or_path <lora_adapter_dir> \
  --template default \
  --finetuning_type lora \
  --export_dir <export_dir> \
  --export_size 2 \
  --export_device cpu \
  --export_legacy_format False \
  --trust_remote_code True

Evaluation

Evaluated with lm-evaluation-harness using vLLM on 4x RTX 4090. Dates (UTC): 2026-01-04.

GSM8K (5-shot)

exact_match (strict-match): 0.6171
exact_match (flexible-extract): 0.5777

C-Eval (valid, 5-shot)

acc: 0.6241
acc_norm: 0.6241

Example command (paths are placeholders):

bash

lm_eval --model vllm \
  --model_args "pretrained=<model_dir>,tensor_parallel_size=4,dtype=auto,gpu_memory_utilization=0.8,max_model_len=4096,enforce_eager=True,trust_remote_code=True" \
  --tasks gsm8k \
  --num_fewshot 5 \
  --batch_size auto

Usage

This repo includes custom modeling code; trust_remote_code=True is required.

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "killer66678/openpangu_7b_lora"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype="auto",
    device_map="auto",
)

Limitations and License Notes

The openPangu license restricts use within the European Union.
If you distribute a product or service based on this model, the license requires specific attribution and trademark notices.
As with any LLM, outputs may be incorrect or biased.

Acknowledgements

本研究的实验与计算工作依托于华为云昇腾AI云服务平台完成，特此对其提供的稳定算力支持表示感谢。