from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct",
    torch_dtype=torch.float16,
    device_map="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "ECHO-9")
tokenizer = AutoTokenizer.from_pretrained("ECHO-9")

# Generate
prompt = "Your prompt here"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

With Ollama (Recommended)

Merge adapter with base model:

python

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
model = PeftModel.from_pretrained(base, "ECHO-9")
merged = model.merge_and_unload()
merged.save_pretrained("./merged_model")

Create Modelfile and import to Ollama

Training Details

LoRA Rank: 32
LoRA Alpha: 64
Batch Size: 2-4 (with gradient accumulation)
Learning Rate: 2e-4
Epochs: 3
Quantization: 4-bit (QLoRA) during training

License

Apache 2.0

nullvektordom/corrupted-triad-echo9

ECHO-9 - CORRUPTED TRIAD

Model Details

Usage

With Transformers + PEFT

With Ollama (Recommended)

Training Details

License

nullvektordom/
corrupted-triad-echo9