You can deploy this model and start earning money today!

emreski1/
EmresGPT-LoRA

Name: EmresGPT-LoRA
Availability: InStock

Views

No views yet

EmresGPT-LoRA

🧠 Fine-tuned LLM based on Mistral-7B-Instruct using EmresKI's internal knowledge base (Wiki + Memory + Manifest).

🧬 Model Overview

EmresGPT-LoRA is a LoRA-finetuned checkpoint of the open-source model mistralai/Mistral-7B-Instruct-v0.2, optimized on structured Markdown knowledge and reflective memory logs from the EmresKI system.

Base model: Mistral-7B-Instruct-v0.2
Technique: PEFT (LoRA)
Training dataset:
- EmresWiki (Markdown-based, topic-driven knowledge)
- Memory logs from EmresKI (questions, task results, reflections)
- Manifest-based system prompt

📚 Training Details

Epochs: 3
Batch size: 2 (gradient_accumulation_steps: 4)
Framework: HuggingFace Transformers + PEFT
Tokens: Mixed prompt/response pairs from dataset.jsonl
Compute: local GPU (>=12GB VRAM recommended)

🔧 Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
from peft import PeftModel
import torch

model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2", torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(model, "emreski1/EmresGPT-LoRA")
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, device=0)
response = pipe("Frage: Was ist Backpropagation?\nAntwort:", max_new_tokens=200)
print(response[0]['generated_text'])

🧠 Example Prompt

Frage: Was ist der Unterschied zwischen Boosting und einem Entscheidungsbaum?
Antwort:

📄 License

This model is licensed under Apache-2.0. You are free to use, modify, and distribute it with proper attribution.

🤝 Contributors

@emreski1 — Project lead, architecture, Wiki engine, self-reflection memory system

🌐 Tags

LoRA EmresGPT Mistral fine-tuning transformers German AI KnowledgeGraph SelfImprovingAI