lora-teuken-fact20 is a LORA adapter for openGPT-X/Teuken-7B-instruct-research-v0.4.
Using the lora-adapter for Teuken 7B it is possible to get much more correct answers on the question:
Wich districts has a city. Wehre city may be e.g. Berlin or Essen.
Only the districts of the 20 biggest cities in germany are considered.
Language is german.
Model Details
Model Description
Teuken 7B (like other LLMs to) has problems with simple facts. E.g. if you ask it for the districts of the german city Dresden you will get varing results.
Using the dataset andy300/cities_bezirk_mp i finetuned openGPT-X/Teuken-7B-instruct-research-v0.4. The result is the LORA Adapter andy300/lora-teuken-fact20.
With the LORA Adapter Teuken 7B is able to answer with 233 correct dstricts from a total of 235 district of the 20 biggest cities in Germany.
Pure Teuken 7B on the other gives 97 correct districts.
Developed by: Andreas Wenzel
Model type: LORA Adapter
Language(s) (NLP): German
License: MIT
Finetuned from model: openGPT-X/Teuken-7B-instruct-research-v0.4
This LORA Adapter improves Teuken 7B's ability to answer questions such as: "Which districts does the city of Berlin have?" Only the 20 largest cities in Germany are considered.
Bias, Risks, and Limitations
Using the LORA adapter changes slightly the properties of Teuken 7B.
How to Get Started with the Model
LORA adapter can be loaded for inference of openGPT-X/Teuken-7B-instruct-research-v0.4 with transformer or vLLM.
Transformer
The model requires the same python libraries as required by Teuken 7B.
Model and tokenizer is used like in the openGPT-X/Teuken-7B-instruct-research-v0.4 example.
Only the LORA-Adapter andy300/teuken-fact20 is added to the openGPT-X/Teuken-7B-instruct-research-v0.4 model with the model.load_adapter method.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model_name = "openGPT-X/Teuken-7B-instruct-research-v0.4"
adapter_name = "andy300/teuken-fact20"
model = AutoModelForCausalLM.from_pretrained(
model_name,
trust_remote_code=True,
torch_dtype=torch.bfloat16
)
model.load_adapter(adapter_name)
model = model.to(device).eval()
tokenizer = AutoTokenizer.from_pretrained(
model_name,
use_fast=False,
trust_remote_code=True,
)
messages = [{"role": "User", "content": "Kannst du mir eine Liste aller Stadtbezirke in München geben?"}]
prompt_ids = tokenizer.apply_chat_template(messages, chat_template="DE", tokenize=True, add_generation_prompt=True, return_tensors="pt")
prediction = model.generate(
prompt_ids.to(model.device),
max_length=512,
do_sample=True,
top_k=50,
top_p=0.95,
temperature=0.7,
num_return_sequences=1,
)
prediction_text = tokenizer.decode(prediction[0].tolist())
print(prediction_text)
