KafkaLM 13b is a Llama2 13b model - further pre-trained on a large German dataset from Björn Plüster and LAION. leo-hessianai-13b - which was finetuned on an ensemble of popular high-quality open-source instruction sets (translated from English to German).
Why Kafka?
The models are proficient, yet creative, and have some tendencies to linguistically push boundaries 😊
Model Details
The purpose of releasing the KafkaLM series is to contribute to the German AI community with a set of fine-tuned LLMs that are easy to use in everyday applications across a variety of tasks.
The main goal was to provide LLMs proficient in German, especially to be used in German-speaking business contexts where English alone is not sufficient.
DPO
The model has been aligned with a german and modified version of the ultra feedback dataset from huggingface.
<|system|>
Du bist ein freundlicher und hilfsbereiter KI-Assistent. Du beantwortest Fragen faktenorientiert und präzise, ohne dabei relevante Fakten auszulassen.</s>
<|user|>
Welche Möglichkeiten der energetischen Sanierung habe ich neben Solar und Energiespeicher?</s>
<|assistant|>
Inference
Getting started with the model is straightforward
python
1import transformers
23model_id ="seedboxai/KafkaLM-13B-German-V0.1-DPO"45model = AutoModelForCausalLM.from_pretrained(model_id, load_in_4bit=True)67tokenizer = AutoTokenizer.from_pretrained(model_id)89tokenizer.padding_side ="right"10tokenizer.pad_token = tokenizer.unk_token
11tokenizer.add_eos_token =False1213defgenerate_prompt(input):14 prompt =''15 sys_prompt ="Du bist ein freundlicher und hilfsbereiter KI-Assistent. Du beantwortest Fragen faktenorientiert und präzise, ohne dabei relevante Fakten auszulassen."1617 prompt +=f"<|system|>\n{sys_prompt.strip()}</s>\n"18 prompt +=f"<|user|>\n{input.strip()}</s>\n"19 prompt +=f"<|assistant|>\n"2021return prompt.strip()222324generate_text = transformers.pipeline(25 model=model, tokenizer=tokenizer,26 return_full_text=True,27 task='text-generation',28 temperature=0.5,29 max_new_tokens=512,30 top_p=0.95,31 top_k=50,32 do_sample=True,33)343536print(generate_text(generate_prompt("Wer ist eigentlich dieser Kafka?")))37
Disclaimer
The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model.
This model should only be used for research purposes. The original Llama2 license and all restrictions of datasets used to train this model apply.
