Quantization made by Richard Erkhov.
Yippie!! It's my birthday in 2 days! So I'm gonna drop this model that I made at 2AM.
It's quite a bit different from what I've been doing recently, but it was pretty fun to work on. :D
Used a different merging technique that I quickly designed & crafted which focused on the math and logical reasoning.
Also fine-tuned this on Self-Play RL algorithms.
Not the best LLM out there but it was pretty fun making this, and coming up with something different.
Borrowed some ideas from my other "distillation" technique, which significantly reduces the number of layers while aiming to retain the output quality.
1import torch
2from transformers import pipeline
3
4pipe = pipeline(
5 "text-generation",
6 model="minchyeom/birthday-llm",
7 model_kwargs={"torch_dtype": torch.bfloat16},
8 device="cuda", # replace with "mps" to run on a Mac device
9)
10
11messages = [
12 {"role": "user", "content": "How many r's are there in the word strawberry?"},
13]
14
15outputs = pipe(messages, max_new_tokens=256)
16assistant_response = outputs[0]["generated_text"][-1]["content"].strip()
17print(assistant_response)
