Based on some research, when a finetuned model is merged with its base model with TIES method, there is possibility the merged model will achieve better output.

UPDATE!!! as 20 November 2024, this model is third best model (number one for Gemma2-9B based model) on HF's Open LLM Leaderboard (with Merge/MoErges hide model unchecked) for LLM model below 10B parameters.

gmonsoon/SahabatAI-Lion-9B-TIES-v1 is a merge of the following models:

DEMO Spaces: HERE

🧩 Configuration

yaml

models:
  - model: GoToCompany/gemma2-9b-cpt-sahabatai-v1-instruct
    parameters:
      weight: 1
      density: 1
  - model: GoToCompany/gemma2-9b-cpt-sahabatai-v1-instruct
    parameters:
      weight: 1
      density: 1
merge_method: ties
base_model: aisingapore/gemma2-9b-cpt-sea-lionv3-instruct
parameters:
  density: 1
  normalize: true
  int8_mask: true
dtype: bfloat16

💻 Usage

python

!pip install -qU transformers accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "gmonsoon/SahabatAI-Lion-9B-TIES-v1"
messages = [{"role": "user", "content": "What is a large language model?"}]

tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	33.70
IFEval (0-Shot)	73.78
BBH (3-Shot)	43.40
MATH Lvl 5 (4-Shot)	19.34
GPQA (0-shot)	9.40
MuSR (0-shot)	19.13
MMLU-PRO (5-shot)	37.19

gmonsoon/SahabatAI-Lion-9B-TIES-v1-GGUF

SahabatAI-Lion-9B-TIES-v1

🧩 Configuration

💻 Usage

Open LLM Leaderboard Evaluation Results

gmonsoon/
SahabatAI-Lion-9B-TIES-v1-GGUF