You can deploy this model and start earning money today!

Kalamazooter/
RatelSlang-Micro-130M

Name: RatelSlang-Micro-130M
Availability: InStock

Kalamazooter/GeminiPhiDutch

state-spaces/mamba-130m-hf

finetune

apache-2.0

autotrain_compatible

text-generation-inference

endpoints_compatible

Views

No views yet

A Tiny Dutch model, just-about semi-coherent

Overview

An experimental fine-tune of mamba-130m using the GeminiPhi Dataset and the dutch-llama-tokenizer by yhavinga

Usage

You need to install transformers from main until transformers=4.39.0 is released.

pip install git+https://github.com/huggingface/transformers@main

We also recommend you to install both causal_conv_1d and mamba-ssm using:

bash

pip install causal-conv1d>=1.2.0
pip install mamba-ssm

If any of these two is not installed, the "eager" implementation will be used. Otherwise the more optimised cuda kernels will be used.

Generation

You can use the classic generate API: setup (For Cuda)

python

from transformers import MambaConfig, MambaForCausalLM, AutoTokenizer
import torch
device = torch.device('cuda:0')
tokenizer = AutoTokenizer.from_pretrained("Kalamazooter/RatelSlang-Micro-130M")
model = MambaForCausalLM.from_pretrained("Kalamazooter/RatelSlang-Micro-130M")
model = model.to(device)

Inference

python

input_ids = tokenizer("**Vraag: Ik heb 4 schapen, per schaap heb ik 3 lammetjes, hoeveel lammetjes heb ik?\n\n Antwoord:", return_tensors="pt").input_ids.to(device)
out = model.generate(input_ids, max_new_tokens=50)
print(tokenizer.batch_decode(out))
['<s> **Vraag: Ik heb 4 schapen, per schaap heb ik 3 lammetjes, hoeveel lammetjes heb ik?\n\n Antwoord:\n\n1. Bereken het aantal lammetjes dat je hebt: 4 schapen x 3 lammetjes per schaap = 12 lammetjes\n2. Bereken het aantal lammetjes dat je hebt: 12 lam']

PEFT finetuning example

In order to finetune using the peft library, it is recommend to keep the model in float32!

python

from datasets import load_dataset
from trl import SFTTrainer
from peft import LoraConfig
from transformers import AutoTokenizer, AutoModelForCausalLM, TrainingArguments
tokenizer = AutoTokenizer.from_pretrained("Kalamazooter/RatelSlang-Micro-130M")
model = AutoModelForCausalLM.from_pretrained("Kalamazooter/RatelSlang-Micro-130M")
dataset = load_dataset("Abirate/english_quotes", split="train")
training_args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    logging_dir='./logs',
    logging_steps=10,
    learning_rate=2e-3
)
lora_config =  LoraConfig(
        r=8,
        target_modules=["x_proj", "embeddings", "in_proj", "out_proj"],
        task_type="CAUSAL_LM",
        bias="none"
)
trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    args=training_args,
    peft_config=lora_config,
    train_dataset=dataset,
    dataset_text_field="quote",
)
trainer.train()