Dataset: berkeruveyik/toxic-speech-annotated-dataset
Languages: English and Turkish
Supervision format: conversational (user input + assistant structured target)

Example target format:

text

number_of_bad_words: 2
tags: Strong Insult
insult_word: stupid, ugly

Training Procedure

The model was trained with supervised fine-tuning using TRL (SFTTrainer) on structured conversation data.

Typical training setup:

max_length=512
num_train_epochs=10
per_device_train_batch_size=8
per_device_eval_batch_size=8
gradient_accumulation_steps=2
learning_rate=2e-5
lr_scheduler_type=cosine
warmup_ratio=0.1
weight_decay=0.01
save_strategy=epoch
eval_strategy=epoch
load_best_model_at_end=True

Inference

For better format consistency, use deterministic decoding (do_sample=False).

python

from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline

MODEL_ID = "YOUR_USERNAME/YOUR_MODEL_REPO"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(MODEL_ID, device_map="auto", dtype="auto")

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)

text = "You are useless and disgusting."
messages = [{"role": "user", "content": text}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

out = pipe(prompt, max_new_tokens=128, do_sample=False)
print(out[0]["generated_text"][len(prompt):])

Intended Use

Toxicity signal extraction for moderation workflows
Annotation support
Safety analysis prototypes for EN/TR text

Limitations

May miss implicit abuse, sarcasm, or context-heavy harassment.
Extracted insult words can be incomplete or noisy.
Performance may drop on long inputs, slang drift, or unseen domains.
Should not be the sole basis for legal, HR, or punitive decisions.

Responsible Use

Keep human review for high-stakes cases.
Monitor false positives and false negatives.
Re-evaluate regularly across both languages and target domains.

Suggested Evaluation Metrics

Tag-level Precision / Recall / F1
Exact-match rate for full structured output
number_of_bad_words MAE
Parse-valid output rate

Credits

Base model: google/gemma-3-1b-it
Dataset: berkeruveyik/toxic-speech-annotated-dataset
License: Apache-2.0

berkeruveyik/toxic-speech-finetune-with-gemma-3-1b-v1

Toxic Speech Structured Analysis (Gemma 3 1B IT)

Model Description

Task

Training Data