You can deploy this model and start earning money today!

oguzatas/
mamba-tr-project-transformer

Name: mamba-tr-project-transformer
Availability: InStock

Views

No views yet

Paper

Open In Colab

Transformer Baseline (GPT-2 Style) - Turkish

This model serves as the baseline for a comparative study between Transformer and Mamba architectures on agglutinative languages (specifically Turkish). It is a decoder-only Transformer model (~111M parameters) trained on the Turkish Wikipedia dataset.

Model Description

Architecture: GPT-2 (Small)
Parameters: ~111 Million
Context Length: 1024
Training Data: Turkish Wikipedia (Nov 2023)
Purpose: To provide a performance benchmark for the Mamba architecture.

Usage

python

from transformers import GPT2LMHeadModel, PreTrainedTokenizerFast

model_id = "oguzatas/mamba-tr-project-transformer"
tokenizer_id = "oguzatas/mamba-tr-project-tokenizer"

tokenizer = PreTrainedTokenizerFast.from_pretrained(tokenizer_id)
model = GPT2LMHeadModel.from_pretrained(model_id)

text = "Türkiye'nin başkenti"
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=20)

print(tokenizer.decode(outputs[0]))