NOTE: MODEL TRAINED ON 5M TOKENS AGAIN IN 14 JUNE 2025, IF YOU DOWNLOADED WEIGHTS BEFORE; REINSTALL!
This model trained on 5M (About 5.000.000) Tokens, With "English" Sentences.
Model IS NOT for QA (UNLIKE ChatGPT or LLama), this model is only Pre-Trained on Large Corpus, so this is a Base Model
Model Details
Model Description
CHECK THE COMMUNITY FOR VERY IMPORTANT UPDATES!
Developed by: Bertug Gunel (Bertuğ Günel)
Funded by [optional]: Nobody
Shared by [optional]: Nobody
Model type: Decoder-Only Transformer
Language(s) (NLP): English
License: CC-BY-NC-4.0
Finetuned from model [optional]: Not Fine-Tuned
Model Sources [optional]
Repository: Cooming Soon!
Paper [optional]:"Attention All You Need", 1706.03762
Demo [optional]: Model is already a demo model.
Uses
This codes, loads the model (BrtGPT-124M-Base), you can use it!
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
# Load Model and Tokenizer
model_name = "Bertug1911/BrtGPT-124m-Base"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
# Input
prompt = "Math is so important because"
# Tokenize it
inputs = tokenizer(prompt, return_tensors="pt")
# Generate with model
output = model.generate(
**inputs,
max_new_tokens=50,
temperature=0.01,
top_k=1,
do_sample=True
)
# Decode output
generated_text = tokenizer.decode(output[0], skip_special_tokens=False)
generated_text = generated_text.replace(" ", "")
generated_text = generated_text.replace("Ġ", " ")
print(generated_text)
API USE:
from gradio_client import Client
client = Client("Bertug1911/BrtGPT-Web-UI")
result = client.predict(
prompt="Hello!!",
temperature=0.5,
top_k=10,
max_new_tokens=15,
api_name="/generate_text"
)
print(result)
USAGE EXAMPLES:
Input
Max New Tokens
Temperature
Output
"Today"
50
0.1
"Today, a complex and multifaceted system, is often viewed as a myriad of the intricacies of the human mind and the intricacies of the human condition. It is believed to be a powerful"
"To stop world hunger, we should"
50
0.1
"To stop world hunger, we should be able to find a more stable and healthy relationship with the body. By doing so, we can make a mealtime and easier to start and maintain a healthy and balance."
"Math is so important because"
50
0.1
"MMath is so important because it's essential to carefully consider and address any potential health concerns that may arise from the condition, as it can lead to a range of health issues. By including the bleeding and potentially causing sympt..."
"To be rich, you should,"
50
0.4
"To be rich you should be on the same time, it's essential to consider the various factors that contribute to your unique needs. For instance, it's crucial to consider that you should be taking a black room,..."
Direct Use
Perhaps the worst part of open source models is that using them is very laborious and requires a lot of processing power, but our model solves both problems:
you can use and download them for FREE and VERY EASILY from the link below!
Model only generates (completes) "English" sentences with "English" tokens. (And contains some Japanese/Chinese tokens.) Don't try with another Languages!
Bias, Risks, and Limitations
Model can generates: "Political" contents, USE YOUR OWN RISK
Recommendations
No big risks or biases!
You can use it freely (But only "Non-commerical")
How to Get Started with the Model
You can use model for generating English texts.
Model is FREE to use.
Training Details
Training Data
NOTE: MODEL TRAINED ON 5M TOKENS AGAIN IN 13 JUNE 2025, IF YOU DOWNLOADED WEIGHTS BEFORE; REINSTALL!
Model trained on: Train.csv (5M tokens, 15000+ lines)
Data Type
Training Type
Tokens (Total)
Status
Raw (sentences)
Pre Training
About 5M (5000K)
FINISHED
Raw (sentences)
Fine-Tuning (For upgrade model performance on tests and usage!)
About 0.1M (100K)
FINISHED ON 17 JUNE
Instruction (Coming soon!)
Instruction Tuning (IFT)
Cooming soon!
SOON! (Probably; in July (5-15))
FINE-TUNING DETAILS:
You can acces fine-tuned model with this link: "https://huggingface.co/Bertug1911/BrtGPT-124m-FineTuned" NOTE: Fine-tuned model has "model.safetensors", this file contains weights (Fine-tuned from this model.)
Training Procedure
Model Trained on: B200 GPU (Nvidia) for 21,5 Minutes.
Training Hyperparameters
Training regime: Training precision is: FP16, Sparsity: OFF
