Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
Qwen2-7B-tldr-preference-sft – AI Model by heli-stand | AlphaNeural AI
You can deploy this model and start earning money today!
heli-stand
/
Qwen2-7B-tldr-preference-sft
like
0
transformers
safetensors
qwen2
text-generation
tldr
text-summarization
conversational
text-generation-inference
endpoints_compatible
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Model Card for Qwen2-7B Supervised Fine-tuned on TLDR Dataset
This model is a supervised fine-tuned version of Qwen2-7B, optimized for text summarization tasks using the TLDR dataset on completions only.
Model Details
Model Description
This is a fine-tuned Qwen2-7B model, trained on the TLDR dataset for text summarization tasks.
Model type:
Transformer-based language model
Language(s) (NLP):
English
License:
apache-2.0
Finetuned from model:
Qwen2-7B
Model Sources
Repository:
https://huggingface.co/Qwen/Qwen2-7B
Training Procedure
Training Data
The model was fine-tuned on the TLDR dataset, which consists of long texts paired with their summarized versions.
Repository:
https://huggingface.co/datasets/trl-internal-testing/tldr-preference-sft-trl-style
Training Hyperparameters
Optimizer:
Adafactor
Learning Rate:
5e-6 (selected from grid search [5e-4, 1e-4, 5e-5, 1e-5, 5e-6, 1e-6])
Batch Size:
64 (16 per_device_train_batch_size * 4 devices)
Epochs:
1 (total 3 epochs)
Padding:
Right-side padding
Training Infrastructure
Hardware:
4 * NVIDIA H100 GPU
Evaluation Results
This checkpoint represents the model with the lowest evaluation loss during the grid search process.
Best Evaluation Loss:
1.751 (reference only)
Achieved at:
Epoch 1, Learning Rate 5e-6
