Beta

Explore
Marketplace
Neural Labs
Chat
Wallet
Docs

Beta

Explore Marketplace Neural Labs Wallet Docs

Build

Manage

My assets Manage wallet Organization

Community

Discord X (Twitter)Telegram

Docs Privacy Policy Terms and Conditions

Proud member of

Proud member of

Docs Cookie Policy

Privacy Policy Terms and Conditions

© 2026 AlphaNeural AI. All rights reserved.

aligned_tinyllama_ultrafeedback_fixed1k_mars – AI Model by payelb | AlphaNeural AI

You can deploy this model and start earning money today!

payelb/
aligned_tinyllama_ultrafeedback_fixed1k_mars

like

0

reward-modeling

TinyLlama/TinyLlama-1.1B-Chat-v1.0

Views

No views yet

Aligned TinyLlama on UltraFeedback (fixed-1k prompt pool)

This model was aligned with TRL PPO using a reward model:

payelb/UltraFeedback_openbmb_deberta_1k_fixed_MARS (tag: mars)

Key settings:

Prompt pool: restricted to the same fixed/selected 1k subset used for RM training (loaded from CSV)
PPO updates: 200
batch size: 4
lr: 1e-05
LoRA: r=16, alpha=32, dropout=0.05