Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
BERT_BiLSTM_civil_comments – AI Model by jfia0107 | AlphaNeural AI
You can deploy this model and start earning money today!
jfia0107
/
BERT_BiLSTM_civil_comments
like
0
mit
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Trained using
https://github.com/jfia0107/PyTorch-Lightning-NLP-lab
and these parameters:
freeze_backbone: True
unfreeze_top_n: 2
scheduler: linear_warmup
warmup_ratio: 0.1
pretrained_model: distilroberta-base
attn_implementation: flash_attention_2
head_dropout: 0.3
head_hidden_size: 128
head_num_layers: 2
batch_size: 2048
max_epochs: 5
optimizer: torch.optim.AdamW
bbone_lr: 0.0001
head_lr: 0.005
weight_decay: 0.01
minority_ratio: 0.05
