You can deploy this model and start earning money today!

Masioki/
gttbsc_distilbert-ft

Name: gttbsc_distilbert-ft
Availability: InStock

transformers

tensorboard

safetensors

single-embedding-sentence-classifier

generated_from_trainer

asapp/slue-phase-2

model-index

endpoints_compatible

Views

No views yet

gttbsc_distilbert-ft

Ground truth text multi-label DAC.
Fined tuned using LoRa.

Model description

Backbone: DistilBert uncased
Pooling: Self attention
Multi-label classification head: 2 dense layers with two dropouts 0.3 and Tanh activation inbetween

Training and evaluation data

Trained on ground truth.
Evaluated on ground truth (GT) and normalized Whisper small transcripts (E2E).

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.00043
train_batch_size: 2
eval_batch_size: 2
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 8
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 20

Framework versions

Transformers 4.41.2
Pytorch 2.3.0+cu121
Datasets 2.19.2
Tokenizers 0.19.1

Masioki/gttbsc_distilbert-ft