You can deploy this model and start earning money today!

tabbas97/
distilbert-base-uncased-finetuned-pubmed-lora-trained-tabbas97

Name: distilbert-base-uncased-finetuned-pubmed-lora-trained-tabbas97
Availability: InStock

peft

safetensors

generated_from_trainer

pubmed-summarization

distilbert/distilbert-base-uncased

adapter

apache-2.0

Views

No views yet

distilbert-base-uncased-finetuned-pubmed-lora-trained-tabbas97

This model is a fine-tuned version of distilbert-base-uncased on the pubmed-summarization dataset. It achieves the following results on the evaluation set:

Loss: 1.9256

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 32
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 4
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
2.1986	0.1667	500	2.0156
2.1414	0.3334	1000	1.9893
2.1247	0.5002	1500	1.9770
2.1106	0.6669	2000	1.9640
2.103	0.8336	2500	1.9548
2.0974	1.0003	3000	1.9519
2.0874	1.1671	3500	1.9506
2.0842	1.3338	4000	1.9470
2.0799	1.5005	4500	1.9406
2.0781	1.6672	5000	1.9363
2.0763	1.8339	5500	1.9371
2.0664	2.0007	6000	1.9311
2.0717	2.1674	6500	1.9277
2.0683	2.3341	7000	1.9247
2.0622	2.5008	7500	1.9290
2.0614	2.6676	8000	1.9170
2.0614	2.8343	8500	1.9239
2.0646	3.0010	9000	1.9211

Framework versions

PEFT 0.11.1
Transformers 4.40.2
Pytorch 2.2.1+cu121
Datasets 2.19.1
Tokenizers 0.19.1

tabbas97/distilbert-base-uncased-finetuned-pubmed-lora-trained-tabbas97

distilbert-base-uncased-finetuned-pubmed-lora-trained-tabbas97

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

tabbas97/
distilbert-base-uncased-finetuned-pubmed-lora-trained-tabbas97