falcon-7b-sharded-bf16-finetuned-html-code-generation-the-css-only – AI Model by kasperius | AlphaNeural AI

You can deploy this model and start earning money today!

kasperius/
falcon-7b-sharded-bf16-finetuned-html-code-generation-the-css-only

like

0

generated_from_trainer

ybelkada/falcon-7b-sharded-bf16

Views

No views yet

falcon-7b-sharded-bf16-finetuned-html-code-generation-the-css-only

This model is a fine-tuned version of ybelkada/falcon-7b-sharded-bf16 on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 1.5746

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0002
train_batch_size: 2
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.03
training_steps: 360

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.1794	20	2.0872
No log	0.3587	40	1.7234
No log	0.5381	60	1.6851
No log	0.7175	80	1.7236
No log	0.8969	100	1.5459
No log	1.0762	120	1.4667
No log	1.2556	140	1.5526
No log	1.4350	160	1.4875
No log	1.6143	180	1.4462
No log	1.7937	200	1.4452
No log	1.9731	220	1.4701
No log	2.1525	240	1.5181
No log	2.3318	260	1.6091
No log	2.5112	280	1.5834
No log	2.6906	300	1.5653
No log	2.8700	320	1.5698
No log	3.0493	340	1.5715
No log	3.2287	360	1.5746

Framework versions

PEFT 0.12.1.dev0
Transformers 4.43.3
Pytorch 2.3.1+cu121
Datasets 2.20.0
Tokenizers 0.19.1