mnoukhov_-_pythia1b-sft-tldr-gguf – AI Model by RichardErkhov | AlphaNeural AI

You can deploy this model and start earning money today!

RichardErkhov/
mnoukhov_-_pythia1b-sft-tldr-gguf

like

0

endpoints_compatible

Views

No views yet

Quantization made by Richard Erkhov.

Request more models

pythia1b-sft-tldr - GGUF

Model creator: https://huggingface.co/mnoukhov/
Original model: https://huggingface.co/mnoukhov/pythia1b-sft-tldr/

Name	Quant method	Size
pythia1b-sft-tldr.Q2_K.gguf	Q2_K	0.39GB
pythia1b-sft-tldr.IQ3_XS.gguf	IQ3_XS	0.44GB
pythia1b-sft-tldr.IQ3_S.gguf	IQ3_S	0.45GB
pythia1b-sft-tldr.Q3_K_S.gguf	Q3_K_S	0.45GB
pythia1b-sft-tldr.IQ3_M.gguf	IQ3_M	0.48GB
pythia1b-sft-tldr.Q3_K.gguf	Q3_K	0.51GB
pythia1b-sft-tldr.Q3_K_M.gguf	Q3_K_M	0.51GB
pythia1b-sft-tldr.Q3_K_L.gguf	Q3_K_L	0.55GB
pythia1b-sft-tldr.IQ4_XS.gguf	IQ4_XS	0.54GB
pythia1b-sft-tldr.Q4_0.gguf	Q4_0	0.56GB
pythia1b-sft-tldr.IQ4_NL.gguf	IQ4_NL	0.56GB
pythia1b-sft-tldr.Q4_K_S.gguf	Q4_K_S	0.56GB
pythia1b-sft-tldr.Q4_K.gguf	Q4_K	0.61GB
pythia1b-sft-tldr.Q4_K_M.gguf	Q4_K_M	0.61GB
pythia1b-sft-tldr.Q4_1.gguf	Q4_1	0.61GB
pythia1b-sft-tldr.Q5_0.gguf	Q5_0	0.66GB
pythia1b-sft-tldr.Q5_K_S.gguf	Q5_K_S	0.66GB
pythia1b-sft-tldr.Q5_K.gguf	Q5_K	0.71GB
pythia1b-sft-tldr.Q5_K_M.gguf	Q5_K_M	0.71GB
pythia1b-sft-tldr.Q5_1.gguf	Q5_1	0.72GB
pythia1b-sft-tldr.Q6_K.gguf	Q6_K	0.78GB
pythia1b-sft-tldr.Q8_0.gguf	Q8_0	1.0GB

Original model description:

license: apache-2.0 base_model: EleutherAI/pythia-1b-deduped tags:

trl
sft
generated_from_trainer model-index:
name: pythia1b-sft-tldr results: []

pythia1b-sft-tldr

This model is a fine-tuned version of EleutherAI/pythia-1b-deduped on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 2.3828

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
distributed_type: multi-GPU
num_devices: 4
gradient_accumulation_steps: 4
total_train_batch_size: 128
total_eval_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
num_epochs: 1
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
2.5278	0.2007	183	2.4199
2.4136	0.4013	366	2.4004
2.3978	0.6020	549	2.3887
2.3813	0.8026	732	2.3828

Framework versions

Transformers 4.41.1
Pytorch 2.2.1+cu121
Datasets 2.19.0
Tokenizers 0.19.1