You can deploy this model and start earning money today!

cxoijve/
Llama-2-7b-chat-hf

Name: Llama-2-7b-chat-hf
Availability: InStock

peft

safetensors

meta-llama/Llama-2-7b-chat-hf

adapter

Views

No views yet

Model Description

NSMC 데이터에 대해 meta-llama/Llama-2-7b-chat-hf 미세튜닝
영화 리뷰 텍스트를 프롬프트에 포함하여 모델에 입력하면 '긍정' 또는 '부정'이라고 예측 텍스트를 직접 생성
NSMC의 train 스플릿 상위 2,000개 이상의 샘플을 학습에 사용
test 스플릿 상위 1,000개의 샘플만 측정

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 1
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 2
optimizer: adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08,
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.03
training_args.logging_steps: 100
training_args.max_steps : 1600
trainable params: 19,988,480 || all params: 6,758,404,096 || trainable%: 0.2957573965106688

Training Results

TrainOutput(global_step=1600, training_loss=0.7892872190475464, metrics={'train_runtime': 5825.2445, 'train_samples_per_second': 0.549, 'train_steps_per_second': 0.275, 'total_flos': 6.51493254365184e+16, 'train_loss': 0.7892872190475464, 'epoch': 1.6})

Accuracy

Llama2: 정확도 0.52

	TP	TN
PP	192	168
PN	317	324

정확도를 향상시키기 위해 여러 차례 노력을 해보았지만 반복해서 오류가 발생하였습니다.

Model Card Authors

cxoijve

cxoijve/Llama-2-7b-chat-hf