You can deploy this model and start earning money today!

Rfy23/
hw-llama-2-7B-nsmc

Name: hw-llama-2-7B-nsmc
Availability: InStock

peft

safetensors

K-intelligence/midm-bitext-S-7B-inst-v1

adapter

Views

No views yet

Model Card for Model ID

KT-AI/midm-bitext-S-7B-inst-v1

Model Details

Model Description

NSMC 영화 리뷰 데이터에 대하여 KT-AI/midm-bitext-S-7B-inst-v1 미세튜닝.
입력 프롬프트를 이용하여 데이터셋의 document(리뷰)가 긍정적인 내용이면 '1'을 부정적인 내용이면 '0'을 예측하도록 함.
train data: nsmc train 상위 2000개 샘플 이용
test data: nsmc test 상위 2000개 샘플 이용

Training Data

'nsmc'

상위 2000개 데이터 이용

Training Procedure

prepare_sample_text에 리뷰를 긍정/부정으로 판단하도록 입력 프롬프트 수정하였음.

Training Hyperparameters

per_device_train_batch_size: 1
per_device_eval_batch_size: 1
learning_rate: 1e-4
gradient_accumulation_steps: 2
optimizer: paged_adamw_32bit
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.03
training_args.logging_steps: 50
training_args.max_steps : 1000
trainable params: trainable params: 16,744,448 || all params: 7,034,347,520 || trainable%: 0.23803839591934178

Results

TrainOutput(global_step=1000, training_loss=1.0208648338317872, metrics={'train_runtime': 1128.0266, 'train_samples_per_second': 1.773, 'train_steps_per_second': 0.887, 'total_flos': 3.1051694997504e+16, 'train_loss': 1.0208648338317872, 'epoch': 1.0})

Accruacy

미세튜닝 후 모델의 정확도:0.61

Rfy23/hw-llama-2-7B-nsmc