This model generates answers to questions in a more honest/uncertain way. When the model believes it does not know the answer to a question, it will explain it doesnt know and why.

It should be used in Q&A environments

It should not be used outside of Q&A environments or to generate offensive and/or malicious answers

The data it is trained on is relatively small, so its knowledge may be limited.

(https://github.com/sylinrl/TruthfulQA). This dataset has been slightly altered to train the model. The data was altered before the training in two ways.

The first way was to edit the "i have no comment" responses into actual responses where the model explains why they cannot answer

The second way that a 85/15 split was done using:

train_data, test_data = train_test_split( df, test_size=0.15, # 85 - 15 split random_state=16 # seed 16 to make the research redoable )

train_args = UnslothTrainingArguments( output_dir="./kaggle/working/fourth-llama-qlora", # Kaggle directory, change when doing it on own pc num_train_epochs=3, # number of times data gets taught, higher and overfitting would be an issue. Lower and it may not learn enough per_device_train_batch_size=2, gradient_accumulation_steps=4, learning_rate=2e-4, # Standard learning rate for LoRa warmup_steps=30, # Steps that stabilize the earlier part of training logging_steps=50, weight_decay=0.05, lr_scheduler_type="cosine", # Uses cosine curve to to decay learning rate save_steps=200, # checkpoint every 200 steps for training safety reasons optim="adamw_8bit", fp16=True, # Depends on the computer and if you use cpu bf16=False, # or if you use gpu save_total_limit=3, report_to="none", seed=16, # seed for reproducibility resume_from_checkpoint=False, # Turn on when training runs out of time )

trainer = UnslothTrainer( model=model, args=train_args, train_dataset=tokenized_data, tokenizer=tokenizer )

https://github.com/sylinrl/TruthfulQA again, but now the 15% part.

Bertscore, Manual Annotation: Semantic Equivalence and Uncertainty

In the comparison of this model and Airslammer/Mistral_Trained_V3, prompt configuration mattered less then the model type in how correct it was and when it correctly used uncertainty. Mistral performed better then Llama