This model is a fine-tuned version of NousResearch/Llama-2-7b-chat-hf, optimized for movie-related conversations. The fine-tuning process was performed using LoRA to efficiently adapt the model while keeping computational requirements manageable. It is designed to improve conversational understanding and response generation for movie-related queries.

Training Details

Hardware Used: Kaggle GPU (T4x2)
Fine-Tuning Framework: Hugging Face Transformers + LoRA
Output Folder: ./results
Number of Epochs: 2
Batch Size:
- Per Device Train: 4
- Per Device Eval: 4
Gradient Accumulation Steps: 1
Gradient Checkpointing: Enabled
Max Gradient Norm: 0.3
Mixed Precision: fp16=False, bf16=False
Optimizer: paged_adamw_32bit
Learning Rate: 2e-5
Weight Decay: 0.001
LR Scheduler Type: cosine
Warmup Ratio: 0.03
Max Steps: -1 (determined by epochs)
Quantization Settings:
- use_4bit = True
- bnb_4bit_compute_dtype = float16
- bnb_4bit_quant_type = nf4
- use_nested_quant = False
LoRA Hyperparameters:
- lora_r = 64
- lora_alpha = 16
- lora_dropout = 0.05
Sequence Length: Dynamic (max_seq_length=None)
Packing: Disabled (packing=False)
Device Map: {"": 0}

Capabilities

Answers movie-related questions with improved accuracy.
Understands movie genres, actors, directors, and plots.
Provides recommendations based on user preferences.

Limitations

May generate incorrect or biased information.
Limited to the knowledge present in the training dataset.
Does not have real-time access to new movie releases.

Usage

You can load and use the model with the following code:

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "HiTruong/Llama-2-chat-finetuned"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

def generate_answer(question):
    inputs = tokenizer(f"<s>[INST] {question} [/INST]", return_tensors="pt", truncation=True, max_length=100).to(model.device)
    with torch.no_grad():
        output = model.generate(**inputs, max_length=75, eos_token_id=tokenizer.eos_token_id)
    response = tokenizer.decode(output[0], skip_special_tokens=True, clean_up_tokenization_spaces=True)
    return response.replace(f"[INST] {question} [/INST]", "").strip().split('.')[0]

input_text = "What are some great sci-fi movies?"

print(generate_answer(input_text))