Frozen Base Model (4-bit)
+ Trainable LoRA ΔW
+ Response-only loss masking
→ Instruction-aligned behavior

python

import torch

# Ensure the model is in evaluation mode
model.eval()

# Function to format prompts consistently with training data
def format_prompt(instruction, context=None):
    if context:
        return f"Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Input:\n{context}\n\n### Response:\n"
    else:
        return f"Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Response:\n"

# Define a few test prompts
test_prompts = [
    {
        "instruction": "Explain the concept of photosynthesis in simple terms.",
        "context": None
    },
    {
        "instruction": "What is the capital of France?",
        "context": None
    },
    {
        "instruction": "Summarize the main idea of the following text:",
        "context": "The quick brown fox jumps over the lazy dog. This sentence is often used to display all letters of the English alphabet, making it a pangram."
    },
    {
        "instruction": "List three benefits of regular exercise.",
        "context": None
    }
]

# Add the new test prompts (assuming `new_test_prompts` is defined as in the previous step)
new_test_prompts = [
    {
        "instruction": "Write a short, imaginative story about a cat who discovers a secret portal to another dimension under its owner's bed.",
        "context": None
    },
    {
        "instruction": "If a train leaves New York at 10 AM traveling at 60 mph and another train leaves Chicago at 11 AM traveling at 50 mph, and the cities are 800 miles apart, at what time do they meet? (Assume they are traveling towards each other on the same track).",
        "context": None
    },
    {
        "instruction": "What is the capital of Australia?",
        "context": None
    },
    {
        "instruction": "Explain the difference between supervised and unsupervised learning in machine learning, and provide an example of when each would be used.",
        "context": None
    },
    {
        "instruction": "Summarize the following passage:",
        "context": "The advent of artificial intelligence has brought forth a new era of technological advancement, impacting various sectors from healthcare to finance. While AI promises increased efficiency and innovative solutions, it also raises ethical concerns regarding job displacement, privacy, and bias in algorithms. Societies worldwide are grappling with how to regulate and integrate AI responsibly, balancing progress with human values. This calls for a multidisciplinary approach involving policymakers, technologists, ethicists, and the public to shape a future where AI serves humanity's best interests."
    }
]
test_prompts.extend(new_test_prompts)

python
# Ensure the base model is in evaluation mode
base_model.eval()

# Function to format prompts consistently with training data
def format_prompt(instruction, context=None):
    if context:
        return f"Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Input:\n{context}\n\n### Response:\n"
    else:
        return f"Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Response:\n"

# Reuse the test_prompts defined previously
# (Assuming test_prompts is available from previous execution or defined globally)

print("\n--- Generating Responses from BASE MODEL ---\n")
with torch.no_grad():
    for i, prompt_data in enumerate(test_prompts):
        instruction = prompt_data["instruction"]
        context = prompt_data["context"]

        formatted_input = format_prompt(instruction, context)

        # Tokenize the input prompt
        inputs = tokenizer(formatted_input, return_tensors="pt").to(base_model.device)

        # Generate response using the BASE MODEL
        outputs = base_model.generate(
            **inputs,
            max_new_tokens=150,
            do_sample=True,
            temperature=0.7,
            top_k=50,
            top_p=0.95,
            eos_token_id=tokenizer.eos_token_id,
            pad_token_id=tokenizer.pad_token_id
        )

        generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
        response = generated_text[len(formatted_input):].strip()

        print(f"### Test Prompt {i+1} (BASE MODEL): ###")
        print(f"Instruction: {instruction}")
        if context:
            print(f"Context: {context}")
        print(f"Base Model Response: {response}\n")

--- Generating Responses from BASE MODEL ---

### Test Prompt 1 (BASE MODEL): ###
Instruction: Explain the concept of photosynthesis in simple terms.
Base Model Response: The sun heats the earth's surface and creates energy through the photochemical process. The photochemical process is the process by which photosynthetic organisms convert light energy into chemical energy in the form of a chemical compound called chlorophyll. Chlorophyll is the key component of the green plant and is responsible for the color of the plant.

Chlorophyll is composed of a chromophore (a blue colored pigment) and a non-chromophore (a yellow colored pigment). The non-chromophore is responsible for the red or purple color of the plant.

The green plant is composed of chlorophyll and the non-chromophore. The green plant uses the chemical energy of the sunlight to synthesize ATP molecules. ATP is a chemical compound

### Test Prompt 2 (BASE MODEL): ###
Instruction: What is the capital of France?
Base Model Response: Paris

### Test Prompt 3 (BASE MODEL): ###
Instruction: Summarize the main idea of the following text:
Context: The quick brown fox jumps over the lazy dog. This sentence is often used to display all letters of the English alphabet, making it a pangram.
Base Model Response: The quick brown fox jumps over the lazy dog.

### Test Prompt 4 (BASE MODEL): ###
Instruction: List three benefits of regular exercise.
Base Model Response: The three benefits of regular exercise are:
1. It reduces your risk of heart disease.
2. It helps you lose weight.
3. It increases your lifespan.

### Test Prompt 5 (BASE MODEL): ###
Instruction: Write a short, imaginative story about a cat who discovers a secret portal to another dimension under its owner's bed.
Base Model Response: The cat, named Gigi, woke up in the middle of the night to find a tiny portal leading to another dimension. She jumped up and down excitedly and her owner, a woman named Lola, came running and picked her up. They spent the day together, going to museums and outings and playing games.

One day, the portal was discovered by a little girl named Lulu. The portal was hidden under the bed and Lulu was afraid to go near it. So the two cats went up to the portal, hoping to find the girl.

But when they went into the portal, they discovered that it was a tunnel under a bridge. Lulu was so scared that she cried. When she looked around,

### Test Prompt 6 (BASE MODEL): ###
Instruction: If a train leaves New York at 10 AM traveling at 60 mph and another train leaves Chicago at 11 AM traveling at 50 mph, and the cities are 800 miles apart, at what time do they meet? (Assume they are traveling towards each other on the same track).
Base Model Response: At 10 AM, the train is traveling 60 mph. It will arrive in Chicago at 11 AM. After traveling for 800 miles, it will reach New York.

### Test Prompt 7 (BASE MODEL): ###
Instruction: What is the capital of Australia?
Base Model Response: Melbourne

### Test Prompt 8 (BASE MODEL): ###
Instruction: Explain the difference between supervised and unsupervised learning in machine learning, and provide an example of when each would be used.
Base Model Response: In machine learning, supervised learning involves learning how to build a predictive model based on a set of data. The model is trained to predict the outcome of new data, and is typically used to improve the quality of the data used for prediction. In contrast, unsupervised learning does not require a model to be trained, and instead provides a method to quickly determine if a given data point belongs to a specific class. This technique can be used to classify new data into a specific class.

### Test Prompt 9 (BASE MODEL): ###
Instruction: Summarize the following passage:
Context: The advent of artificial intelligence has brought forth a new era of technological advancement, impacting various sectors from healthcare to finance. While AI promises increased efficiency and innovative solutions, it also raises ethical concerns regarding job displacement, privacy, and bias in algorithms. Societies worldwide are grappling with how to regulate and integrate AI responsibly, balancing progress with human values. This calls for a multidisciplinary approach involving policymakers, technologists, ethicists, and the public to shape a future where AI serves humanity's best interests.
Base Model Response: The following passage is about how to regulate and integrate artificial intelligence responsibly:

Societies worldwide are grappling with how to regulate and integrate artificial intelligence responsibly. The AI revolution has brought forth a new era of technological advancement, impacting various sectors from healthcare to finance. While AI promises increased efficiency and innovative solutions, it also raises ethical concerns regarding job displacement, privacy, and bias in algorithms. Societies worldwide are grappling with how to regulate and integrate AI responsibly, balancing progress with human values. This calls for a multidisciplinary approach involving policymakers, technologists, ethicists, and the public to shape a future where AI serves humanity's best interests.

The AI revolution has brought forth a new era of technological advancement, impacting

python
# Ensure the LoRA-tuned model is in evaluation mode
model.eval()

# Reuse the format_prompt function and test_prompts

print("\n--- Generating Responses from LO-RA TUNED MODEL ---\n")
with torch.no_grad():
    for i, prompt_data in enumerate(test_prompts):
        instruction = prompt_data["instruction"]
        context = prompt_data["context"]

        formatted_input = format_prompt(instruction, context)

        # Tokenize the input prompt
        inputs = tokenizer(formatted_input, return_tensors="pt").to(model.device)

        # Generate response using the LO-RA TUNED MODEL
        outputs = model.generate(
            **inputs,
            max_new_tokens=150,
            do_sample=True,
            temperature=0.7,
            top_k=50,
            top_p=0.95,
            eos_token_id=tokenizer.eos_token_id,
            pad_token_id=tokenizer.pad_token_id
        )

        generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
        response = generated_text[len(formatted_input):].strip()

        print(f"### Test Prompt {i+1} (LO-RA MODEL): ###")
        print(f"Instruction: {instruction}")
        if context:
            print(f"Context: {context}")
        print(f"LoRA Model Response: {response}\n")

--- Generating Responses from LO-RA TUNED MODEL ---

### Test Prompt 1 (LO-RA MODEL): ###
Instruction: Explain the concept of photosynthesis in simple terms.
LoRA Model Response: Photosynthesis is the process of capturing energy from the sun and converting it into sugars. The process can be broken down into two phases:

First, light is captured by the leaves of plants and converted into sugars.

Second, the sugars are used to make energy for the plant.

### Test Prompt 2 (LO-RA MODEL): ###
Instruction: What is the capital of France?
LoRA Model Response: Paris is the capital of France.

### Test Prompt 3 (LO-RA MODEL): ###
Instruction: Summarize the main idea of the following text:
Context: The quick brown fox jumps over the lazy dog. This sentence is often used to display all letters of the English alphabet, making it a pangram.
LoRA Model Response: The quick brown fox jumps over the lazy dog.

### Test Prompt 4 (LO-RA MODEL): ###
Instruction: List three benefits of regular exercise.
LoRA Model Response: 1. Regular exercise helps you stay healthy.
2. Regular exercise helps you burn calories.
3. Regular exercise helps you maintain your body weight.

### Test Prompt 5 (LO-RA MODEL): ###
Instruction: Write a short, imaginative story about a cat who discovers a secret portal to another dimension under its owner's bed.
LoRA Model Response: "Cat," the owner of the mysterious portal, had been feeling the cold all night and had been trying to warm up. He had no idea what to do so he opened the door and started to investigate.

"There is a portal under the bed!" he cried.

"Oh, goody! I was hoping that you would discover it!" said his owner, relieved.

"It is a portal! And it is open! I can go through!"

"I thought you were asleep!"

"I am! I was having a nightmare."

"A nightmare? I don't believe it! You are dreaming! You are dreaming!"

"I am dreaming! And I can go through

### Test Prompt 6 (LO-RA MODEL): ###
Instruction: If a train leaves New York at 10 AM traveling at 60 mph and another train leaves Chicago at 11 AM traveling at 50 mph, and the cities are 800 miles apart, at what time do they meet? (Assume they are traveling towards each other on the same track).
LoRA Model Response: At 10 AM, they will meet at an intersection at a point in time of 10:00 AM.

### Test Prompt 7 (LO-RA MODEL): ###
Instruction: What is the capital of Australia?
LoRA Model Response: Canberra is the capital of Australia

### Test Prompt 8 (LO-RA MODEL): ###
Instruction: Explain the difference between supervised and unsupervised learning in machine learning, and provide an example of when each would be used.
LoRA Model Response: In machine learning, unsupervised learning is used for training or testing data, and supervised learning is used for training or testing data. In supervised learning, the machine learning system is given the dataset, and the goal is to find a model that is able to predict the target data. In unsupervised learning, the machine learning system is given the dataset, and the goal is to find an unknown model that can identify the target data.

In machine learning, unsupervised learning is used for training or testing data, and supervised learning is used for training or testing data. In supervised learning, the machine learning system is given the dataset, and the goal is to find a model that is able to predict the target data. In unsupervised learning, the

### Test Prompt 9 (LO-RA MODEL): ###
Instruction: Summarize the following passage:
Context: The advent of artificial intelligence has brought forth a new era of technological advancement, impacting various sectors from healthcare to finance. While AI promises increased efficiency and innovative solutions, it also raises ethical concerns regarding job displacement, privacy, and bias in algorithms. Societies worldwide are grappling with how to regulate and integrate AI responsibly, balancing progress with human values. This calls for a multidisciplinary approach involving policymakers, technologists, ethicists, and the public to shape a future where AI serves humanity's best interests.
LoRA Model Response: Artificial intelligence has brought forth a new era of technological advancement, impacting various sectors from healthcare to finance. While AI promises increased efficiency and innovative solutions, it also raises ethical concerns regarding job displacement, privacy, and bias in algorithms. Societies worldwide are grappling with how to regulate and integrate AI responsibly, balancing progress with human values. This calls for a multidisciplinary approach involving policymakers, technologists, ethicists, and the public to shape a future where AI serves humanity's best interests.

While it has been a boon to many industries, there are also many societal and ethical issues associated with AI. For example, AI has been used to create a new class of criminals known as the "robo-criminal." This type of