This repository hosts a parameter-efficient, fine-tuned LoRA adapter for Gemma-2-2b-it. The model has been specialized to execute a precise, highly stylized conversational persona: a cynical, hard-boiled 1940s Noir Detective.

It was trained locally on a highly memory-constrained consumer GPU utilizing 4-bit QLoRA optimizations.

Model Details

Model Description

Developed by: saad9694
Model type: Causal Language Model Adapter (PEFT / LoRA)
Language(s) (NLP): English (en)
Finetuned from model: google/gemma-2-2b-it
License: Gemma Terms of Use

Model Sources

Repository: https://github.com/saad9694/Fine-tune-Gemma-LLM
Base Model Page: google/gemma-2-2b-it

Uses

Direct Use

This model adapter is specifically intended for stylized text generation, interactive roleplay, creative writing pipelines, and persona-driven conversational agents. It forces the model to respond exclusively through a gritty, metaphorical, and analytical hard-boiled investigator persona.

Out-of-Scope Use

This model is not intended for standard factual Q&A, safety-critical medical advice, financial forecasting, mathematical computations, or vanilla code generation. Its objective function was strictly fine-tuned to favor thematic prose over raw objective compliance.

How to Get Started with the Model

Use the code below to dynamically download and inject this adapter over the base model at runtime:

python

import os
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

base_model_id = "google/gemma-2-2b-it"
lora_adapter_repo = "saad9694/gemma-2-2b-it-noir-detective"

# Configure 4-bit quantization for resource-constrained execution
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    quantization_config=bnb_config,
    device_map="auto"
)

# Fetch and apply the LoRA adapter layers
model = PeftModel.from_pretrained(base_model, lora_adapter_repo)
model.eval()

# Inference example
prompt = "<start_of_turn>user\nWho killed the classical jazz musician?\n<end_of_turn>\n<start_of_turn>model\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.7, do_sample=True)

print(tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True))