You can deploy this model and start earning money today!

Shlok307/
gemma-3-1b-it-cyber-lora

Name: gemma-3-1b-it-cyber-lora
Availability: InStock

Views

No views yet

Model Card for Model ID

This model is a LoRA adapter fine-tuned on top of google/gemma-3-1b-it, optimized for safe cybersecurity explanations, vulnerability analysis, and general security guidance. It improves the base Gemma model’s performance on technical cyber topics while keeping responses safe, helpful, and beginner-friendly.

Model Description

This is a lightweight LoRA (QLoRA) adapter trained on a curated dataset of cybersecurity conversations containing system, user, and assistant messages.
The fine-tuning process enhances Gemma-3-1B to be:

Better at explaining security concepts
More fluent in cybersecurity terminology
More structured and coherent in its answers
More helpful, safer, and suitable for educational use

The underlying base model remains google/gemma-3-1b-it.

Credits

Developed by: Shlok Talhar
Finetuned from: google/gemma-3-1b-it
Model type: LoRA adapter for causal language modeling
Language: English
License: Gemma License
Training method: QLoRA using PEFT

Model Sources

Repository: https://huggingface.co/Shlok307/gemma-3-1b-it-cyber-lora
Base Model: https://huggingface.co/google/gemma-3-1b-it

How to Get Started with the Model

Use the code below to get started with the model.

!pip install -U bitsandbytes accelerate

python

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

base = "google/gemma-3-1b-it"
adapter = "Shlok307/gemma-3-1b-it-cyber-lora"

# 4-bit quantization (recommended)
quant_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype="float16",
    bnb_4bit_quant_type="nf4"
)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(base)

# Load base model in 4-bit
model = AutoModelForCausalLM.from_pretrained(
    base,
    quantization_config=quant_config,
    device_map="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(model, adapter)
model.eval()

# Example prompt
prompt = (
    "System: You are a straight forward cybersecurity and Fraud detection assistant.\n"
    "User: Explain how SQL injection works on website.\n"
    "Assistant:"
)

# Tokenize
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# Inference
with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=450,
        temperature=0.7
    )

print(tokenizer.decode(output[0], skip_special_tokens=True))

📊 Model Parameters

Total parameters: 656,968,832
LoRA trainable parameters: 5,963,776
Percentage of model fine-tuned: 0.9078%
Parameter-efficient tuning: Yes (LoRA / QLoRA)

Prompt Format

System: <system>
User: <user>
Assistant: <assistant>

Summary of Improvements

Strong improvements in concept explanations
Better consistency and clarity on cybersecurity tasks
Safe outputs with reduced harmful behavior