This model is a fine-tuned version of the unsloth/Llama-3.2-3B-bnb-4bit, developed to act as SARA—the Security Awareness and Resilience Assistant. SARA is optimized to be a lightweight, offline-friendly AI assistant capable of running on low-spec laptops, designed to provide practical cybersecurity advice in a conversational style.
This model is intended for providing cybersecurity information and guidance to general users in an accessible, offline-friendly way.
Direct Use
This model can be used as an offline assistant for basic cybersecurity questions, answering common queries in a conversational format. It is ideal for use cases where an internet connection is not available or where low-spec hardware constraints apply.
Out-of-Scope Use
This model should not be used for professional or critical cybersecurity advice, as it is designed for general guidance and may lack the specificity required for advanced technical issues. It is also not suitable for providing nuanced advice in areas outside basic cybersecurity practices.
Bias, Risks, and Limitations
While SARA is optimized for basic cybersecurity education, it has limitations in depth and may lack the ability to answer highly technical questions. Additionally, it may be limited in handling complex, nuanced queries due to its lightweight design and quantized 4-bit structure.
Recommendations
Users should consider SARA as an educational tool rather than a replacement for professional cybersecurity advice. Further fine-tuning could help improve the model's handling of diverse inputs and conversational depth, making it more robust for varied user needs.
How to Get Started with the Model
python
1from transformers import AutoModelForCausalLM, AutoTokenizer
23# Load model and tokenizer4model = AutoModelForCausalLM.from_pretrained("EryriLabs/Llama-3.2-SARA-3b", device_map="auto")5tokenizer = AutoTokenizer.from_pretrained("EryriLabs/Llama-3.2-SARA-3b")67# Sample question8input_text ="What make a strong password?"910# Tokenize and generate response11inputs = tokenizer(input_text, return_tensors="pt").to("cuda")12outputs = model.generate(inputs["input_ids"], max_length=50)13response = tokenizer.decode(outputs[0], skip_special_tokens=True)1415print(response)
Training Details
Training Data
The model was fine-tuned on a custom Q&A-style dataset centered on cybersecurity fundamentals, such as creating a strong password, using 2-Step Verification etc.
Training Procedure
The fine-tuning was conducted on a system with an Intel i9 12900k CPU, an NVIDIA GeForce RTX 4090 GPU, and 32GB RAM. Unsloth’s 4-bit quantization (bnb-4bit) was applied to keep the model compact and efficient for low-spec laptop deployment.
Training Hyperparameters
Training regime: Mixed precision with 4-bit quantization (bnb-4bit)
Speeds, Sizes, Times [optional]
Training took approximately 10 minutes, with additional fine-tuning recommended for improved performance, especially for handling varied text inputs and enhancing conversational depth.
Evaluation
Testing Data, Factors & Metrics
Testing Data
Testing was conducted on a dataset of common cybersecurity questions to evaluate the model’s responsiveness and accuracy for general use cases.
Factors
The model was evaluated based on its ability to provide clear, direct answers to basic cybersecurity questions.
Metrics
The main evaluation metric was response accuracy for typical cybersecurity queries.
Results
The model performs adequately for its intended purpose, with room for improvement in response handling and input variability.
Summary
SARA functions well for basic cybersecurity guidance but requires additional fine-tuning to better handle diverse inputs and enhance conversational flow.
