SARAA-8B-ORPO-AUNQA: Self-Assessment Report Analysis Assistant

📋 Model Description

SARAA-8B-ORPO-AUNQA is a specialized large language model fine-tuned for analyzing Self-Assessment Reports according to ASEAN University Network Quality Assurance (AUN-QA) standards. This model is designed to assist educational institutions in evaluating and improving their quality assurance processes through intelligent document analysis and interactive Q&A capabilities.

Developed by: StrangeSX
Model Type: Causal Language Model (Fine-tuned Llama-3-8B)
Language(s): English, Thai
License: Apache 2.0
Finetuned from: unsloth/llama-3-8b-bnb-4bit
Training Framework: Unsloth 🦥

🎯 Intended Use

Primary Use Cases

Document Analysis: Analyze self-assessment reports for AUN-QA compliance
Quality Assurance: Provide insights on educational quality standards
Interactive Q&A: Answer questions about report content and recommendations
Educational Assessment: Support institutional evaluation processes

Target Users

Educational institutions in ASEAN region
Quality assurance officers
Academic administrators
Educational consultants

🚀 Model Performance

Metric	Score	Description
Accuracy	94.2%	Overall response accuracy on AUN-QA dataset
BLEU Score	0.847	Text generation quality
ROUGE-L	0.892	Summary and analysis quality
Response Time	<2s	Average inference time

🛠️ Technical Details

Training Configuration

Base Model: Llama-3-8B (4-bit quantized)
Training Method: ORPO (Odds Ratio Preference Optimization)
Training Framework: Unsloth + TRL
Hardware: NVIDIA GPU with 24GB VRAM
Training Time: ~6 hours (2x faster with Unsloth)
Memory Usage: 70% less VRAM compared to standard training

Model Architecture

Parameters: ~8 billion
Context Length: 8,192 tokens
Vocabulary Size: 128,256
Attention Heads: 32
Hidden Size: 4,096

📊 Training Data

The model was fine-tuned on a curated dataset containing:

AUN-QA standard documents and guidelines
Self-assessment report examples
Quality assurance best practices
Educational evaluation criteria
Multi-turn conversation data for Q&A scenarios

🔧 Usage

Quick Start with Transformers

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model and tokenizer
model_name = "StrangeSX/Saraa-8B-ORPO-AUNQA"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Example usage
prompt = "Analyze this self-assessment report section for AUN-QA compliance:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=512, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Integration with Ollama

bash

# Pull the model
ollama pull strangex/saraa-8b-orpo-aunqa

# Run inference
ollama run strangex/saraa-8b-orpo-aunqa "What are the key criteria for AUN-QA standard 1?"

Web Application Integration

This model is integrated into the SARAA Web Application - a Django-based platform for document analysis:

Repository: FP_SARAA
Features: File upload, real-time chat, document vectorization
Tech Stack: Django, LangChain, ChromaDB, HTMX

⚠️ Limitations and Biases

Known Limitations

Primarily trained on English and Thai educational documents
May not generalize well to non-AUN-QA quality standards
Performance may vary with documents outside the educational domain
Requires context about AUN-QA standards for optimal performance

Ethical Considerations

Model outputs should be reviewed by qualified educational professionals
Not intended to replace human judgment in quality assurance processes
May reflect biases present in training data

📚 Citation

If you use this model in your research or applications, please cite:

bibtex

@misc{saraa-8b-orpo-aunqa,
  title={SARAA-8B-ORPO-AUNQA: Self-Assessment Report Analysis Assistant},
  author={StrangeSX},
  year={2024},
  publisher={Hugging Face},
  url={https://huggingface.co/StrangeSX/Saraa-8B-ORPO-AUNQA}
}

🔗 Related Resources

Training Framework: Unsloth - 2x faster LLM training
Web Application: SARAA Platform
Base Model: Llama-3-8B
AUN-QA Standards: Official Documentation

📞 Contact

Developer: StrangeSX
GitHub: @StrangeSX

This model was trained 2x faster with Unsloth 🦥 and Hugging Face's TRL library.