~96,000 legal documents: Official sources such as laws, decrees, circulars,...
~48,000 legal news articles: Collected from online legal news portals, featuring case studies, legal interpretations,...

Training Configuration

The model was trained using full-parameter fine-tuning (no quantization or LoRA). Below is the training setup used for continual pretraining:

🔧 Model & Tokenization

Base model: Qwen/Qwen3-4B
Maximum sequence length: 4096
Block size: 4096

All texts were collected from publicly available and legally permitted sources, then preprocessed to ensure quality and consistency for domain adaptation.

🚀 Example Usage

python

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("VLSP2025-LegalSML/qwen3-4b-legal-pretrain")
model = AutoModelForCausalLM.from_pretrained("VLSP2025-LegalSML/qwen3-4b-legal-pretrain")

🧑‍💼 Maintainers

This model is developed and maintained by the VLSP 2025 LegalSLM Task Organizers.

For inquiries, please contact: leanhcuong@tdtu.edu.vn

⚠️ License & Usage

This model is released for research purposes only under the scope of the VLSP 2025 Evaluation Campaign. Any use outside the competition must comply with relevant data and model licensing agreements.