This model is a merged Hugging Face-format checkpoint built from a LoRA fine-tune workflow. The goal is to improve answer quality for Spanish legal Q&A about the Costa Rican Codigo de Trabajo.

Developed by: josoroma
Funded by [optional]: Self-funded
Shared by [optional]: josoroma
Model type: Causal language model (instruction-tuned, merged from LoRA)
Language(s) (NLP): Spanish (es)
License: Gemma license terms apply (base-model license)
Finetuned from model [optional]: unsloth/gemma-4-31B-it

Model Sources [optional]

Repository: https://huggingface.co/josoroma/gemma-4-codigo-trabajo-finetune
Codebase: https://github.com/josoroma/codigo-trabajo-dagster-pipeline
Codebase description: Dagster pipeline and supporting workflow used to fetch/chunk source law text, generate grounded synthetic entries, validate quote-level traceability, and export training-ready datasets.

Uses

Direct Use

Spanish legal question answering and explanation tasks related to Costa Rican labor law.
Educational and drafting support where human legal review is always required.

Downstream Use [optional]

Legal assistant prototypes.
Retrieval-augmented systems that cite official legal text.
Dataset generation pipelines for legal instruction tuning.

Out-of-Scope Use

Legal advice without professional review.
High-stakes or fully autonomous legal decisions.
Use outside the legal or jurisdictional scope represented in training data.

Bias, Risks, and Limitations

Outputs may be incomplete, outdated, or jurisdiction-specific.
The model can hallucinate legal citations or procedural details.
Training focus on one legal domain can reduce generalization to unrelated tasks.

Recommendations

Require expert human validation for all legal conclusions.
Ask for article references and verify them against official sources.
Use conservative generation settings for legal tasks.

How to Get Started with the Model

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "josoroma/gemma-4-codigo-trabajo-finetune"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
        model_id,
        torch_dtype=torch.bfloat16,
        device_map="auto",
)

prompt = "Explica en que consiste el articulo 1 del Codigo de Trabajo de Costa Rica."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.inference_mode():
        outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.2, top_p=0.95)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

Primary dataset: Costa Rican labor-law instruction data (codigo_trabajo.jsonl).
Core fields: instruction, input, output.
Traceability fields included in data assets: source_quote, source_url, article, chunk_id.

Training Procedure

Preprocessing [optional]

JSONL rows mapped into chat-style conversation pairs.
Prompts rendered with Gemma chat template before SFT.

Training Hyperparameters

Training regime: 4-bit base loading + LoRA fine-tuning + merged checkpoint export
Per-device batch size: 1
Gradient accumulation steps: 4
Max steps: 60
Learning rate: 2e-4
Optimizer: adamw_8bit

Speeds, Sizes, Times [optional]

Merged checkpoint generation completed on RunPod A100 80GB.
Upload included large LFS artifacts (multi-GB safetensors shards).

Evaluation

Testing Data, Factors & Metrics

Testing Data

Manual prompt checks against known labor-law articles.

Factors

Spanish legal phrasing.
Article-specific explanation quality.
Citation/grounding behavior.

Metrics

Qualitative review only in this release.

Results

Early qualitative results indicate improved domain relevance for Costa Rican labor-law prompts compared with base behavior.

Summary

This release is suitable for experimentation and assistant-style workflows, with mandatory expert review for any real legal use.

Model Examination [optional]

Formal interpretability analysis was not performed for this version.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator: https://mlco2.github.io/impact#compute

Hardware Type: NVIDIA A100 80GB
Hours used: More Information Needed
Cloud Provider: RunPod
Compute Region: More Information Needed
Carbon Emitted: More Information Needed

Technical Specifications [optional]

Model Architecture and Objective

Base architecture: Gemma 4 31B instruct variant.
Objective: Instruction-following in Spanish legal domain with merged LoRA weights.

Compute Infrastructure

Training and merge workflow executed across Kaggle (training workflow development) and RunPod (merge/export workflow).

Hardware

Kaggle dual T4 for notebook-stage workflow.
RunPod single A100 80GB for merge.

Software

Python, Transformers, TRL, Unsloth, bitsandbytes, safetensors.

Citation [optional]

BibTeX:

bibtex

@misc{josoroma_gemma4_codigo_trabajo_finetune,
    title={gemma-4-codigo-trabajo-finetune},
    author={josoroma},
    year={2026},
    howpublished={\url{https://huggingface.co/josoroma/gemma-4-codigo-trabajo-finetune}}
}

APA:

josoroma. (2026). gemma-4-codigo-trabajo-finetune. Hugging Face. https://huggingface.co/josoroma/gemma-4-codigo-trabajo-finetune

Glossary [optional]

LoRA: Low-Rank Adaptation for parameter-efficient fine-tuning.
SFT: Supervised fine-tuning.
LFS: Git Large File Storage for large model artifacts.

More Information [optional]

This card will be updated with quantitative evaluation and additional governance details in future revisions.

Model Card Authors [optional]

josoroma

Model Card Contact

Hugging Face: https://huggingface.co/josoroma