from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model and tokenizer
repo = "hongjoong-entec/gemma-3-4b-tax-law-lora"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token
model = AutoModelForCausalLM.from_pretrained(
    repo, 
    torch_dtype=torch.float16, 
    trust_remote_code=True
)

# Prompt template for best results
prompt_style = """Below is an instruction that describes a task, paired with an input that provides further context.
Write a response that appropriately completes the request.
Respond in Korean and provide only the final answer without chain-of-thought, analysis, section headers, or disclaimers.
If needed, include a brief basis in one concise sentence.

### Question:
{}

### Response:
{}"""

# Example usage
question = "접대비 한도는 어떻게 계산하나요?"
input_text = prompt_style.format(question, "")
inputs = tokenizer([input_text], return_tensors="pt")

# Filter unwanted outputs
bad_words = ["Disclaimer:", "Chain of Thought:", "### Chain of Thought:", "### Reasoning:", "### Analysis:"]
bad_words_ids = tokenizer(bad_words, add_special_tokens=False).input_ids

# Generate response
outputs = model.generate(
    input_ids=inputs.input_ids,
    attention_mask=inputs.attention_mask,
    max_new_tokens=1024,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id,
    bad_words_ids=bad_words_ids,
    temperature=0.01,
    use_cache=True,
)

# Decode only generated part
gen_only = outputs[0, inputs.input_ids.shape[-1]:]
answer = tokenizer.decode(gen_only, skip_special_tokens=True).strip()
print(answer)

Training Details

Training Data

Internal processed Korean tax-law QA subset
Dataset format: legalqa_fino1_format.jsonl
Focus: Korean tax regulations and Q/A pairs

Training Procedure (LoRA)

Base model: google/gemma-3-4b-it
Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Hyperparameters:
- r=64
- alpha=16
- dropout=0.05
- Learning rate: 2e-4
- Batch size: 1 (with gradient accumulation)

Evaluation

Qualitative evaluation on Korean tax-law prompts
ROUGE scoring against reference answers
Manual review for accuracy and relevance

Limitations

⚠️ Important Disclaimers:

This model is for informational purposes only and does not constitute legal advice
Always consult with qualified tax professionals for official guidance
Outputs may be outdated or incomplete and require human validation
Tax laws change frequently - verify current regulations

Model Performance

The model demonstrates good performance on Korean tax-related queries but should be used as a starting point for research rather than definitive answers.

itcen-entec/gemma-3-4b-tax-law-lora

Gemma 3 4B Tax Law LoRA (Korean)

Model Details

Uses

Direct Use

Out-of-Scope Use

How to Get Started with the Model

Training Details

Training Data

Training Procedure (LoRA)

Evaluation

Limitations

Model Performance

itcen-entec/
gemma-3-4b-tax-law-lora