LLM-JP-3.6B LoRA Adapter for Task Generation This repository contains a LoRA adapter trained on the LLM-JP-3.6B base model for generating responses to specific tasks. The adapter is optimized for Japanese language task completion.

Base Model: llm-jp/llm-jp-3-13b Architecture: LoRA adapter using QLoRA for efficient fine-tuning Quantization: 4-bit quantization (NF4) using bitsandbytes Language: Japanese License: [Base model license applies]

Here's how to use this model: from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig from peft import PeftModel import torch

bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, )

base_model = AutoModelForCausalLM.from_pretrained( "llm-jp/llm-jp-3-13b", quantization_config=bnb_config, device_map="auto", token=YOUR_HF_TOKEN )

tokenizer = AutoTokenizer.from_pretrained( "llm-jp/llm-jp-3-13b", trust_remote_code=True, token=YOUR_HF_TOKEN )

model = PeftModel.from_pretrained(base_model, ADAPTER_ID, token=YOUR_HF_TOKEN)

prompt = """ [your input here] """

input_ids = tokenizer(prompt, return_tensors="pt").to(model.device) outputs = model.generate( **input_ids, max_new_tokens=512, do_sample=False, repetition_penalty=1.2, ) response = tokenizer.decode(outputs[0][input_ids.input_ids.size(1):], skip_special_tokens=True)

The model expects input in the following format: Input = """" [task description or question] """ Generation Parameters = [max_new_tokens: int do_sample: bool repetition_penalty: float]

Default generation parameters: max_new_tokens: 512 do_sample: False repetition_penalty: 1.2

The model requires 4-bit quantization for efficient inference Responses are generated in Japanese Maximum context length is determined by the base model's limitations

This model was fine-tuned on a task-specific dataset. The training data format follows: json{ "task_id": "unique_identifier", "input": "task_description", "output": "generated_response" }

The model generates responses in JSONL format with the following structure: json{ "task_id": "unique_identifier", "input": "original_input", "output": "generated_response" }

transformers peft torch bitsandbytes tqdm

If you use this model, please cite the original LLM-JP paper and this adaptation.

This adapter inherits the license from the base LLM-JP model. Please refer to the base model's license for usage terms and conditions.