You can deploy this model and start earning money today!

Ludacris919/
llm-jp-3-13b-finetune-combined-second

Name: llm-jp-3-13b-finetune-combined-second
Availability: InStock

Views

No views yet

Model Card for Model ID

Model Details

Model Description

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

Developed by: [More Information Needed]
Funded by [optional]: [More Information Needed]
Shared by [optional]: [More Information Needed]
Model type: [More Information Needed]
Language(s) (NLP): [More Information Needed]
License: [More Information Needed]
Finetuned from model [optional]: [More Information Needed]

Model Sources [optional]

Repository: [More Information Needed]
Paper [optional]: [More Information Needed]
Demo [optional]: [More Information Needed]

Uses

Direct Use

[More Information Needed]

Downstream Use [optional]

[More Information Needed]

Out-of-Scope Use

[More Information Needed]

Bias, Risks, and Limitations

[More Information Needed]

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

Training Details

Training Data

[More Information Needed]

Training Procedure

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

Training regime: [More Information Needed]

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

[More Information Needed]

Metrics

[More Information Needed]

Results

[More Information Needed]

Summary

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: [More Information Needed]
Hours used: [More Information Needed]
Cloud Provider: [More Information Needed]
Compute Region: [More Information Needed]
Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

[More Information Needed]

Sample Use

以下は以下はELYZA-tasks-100-TV_0.jsonの解答のためのコードです。

'''python from transformers import ( AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, TrainingArguments, logging, ) from peft import ( LoraConfig, PeftModel, get_peft_model, ) import os, torch, gc from datasets import load_dataset, Dataset, concatenate_datasets import json import bitsandbytes as bnb from trl import SFTTrainer

データセットの読み込みと結合(pathは各自変更)

data_files = [ "./ichikara-instruction-003-001-1.json", "./ichikara-instruction-003-001-2.1.json", "./ichikara-instruction-003-001-2.2.json", "./ichikara-instruction-003-001-5.1.json", "./ichikara-instruction-003-001-5.2.json" ]

データセットを格納するリスト

datasets = []

for file_path in data_files: try: print(f"Processing {file_path}") with open(file_path, 'r', encoding='utf-8') as f: data = json.load(f) if not isinstance(data, list): data = [data] dataset = Dataset.from_list(data) datasets.append(dataset) print(f"Successfully loaded {len(dataset)} examples from {file_path}") except Exception as e: print(f"Error processing {file_path}: {str(e)}") continue

データセットの結合

combined_dataset = concatenate_datasets(datasets) print(f"Combined dataset size: {len(combined_dataset)} examples")

モデルの設定

base_model_id = "models/models--llm-jp--llm-jp-3-13b/snapshots/cd3823f4c1fcbb0ad2e2af46036ab1b0ca13192a" new_model_id = "llm-jp-3-13b-finetune-combined-second"

量子化の設定

bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, )

モデルとトークナイザーの読み込み

model = AutoModelForCausalLM.from_pretrained( base_model_id, quantization_config=bnb_config, device_map="auto" )

tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True)

LoRAの設定

def find_all_linear_names(model): cls = bnb.nn.Linear4bit lora_module_names = set() for name, module in model.named_modules(): if isinstance(module, cls): names = name.split('.') lora_module_names.add(names[0] if len(names) == 1 else names[-1]) if 'lm_head' in lora_module_names: lora_module_names.remove('lm_head') return list(lora_module_names)

modules = find_all_linear_names(model)

peft_config = LoraConfig( r=16, lora_alpha=32, lora_dropout=0.05, bias="none", task_type="CAUSAL_LM", target_modules=modules, )

model = get_peft_model(model, peft_config)

データの前処理

prompt = """### 指示 {}

回答

{}"""

def formatting_prompts_func(examples): input = examples["text"] output = examples["output"] text = prompt.format(input, output) + tokenizer.eos_token return {"formatted_text": text}

formatted_dataset = combined_dataset.map( formatting_prompts_func, num_proc=4, )

より大きなデータセット用にトレーニング設定を調整

training_arguments = TrainingArguments( output_dir=new_model_id, per_device_train_batch_size=1, gradient_accumulation_steps=4, optim="paged_adamw_32bit", num_train_epochs=1, # エポック数を増やして学習を強化 logging_strategy="steps", logging_steps=100, warmup_steps=200, # より長いウォームアップ期間 save_steps=200, save_total_limit=3, max_steps=-1, learning_rate=1e-5, fp16=False, bf16=False, seed=3407, group_by_length=True, report_to="none" )

トレーナーの設定と学習の実行

trainer = SFTTrainer( model=model, train_dataset=formatted_dataset, peft_config=peft_config, max_seq_length=512, dataset_text_field="formatted_text", tokenizer=tokenizer, args=training_arguments, packing=False, )

model.config.use_cache = False trainer.train()

推論

from tqdm import tqdm import torch import json import re

タスクデータの読み込み

task_data = [] with open("./elyza-tasks-100-TV_0.jsonl", "r") as f: item = "" for line in f: line = line.strip() item += line if item.endswith("}"): task_data.append(json.loads(item)) item = ""

バッチ処理のための関数

def batch_generate(prompts, model, tokenizer, batch_size=4): results = []

for i in tqdm(range(0, len(prompts), batch_size)):
    batch = prompts[i:i + batch_size]
    # バッチ処理用の入力を準備
    encoded_inputs = tokenizer(
        batch,
        padding=True,
        return_tensors="pt",
        add_special_tokens=False
    )
    
    input_ids = encoded_inputs['input_ids'].to(model.device)
    attention_mask = encoded_inputs['attention_mask'].to(model.device)
    
    with torch.no_grad():
        outputs = model.generate(
            input_ids=input_ids,
            attention_mask=attention_mask,
            max_new_tokens=256,
            do_sample=False,
            num_beams=2,
            repetition_penalty=1.2,
            pad_token_id=tokenizer.eos_token_id
        )
    
    # バッチ内の各出力をデコード
    for j, output in enumerate(outputs):
        decoded = tokenizer.decode(output[input_ids[j].size(0):], 
                                 skip_special_tokens=True)
        results.append(decoded)

return results

プロンプトの準備

prompts = [f"### 指示\n{data['input']}\n### 回答\n" for data in task_data]

バッチ処理で推論を実行

outputs = batch_generate(prompts, model, tokenizer, batch_size=4)

結果の保存

results = [] for i, data in enumerate(task_data): results.append({ "task_id": data["task_id"], "input": data["input"], "output": outputs[i] })

結果をjsonlファイルとして保存

jsonl_id = re.sub(".*/", "", new_model_id) output_file = f"./{jsonl_id}-outputs.jsonl" with open(output_file, 'w', encoding='utf-8') as f: for result in results: json.dump(result, f, ensure_ascii=False) f.write('\n') print(f"Results saved to {output_file}")

Hugging Face Token

HF_TOKEN = "your_token"

モデルの保存

model.push_to_hub(new_model_id, token=HF_TOKEN, private=True) tokenizer.push_to_hub(new_model_id, token=HF_TOKEN, private=True)

Ludacris919/llm-jp-3-13b-finetune-combined-second

Model Card for Model ID

Model Details

Model Description

Model Sources [optional]

Uses

Direct Use

Downstream Use [optional]

Out-of-Scope Use

Bias, Risks, and Limitations

Recommendations

How to Get Started with the Model

Training Details

Training Data

Training Procedure

Preprocessing [optional]

Training Hyperparameters

Speeds, Sizes, Times [optional]

Evaluation

Testing Data, Factors & Metrics

Testing Data

Factors

Metrics

Results

Summary

Model Examination [optional]

Environmental Impact

Technical Specifications [optional]

Model Architecture and Objective

Compute Infrastructure

Hardware

Software

Citation [optional]

Glossary [optional]

More Information [optional]

Model Card Authors [optional]

Model Card Contact

Sample Use

データセットの読み込みと結合(pathは各自変更)

データセットを格納するリスト

データセットの結合

モデルの設定

量子化の設定

モデルとトークナイザーの読み込み

LoRAの設定

データの前処理

回答

より大きなデータセット用にトレーニング設定を調整

トレーナーの設定と学習の実行

推論

タスクデータの読み込み

バッチ処理のための関数

プロンプトの準備

バッチ処理で推論を実行

結果の保存

結果をjsonlファイルとして保存

Hugging Face Token

モデルの保存

Ludacris919/
llm-jp-3-13b-finetune-combined-second