You can deploy this model and start earning money today!

sugiyama34/
llm-jp-3-13b-finetune

Name: llm-jp-3-13b-finetune
Availability: InStock

Views

No views yet

Model Card for llm-jp-3-13b-finetune

This model is a fine-tuned version of llm-jp/llm-jp-3-13b, using the Hugging Face Transformers library, created for the submission of the final exam in "大規模言語モデル Deep Learning 応用講座 2024 Fall".

Model Details

This is quantized and fine-tuned model from llm-jp/llm-jp-3-13b.

What have been done:

4b quantization
LoRA fine-tuning with "Ichikara Instruction" dataset

Model Description

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

Developed by: @sugiyama34
Funded by [optional]: -
Shared by [optional]: -
Model type: AutoModelForCausalLM
Language(s) (NLP): Japanese
License: CC-BY-NC-SA, Apache License, Version 2.0
Finetuned from model [optional]: llm-jp/llm-jp-3-13b

instruction dataset "Ichikara Instruction" is given by: 関根聡, 安藤まや, 後藤美知子, 鈴木久美, 河原大輔, 井之上直也, 乾健太郎. ichikara-instruction: LLMのための日本語インストラクションデータの構築. 言語処理学会第30回年次大会(2024)

Model Sources [optional]

Repository: -
Paper [optional]: -
Demo [optional]: -

Uses

Only for submission of final exam.

Direct Use

Only for submission of final exam.

Downstream Use [optional]

Only for submission of final exam.

Out-of-Scope Use

Anything other than my submission of final exam.

Bias, Risks, and Limitations

I'm not sure right now, I need to study hard.

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

松尾研 LLM 講座 2024 Fall の最終課題の評価を動かすためのコードは以下です。

In the Google Colab. env. with GPU, use the code below to get started with the model.

When running the following code,

replace "YOUR TOKEN" below with your hugging face token,
replace PATH TO YOUR DATA below with the path to the evaluation data

After running, we can get jsonl file in the "files" in the Colab.

python

# install necessary libraries
!pip install -U pip
!pip install -U transformers
!pip install -U bitsandbytes

# import
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
)
import os, torch, gc, re
import bitsandbytes as bnb
import json
from tqdm import tqdm

# setting model id
base_model_id = "llm-jp/llm-jp-3-13b"
new_model_id = "llm-jp-3-13b-finetune" # evaluation に利用するモデルの名称

# YOUR HUGGING FACE TOKEN HERE
HF_TOKEN = "YOUR TOKEN"

# BitsAndBytesConfig
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

# load model
model_name = 'sugiyama34/llm-jp-3-13b-finetune'
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    quantization_config=bnb_config,
    device_map='auto'
)

# load data
datasets = []

elyza_tasks_100_tv_0_path = "PATH TO YOUR DATA"

with open(elyza_tasks_100_tv_0_path, "r") as f:
    item = ""
    for line in f:
      line = line.strip()
      item += line
      if item.endswith("}"):
        datasets.append(json.loads(item))
        item = ""

# inference
results = []
for data in tqdm(datasets):

  input = data["input"]

  prompt = f"""### 指示
  {input}
  ### 回答
  """

  tokenized_input = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt").to(model.device)
  attention_mask = torch.ones_like(tokenized_input)

  with torch.no_grad():
      outputs = model.generate(
          tokenized_input,
          attention_mask=attention_mask,
          max_new_tokens=100,
          do_sample=False,
          repetition_penalty=1.2,
          pad_token_id=tokenizer.eos_token_id
      )[0]
  output = tokenizer.decode(outputs[tokenized_input.size(1):], skip_special_tokens=True)

  results.append({"task_id": data["task_id"], "input": input, "output": output})

# export output data
jsonl_id = re.sub(".*/", "", new_model_id)
with open(f"./{jsonl_id}-outputs.jsonl", 'w', encoding='utf-8') as f:
    for result in results:
        json.dump(result, f, ensure_ascii=False)  # ensure_ascii=False for handling non-ASCII characters
        f.write('\n')

Training Details

Training Data

ichikara-instruction: LLMのための日本語インストラクションデータ – LIAT-AIP homepage instruction dataset "Ichikara Instruction" is given by: 関根聡, 安藤まや, 後藤美知子, 鈴木久美, 河原大輔, 井之上直也, 乾健太郎. ichikara-instruction: LLMのための日本語インストラクションデータの構築. 言語処理学会第30回年次大会(2024)

Training Procedure

I'm not sure right now, I need to study hard.

Preprocessing [optional]

The data of ichikara instruction dataset is transformed into the following format:

python

prompt = """### 指示
{input}
### 回答
{output}"""

And then feeded into LoRA training.

Training Hyperparameters

I'm not sure right now, I need to study hard.

Training regime: [More Information Needed]

Speeds, Sizes, Times [optional]

I'm not sure right now, I need to study hard.

Evaluation

I'm not sure right now, I need to study hard.

Testing Data, Factors & Metrics

Testing Data

I'm not sure right now, I need to study hard.

Factors

I'm not sure right now, I need to study hard.

Metrics

I'm not sure right now, I need to study hard.

Results

I'm not sure right now, I need to study hard.

Summary

I'm not sure right now, I need to study hard.

Model Examination [optional]

I'm not sure right now, I need to study hard.

Environmental Impact

I'm not sure right now, I need to study hard.

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: [More Information Needed]
Hours used: [More Information Needed]
Cloud Provider: [More Information Needed]
Compute Region: [More Information Needed]
Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

I'm not sure right now, I need to study hard.

Compute Infrastructure

I'm not sure right now, I need to study hard.

Hardware

I'm not sure right now, I need to study hard.

Software

I'm not sure right now, I need to study hard.

Citation [optional]

No paper for this work.

Glossary [optional]

not yet written

More Information [optional]

not yet written

Model Card Authors [optional]

Satoshi Sugiyama

Model Card Contact

not given here