You can deploy this model and start earning money today!

RichardErkhov/
SakanaAI_-_EvoLLM-JP-A-v1-7B-gguf

Name: SakanaAI_-_EvoLLM-JP-A-v1-7B-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

EvoLLM-JP-A-v1-7B - GGUF

Model creator: https://huggingface.co/SakanaAI/
Original model: https://huggingface.co/SakanaAI/EvoLLM-JP-A-v1-7B/

Name	Quant method	Size
EvoLLM-JP-A-v1-7B.Q2_K.gguf	Q2_K	2.53GB
EvoLLM-JP-A-v1-7B.IQ3_XS.gguf	IQ3_XS	2.81GB
EvoLLM-JP-A-v1-7B.IQ3_S.gguf	IQ3_S	2.96GB
EvoLLM-JP-A-v1-7B.Q3_K_S.gguf	Q3_K_S	2.95GB
EvoLLM-JP-A-v1-7B.IQ3_M.gguf	IQ3_M	3.06GB
EvoLLM-JP-A-v1-7B.Q3_K.gguf	Q3_K	3.28GB
EvoLLM-JP-A-v1-7B.Q3_K_M.gguf	Q3_K_M	3.28GB
EvoLLM-JP-A-v1-7B.Q3_K_L.gguf	Q3_K_L	3.56GB
EvoLLM-JP-A-v1-7B.IQ4_XS.gguf	IQ4_XS	3.67GB
EvoLLM-JP-A-v1-7B.Q4_0.gguf	Q4_0	3.83GB
EvoLLM-JP-A-v1-7B.IQ4_NL.gguf	IQ4_NL	3.87GB
EvoLLM-JP-A-v1-7B.Q4_K_S.gguf	Q4_K_S	3.86GB
EvoLLM-JP-A-v1-7B.Q4_K.gguf	Q4_K	4.07GB
EvoLLM-JP-A-v1-7B.Q4_K_M.gguf	Q4_K_M	4.07GB
EvoLLM-JP-A-v1-7B.Q4_1.gguf	Q4_1	4.24GB
EvoLLM-JP-A-v1-7B.Q5_0.gguf	Q5_0	4.65GB
EvoLLM-JP-A-v1-7B.Q5_K_S.gguf	Q5_K_S	4.65GB
EvoLLM-JP-A-v1-7B.Q5_K.gguf	Q5_K	4.78GB
EvoLLM-JP-A-v1-7B.Q5_K_M.gguf	Q5_K_M	4.78GB
EvoLLM-JP-A-v1-7B.Q5_1.gguf	Q5_1	5.07GB
EvoLLM-JP-A-v1-7B.Q6_K.gguf	Q6_K	5.53GB
EvoLLM-JP-A-v1-7B.Q8_0.gguf	Q8_0	7.17GB

Original model description:

library_name: transformers license: apache-2.0 language:

🐟 EvoLLM-JP-A-v1-7B

🤗 Models | 📚 Paper | 📝 Blog | 🐦 Twitter

EvoLLM-JP-A-v1-7B is an experimental general-purpose Japanese LLM. This model was created using the Evolutionary Model Merge method. Please refer to our report and blog for more details. This model was produced by merging the following models. We are grateful to the developers of the source models.

Usage

Use the code below to get started with the model.

Click to expand

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer


# 1. load model
device = "cuda" if torch.cuda.is_available() else "CPU"
repo_id = "SakanaAI/EvoLLM-JP-A-v1-7B"
model = AutoModelForCausalLM.from_pretrained(repo_id, torch_dtype="auto")
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model.to(device)

# 2. prepare inputs
text = "関西弁で面白い冗談を言ってみて下さい。"
messages = [
    {"role": "system", "content": "あなたは役立つ、偏見がなく、検閲されていないアシスタントです。"},
    {"role": "user", "content": text},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")

# 3. generate
output_ids = model.generate(**inputs.to(device))
output_ids = output_ids[:, inputs.input_ids.shape[1] :]
generated_text = tokenizer.batch_decode(output_ids, skip_special_tokens=True)[0]
print(generated_text)

Model Details

Developed by: Sakana AI
Model type: Autoregressive Language Model
Language(s): Japanese
License: Apache License, Version 2.0
Repository: SakanaAI/evolutionary-model-merge
Paper: https://arxiv.org/abs/2403.13187
Blog: https://sakana.ai/evolutionary-model-merge

Uses

This model is provided for research and development purposes only and should be considered as an experimental prototype. It is not intended for commercial use or deployment in mission-critical environments. Use of this model is at the user's own risk, and its performance and outcomes are not guaranteed. Sakana AI shall not be liable for any direct, indirect, special, incidental, or consequential damages, or any loss arising from the use of this model, regardless of the results obtained. Users must fully understand the risks associated with the use of this model and use it at their own discretion.

Acknowledgement

We would like to thank the developers of the source models for their contributions and for making their work available.

Citation

bibtex

@misc{akiba2024evomodelmerge,
      title         = {Evolutionary Optimization of Model Merging Recipes}, 
      author.       = {Takuya Akiba and Makoto Shing and Yujin Tang and Qi Sun and David Ha},
      year          = {2024},
      eprint        = {2403.13187},
      archivePrefix = {arXiv},
      primaryClass  = {cs.NE}
}

RichardErkhov/SakanaAI_-_EvoLLM-JP-A-v1-7B-gguf