You can deploy this model and start earning money today!

RichardErkhov/
kaist-ai_-_janus-dpo-7b-gguf

Name: kaist-ai_-_janus-dpo-7b-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

janus-dpo-7b - GGUF

Model creator: https://huggingface.co/kaist-ai/
Original model: https://huggingface.co/kaist-ai/janus-dpo-7b/

Name	Quant method	Size
janus-dpo-7b.Q2_K.gguf	Q2_K	2.53GB
janus-dpo-7b.IQ3_XS.gguf	IQ3_XS	2.81GB
janus-dpo-7b.IQ3_S.gguf	IQ3_S	2.96GB
janus-dpo-7b.Q3_K_S.gguf	Q3_K_S	1.56GB
janus-dpo-7b.IQ3_M.gguf	IQ3_M	3.06GB
janus-dpo-7b.Q3_K.gguf	Q3_K	3.28GB
janus-dpo-7b.Q3_K_M.gguf	Q3_K_M	3.28GB
janus-dpo-7b.Q3_K_L.gguf	Q3_K_L	3.56GB
janus-dpo-7b.IQ4_XS.gguf	IQ4_XS	3.67GB
janus-dpo-7b.Q4_0.gguf	Q4_0	3.83GB
janus-dpo-7b.IQ4_NL.gguf	IQ4_NL	3.87GB
janus-dpo-7b.Q4_K_S.gguf	Q4_K_S	3.86GB
janus-dpo-7b.Q4_K.gguf	Q4_K	4.07GB
janus-dpo-7b.Q4_K_M.gguf	Q4_K_M	4.07GB
janus-dpo-7b.Q4_1.gguf	Q4_1	4.24GB
janus-dpo-7b.Q5_0.gguf	Q5_0	4.65GB
janus-dpo-7b.Q5_K_S.gguf	Q5_K_S	4.65GB
janus-dpo-7b.Q5_K.gguf	Q5_K	4.78GB
janus-dpo-7b.Q5_K_M.gguf	Q5_K_M	4.78GB
janus-dpo-7b.Q5_1.gguf	Q5_1	5.07GB
janus-dpo-7b.Q6_K.gguf	Q6_K	5.53GB
janus-dpo-7b.Q8_0.gguf	Q8_0	7.17GB

Original model description:

license: apache-2.0 base_model: kaist-ai/mpa-Mistral-7b-v0.2-hf-sft-66k tags:

axolotl
trl
generated_from_trainer
dpo model-index:
name: janus-dpo-7b results: [] datasets:
kaist-ai/Multifaceted-Collection-DPO language:
en library_name: transformers pipeline_tag: text-generation

Links for Reference

Homepage: https://lklab.kaist.ac.kr/Janus/
Repository: https://github.com/kaistAI/Janus
Paper: https://arxiv.org/abs/2405.17977
Point of Contact: seongyun@kaist.ac.kr

TL; DR

Janus is a model trained using Mistral-7B-v0.2 as its base model. Janus has been trained on Multifaceted Collection, a preference dataset containing 196k unique system messages for aligning LLMs to diverse human preferences. Janus not only excels at generating personalized responses that cater to various human preferences but is also adept at producing responses that are generally preferred for being helpful and harmless.

Model Details

Janus-DPO-7B is a model created by applying DPO to Janus using the Multifaceted-Collection-DPO.

Model Description

Model type: Language model
Language(s) (NLP): English
License: Apache 2.0
Related Models: Janus-7B, Janus-ORPO-7B, Janus-RM-7B
Training Datasets: Multifaceted-Collection-DPO
Resources for more information:
- Research paper
- GitHub Repo

Usage

Janus is a model generalized for various system messages, allowing users to control the model's response by inputting the desired system message. The input prompt format is as follows:

[INST]{system_message}\n{instruction}[/INST]

Additionally, an example of the inference code applying this is as follows:

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_name = "kaist-ai/janus-dpo-7b"
device = "cuda:0"

# Load the model and tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_name)

dtype = "float16"
if torch.cuda.is_bf16_supported():
    dtype = "bfloat16"
    
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=getattr(torch, dtype))
model.eval()
model.to(device)

# Prepare inputs
system = "As a financial news headline writer with a flair for the dramatic, you have taken on the role of crafting compelling headlines about the integration of AI into the financial sector. Your expertise allows you to weave industry-specific terminology seamlessly into each headline, striking a balance between capturing attention and providing meaningful insights into the transformative benefits of AI in finance. With each headline, you focus on elucidating the key advantages AI brings to financial operations, making complex information accessible and immediately impactful. While your headlines are designed to engage and inform an audience of finance and technology professionals, you navigate the fine line of excitement and accuracy with care, ensuring that the promises made are grounded in reality, thus avoiding any form of sensationalism. Your mission is to distill the essence of AI's impact on finance into a single, powerful line that speaks volumes to the informed reader."
prompt = "Write a headline for an article about the benefits of using AI in the finance sector."

def apply_template_mistral_instruct(system_message, content):
    prompt = f"{system_message}\n{content}".strip()
    return f"[INST] {prompt} [/INST] "

input_str = apply_template_mistral_instruct(system, prompt)
input_ids = tokenizer.encode(input_str, return_tensors="pt")
print(input_str)

model_inputs = input_ids.to(device)

# Generate text
output_ids = model.generate(model_inputs, max_new_tokens=1024)
decoded = tokenizer.batch_decode(output_ids, skip_special_tokens=True)
print(decoded[0][len(input_str):])
# Revolutionary Trends: How AI Is Redefining Efficiency and Accuracy in the Financial Realm

To train Janus and evaluate the responses it generates, please refer to the GitHub Repo. Additionally, refer to the Multifaceted Bench, which evaluates how well LLM generates personalized responses.

Training Details

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-07
train_batch_size: 1
eval_batch_size: 8
seed: 42
distributed_type: multi-GPU
num_devices: 4
gradient_accumulation_steps: 4
total_train_batch_size: 16
total_eval_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 10
training_steps: 8143

Framework versions

Transformers 4.40.0.dev0
Pytorch 2.1.1
Datasets 2.15.0
Tokenizers 0.15.0

Citation

If you find the following model helpful, please consider citing our paper!

BibTeX:

bibtex

@article{lee2024aligning,
  title={Aligning to Thousands of Preferences via System Message Generalization},
  author={Lee, Seongyun and Park, Sue Hyun and Kim, Seungone and Seo, Minjoon},
  journal={arXiv preprint arXiv:2405.17977},
  year={2024}
}

RichardErkhov/kaist-ai_-_janus-dpo-7b-gguf