You can deploy this model and start earning money today!

RichardErkhov/
MoxoffSpA_-_Moxoff-Phi3Mini-PPO-gguf

Name: MoxoffSpA_-_Moxoff-Phi3Mini-PPO-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Moxoff-Phi3Mini-PPO - GGUF

Model creator: https://huggingface.co/MoxoffSpA/
Original model: https://huggingface.co/MoxoffSpA/Moxoff-Phi3Mini-PPO/

Name	Quant method	Size
Moxoff-Phi3Mini-PPO.Q2_K.gguf	Q2_K	1.32GB
Moxoff-Phi3Mini-PPO.IQ3_XS.gguf	IQ3_XS	1.51GB
Moxoff-Phi3Mini-PPO.IQ3_S.gguf	IQ3_S	1.57GB
Moxoff-Phi3Mini-PPO.Q3_K_S.gguf	Q3_K_S	1.57GB
Moxoff-Phi3Mini-PPO.IQ3_M.gguf	IQ3_M	1.73GB
Moxoff-Phi3Mini-PPO.Q3_K.gguf	Q3_K	1.82GB
Moxoff-Phi3Mini-PPO.Q3_K_M.gguf	Q3_K_M	1.82GB
Moxoff-Phi3Mini-PPO.Q3_K_L.gguf	Q3_K_L	1.94GB
Moxoff-Phi3Mini-PPO.IQ4_XS.gguf	IQ4_XS	1.93GB
Moxoff-Phi3Mini-PPO.Q4_0.gguf	Q4_0	2.03GB
Moxoff-Phi3Mini-PPO.IQ4_NL.gguf	IQ4_NL	2.04GB
Moxoff-Phi3Mini-PPO.Q4_K_S.gguf	Q4_K_S	2.04GB
Moxoff-Phi3Mini-PPO.Q4_K.gguf	Q4_K	2.23GB
Moxoff-Phi3Mini-PPO.Q4_K_M.gguf	Q4_K_M	2.23GB
Moxoff-Phi3Mini-PPO.Q4_1.gguf	Q4_1	2.24GB
Moxoff-Phi3Mini-PPO.Q5_0.gguf	Q5_0	2.46GB
Moxoff-Phi3Mini-PPO.Q5_K_S.gguf	Q5_K_S	2.46GB
Moxoff-Phi3Mini-PPO.Q5_K.gguf	Q5_K	2.62GB
Moxoff-Phi3Mini-PPO.Q5_K_M.gguf	Q5_K_M	2.62GB
Moxoff-Phi3Mini-PPO.Q5_1.gguf	Q5_1	2.68GB
Moxoff-Phi3Mini-PPO.Q6_K.gguf	Q6_K	2.92GB
Moxoff-Phi3Mini-PPO.Q8_0.gguf	Q8_0	3.78GB

Original model description:

license: mit language:

en library_name: transformers tags:
ppo
phi3
chatml datasets: argilla/ultrafeedback-binarized-preferences-cleaned metrics:
hellaswag
arc_challenge
m_mmlu 5 shot

Model Information

Moxoff-Phi3Mini-PPO is an updated version of Phi-3-mini-128k-instruct, aligned with PPO.

It's trained on ultrafeedback-binarized-preferences-cleaned.

Evaluation

We evaluated the model using the same test sets as used for the Open LLM Leaderboard

hellaswag acc_norm	arc_challenge acc_norm	m_mmlu 5-shot acc	Average
0.7044	0.4701	0.5814	0.5833

Usage

Be sure to install these dependencies before running the program

!pip install transformers torch sentencepiece

python

from transformers import AutoModelForCausalLM, AutoTokenizer

device = "cpu" # if you want to use the gpu make sure to have cuda toolkit installed and change this to "cuda"

model = AutoModelForCausalLM.from_pretrained("MoxoffSpA/Moxoff-Phi3Mini-PPO")
tokenizer = AutoTokenizer.from_pretrained("MoxoffSpA/Moxoff-Phi3Mini-PPO")

question = """Quanto è alta la torre di Pisa?"""
context = """
La Torre di Pisa è un campanile del XII secolo, famoso per la sua inclinazione. Alta circa 56 metri.
"""

prompt = f"Domanda: {question}, contesto: {context}"

messages = [
    {"role": "user", "content": prompt}
]

encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")

model_inputs = encodeds.to(device)
model.to(device)

generated_ids = model.generate(
    model_inputs, # The input to the model
    max_new_tokens=128, # Limiting the maximum number of new tokens generated
    do_sample=True, # Enabling sampling to introduce randomness in the generation
    temperature=0.1, # Setting temperature to control the randomness, lower values make it more deterministic
    top_p=0.95, # Using nucleus sampling with top-p filtering for more coherent generation       
    eos_token_id=tokenizer.eos_token_id # Specifying the token that indicates the end of a sequence
)

decoded_output = tokenizer.decode(generated_ids[0], skip_special_tokens=True)
trimmed_output = decoded_output.strip()
print(trimmed_output)

Bias, Risks and Limitations

Moxoff-Phi3Mini-PPO has not been aligned to human preferences for safety within the RLHF phase or deployed with in-the-loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). It is also unknown what the size and composition of the corpus was used to train the base model, however it is likely to have included a mix of Web data and technical sources like books and code.

Links to resources

ultrafeedback-binarized-preferences-cleaned dataset: https://huggingface.co/datasets/argilla/ultrafeedback-binarized-preferences-cleaned
Phi-3-mini-128k-instruct model: https://huggingface.co/microsoft/Phi-3-mini-128k-instruct
Open LLM Leaderbord: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard

The Moxoff Team

Jacopo Abate, Marco D'Ambra, Dario Domanin, Luigi Simeone, Gianpaolo Francesco Trotta

RichardErkhov/MoxoffSpA_-_Moxoff-Phi3Mini-PPO-gguf