You can deploy this model and start earning money today!

RichardErkhov/
refuelai_-_Llama-3-Refueled-gguf

Name: refuelai_-_Llama-3-Refueled-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Llama-3-Refueled - GGUF

Model creator: https://huggingface.co/refuelai/
Original model: https://huggingface.co/refuelai/Llama-3-Refueled/

Name	Quant method	Size
Llama-3-Refueled.Q2_K.gguf	Q2_K	2.96GB
Llama-3-Refueled.Q3_K_S.gguf	Q3_K_S	3.41GB
Llama-3-Refueled.Q3_K.gguf	Q3_K	3.74GB
Llama-3-Refueled.Q3_K_M.gguf	Q3_K_M	3.74GB
Llama-3-Refueled.Q3_K_L.gguf	Q3_K_L	4.03GB
Llama-3-Refueled.IQ4_XS.gguf	IQ4_XS	4.18GB
Llama-3-Refueled.Q4_0.gguf	Q4_0	4.34GB
Llama-3-Refueled.IQ4_NL.gguf	IQ4_NL	4.38GB
Llama-3-Refueled.Q4_K_S.gguf	Q4_K_S	4.37GB
Llama-3-Refueled.Q4_K.gguf	Q4_K	4.58GB
Llama-3-Refueled.Q4_K_M.gguf	Q4_K_M	4.58GB
Llama-3-Refueled.Q4_1.gguf	Q4_1	4.78GB
Llama-3-Refueled.Q5_0.gguf	Q5_0	5.21GB
Llama-3-Refueled.Q5_K_S.gguf	Q5_K_S	5.21GB
Llama-3-Refueled.Q5_K.gguf	Q5_K	5.34GB
Llama-3-Refueled.Q5_K_M.gguf	Q5_K_M	5.34GB
Llama-3-Refueled.Q5_1.gguf	Q5_1	5.65GB
Llama-3-Refueled.Q6_K.gguf	Q6_K	6.14GB
Llama-3-Refueled.Q8_0.gguf	Q8_0	7.95GB

Original model description:

license: cc-by-nc-4.0 language:

en library_name: transformers tags:
data labeling

Model Details

RefuelLLM-2-small, aka Llama-3-Refueled, is a Llama3-8B base model instruction tuned on a corpus of 2750+ datasets, spanning tasks such as classification, reading comprehension, structured attribute extraction and entity resolution. We're excited to open-source the model for the community to build on top of.

More details about RefuelLLM-2 family of models
You can also try out the models in our LLM playground

Model developers - Refuel AI

Input - Text only.

Output - Text only.

Architecture - Llama-3-Refueled is built on top of Llama-3-8B-instruct which is an auto-regressive language model that uses an optimized transformer architecture.

Release Date - May 8, 2024.

License - CC BY-NC 4.0

How to use

This repository contains weights for Llama-3-Refueled that are compatible for use with HuggingFace. See the snippet below for usage with Transformers:

python

>>> import torch
>>> from transformers import AutoModelForCausalLM, AutoTokenizer

>>> model_id = "refuelai/Llama-3-Refueled"
>>> tokenizer = AutoTokenizer.from_pretrained(model_id)
>>> model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")

>>> messages = [{"role": "user", "content": "Is this comment toxic or non-toxic: RefuelLLM is the new way to label text data!"}]

>>> inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to("cuda")

>>> outputs = model.generate(inputs, max_new_tokens=20)
>>> print(tokenizer.decode(outputs[0]))

Training Data

The model was both trained on over 4 Billion tokens, spanning 2750+ NLP tasks. Our training collection consists majorly of:

Human annotated datasets like Flan, Task Source, and the Aya collection
Synthetic datasets like OpenOrca, OpenHermes and WizardLM
Proprietary datasets developed or licensed by Refuel AI

Benchmarks

In this section, we report the results for Refuel models on our benchmark of labeling tasks. For details on the methodology see here.

Provider	Model	LLM Output Quality (by task type)
		Overall	Classification	Reading Comprehension	Structure Extraction	Entity Matching
Refuel	RefuelLLM-2	83.82%	84.94%	76.03%	88.16%	92.00%
OpenAI	GPT-4-Turbo	80.88%	81.77%	72.08%	84.79%	97.20%
Refuel	RefuelLLM-2-small (Llama-3-Refueled)	79.67%	81.72%	70.04%	84.28%	92.00%
Anthropic	Claude-3-Opus	79.19%	82.49%	67.30%	88.25%	94.96%
Meta	Llama3-70B-Instruct	78.20%	79.38%	66.03%	85.96%	94.13%
Google	Gemini-1.5-Pro	74.59%	73.52%	60.67%	84.27%	98.48%
Mistral	Mixtral-8x7B-Instruct	62.87%	79.11%	45.56%	47.08%	86.52%
Anthropic	Claude-3-Sonnet	70.99%	79.91%	45.44%	78.10%	96.34%
Anthropic	Claude-3-Haiku	69.23%	77.27%	50.19%	84.97%	54.08%
OpenAI	GPT-3.5-Turbo	68.13%	74.39%	53.21%	69.40%	80.41%
Meta	Llama3-8B-Instruct	62.30%	68.52%	49.16%	65.09%	63.61%

Limitations

The Llama-3-Refueled does not have any moderation mechanisms. We're looking forward to engaging with the community on ways to make the model finely respect guardrails, allowing for deployment in environments requiring moderated outputs.

RichardErkhov/refuelai_-_Llama-3-Refueled-gguf