You can deploy this model and start earning money today!

RichardErkhov/
Vezora_-_Agent-7b-v1-128k-gguf

Name: Vezora_-_Agent-7b-v1-128k-gguf
Availability: InStock

endpoints_compatible

gguf

template

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Agent-7b-v1-128k - GGUF

Model creator: https://huggingface.co/Vezora/
Original model: https://huggingface.co/Vezora/Agent-7b-v1-128k/

Name	Quant method	Size
Agent-7b-v1-128k.Q2_K.gguf	Q2_K	2.53GB
Agent-7b-v1-128k.IQ3_XS.gguf	IQ3_XS	2.81GB
Agent-7b-v1-128k.IQ3_S.gguf	IQ3_S	2.96GB
Agent-7b-v1-128k.Q3_K_S.gguf	Q3_K_S	2.95GB
Agent-7b-v1-128k.IQ3_M.gguf	IQ3_M	3.06GB
Agent-7b-v1-128k.Q3_K.gguf	Q3_K	3.28GB
Agent-7b-v1-128k.Q3_K_M.gguf	Q3_K_M	3.28GB
Agent-7b-v1-128k.Q3_K_L.gguf	Q3_K_L	3.56GB
Agent-7b-v1-128k.IQ4_XS.gguf	IQ4_XS	3.67GB
Agent-7b-v1-128k.Q4_0.gguf	Q4_0	3.83GB
Agent-7b-v1-128k.IQ4_NL.gguf	IQ4_NL	3.87GB
Agent-7b-v1-128k.Q4_K_S.gguf	Q4_K_S	3.86GB
Agent-7b-v1-128k.Q4_K.gguf	Q4_K	4.07GB
Agent-7b-v1-128k.Q4_K_M.gguf	Q4_K_M	4.07GB
Agent-7b-v1-128k.Q4_1.gguf	Q4_1	4.24GB
Agent-7b-v1-128k.Q5_0.gguf	Q5_0	4.65GB
Agent-7b-v1-128k.Q5_K_S.gguf	Q5_K_S	4.65GB
Agent-7b-v1-128k.Q5_K.gguf	Q5_K	4.78GB
Agent-7b-v1-128k.Q5_K_M.gguf	Q5_K_M	4.78GB
Agent-7b-v1-128k.Q5_1.gguf	Q5_1	5.07GB
Agent-7b-v1-128k.Q6_K.gguf	Q6_K	5.53GB
Agent-7b-v1-128k.Q8_0.gguf	Q8_0	7.17GB

Original model description:

license: apache-2.0 datasets:

m-a-p/Code-Feedback

My Kofi

Model Overview

The base model used for training is CallComply/openchat-3.5-0106-128k, which features a context length of 128k. This model was trained on 31,000 examples from the m-a-p/Code-Feedback dataset. This dataset aids the model in interactive code performance, enabling it to self-improve with interpreter and human feedback. It is ideal for applications like TaskWeaver, which helps automatically build code, or OpenInterpreter, which assists you in writing code or serves as a general agent.

The reason for choosing this base model is its long context length and strong performance in every category, specifically coding. This model was trained to output up to 8192 tokens, and still inherits its 128k context window from its base model. Making this is the best open source generalized agent model.

Additional Information

During training, we filtered examples that were over 4096 tokens and under 8192 tokens. The end result was 31k examples. The model performed exceptionally well during testing, prompting us to release it, although we are planning on benchmarking this model.

The model is modified and trained back to the standard Mistral EOS token </s>.

Training consisted of:

Epochs = 3
learning_rate = .0002
per_device_train_batch_size = 4
gradient_accumulation_steps = 8
warmup_steps = 10

This model is trained using the Unsloth qlora. Big thank you to Unsloth for releasing their free tier training library, allowing me to train 30k (total 90k with 3 epochs) examples with 8192 tokens of output on a single 3090. Completing training in just 37 hours.

Thank you to the creators of the open feedback dataset, which they decided to open source. Thank you <3.

Benchmarks coming soon, but real world use case for this model is terrific!

Prompt Template

###Human: Write a python script....
###Assistant: python.....

EOS = </s>.

Trained by Nicolas Mejia Petit Twitter: https://twitter.com/mejia_petit

RichardErkhov/Vezora_-_Agent-7b-v1-128k-gguf

Original model description:

Model Overview

Additional Information

Prompt Template

RichardErkhov/
Vezora_-_Agent-7b-v1-128k-gguf