You can deploy this model and start earning money today!

RichardErkhov/
vilm_-_vulture-40b-gguf

Name: vilm_-_vulture-40b-gguf
Availability: InStock

endpoints_compatible

gguf

template

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

vulture-40b - GGUF

Model creator: https://huggingface.co/vilm/
Original model: https://huggingface.co/vilm/vulture-40b/

Name	Quant method	Size
vulture-40b.Q2_K.gguf	Q2_K	14.52GB
vulture-40b.IQ3_XS.gguf	IQ3_XS	16.32GB
vulture-40b.IQ3_S.gguf	IQ3_S	16.85GB
vulture-40b.Q3_K_S.gguf	Q3_K_S	16.85GB
vulture-40b.IQ3_M.gguf	IQ3_M	17.64GB
vulture-40b.Q3_K.gguf	Q3_K	18.5GB
vulture-40b.Q3_K_M.gguf	Q3_K_M	18.5GB
vulture-40b.Q3_K_L.gguf	Q3_K_L	19.9GB
vulture-40b.IQ4_XS.gguf	IQ4_XS	20.98GB
vulture-40b.Q4_0.gguf	Q4_0	21.89GB
vulture-40b.IQ4_NL.gguf	IQ4_NL	22.11GB
vulture-40b.Q4_K_S.gguf	Q4_K_S	21.89GB
vulture-40b.Q4_K.gguf	Q4_K	23.46GB
vulture-40b.Q4_K_M.gguf	Q4_K_M	23.46GB
vulture-40b.Q4_1.gguf	Q4_1	24.27GB
vulture-40b.Q5_0.gguf	Q5_0	26.64GB
vulture-40b.Q5_K_S.gguf	Q5_K_S	26.64GB
vulture-40b.Q5_K.gguf	Q5_K	28.2GB
vulture-40b.Q5_K_M.gguf	Q5_K_M	28.2GB
vulture-40b.Q5_1.gguf	Q5_1	29.01GB
vulture-40b.Q6_K.gguf	Q6_K	31.68GB
vulture-40b.Q8_0.gguf	Q8_0	40.88GB

Original model description:

license: apache-2.0 language:

ru
en
de
es
it
ja
vi
zh
fr
pt
id
ko pipeline_tag: text-generation

🌍 Vulture-40B

Vulture-40B is a further fine-tuned causal Decoder-only LLM built by Virtual Interactive (VILM), on top of the famous Falcon-40B by TII. We collected a new dataset from news articles and Wikipedia's pages of 12 languages (Total: 80GB) and continue the pretraining process of Falcon-40B. Finally, we construct a multilingual instructional dataset following Alpaca's techniques.

Technical Report coming soon 🤗

Prompt Format

The reccomended model usage is:

A chat between a curious user and an artificial intelligence assistant.

USER:{user's question}<|endoftext|>ASSISTANT:

Model Details

Model Description

Developed by: https://www.tii.ae
Finetuned by: Virtual Interactive
Language(s) (NLP): English, German, Spanish, French, Portugese, Russian, Italian, Vietnamese, Indonesian, Chinese, Japanese and Korean
Training Time: 1,800 A100 Hours

Acknowledgement

Thanks to TII for the amazing Falcon as the foundation model.
Big thanks to Google for their generous Cloud credits.

Out-of-Scope Use

Production use without adequate assessment of risks and mitigation; any use cases which may be considered irresponsible or harmful.

Bias, Risks, and Limitations

Vulture-40B is trained on a large-scale corpora representative of the web, it will carry the stereotypes and biases commonly encountered online.

Recommendations

We recommend users of Vulture-40B to consider finetuning it for the specific set of tasks of interest, and for guardrails and appropriate precautions to be taken for any production use.

How to Get Started with the Model

To run inference with the model in full bfloat16 precision you need approximately 4xA100 80GB or equivalent.

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model = "vilm/vulture-40B"

tokenizer = AutoTokenizer.from_pretrained(model)
m = AutoModelForCausalLM.from_pretrained(model, torch_dtype=torch.bfloat16, device_map="auto" )

prompt = "A chat between a curious user and an artificial intelligence assistant.\n\nUSER:Thành phố Hồ Chí Minh nằm ở đâu?<|endoftext|>ASSISTANT:"

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

output = m.generate(input_ids=inputs["input_ids"],
                    attention_mask=inputs["attention_mask"],
                    do_sample=True,
                    temperature=0.6,
                    top_p=0.9,
                    max_new_tokens=50,)
output = output[0].to("cpu")
print(tokenizer.decode(output))

Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more quants, at much higher speed, than I would otherwise be able to.

RichardErkhov/vilm_-_vulture-40b-gguf