You can deploy this model and start earning money today!

RichardErkhov/
maywell_-_Synatra-kiqu-10.7B-gguf

Name: maywell_-_Synatra-kiqu-10.7B-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Synatra-kiqu-10.7B - GGUF

Model creator: https://huggingface.co/maywell/
Original model: https://huggingface.co/maywell/Synatra-kiqu-10.7B/

Name	Quant method	Size
Synatra-kiqu-10.7B.Q2_K.gguf	Q2_K	3.73GB
Synatra-kiqu-10.7B.IQ3_XS.gguf	IQ3_XS	4.14GB
Synatra-kiqu-10.7B.IQ3_S.gguf	IQ3_S	4.37GB
Synatra-kiqu-10.7B.Q3_K_S.gguf	Q3_K_S	4.34GB
Synatra-kiqu-10.7B.IQ3_M.gguf	IQ3_M	4.51GB
Synatra-kiqu-10.7B.Q3_K.gguf	Q3_K	4.84GB
Synatra-kiqu-10.7B.Q3_K_M.gguf	Q3_K_M	4.84GB
Synatra-kiqu-10.7B.Q3_K_L.gguf	Q3_K_L	5.26GB
Synatra-kiqu-10.7B.IQ4_XS.gguf	IQ4_XS	5.43GB
Synatra-kiqu-10.7B.Q4_0.gguf	Q4_0	5.66GB
Synatra-kiqu-10.7B.IQ4_NL.gguf	IQ4_NL	5.72GB
Synatra-kiqu-10.7B.Q4_K_S.gguf	Q4_K_S	5.7GB
Synatra-kiqu-10.7B.Q4_K.gguf	Q4_K	6.02GB
Synatra-kiqu-10.7B.Q4_K_M.gguf	Q4_K_M	6.02GB
Synatra-kiqu-10.7B.Q4_1.gguf	Q4_1	6.27GB
Synatra-kiqu-10.7B.Q5_0.gguf	Q5_0	6.89GB
Synatra-kiqu-10.7B.Q5_K_S.gguf	Q5_K_S	6.89GB
Synatra-kiqu-10.7B.Q5_K.gguf	Q5_K	7.08GB
Synatra-kiqu-10.7B.Q5_K_M.gguf	Q5_K_M	7.08GB
Synatra-kiqu-10.7B.Q5_1.gguf	Q5_1	7.51GB
Synatra-kiqu-10.7B.Q6_K.gguf	Q6_K	8.2GB
Synatra-kiqu-10.7B.Q8_0.gguf	Q8_0	10.62GB

Original model description:

license: cc-by-sa-4.0

Join our discord

Server Link

License

cc-by-sa-4.0

Model Details

Base Model
maywell/Synatra-10.7B-v0.4

Trained On
A100 80GB * 8

Sionic AI에서 GPU 자원을 지원받아 제작되었습니다.

Instruction format

It follows Alpaca format.

Model Benchmark

TBD

Implementation Code

Since, chat_template already contains insturction format above. You can use the code below.

python

from transformers import AutoModelForCausalLM, AutoTokenizer

device = "cuda" # the device to load the model onto

model = AutoModelForCausalLM.from_pretrained("maywell/Synatra-kiqu-10.7B")
tokenizer = AutoTokenizer.from_pretrained("maywell/Synatra-kiqu-10.7B")

messages = [
    {"role": "user", "content": "엔비디아는 뭐 하는 기업이야?"},
]

encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")

model_inputs = encodeds.to(device)
model.to(device)

generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)
decoded = tokenizer.batch_decode(generated_ids)
print(decoded[0])

RichardErkhov/maywell_-_Synatra-kiqu-10.7B-gguf