You can deploy this model and start earning money today!

DanVP/
hy-mt-xianxia-lora-vi-gguf

Name: hy-mt-xianxia-lora-vi-gguf
Availability: InStock

tencent/HY-MT1.5-1.8B

adapter

other

Views

No views yet

HY-MT Xianxia LoRA VI GGUF

GGUF LoRA adapter for DanVP/hy-mt-xianxia-lora-vi, intended for use with the official tencent/HY-MT1.5-1.8B-GGUF base model.

Files

hy-mt-xianxia-lora-vi-f16.gguf - f16 LoRA adapter converted from the PEFT adapter in DanVP/hy-mt-xianxia-lora-vi.

Recommended Base

For CPU demos, start with:

tencent/HY-MT1.5-1.8B-GGUF/HY-MT1.5-1.8B-Q4_K_M.gguf

Higher quality local runs can try Q6_K or Q8_0 if RAM and latency allow.

llama.cpp Usage

bash

llama-cli \
  -m HY-MT1.5-1.8B-Q4_K_M.gguf \
  --lora hy-mt-xianxia-lora-vi-f16.gguf \
  -p "<|prompt|>" \
  -n 1024 \
  --temp 0

This model uses HY-MT chat special tokens. The demo Space formats prompts as:

<｜hy_begin▁of▁sentence｜><｜hy_User｜>{instruction_and_source}<｜hy_Assistant｜>

License

This adapter is a derivative of Tencent HY-MT1.5 and inherits the Tencent HY Community License. See the base model license for full terms and restrictions.

DanVP/hy-mt-xianxia-lora-vi-gguf