Views
No views yet
1./llama-server \
2 -m /path/to/model.gguf \
3 -c 2048 \ # Context length
4 --host 0.0.0.0 \ # Allow remote connections
5 --port 8080 \ # Server port
6 --n-gpu-layers 35 # GPU acceleration (if available)API URL: http://localhost:8080
Model: (leave empty)
API Type: llama.cpp1{
2 "temperature": 0.7,
3 "max_tokens": 512,
4 "top_p": 0.9
5}1Context Length: 2048
2GPU Offload: Recommended (enable if available)
3Batch Size: 512| Filename | Precision | Size | Characteristics |
|---|---|---|---|
| mechat.gguf | FP16 | [12.5GB] | Full original model precision |
--n-gpu-layers 35 for GPU acceleration (requires CUDA-enabled build)--mlock to prevent swapping1@misc{qiu2023smile,
2 title={SMILE: Single-turn to Multi-turn Inclusive Language Expansion via ChatGPT for Mental Health Support},
3 author={Huachuan Qiu and Hongliang He and Shuai Zhang and Anqi Li and Zhenzhong Lan},
4 year={2023},
5 eprint={2305.00450},
6 archivePrefix={arXiv},
7 primaryClass={cs.CL}
8}