ChatGLM3-6B-GGML is a quantized version of ChatGLM3-6B that can run on CPU servers.
1git clone --recursive https://github.com/li-plus/chatglm.cpp.git
2
3python3 -m pip install torch tabulate tqdm transformers accelerate sentencepiece
4pip install -U 'chatglm-cpp[api]'
You can generate dialogue by invoking the ChatGLM3-6B model with the following code:
1>>> import chatglm_cpp
2>>>
3>>> pipeline = chatglm_cpp.Pipeline("./chatglm-ggml.bin")
4>>> pipeline.chat([chatglm_cpp.ChatMessage(role="user", content="你好")])
5ChatMessage(role="assistant", content="你好！我是人工智能助手 ChatGLM-6B，很高兴见到你，欢迎问我任何问题。", tool_calls=[])
关于更多的使用说明，包括如何运行命令行和网页版本的 DEMO，请参考我的
文章。
For more usage instructions, including how to run the command line and web versions of the DEMO, please refer to my
article.
本仓库的代码依照 MIT 协议开源，ChatGLM3-6B 模型的权重的使用则需要遵循
Model License。
The code in this repository is open-sourced under the MIT LICENSE, while the use of the ChatGLM3-6B model weights needs to comply with the
Model License.
