This is the
WizardMath-13B-V1.0 model in MLC format
q4f16_1.
The model can be used for projects
MLC-LLM and
WebLLM.
Here are some examples of using this model in MLC LLM.
Before running the examples, please install MLC LLM by following the
installation documentation.
1from mlc_llm import MLCEngine
2
3# Create engine
4model = "HF://mlc-ai/WizardMath-13B-V1.0-q4f16_1-MLC"
5engine = MLCEngine(model)
6
7# Run chat completion in OpenAI API.
8for response in engine.chat.completions.create(
9 messages=[{"role": "user", "content": "What is the meaning of life?"}],
10 model=model,
11 stream=True,
12):
13 for choice in response.choices:
14 print(choice.delta.content, end="", flush=True)
15print("\n")
16
17engine.terminate()
For more information on MLC LLM project, please visit our
documentation and
GitHub repo.
