Views
No views yet
| Prompt | Response | Correct? |
|---|---|---|
| What is the capital of France? | "The capital of France is Paris. Paris is not only the larges" | ✅ |
| What is 2+2? Just the number. | "The sum of 2 and 2 is 4. This is a basic arithmetic operatio" | ✅ |
| Attribute | Value |
|---|---|
| Base Model | microsoft/Phi-3.5-mini-instruct |
| File Size | 2685 MB |
| Format | GGUF |
| Chat Format | chatml |
| CPU Speed | 7.0 tokens/sec |
| License | mit |
1from llama_cpp import Llama
2
3llm = Llama(model_path="model.gguf", chat_format="chatml", n_ctx=512, n_threads=4, verbose=False)
4response = llm.create_chat_completion(
5 messages=[{"role": "user", "content": "What is the capital of France?"}],
6 max_tokens=50,
7)
8print(response["choices"][0]["message"]["content"])1from dispatchai import load_model
2model = load_model("Phi-3.5-mini-instruct-Q5-mobile", backend="gguf")
3print(model.chat("Hello!"))