Views
No views yet
curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install.sh | bash -s -- --plugins wasi_nn-ggmlcurl -LO https://github.com/second-state/llama-utils/raw/main/chat/llama-chat.wasmwasmedge --dir .:. \
--nn-preload default:GGML:AUTO:llama-2-7b-chat-wasm-q5_k_m.gguf \
llama-chat.wasm --reverse-prompt "</s>"[USER]:
What is Wasm?
[ASSISTANT]:
Wasm is a binary instruction format for a stack-based virtual machine. It's a portable target for the WebAssembly runtime.
[USER]:
What is WASI-NN?
[ASSISTANT]:
WASI-NN is a Wasm system interface for neural networks. It allows Wasm apps to use and integrate with different neural network frameworks.
[USER]:
Which runtimes support it?
[ASSISTANT]:
Wasm is supported by a number of runtimes including WasmEdge, Wasmtime, and Wasmer.
[USER]:
How do I run AI workloads in serverless functions?
[ASSISTANT]:
You can use Wasm to run AI workloads in serverless functions. WasmEdge supports running AI workloads using the WASI-NN interface.Thellama-2-7b-chat-wasm-overfit-q5_k_m.gguffile is the fine-tuned model at epoch 25. It has a training loss of 0.03, and is probably over-fitted. You can try the above questions and see it give poor answers. We believe that training loss at 0.05 to 0.1 is optimal for this model.