Views
No views yet
1import * as webllm from "@mlc-ai/web-llm";
2
3const engine = new webllm.MLCEngine();
4
5await engine.reload("https://huggingface.co/UMASHIKA/smollm2-360m-japanese-finetuned-01", {
6 model_lib: "https://huggingface.co/UMASHIKA/smollm2-360m-japanese-finetuned-01/resolve/main/smollm2-360m-japanese-q4f32_1-ctx8192-webgpu.wasm"
7});
8
9const response = await engine.chat.completions.create({
10 messages: [{ role: "user", content: "こんにちは！" }]
11});
12
13console.log(response.choices[0].message.content);1const appConfig = {
2 model_list: [
3 {
4 model: "https://huggingface.co/UMASHIKA/smollm2-360m-japanese-finetuned-01",
5 model_id: "smollm2-360m-japanese",
6 model_lib: "https://huggingface.co/UMASHIKA/smollm2-360m-japanese-finetuned-01/resolve/main/smollm2-360m-japanese-q4f32_1-ctx8192-webgpu.wasm",
7 overrides: {
8 context_window_size: 8192,
9 temperature: 0.7,
10 top_p: 0.9
11 }
12 }
13 ]
14};
15
16const engine = new webllm.MLCEngine();
17await engine.reload("smollm2-360m-japanese", appConfig);| ファイル | サイズ | 説明 |
|---|---|---|
params_shard_*.bin | 196MB (7ファイル) | 量子化された重みパラメータ |
smollm2-360m-japanese-q4f32_1-ctx8192-webgpu.wasm | 5.7MB | カスタムコンパイル済みWASM |
mlc-chat-config.json | 2.3KB | MLC設定ファイル |
tokenizer.json | 3.4MB | トークナイザー |
tokenizer_config.json | 3.8KB | トークナイザー設定 |
1# Phase 1: WebLLM変換
2mlc_llm convert_weight models/smollm2-360m-finetuned-02 \
3 --quantization q4f32_1 \
4 --output models/smollm2-360m-finetuned-02/webllm-output/smollm2-360m-finetuned-02-q4f32_1-MLC
5
6mlc_llm gen_config models/smollm2-360m-finetuned-02 \
7 --quantization q4f32_1 \
8 --conv-template chatml \
9 --context-window-size 8192 \
10 --output models/smollm2-360m-finetuned-02/webllm-output/smollm2-360m-finetuned-02-q4f32_1-MLC/mlc-chat-config.json
11
12# Phase 2: WASMコンパイル
13mlc_llm compile models/smollm2-360m-finetuned-02/webllm-output/smollm2-360m-finetuned-02-q4f32_1-MLC/mlc-chat-config.json \
14 --device webgpu \
15 --opt O3 \
16 --system-lib-prefix smollm2_360m_japanese \
17 -o smollm2-360m-japanese-q4f32_1-ctx8192-webgpu.wasm