Views
No views yet
1import { CreateMLCEngine } from "@mlc-ai/web-llm";
2
3const engine = await CreateMLCEngine(
4 "https://huggingface.co/UMASHIKA/gemma3-270m-japanese-webllm-03",
5 {
6 initProgressCallback: (progress) => {
7 console.log(`Loading: ${(progress.progress * 100).toFixed(1)}%`);
8 }
9 }
10);
11
12const reply = await engine.chat.completions.create({
13 messages: [{ role: "user", content: "こんにちは!" }]
14});
15
16console.log(reply.choices[0].message.content);1import * as webllm from "@mlc-ai/web-llm";
2
3const customModel: webllm.ModelRecord = {
4 model: "https://huggingface.co/UMASHIKA/gemma3-270m-japanese-webllm-03",
5 model_id: "gemma3-270m-japanese-03",
6 model_lib: "https://huggingface.co/UMASHIKA/gemma3-270m-japanese-webllm-03/resolve/main/gemma3-270m-japanese-q4f32_1-ctx4k-webgpu.wasm",
7 vram_required_MB: 900,
8 low_resource_required: true,
9};
10
11const appConfig: webllm.AppConfig = {
12 model_list: [...webllm.prebuiltAppConfig.model_list, customModel]
13};
14
15const engine = await webllm.CreateMLCEngine(
16 "gemma3-270m-japanese-03",
17 { appConfig }
18);| デバイス | ロード時間 | 推論速度 | VRAM使用量 |
|---|---|---|---|
| iPhone 15 Pro | 5-10秒 | 30-50 tokens/s | ~900MB |
| Desktop (WebGPU) | 3-8秒 | 40-80 tokens/s | ~900MB |
.
├── mlc-chat-config.json ← WebLLM設定
├── gemma3-270m-japanese-q4f32_1-ctx4k-webgpu.wasm ← カスタムWASM
├── params_shard_0.bin ← 量子化済み重み
├── params_shard_1.bin
├── params_shard_2.bin
├── params_shard_3.bin
├── tokenizer.model ← トークナイザー
├── tokenizer.json
└── ...gemma3-270m-japanese-q4f32_1-ctx4k-webgpu.wasm