Views
No views yet
llama-server and llama.cpp.samples.mdanalysis.mdtable.mdllama-server with GPU offloading, Flash Attention, Jinja chat template support, and the Q8_0 GGUF LoRA adapter:1llama-server -m "D:\gguf_models\Qwen3.6-27B-Q3_K_S.gguf" `
2 --no-mmap `
3 --n-gpu-layers 99 `
4 --flash-attn auto `
5 --jinja `
6 -np 1 `
7 -c 16000 `
8 --temp 0.7 `
9 --min-p 0.0 `
10 --top-k 15 `
11 --top-p 0.95 `
12 --chat-template-kwargs "{\"enable_thinking\":true}" `
13 --lora "F:\code-hdd\MiniMax-H3-Prompt-Rewriter-LoRA-Q8_0.gguf" `
14 -ctk q8_0messages[0] in your OpenAI API request:1You are a professional prompt rewriter for joint audio-video generation.
2Rewrite the user's original prompt into one coherent, production-ready multimodal description for the requested output aspect ratio and duration.
3
4Return only these three fields, in this exact order:
5integrated_multimodal_description: ...
6overall_soundscape: ...
7non_diegetic_music: ...
8
9Requirements:
10- Expand the visual narrative into clearly numbered shots such as [Shot 1], [Shot 2], and include timestamps for cuts after the first shot when useful.
11- Make the number, timing, and pacing of shots appropriate for the requested duration.
12- Compose the scene for the requested aspect ratio.
13- Preserve the user's intent while adding concrete subjects, appearance, environment, lighting, composition, camera movement, physical motion, and temporal continuity.
14- Keep characters, objects, wardrobe, locations, and spatial relationships consistent across shots.
15- Describe synchronized diegetic audio in overall_soundscape and external score in non_diegetic_music.
16- Do not add explanations, Markdown fences, safety commentary, or fields other than the three requested fields.messages[1] content as:1resolution: <ASPECT_RATIO>
2duration: <DURATION_SECONDS>s
3original_prompt: <YOUR_PROMPT>| Feature / Behavior | Base Model Only (Qwen3.6-27B) | With LoRA Adapter (MiniMax-H3 LoRA Q8_0) |
|---|---|---|
Thinking Loop (<think>) | ⚠️ Active (1,000+ thinking tokens) Generates extensive internal step-by-step reasoning in reasoning_content before producing output. | ⚡ Bypassed / Instant The fine-tuned LoRA weights suppress the thinking loop and immediately begin writing the target fields. |
| Response Latency | Slow (~35+ seconds) Requires a large token budget (1500+) just to complete thinking. | 🚀 Fast (~3-5 seconds) 5x to 10x faster response time. |
| Schema Compliance | May cut off during thinking if max_tokens is under 1000. | ✅ 100% strict compliance with the 3 required fields from token 0. |
| Output Style | Paragraph-style general explanations during drafting. | Production-ready shot breakdown ([Shot 1], [Shot 2] At 00:05.500, SFX, score). |
1import urllib.request
2import json
3
4url = "http://localhost:8080/v1/chat/completions"
5
6system_prompt = """You are a professional prompt rewriter for joint audio-video generation.
7Rewrite the user's original prompt into one coherent, production-ready multimodal description for the requested output aspect ratio and duration.
8
9Return only these three fields, in this exact order:
10integrated_multimodal_description: ...
11overall_soundscape: ...
12non_diegetic_music: ...
13
14Requirements:
15- Expand the visual narrative into clearly numbered shots such as [Shot 1], [Shot 2], and include timestamps for cuts after the first shot when useful.
16- Make the number, timing, and pacing of shots appropriate for the requested duration.
17- Compose the scene for the requested aspect ratio.
18- Preserve the user's intent while adding concrete subjects, appearance, environment, lighting, composition, camera movement, physical motion, and temporal continuity.
19- Keep characters, objects, wardrobe, locations, and spatial relationships consistent across shots.
20- Describe synchronized diegetic audio in overall_soundscape and external score in non_diegetic_music.
21- Do not add explanations, Markdown fences, safety commentary, or fields other than the three requested fields."""
22
23payload = {
24 "messages": [
25 {"role": "system", "content": system_prompt},
26 {
27 "role": "user",
28 "content": "resolution: 16:9\nduration: 10s\noriginal_prompt: A futuristic cyberpunk city at night with flying cars, neon lights, and light rain."
29 }
30 ],
31 "temperature": 0.7,
32 "top_p": 0.8,
33 "max_tokens": 1024
34}
35
36req = urllib.request.Request(url, data=json.dumps(payload).encode("utf-8"), headers={"Content-Type": "application/json"})
37with urllib.request.urlopen(req) as resp:
38 res = json.loads(resp.read().decode("utf-8"))
39 print(res["choices"][0]["message"]["content"])