Views
No views yet
1# pip install mlx-vlm
2from mlx_vlm import load, generate
3
4model, processor = load("tokimoa/ovisocr2-mlx-4bit")
5prompt = "Extract all readable content from the image in natural human reading order and output the result as a single Markdown document. For charts or images, represent them using an HTML image tag: <img src=\"images/bbox_{left}_{top}_{right}_{bottom}.jpg\" />, where left, top, right, bottom are bounding box coordinates scaled to [0, 1000). Format formulas as LaTeX. Format tables as HTML: <table>...</table>. Transcribe all other text as standard Markdown. Preserve the original text without translation or paraphrasing."
6messages = [{"role":"user","content":[{"type":"image"},{"type":"text","text":prompt}]}]
7text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
8out = generate(model, processor, text, image=["page.png"], max_tokens=4096, temperature=0.0, verbose=False)
9print(out.text)ATH-MaaS/OvisOCR2（bf16、Qwen3.5系 linear+full attentionハイブリッド）mlx-vlm 0.6.7 convert -q --q-bits 4（実効5.86bpw、622MB）generation_config.json が同梱されておらず、config.json の eos_token_id は <|endoftext|> (248044) を指しています。実際のターン終端は <|im_end|> (248046) のため、transformersの generate() にそのまま渡すと停止せず文書を繰り返し生成します。eos_token_id=248046 を明示してください（mlx-vlm・vLLMはtokenizer側のEOSを使うため影響なし）。