You can deploy this model and start earning money today!

mohameddalii/
ocr-mt-5090

Name: ocr-mt-5090
Availability: InStock

Views

No views yet

OCR + Qwen3-4B on one RTX 5090

Serve PaddleOCR-VL and Qwen/Qwen3-4B-Instruct-2507 on the same 32GB 5090 without quantization. This repo is the serve kit (scripts + engines), not a new weight dump — vLLM still pulls the official model repos.

Proven together on a Vast.ai 5090 (bf16):

Service	GPU share	Context	Parallel seqs	Accuracy (colocated)
Qwen3-MT	0.38 (~12 GB)	2048	8	64/64 translation suite, chrF 71.7
PaddleOCR-VL	0.48 (~15 GB)	8192	8	12/12 multilingual, 85/85 fields

Start Qwen first, then OCR. Both vLLM processes budget --gpu-memory-utilization against the full 32GB, not remaining memory.

Repo: https://huggingface.co/mohameddalii/ocr-mt-5090

Pull

bash

hf download mohameddalii/ocr-mt-5090 --local-dir ./ocr-mt-5090
cd ocr-mt-5090

Generic 5090 (any Linux box)

bash

python3 -m venv .venv && source .venv/bin/activate
uv pip install -r requirements.txt
# Blackwell: if CUDA tensors fail, reinstall torch with cu128+
# uv pip install torch --index-url https://download.pytorch.org/whl/cu128

bash scripts/prefetch.sh          # optional, caches both models
bash scripts/serve_both.sh        # MT then OCR; leaves both in the foreground wait

Endpoints:

OCR: http://127.0.0.1:18000/v1
Translation: http://127.0.0.1:18100/v1

bash

python python/ocr.py scan.png
python python/translate.py --from English --to Arabic "The total is 12 dollars."

Vast.ai 5090

On a Vast base/pytorch image with supervisor + Caddy:

bash

cd /workspace
hf download mohameddalii/ocr-mt-5090 --local-dir /workspace/ocr-mt-5090
bash /workspace/ocr-mt-5090/vast/install_vast.sh

That copies supervisor wrappers, adds PaddleOCR-VL (10100→18000) and Qwen3-MT (10200→18100) to /etc/portal.yaml, starts MT, waits, then starts OCR.

Need two free normal ports at instance create time (defaults 10100 and 10200). Public URLs come from vast-capabilities direct_url plus $OPEN_BUTTON_TOKEN.

Environment knobs

Variable	Default	Role
`MT_GPU_UTIL`	`0.38`	Qwen GPU fraction
`OCR_GPU_UTIL`	`0.48`	OCR GPU fraction
`MT_PORT` / `OCR_PORT`	`18100` / `18000`	bind ports
`HF_HOME`	`./.hf_home`	model cache
`VENV` / `MT_VENV` / `OCR_VENV`	`.venv`	python env

Do not raise both utils so they sum past ~0.90 — the card will OOM. Do not switch to 4-bit; that is what this split avoids.

Layout

scripts/serve_both.sh      start both (MT first)
scripts/serve_mt.sh
scripts/serve_ocr.sh
scripts/prefetch.sh
python/ocr_engine.py       parallel crops + OCR: prompt
python/mt_engine.py        glossary + copy-spans
python/ocr.py / translate.py
vast/install_vast.sh       supervisor + portal