JiRack Coder Reasoning 8B INT4
A fast and efficient coding assistant with a clean built-in web UI, powered by Qwen3.0-Coder-8B-Instruct and optimized using Microsoft ONNX Runtime.
- JiRack is a cloud-ready model that helps save money on cloud infrastructure. It can be used as an expert model in RAG deployments, with the ONNX JiRack Java server as an alternative.
- Subscription: $1 per month per user (updated license for non-company use).
- Corp Subscription: $3 per month per user (updated license for company use).
- It works without subscription but send message about subscription
Quick Start
Watch JiRack Coder Reasoning 8B in action:
Run with Docker
Default CPU
1docker run -d \
2 --name jirack_coder_reasoing_8b \
3 -p 7869:7869 \
4 --restart unless-stopped \
5 cmsmanhattan/jirack_coder_8b_int4_qwenbase:latest
Multi CPU
1docker run -d \
2 --name jirack_coder_reasoing_8b \
3 -p 7869:7869 \
4 --restart unless-stopped \
5 --memory=20g \
6 --cpus=12 \
7 cmsmanhattan/jirack_coder_8b_int4_qwenbase:latest
GPU (Coming soon)
1docker run -d \
2 --name jirack_coder_reasoing_8b \
3 -p 7869:7869 \
4 --gpus all \
5 --restart unless-stopped \
6 cmsmanhattan/jirack_coder_8b_int4_gpu_qwenbase:latest
Docker Compose Example
1services:
2 jirack:
3 image: cmsmanhattan/jirack_coder_8b_int4_qwenbase:latest
4 container_name: jirack_onnx_service
5 ports:
6 - "7869:7869"
7 volumes:
8 - .:/app
9 - ./web:/app/web
10 environment:
11 - MAX_TOKENS=1024
12 - TEMPERATURE=0.7
13 - TOP_P=0.9
14 - DEFAULT_STREAM=False
15 - INTRA_THREADS=4
16 - USE_ENV_ALLOCATOR=1
17 deploy:
18 resources:
19 limits:
20 memory: 16g
Access the UI
Once the container is running, open your browser and navigate to:
http://localhost:7869
This opens the JiRack Coder UI — a clean web interface designed for coding.
Changing the Port
The listening port can be easily modified directly from the Settings panel within the JiRack Coder UI.
Licensing
- The JiRack Coder Reasoning 8B model is provided under a commercial license ($12 per user per year).
- All JiRack UI clients are provided under a commercial license.
- However, the UI clients can be used for free when running together with the official JiRack Docker containers, as long as they are not redistributed separately.
JiRack Coder 32B is available exclusively under a commercial enterprise license.
For commercial licensing, cluster deployment, or enterprise use, please contact us.
-
-
Live email chat with the model: support@cmsmanhattan.com
Hardware Recommendations for AMD Systems
Note: This model is heavier than JiRack Coder 7B INT8.
Recommended Hardware for JiRack Coder Reasoning 8B INT4 (single Docker container)
| Use Case | CPU | GPU (ROCm) | VRAM / RAM | Expected Speed | Recommendation |
|---|
| Recommended | Ryzen 7 7700 / 9700X | RX 7900 XTX / 7900 XT | 24GB VRAM | 50-75 tokens/s | Best choice |
| High Performance | Ryzen 9 7950X / 9950X | RX 7900 XTX | 24GB+ VRAM | 65-90 tokens/s | Excellent |
| Enterprise | EPYC 7003/9004 series | MI300X or 2x RX 7900 XTX | 48GB+ VRAM | 90-140 tokens/s | For 32B model |
| Budget Option | Ryzen 5 7600 / 9600X | RX 7800 XT (16GB) | 16GB VRAM | 35-50 tokens/s | Acceptable |
Important Memory Notes
Even though the 8B INT4 model itself takes approximately 5–6 GB, we recommend at least 24GB VRAM for the following reasons:
- KV-cache consumption during generation, especially with long context
- ONNX Runtime overhead and temporary buffers
- System stability and avoiding out-of-memory errors
- Room for larger context windows
Minimum recommended: 24GB VRAM (RX 7900 series)
Ideal: 24–32GB VRAM
For pure CPU inference (no GPU), we recommend at least 64GB system RAM (Ryzen 9 7950X/9950X).
I added the default model in full FP32 precision. This serves as the base for quantization, allowing us to find the optimal balance between model size and performance.
📧 Contact & Licensing
For joint venture opportunities, hardware integration, or licensing inquiries:
- Email: grabko@cmsmanhattan.com
- Phone: +1 (516) 777-0945
- Location: New York, USA
