Views
No views yet
[!IMPORTANT] NVFP4 quantization requires NVIDIA Blackwell architecture (RTX 50-series, GB200, B100/B200, etc.). This format is designed to maximize the efficiency of Blackwell's native FP4 Tensor Cores. If you are using Ampere, Ada Lovelace, or Hopper, please use a standard BF16 or AWQ/GPTQ variant.
Ready to experience the freedom of unrestricted AI? Join the waitlist at uncensoredgpt.ai — limited spots available.
vLLM >= 0.20.0. For maximum performance on the RTX 5090, use the following configuration.1vllm serve NeuralNet-Hub/gemma-4-31B-it-abliterated-uncensored-NVFP4 \
2 --quantization nvfp4 \
3 --dtype bfloat16 \
4 --kv-cache-dtype fp8 \
5 --max-model-len 150000 \
6 --reasoning-parser gemma4 \
7 --enable-auto-tool-choice \
8 --tool-call-parser gemma41# Deploy with: vllm serve --config config.yaml
2# Optimized for NVIDIA RTX 5090 (Blackwell)
3# Target: ~15 parallel requests at 150k context length
4
5model: NeuralNet-Hub/gemma-4-31B-it-abliterated-uncensored-NVFP4
6kv-cache-dtype: fp8
7gpu-memory-utilization: 0.95
8max-model-len: 150000
9max-num-batched-tokens: 4096
10tensor-parallel-size: 1
11
12# Parsing Configuration
13reasoning-parser: gemma4
14enable-auto-tool-choice: true
15tool-call-parser: gemma4
16
17# Infrastructure settings
18download-dir: /workspace/models
19host: 127.0.0.1
20port: 180001from openai import OpenAI
2
3client = OpenAI(base_url="http://localhost:18000/v1", api_key="EMPTY")
4
5messages = [{"role": "user", "content": "Explain the concept of quantum entanglement to a 10-year-old."}]
6
7response = client.chat.completions.create(
8 model="NeuralNet-Hub/gemma-4-31B-it-abliterated-uncensored-NVFP4",
9 messages=messages,
10 max_tokens=4096,
11 temperature=0.7,
12 top_p=0.9,
13)
14print(response.choices[0].message.content)1messages = [
2 {
3 "role": "user",
4 "content": [
5 {"type": "image_url", "image_url": {"url": "https://example.com/diagram.jpg"}},
6 {"type": "text", "text": "Analyze this technical diagram and summarize the key components."}
7 ]
8 }
9]
10
11response = client.chat.completions.create(
12 model="NeuralNet-Hub/gemma-4-31B-it-abliterated-uncensored-NVFP4",
13 messages=messages,
14 max_tokens=2048,
15)pip install -U "huggingface_hub[cli]"huggingface-cli download NeuralNet-Hub/gemma-4-31B-it-abliterated-uncensored-NVFP4 --local-dir ./gemma-4-31B-NVFP4[!WARNING] If deploying on older architectures (Ampere/Ada/Hopper), ensure you use a compatible quantization format. NVFP4 is specifically engineered to utilize the new hardware capabilities of the Blackwell generation.