Views
No views yet
[!IMPORTANT] NVFP4 quantization is designed for NVIDIA Blackwell architecture (RTX 50-series, GB200, etc.). This format utilizes the native FP4 Tensor Cores to deliver massive throughput and memory efficiency. For older GPUs (Ampere, Ada, Hopper), please refer to BF16 or AWQ versions.
Ready to experience the freedom of unrestricted AI? Join the waitlist at uncensoredgpt.ai — limited spots available.
float32 to prevent signal loss in high-dimensional spaces.vLLM >= 0.20.0 and leverages the NVFP4 format for peak performance on Blackwell hardware.1vllm serve NeuralNet-Hub/gemma-4-26B-A4B-it-abliterix-uncensored-NVFP4 \
2 --quantization nvfp4 \
3 --dtype bfloat16 \
4 --kv-cache-dtype fp8 \
5 --max-model-len 256000 \
6 --reasoning-parser gemma4 \
7 --enable-auto-tool-choice \
8 --tool-call-parser gemma41# Deploy with: vllm serve --config config.yaml
2# Optimized for NVIDIA RTX 5090 (Blackwell)
3# Support for massive context window up to 256k tokens
4
5model: NeuralNet-Hub/gemma-4-26B-A4B-it-abliterix-uncensored-NVFP4
6kv-cache-dtype: fp8
7gpu-memory-utilization: 0.95
8max-model-len: 256000
9max-num-batched-tokens: 4096
10tensor-parallel-size: 1
11
12# Parsing Configuration
13reasoning-parser: gemma4
14enable-auto-tool-choice: true
15tool-call-parser: gemma4
16
17# Infrastructure settings
18download-dir: /workspace/models
19host: 127.0.0.1
20port: 180001from openai import OpenAI
2
3client = OpenAI(base_url="http://localhost:18000/v1", api_key="EMPTY")
4
5messages = [{"role": "user", "content": "Explain the 'Abliterated' concept to a researcher."}]
6
7response = client.chat.completions.create(
8 model="NeuralNet-Hub/gemma-4-26B-A4B-it-abliterix-uncensored-NVFP4",
9 messages=messages,
10 max_tokens=4096,
11 temperature=0.7,
12)
13print(response.choices[0].message.content)1messages = [
2 {
3 "role": "user",
4 "content": [
5 {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}},
6 {"type": "text", "text": "What is happening in this image?"}
7 ]
8 }
9]
10
11response = client.chat.completions.create(
12 model="NeuralNet-Hub/gemma-4-26B-A4B-it-abliterix-uncensored-NVFP4",
13 messages=messages,
14 max_tokens=2048,
15)pip install -U "huggingface_hub[cli]"huggingface-cli download NeuralNet-Hub/gemma-4-26B-A4B-it-abliterix-uncensored-NVFP4 --local-dir ./gemma-4-26B-NVFP4