You can deploy this model and start earning money today!

nightmedia/
granite-4.1-3B-TNG-Holodeck-V2B-Heretic-q8-hi-mlx

Name: granite-4.1-3B-TNG-Holodeck-V2B-Heretic-q8-hi-mlx
Availability: InStock

Views

No views yet

granite-4.1-3B-TNG-Holodeck-V2B-Heretic-q8-hi-mlx

This model is a NuSLERP merge of:

treadon/granite-4.1-3b-Abliterated-AND-Disinhibited
nightmedia/granite-4.1-3B-TNG-Coder-Heretic-V2
nightmedia/granite-4.1-3B-TNG-Holodeck-V2B-Heretic
ermiaazarkhalili/Granite-4.1-3B-SFT-Claude-Opus-Reasoning-Unsloth

brainwaves

quant    arc   arc/e boolq hswag obkqa piqa  wino
bf16     0.514,0.720,0.846
q8-hi    0.520,0.716,0.847,0.704,0.422,0.791,0.687

Quant    Perplexity      Peak Memory   Tokens/sec
bf16     4.235 ± 0.027   9.74 GB       2031
mxfp8    4.756 ± 0.031   6.58 GB       1665
q8-hi    4.238 ± 0.027   6.90 GB       1613

Model components

granite-4.1-3B-TNG-Coder-Heretic-V2

brainwaves

quant    arc   arc/e boolq hswag obkqa piqa  wino
bf16     0.510,0.717,0.846,0.694,0.422,0.786,0.669
q8-hi    0.515,0.718,0.848,0.694,0.426,0.784,0.668
Quant    Perplexity      Peak Memory   Tokens/sec
bf16     4.425 ± 0.029   9.74 GB       1996
q8-hi    4.423 ± 0.029   6.90 GB       1721

ermiaazarkhalili/Granite-4.1-3B-SFT-Claude-Opus-Reasoning-Unsloth

brainwaves

         arc   arc/e boolq hswag obkqa piqa  wino
q8-hi    0.417,0.585,0.651,0.682,0.390,0.781,0.669

Parent model

treadon/granite-4.1-3b-Abliterated-AND-Disinhibited

brainwaves

quant    arc   arc/e boolq hswag obkqa piqa  wino
mxfp8    0.405,0.598,0.843,0.520,0.442,0.713,0.582

Baseline model

ibm-granite/granite-4.1-3B

brainwaves

quant    arc   arc/e boolq hswag obkqa piqa  wino
mxfp8    0.406,0.581,0.821,0.484,0.434,0.712,0.559

The granite-4.1-3B-TNG-Coder-Heretic-V2 model was trained on the Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-qx86-hi-mlx with a custom Holodeck template that generated the characters.

This model is deeply immersed in Star Trek lore, with an intimate understanding of DS9 station operations and character strengths and weaknesses. It has not been explicitly trained on Star Trek, just on what the large model knows about Star Trek.

The traces have been generated using a custom system prompt, and contain general backend engineering questions in Haskell, Python, Golang, etc..

Additionally to the 544 TNG-flavored training traces, there are 986 questions from Polaris Alpha, distilled from the MTP-Holo3-Qwopus.

The model can deliver output in a variety of formats depending on the type of question and complexity.

More details coming soon.

-G

Model recipe

recipe

models:
  - model: granite-4.1-3B-TNG-Coder-Heretic-V2
    parameters:
      weight: 1.6
  - model: ermiaazarkhalili/Granite-4.1-3B-SFT-Claude-Opus-Reasoning-Unsloth
    parameters:
      weight: 0.4
merge_method: nuslerp
dtype: bfloat16
name: granite-4.1-3B-TNG-Holodeck-V2B-Heretic

Use with mlx

pip install mlx-lm

python

from mlx_lm import load, generate

model, tokenizer = load("granite-4.1-3B-TNG-Holodeck-V2B-Heretic-q8-hi-mlx")

prompt = "hello"

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True, return_dict=False,
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)

nightmedia/granite-4.1-3B-TNG-Holodeck-V2B-Heretic-q8-hi-mlx

granite-4.1-3B-TNG-Holodeck-V2B-Heretic-q8-hi-mlx

Model components

Parent model

Baseline model

Model recipe

Use with mlx

nightmedia/
granite-4.1-3B-TNG-Holodeck-V2B-Heretic-q8-hi-mlx