Converted via llm compressor, with 200 examples from henrik3/sweep-calibration for calibration (which is to be fair not best quality but should be fine)

Performance

Sglang & RTX PRO 4500 (800GB/s bandwidth, Blackwell)

Command:

python3 -m sglang.launch_server --model-path henrik3/sweep-next-edit-v2-7B-NVFP4 --port 8000 --host 0.0.0.0 --trust-remote-code --mem-fraction-static 0.8 --context-length 16384 --speculative-algorithm NGRAM --speculative-num-draft-tokens 4

Stats

~9100 input tokens
~200 output tokens
~300ms response time

Find the original model here: https://huggingface.co/sweepai/sweep-next-edit-v2-7B