python3 -m vllm.entrypoints.openai.api_server --model=naver-hyperclovax/HyperCLOVAX-SEED-Think-14B --trust_remote_code --port=8000 --speculative-config='{"method": "arctic","model": "K-Compression/Arctic-LSTM-Speculator-HyperCLOVAX-SEED-Think-14B"}'

Performance

We compare the output token throughput (tokens/s) of vLLM-based standard decoding and speculative decoding for HyperCLOVA X SEED 14B Think on a single H100 GPU as shown below:

HyperCLOVA X SEED 14B Think	ShareGPT (tokens/s)
No speculation	84.40
Arctic Speculator	115.94((1.4x faster))

License

The model is licensed under HyperCLOVA X SEED Model License Agreement