SEA-LION (Southeast Asian Languages In One Network) is a collection of Large Language Models (LLMs) which have been pretrained and instruct-tuned for the Southeast Asia (SEA) region.
Qwen-SEA-LION-v4-32B-IT excels at Southeast Asian (SEA) tasks when compared to other open models with fewer than 200 billion parameters, and demonstrates performance comparable to that of larger and top closed models.
This repository contains the Qwen-SEA-LION-v4-32B-IT model exported to OpenVINO™ IR format with weights compressed to INT8 using Optimum-Intel and NNCF. Compared to the INT4 variant, this version preserves higher weight precision at a larger memory footprint, while still enabling on-device inference on Intel AI PCs.
For tokenization, the model employs the default tokenizer used in Qwen3-32B.
1import huggingface_hub as hf_hub
23model_id ="aisingapore/Qwen-SEA-LION-v4-32B-IT-OV-8BIT"4model_path ="Qwen-SEA-LION-v4-32B-IT-OV-8BIT"56hf_hub.snapshot_download(model_id, local_dir=model_path)
3. Run model inference
py
1import openvino_genai as ov_genai
23pipe = ov_genai.LLMPipeline(model_path, device="GPU")45prompt ="What are some traditional dishes across Southeast Asia?"6result = pipe.generate(prompt, max_new_tokens=200)7print(result)
More GenAI usage examples can be found in OpenVINO GenAI library docs and samples
This work was made possible through a collaboration with Intel. We deeply appreciate the support and tools provided.
The SEA-LION project is supported by the National Research Foundation Singapore and the Infocomm Media Development Authority (IMDA), Singapore under its National Large Language Model Funding Initiative.
