HyperCLOVA-X-SEED-Vision-Instruct-3B-Llamafied is based on a model developed by NAVER that can understand and generate text.
It demonstrates competitive performance on major benchmarks related to Korean language and culture. In addition, it supports a context length of up to 16k tokens, enabling it to handle a wide range of tasks.
The training data for HyperCLOVA-X-SEED-Vision-Instruct-3B-Llamafied consists of diverse sources, including high-quality datasets. The training process was carried out in four main stages: Pretraining Stage 1, where the model learns from a large volume of documents; Pretraining Stage 2, which focuses on additional training with high-quality data; Rejection sampling Fine-Tuning (RFT), aimed at enhancing the model’s knowledge across various domains and its complex reasoning abilities; and Supervised Fine-Tuning (SFT), which improves the model’s instruction-following capabilities. Furthermore, due to the characteristics of smaller models, vulnerability to long-context handling was observed. To address this, reinforcement for long-context understanding was incorporated from the pretraining stages through to the SFT stage, enabling the model to stably support context lengths of up to 16k tokens.
1from transformers import AutoModelForCausalLM, AutoTokenizer
2model = AutoModelForCausalLM.from_pretrained("/path/to/ckpt")
3tokenizer = AutoTokenizer.from_pretrained("/path/to/ckpt")
4
5chat = [
6 {"role": "tool_list", "content": ""},
7 {"role": "system", "content": "- AI 언어모델의 이름은 \"CLOVA X\" 이며 네이버에서 만들었다.\n- 오늘은 2025년 04월 24일( 목)이다."},
8 {"role": "user", "content": "슈뢰딩거 방정식과 양자역학의 관계를 최대한 자세히 알려줘."},
9]
10
11inputs = tokenizer.apply_chat_template(chat, add_generation_prompt=True, return_dict=True, return_tensors="pt")
12output_ids = model.generate(**inputs, max_length=1024, stop_strings=["<|endofturn|>", "<|stop|>"], tokenizer=tokenizer)
13print(tokenizer.batch_decode(output_ids))
