In this case the tokenizer is the yi_tokenizer, loading it requires trust_remote_code=True
Have some fun with this fellow.
It can eat a lot of vram, depends on settings to make it useable on two 24 gb vram gpus:
Without fused attention, it's 27 gb vram, will need some if if yo do stuff.
image.png
You can also let if have fused attention and just reduce the max_seq_length to something way smaller yet still useful
image.png
License
The Yi series models are fully open for academic research and free commercial usage with permission via applications. All usage must adhere to the Model License Agreement 2.0. To apply for the official commercial license, please contact us (yi@01.ai).
Prompt Example:
### System:
You are an AI assistant. User will give you a task. Your goal is to complete the task as faithfully as you can. While performing the task think step-by-step and justify your steps.
### Instruction:
How do you fine tune a large language model?
### Response:
