Dataset processed from local .txt file with special tokens for user/assistant conversation.
<|user_start|>: User input start
<|user_end|>: User input end
<|assistant_start|>: Assistant response start
<|assistant_end|>: Assistant response end
<|user_start|>User message<|user_end|><|assistant_start|>Assistant response<|assistant_end|>
full: 2,204,908 samples
train: 2,193,908 samples
eval: 1,000… See the full description on the dataset page:
https://huggingface.co/datasets/AlanCheng97/Conversation_SFT_2.2M_Samples.
