You should mask everything except the last turn. The only part that matters to teach the model is the last turn, as you are teaching it to always output thinking, no matter what the user feeds it.
This is what I used to generate the dataset, so that it's setup to be trained like R1:
import requests
import json
import time
import pandas as pd
from datasets import load_dataset
from tqdm import tqdm
import re
import… See the full description on the dataset page:
https://huggingface.co/datasets/PJMixers-Dev/MinervaAI_Aesir-Preview-Anon-qwq-all.
