You can deploy this model and start earning money today!

kleinpanic93/
canvas-calendar-agent-v7-dpo

Name: canvas-calendar-agent-v7-dpo
Availability: InStock

Views

No views yet

Canvas Calendar Agent v7 — DPO

Fine-tuned google/gemma-4-E2B-it (2.7B params) for the Canvas Calendar Agent project. Trained with Direct Preference Optimization (Rafailov et al., 2023, arXiv:2305.18290) on top of an SFT-trained checkpoint.

Intended use

Reads a student's Canvas LMS state (assignments, courses, deadlines) and their calendar, then produces concrete, actionable scheduling plans (study blocks, exam prep, rescheduling). Speaks the native Gemma-4 tool-call format (<|tool_call>call:tool.name{args}<tool_call|>) for 18 Canvas/Calendar/Study tools.

Training procedure

Stage	Method	Data	Wall time
SFT	Full-parameter fine-tuning	181 trajectory rows from Canvas sessions	~12 min
DPO	β=0.1, sigmoid loss, frozen-snapshot reference	1,071 preference pairs labeled by Gemma-4-31B-IT-NVFP4 teacher (3-vote majority at temp=0)	9:03

DPO hyperparameters

python

DPOConfig(
    loss_type="sigmoid",          # original DPO loss (Rafailov §4)
    beta=0.1,                     # KL implicit coefficient
    num_train_epochs=1,
    per_device_train_batch_size=1,
    gradient_accumulation_steps=8,  # effective batch = 8
    learning_rate=5e-6,
    bf16=True,
    sync_ref_model=False,         # freeze ref at init
    precompute_ref_log_probs=True,
    optim="adamw_torch",
)

Reference policy: snapshot of policy (= SFT model) at training init, frozen via precompute_ref_log_probs=True. Mathematically equivalent to the paper's π_ref = π_SFT.

Training metrics

Metric	Final value
`train_loss`	0.2229
`rewards/accuracies`	0.9032
`rewards/margins`	5.142
`rewards/chosen`	-3.362
`rewards/rejected`	-8.504
`logps/chosen`	-238.5
`logps/rejected`	-328.9
`entropy`	1.01

Random preference accuracy is 0.5; our DPO model correctly ranks 90.3% of held-out preference pairs.

Model architecture

Base: google/gemma-4-E2B-it (2.7B params, decoder-only Transformer)
Precision: bf16 weights, full-parameter fine-tune (no LoRA / no quantization)
Context: 4,096 tokens during training, 8,192 supported at inference
Tool-call format: native Gemma-4 delimiters (<|tool_call>call:NAME{ARGS}<tool_call|>)

Hardware

NVIDIA DGX Spark (Grace-Blackwell GB10 SoC, 122 GiB unified memory, SM121, aarch64). Single GPU. Container: nvcr.io/nvidia/pytorch:25.11-py3. TRL 1.1, Transformers 4.47+, PyTorch 2.10 nightly.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "kleinpanic93/canvas-calendar-agent-v7-dpo",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("kleinpanic93/canvas-calendar-agent-v7-dpo")

messages = [
    {"role": "system", "content": "You are a Canvas calendar agent. ..."},
    {"role": "user", "content": "What assignments do I have due this week?"},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
out = model.generate(inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=False))

The output will contain <|tool_call>...<tool_call|> delimiters that the canvas_sdk.tool_parser decodes into structured tool calls.

For end-to-end agent usage with auto-download:

bash

pip install canvas-sdk[autodownload]
python -m canvas_sdk.demo "Plan my study schedule for next week"

Data ethics + PII handling

Training data was anonymized through a two-pass process before inclusion:

CRN scrubbing: course identifiers replaced with stable @COURSE_n tokens.
PII pass: emails, phones, professor/student names, building/room references replaced with @PROF_EMAIL/@CONTACT_EMAIL/@PHONE/@PROFn/@STUDENTn/@LOCn. spaCy NER + custom regex.

The training data is the companion dataset.

Limitations

Only trained on 1,071 preference pairs and 181 trajectory rows. Coverage of diverse Canvas course structures is limited.
All training contributors used Virginia Tech instances of Canvas LMS — generalization to other institutions is untested.
Tool-call format is Gemma-4 specific. Will not work with chat templates that don't preserve <|tool_call>...<tool_call|> delimiters.
2.7B model — qualitative reasoning ability is below larger models. The 31B-IT teacher (used for labeling) is sometimes too good at distinguishing pairs the smaller policy cannot perfectly reproduce.

Citation

If you use this model, please cite:

bibtex

@article{rafailov2023dpo,
  title={Direct Preference Optimization: Your Language Model is Secretly a Reward Model},
  author={Rafailov, Rafael and Sharma, Archit and Mitchell, Eric and Ermon, Stefano and Manning, Christopher D and Finn, Chelsea},
  journal={NeurIPS},
  year={2023},
  url={https://arxiv.org/abs/2305.18290}
}

License

Apache 2.0 (matches Gemma-4 base license).

Project

Source: kleinpanic/CS3704-DPO-SSOT — training pipeline
Demo + SDK: kleinpanic/CS3704-Canvas-Project — agentic harness, GitHub Pages demo