PEFT LoRA adapter trained with GRPO (group-relative policy optimization) from human yes/no feedback on robot emotion displays. This adapter sits on top of the SFT classifier:

annamanaseryan/cozmo-emotion-sft

What this is / is not

This repo	Not included here
LoRA weights (`adapter_model.safetensors`)	Full DeBERTa weights
`adapter_config.json`	SFT `best_model.bin` (required separately)

You need all three:

microsoft/deberta-v3-base (encoder backbone, from Hugging Face)
annamanaseryan/cozmo-emotion-sft (best_model.bin + tokenizer + EmotionClassifier)
This LoRA adapter

Input format (must match training)

text

HUMAN: {human_text}
ROBOT: {robot_text}

Labels (0–5)

Id	Description
0	anger_frustration
1	interest_desire
2	confusion_sorrow_boredom
3	joy_hope
4	understanding_gratitude_relief
5	disgust_surprise_alarm_fear

See emotion_labels.yaml.

LoRA config

r=8, lora_alpha=16, lora_dropout=0.05
Targets: query_proj, key_proj, value_proj
Checkpoint: training step 061

Install

pip install torch transformers peft pyyaml huggingface_hub

Usage (SFT + GRPO)

python

import sys
import torch
import yaml
from huggingface_hub import snapshot_download
from transformers import AutoTokenizer
from peft import PeftModel

SFT_REPO = "annamanaseryan/cozmo-emotion-sft"
GRPO_REPO = "annamanaseryan/cozmo-emotion-grpo"
device = "cuda" if torch.cuda.is_available() else "cpu"

sft_dir = snapshot_download(SFT_REPO)
grpo_dir = snapshot_download(GRPO_REPO)

sys.path.insert(0, sft_dir)
from modeling_emotion import EmotionClassifier

with open(f"{sft_dir}/emotion_labels.yaml") as f:
    cfg = yaml.safe_load(f)
id2desc = {int(i): d["description"] for i, d in cfg["emotions"].items()}

tokenizer = AutoTokenizer.from_pretrained(sft_dir)
base = EmotionClassifier(n_classes=6, model_name="microsoft/deberta-v3-base")
base.load_state_dict(torch.load(f"{sft_dir}/best_model.bin", map_location="cpu"))
model = PeftModel.from_pretrained(base, grpo_dir)
model.to(device).eval()

def predict(human: str, robot: str, max_length: int = 256):
    text = f"HUMAN: {human}\nROBOT: {robot}"
    inputs = tokenizer(
        text,
        return_tensors="pt",
        max_length=max_length,
        padding="max_length",
        truncation=True,
    ).to(device)
    with torch.no_grad():
        logits = model(**inputs)
        probs = torch.softmax(logits, dim=-1).cpu().squeeze()
    out = {id2desc[i]: float(probs[i]) for i in range(len(probs))}
    pred = max(out, key=out.get)
    return pred, out

label, dist = predict("Do you have empathy?", "i try to understand.")
print(label)
print(dist)

A copy of this script is in inference_example.py.