Temporal span prediction using Google's T5Gemma architecture (270M parameters). Encoder-decoder model combining Gemma weights with T5 architecture. Demonstrates competitive performance with the much smaller flan-t5-small.

Training Details

Base Model: google/t5gemma-2-270m-270m
Architecture: T5ForConditionalGeneration
Dataset: Pieces/temporal-span-prediction-v-0-3-1-quality (41,927 train / ~5K val / ~5K test)
Training Steps: 50,000
Learning Rate: 1e-4
Batch Size: 8 × 4
Mixed Precision: bf16
Hardware: NVIDIA RTX 4090 (24 GB)

Results

Metric	Value
exact_match	~51.8%
tmr	54.35%
avg_iou	~65.7%

Usage

python

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained('Pieces/time-mapping-t5gemma-270m-best')
model = AutoModelForSeq2SeqLM.from_pretrained('Pieces/time-mapping-t5gemma-270m-best')
inputs = tokenizer('Map: tomorrow at 3pm | ref_time: 2026-01-15T10:00:00 | tz: UTC', return_tensors='pt')
outputs = model.generate(**inputs)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)

Part of the TIME-Module Project

This model is part of the TIME (Temporal Intent, Mapping, and Extraction) module, a suite of models for understanding and processing temporal information in natural language.

Related models:

Pieces/time-classification-flan-t5-small-split-best — Intent classification
Pieces/time-mapping-flan-t5-small-quality-best — Span prediction (best)
Pieces/time-mapping-t5gemma-270m-best — Span prediction (T5Gemma)

Citation

bibtex

@software{time_module,
  title={TIME-Module: Temporal Intent, Mapping, and Extraction},
  author={Pieces},
  year={2026},
  url={https://huggingface.co/Pieces}
}