Temporal span prediction using decoder-only Gemma-3-270M. While encoder-decoder models (flan-t5-small) outperform this model on temporal span prediction, this demonstrates the capability of small decoder-only models on structured generation tasks.

Training Details

Base Model: google/gemma-3-270m
Architecture: GemmaForCausalLM
Dataset: Pieces/temporal-span-prediction-v-0-3-1-quality (41,927 train / ~5K val / ~5K test)
Training Steps: 95,000
Learning Rate: 3e-4
Batch Size: 4 × 8
Mixed Precision: bf16
Hardware: NVIDIA RTX 4090 (24 GB)

Results

Metric	Value
exact_match	~39.5%
tmr	42.50%
avg_iou	~58.8%

Usage

python

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained('Pieces/time-mapping-gemma3-270m-best')
model = AutoModelForSeq2SeqLM.from_pretrained('Pieces/time-mapping-gemma3-270m-best')
inputs = tokenizer('Map: tomorrow at 3pm | ref_time: 2026-01-15T10:00:00 | tz: UTC', return_tensors='pt')
outputs = model.generate(**inputs)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)

Part of the TIME-Module Project

This model is part of the TIME (Temporal Intent, Mapping, and Extraction) module, a suite of models for understanding and processing temporal information in natural language.

Related models:

Pieces/time-classification-flan-t5-small-split-best — Intent classification
Pieces/time-mapping-flan-t5-small-quality-best — Span prediction (best)
Pieces/time-mapping-t5gemma-270m-best — Span prediction (T5Gemma)

Citation

bibtex

@software{time_module,
  title={TIME-Module: Temporal Intent, Mapping, and Extraction},
  author={Pieces},
  year={2026},
  url={https://huggingface.co/Pieces}
}