prajwalmani/

t5-small-trivia-qa

Base Model: t5-small (60 million parameters)

Task: Abstractive Question Answering (short-form trivia)

Training Data: ~20,000 samples from TriviaQA

Expected Output: Answers are typically 1–3 words

Epochs: 3

Batch Size: 16

Hardware: NVIDIA GTX 1050 (4GB VRAM)

Limited Factual Memory: The t5-small model is not large enough to store a vast amount of "world knowledge."

Small Dataset: Training on only 20k examples is insufficient for the model to learn facts it hasn't seen.

No Retrieval: This is a standard (non-RAG) model. It cannot "look up" answers from an external source like Google or Wikipedia.

Potential for Hallucination: The model may guess or provide a confident-sounding but incorrect answer, especially for questions outside its training data.

Implement RAG: Add a retrieval-augmented generation (RAG) pipeline. This would allow the model to search a knowledge base (e.g., Google, Wikipedia) for relevant context before formulating an answer.

Use a Larger Model: Start with a more capable base model, such as Flan-T5-Large, Flan-T5-XL, or a modern decoder-based model (e.g., Mistral, Llama 3).

Use the Full Dataset: Train on the complete TriviaQA dataset.

Prompt Engineering: Use stricter, more detailed prompts to force the model to generate only short, precise answers.