You can deploy this model and start earning money today!

igmoiiz/
ResearchMate-Qwen2.5-3B

Name: ResearchMate-Qwen2.5-3B
Availability: InStock

Views

No views yet

ResearchMate-Qwen2.5-3B

ResearchMate-Qwen2.5-3B is a domain-specialized Large Language Model designed to assist researchers, students, and practitioners with scientific literature.

The model is instruction fine-tuned from Qwen2.5-3B-Instruct using QLoRA and Unsloth. Instead of functioning as a general-purpose chatbot, ResearchMate focuses on understanding scientific papers and responding to research-oriented instructions.

This release represents Version 1 of the ResearchMate project.

Model Overview

Property	Value
Model	Qwen2.5-3B-Instruct
Fine-tuning	QLoRA
Framework	Unsloth
Parameter Count	3B
Quantization	4-bit
PEFT	LoRA
Primary Domain	Scientific Literature
Language	English
Version	1.0

Project Goal

ResearchMate aims to provide a lightweight, open-source scientific assistant capable of understanding research papers and responding to academic instructions.

The objective is not to replace Retrieval-Augmented Generation systems or search engines, but to improve a language model's understanding of scientific writing through supervised instruction fine-tuning.

Version 1 focuses on instruction tuning without retrieval.

Supported Tasks

ResearchMate has been instruction-tuned for several scientific literature tasks including:

Scientific Question Answering
Paper Summarization
Abstract Explanation
Beginner-Friendly Concept Explanation
Keyword Extraction
Research Field Identification
Scientific TL;DR Generation
Contribution Identification
Method Identification

Training Method

The model was fine-tuned using:

Unsloth
QLoRA
PEFT LoRA
4-bit Quantization

Training was performed on Kaggle GPUs to reduce memory requirements while maintaining strong instruction-following capabilities.

Dataset Construction

ResearchMate does not train directly on raw datasets.

Instead, a dedicated dataset-building pipeline converts multiple scientific sources into a unified instruction dataset.

The preprocessing pipeline is independent of model training.

Dataset pipeline:

Scientific Dataset

↓

Validation

↓

Schema Conversion

↓

Cleaning

↓

Merge

↓

JSONL Export

↓

Training

Dataset Schema

Every training example follows the same structure:

json

{
    "instruction": "...",
    "input": "...",
    "output": "...",
    "source": "...",
    "task": "..."
}

This unified schema allows new scientific datasets to be added without modifying the training pipeline.

Data Sources

Version 1 uses the following sources:

PubMedQA

Purpose:

Scientific Question Answering

Configuration:

pqa_labeled

Fields used:

Question
Context
Long Answer

arXiv

Scientific papers collected through the official arXiv API.

Paper metadata including titles, abstracts and categories were converted into instruction-response pairs for research-oriented tasks.

Training Configuration

Parameter	Value
Fine-tuning Method	QLoRA
PEFT	LoRA
Quantization	4-bit
Framework	Unsloth
Optimizer	AdamW
Epochs	2
Batch Size	2
Gradient Accumulation	4
Sequence Length	2048

Evaluation

The model was evaluated against the original Qwen2.5-3B-Instruct model.

Evaluation included:

Scientific QA
Response quality inspection
ROUGE (where applicable)
Latency comparison
Hallucination inspection
Task-wise qualitative comparison

Example Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "igmoiiz/ResearchMate-Qwen2.5-3B"

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto"
)

messages = [
    {
        "role": "user",
        "content": "Explain transfer learning in simple language."
    }
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=256
)

print(
    tokenizer.decode(
        outputs[0][inputs["input_ids"].shape[1]:],
        skip_special_tokens=True
    )
)

Example Prompts

Scientific Question Answering

Answer the scientific question using the provided context.

Question:
What is transfer learning?

Summarization

Summarize the following scientific abstract.

Beginner Explanation

Explain this abstract in simple language.

Keyword Extraction

Extract the important keywords from this paper.

Research Field Identification

Identify the research field of this paper.

Intended Uses

Suitable for:

Research assistants
Literature exploration
Educational tools
Scientific tutoring
Academic chatbots
Research paper preprocessing
Scientific writing assistance

Limitations

ResearchMate Version 1 has several limitations.

No Retrieval-Augmented Generation (RAG)
No citation verification
No PDF parsing
No web search
No factual verification beyond the model's learned parameters
Performance depends on the quality and diversity of the instruction dataset
May generate incorrect or outdated scientific information

Users should verify important scientific claims using authoritative sources.

Future Work

Planned improvements include:

Retrieval-Augmented Generation (RAG)
Citation-aware responses
PDF ingestion
Larger scientific datasets
Multi-turn research conversations
Paper recommendation
Local deployment using Ollama and vLLM
Improved evaluation benchmarks
Expanded scientific domains

Repository Structure

Notebook 1
Dataset Builder

↓

Notebook 2
QLoRA Fine-tuning

↓

Notebook 3
Inference

↓

Notebook 4
Evaluation

Citation

If you use this model in your work, please cite the repository.

@software{researchmate2026,
  title={ResearchMate-Qwen2.5-3B},
  author={Moiz Baloch},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/igmoiiz/ResearchMate-Qwen2.5-3B}
}

Acknowledgements

This project builds upon the work of:

Alibaba Qwen Team
Unsloth AI
Hugging Face
PubMedQA
arXiv

Their open-source contributions made this project possible.

Contact

Author: Moiz Baloch

GitHub: https://github.com/igmoiiz

Hugging Face: https://huggingface.co/igmoiiz