A Fine-tuned Phi-2 Model Optimized for STEM Knowledge
Science, Technology, Engineering, Mathematics, and Ethics
Model Description
PHI-2-STEM-261125 is a fine-tuned version of Microsoft's Phi-2 (2.78B parameters) specifically optimized for generating accurate and comprehensive explanations across multiple STEM domains. The model was trained using INT8 quantization to enable efficient training on consumer-grade GPUs.
Key Features
Multi-domain STEM expertise: Mathematics, Physics, Chemistry, Biology, and Ethics
Efficient training: INT8 quantization enables training on 4GB VRAM GPUs
Epoch 1: Loss ~2.07 (initial)
Epoch 2: Loss ~1.85
Epoch 3: Loss ~1.65
Epoch 4: Loss ~1.58
Epoch 5: Loss ~1.54 (final)
Dataset
Overview
The model was trained on a curated dataset of 18 expert-written examples covering 11 specialized STEM domains. Each example provides a concise, technically accurate explanation of fundamental concepts.
Domain Distribution
Domain
Examples
Topics Covered
Mathematics
3
Fundamental Theorem of Calculus, Riemann Hypothesis, Gödel's Incompleteness Theorems
Organic Chemistry
2
SN2 Reaction Mechanism, Molecular Orbital Theory (Benzene)
1# Good prompts2prompts =[3"Explain the Fundamental Theorem of Calculus:",4"What is the SN2 reaction mechanism in organic chemistry?",5"Describe quantum entanglement:",6"What are the four principles of bioethics?",7"Explain Kant's Categorical Imperative:",8]910# Structured format (also works well)11prompt ="""Question: What is the Heisenberg Uncertainty Principle?
12Answer:"""
Example Outputs
Quantum Physics
Prompt: "Explain quantum entanglement:"
Output:
Quantum entanglement describes correlations between particles that persist regardless of spatial separation. When particles are entangled, measuring one particle's state instantly determines the other's state. This phenomenon, verified experimentally through Bell inequality violations, challenges classical notions of locality and has applications in quantum computing and cryptography.
Organic Chemistry
Prompt: "What is the SN2 mechanism?"
Output:
The SN2 (bimolecular nucleophilic substitution) reaction proceeds through a single concerted step. The nucleophile attacks the electrophilic carbon from the backside, opposite to the leaving group. This results in inversion of stereochemistry (Walden inversion). The reaction is favored with primary substrates, good leaving groups, strong nucleophiles, and polar aprotic solvents.
Ethics
Prompt: "What is Kant's Categorical Imperative?"
Output:
Kant's Categorical Imperative is a fundamental principle in deontological ethics. It states: "Act only according to that maxim whereby you can at the same time will that it should become a universal law." This means moral actions must be universalizable - if everyone acted the same way, the action should remain coherent and beneficial. It emphasizes duty and rational moral principles over consequences.
Intended Use
Primary Use Cases
Educational Content Generation: Creating explanations of STEM concepts for learning materials
Research Assistance: Generating initial drafts of scientific explanations
Tutoring Systems: Providing explanations in AI-assisted learning platforms
Knowledge Retrieval: Answering questions about fundamental STEM concepts
Content Augmentation: Enhancing educational content with accurate explanations
Target Users
Educators and teachers
Students (undergraduate and graduate level)
Science communicators
EdTech developers
Researchers exploring LLM capabilities in STEM
Limitations
Known Limitations
Small Training Dataset: Only 18 examples, limiting coverage of STEM topics
Domain Specificity: Best performance on topics similar to training data
No Real-time Information: Knowledge cutoff based on base model (Phi-2)
Mathematical Reasoning: May struggle with complex mathematical derivations
Hallucination Risk: May generate plausible-sounding but incorrect information
Language: English only
Out-of-Scope Use Cases
Medical diagnosis or treatment recommendations
Legal advice
Financial decisions
Safety-critical applications
Generating content presented as human-written without disclosure
Recommendations
Always verify generated content against authoritative sources
Use as a starting point, not as definitive truth
Human review required for any published or educational content
Not suitable for generating content on topics outside training domains
Ethical Considerations
Bias and Fairness
The model inherits biases from the base Phi-2 model and training data
Training data reflects Western academic perspectives on STEM
Limited representation of non-Western scientific traditions
Environmental Impact
Training was performed on consumer hardware (RTX 3050)
Estimated carbon footprint: ~0.5 kg CO2 (30 minutes on 75W GPU)
