This model can be used for mathematical reasoning and problem-solving tasks. It specializes in step-by-step mathematical problem solving and educational assistance.

Downstream Use

Can be used for various mathematical and reasoning applications including:

Mathematical problem solving
Step-by-step reasoning
Educational math assistance
Complex mathematical calculations

Bias, Risks, and Limitations

This model inherits the biases and limitations of the base Phi-4 model. Users should be aware of:

Potential generation of biased or harmful content
Hallucination in factual information
Performance may vary on different types of prompts
Mathematical reasoning accuracy may vary with problem complexity

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.

Use appropriate safety filtering for production applications
Validate mathematical solutions generated by the model
Consider the model's training data and potential biases
Test thoroughly on your specific use case

How to Get Started with the Model

Use the code below to get started with the model.

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "eastbrick/ank-fv1",
    torch_dtype=torch.float16,
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained("eastbrick/ank-fv1")

# Prepare input
prompt = "What is the integral of x^3 + 2x^2 - 5x + 3?"#example question
inputs = tokenizer(prompt, return_tensors="pt")

# Generate
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=256,#can be kept anything based on the input
        do_sample=True,
        pad_token_id=tokenizer.eos_token_id
    )

# Decode response
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Training Details

Training Data

This model was fine-tuned on a filtered subset of the DeepMath-103k dataset, containing approximately 52,000 harder mathematical problems selected for increased difficulty and complexity.

Training Procedure

Preprocessing

The data was filtered based on difficulty level of the questions in order to take only quality questions which can be challenging to solve specifically for phi-4.

Training Hyperparameters

Training regime: LoRA (Low-Rank Adaptation) fine-tuning
Base model: Phi-4 14B
Training time: ~12 hours
Final training loss: ~0.40
Framework: Transformers + PEFT
Training method: LoRA fine-tuning with subsequent merge

Speeds, Sizes, Times

Training time: 12 hours
Hardware: NVIDIA RTX 4090
Model size: ~14B parameters
Training dataset size: 52,000 mathematical problems

Evaluation

35% improvement in accuracy on top of base model in the AIME-2025 questions
Better step by step solutions.

Technical Specifications

Model Architecture and Objective

This is a fully merged model with the following specifications:

Model size: ~14B parameters
Specialization: Mathematical reasoning and problem-solving
Base architecture: Phi-4 14B
Context length: 16384 tokens

Compute Infrastructure

Hardware

Training: NVIDIA RTX 4090
Power consumption: ~450W average during training
Total energy: ~5.4 kWh

Software

Framework: PyTorch + Transformers + PEFT
Training: LoRA fine-tuning framework

Citation

BibTeX:

bibtex

@misc{ank-v1-2025,
  title={ANK-v1: Fine-tuned Phi-4 14B with LoRA for Mathematical Reasoning},
  author={eastbrick},
  year={2025},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/eastbrick/ank-fv1}}
}

APA:

eastbrick. (2025). ANK-fv1: Fine-tuned Phi-4 14B with LoRA for Mathematical Reasoning. Hugging Face. https://huggingface.co/eastbrick/ank-v1

Model Card Authors

eastbrick

This model card was generated to provide transparency about the model's capabilities, limitations, and intended use cases.