This is a from-scratch implementation of a High-Performance Deep Learning Engine.
It bypasses standard frameworks (PyTorch/TensorFlow) to use a custom Virtual Memory Manager (VMM) and a Tape-based Autograd system.
Key Features:
Zero-Copy Autograd: Minimal memory overhead via pre-allocated VMM buffers.
Flash-Attention-2: Tiled attention implementation for $O(N)$ memory efficiency.
RoPE: Rotary Positional Embeddings for infinite sequence context.
LoRA Adaptation: Intelligence contained in a ~120KB Low-Rank weight matrix.
Architecture
Designed to surpass the CMU 11-785 (Deep Learning) curriculum within a 10-day execution window.
