newnet — Neural Network Engine from Scratch in C++

Quick Start

Architecture

newnet/
├── core/
│   ├── tensor.hpp      ← Tensor class (data + grad + shape)
│   └── backend.hpp     ← All math ops (matmul, relu, sigmoid, etc.)
│                          Swap THIS file for SYCL/GPU backend.
├── layers/
│   ├── layer.hpp       ← Abstract base class
│   └── dense.hpp       ← Fully connected layer (forward + backward)
├── graph/
│   ├── graph.hpp       ← Sequential model (chains layers)
│   └── optimizer.hpp   ← SGD optimizer
├── loss/
│   └── loss.hpp        ← MSE loss
└── main.cpp            ← XOR training example with progress bar

Design Principles

1. Loose coupling: Backend is one file. Swap it for SYCL/AdaptiveCpp without touching neural network code.
2. No dependencies: Compiles with plain g++. No cmake, no libraries.
3. Every line readable: Written for humans learning how NNs work internally.
4. Hardcoded derivatives: Each layer knows its own gradient formula. No autograd, no symbolic math.

Backend Abstraction

• matmul (multi-threaded)
• relu_forward / relu_backward
• sigmoid_forward / sigmoid_backward
• transpose, add, scale, sum_columns

Roadmap

• Tensor class
• Dense layer with forward/backward
• SGD optimizer
• MSE loss
• XOR training (proof of concept)
• MNIST training
• Skip connections (ResNet-style)
• Conv2D layer
• Adam optimizer
• L-BFGS (second-order optimizer)
• SYCL/AdaptiveCpp backend
• Vision Transformer

Papers & References

• Backpropagation: Rumelhart et al., 1986
• Xavier initialization: Glorot & Bengio, 2010
• Adam: Kingma & Ba, 2014
• ResNet skip connections: He et al., 2015

License

Generated by ML Intern

• Try ML Intern: https://smolagents-ml-intern.hf.space
• Source code: https://github.com/huggingface/ml-intern

Usage