Bigram Character-Level Language Model: Makemore (Part 1)
Introduced to the concept of a bigram character-level language model, this repository explores its training, sampling, and evaluation processes. The model evaluation was conducted using the Negative Log Likelihood (NLL) loss to assess its quality.
Overview
The model was trained in two distinct ways, both yielding identical results:
Frequency-Based Approach: Directly counting and normalizing bigram frequencies.
Gradient-Based Optimization: Optimizing the counts matrix using a gradient-based framework guided by minimizing the NLL loss.
This demonstrated that both methods converge to the same result, showcasing their equivalence in achieving the desired outcome.
