Framework: nanoGPT (Karpathy's)
Dataset: SEC-EDGAR filings (financial disclosures, annual/quarterly reports)
Tokenizer: GPT-2 BPE (tiktoken)

Usage

python

from transformers import GPT2LMHeadModel, GPT2Tokenizer

model = GPT2LMHeadModel.from_pretrained("lzwjava/sec-edgar-gpt-124m-hf")
tokenizer = GPT2Tokenizer.from_pretrained("lzwjava/sec-edgar-gpt-124m-hf")

prompt = "The company reported total revenue of"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.8, top_k=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Intended Use

This model is trained for research and educational purposes — demonstrating nanoGPT training on domain-specific financial text. It is not suitable for production financial analysis or advice.

Limitations

Trained on a subset of SEC filings; may not generalize to all financial domains
No RLHF or instruction tuning — raw language model only
Generated text may contain factual inaccuracies