FedInsight is a complete RAG (Retrieval-Augmented Generation) project that lets you "talk" to Federal Reserve documents. Ask questions in natural language and get accurate quotes + synthesized answers with sources.
Why this is useful
Analysts and traders want to quickly find what the Fed said about inflation, interest rates, or the labor market in specific periods.
Researchers study the evolution of monetary policy rhetoric over time.
It's just interesting: "What did the Fed say about the 2023 banking crisis?"
Current capabilities
✅ Full FOMC Minutes scraper (with metadata, resume support, logging)
✅ PDF parsing → clean text + metadata (PyMuPDF, column-aware)
✅ Vector database: ChromaDB (local default) or pgvector (PostgreSQL)
✅ RAG pipeline (retrieval + generation with source attribution)
✅ Streamlit chat interface
✅ Fully local operation (no mandatory API keys)
🔄 Beige Book scraper (planned)
Quick start
bash
1git clone https://github.com/murai1998/FedInsight.git
2cd FedInsight
34python -m venv .venv
5source .venv/bin/activate # or .venv\Scripts\Activate.ps1 on Windows (PowerShell)67pip install -r requirements.txt
NVIDIA GPU (optional but recommended): the default torch wheel is CPU-only.
For GPU acceleration install the matching CUDA build, e.g. for an RTX 50-series
(Blackwell) card:
The embedder auto-detects CUDA and uses it when available.
Usage — the full pipeline, step by step
Run everything from the project root with the virtualenv activated (so the
correct Python/torch is used). On Windows you can also prefix commands with
.\.venv\Scripts\python.exe instead of activating.
Optional configuration lives in .env (copy from .env.example):
cp .env.example .env # then edit as needed (vector backend, LLM provider, keys)
1. Start the vector database (pgvector, the default backend)
docker compose up -d # PostgreSQL + pgvector on localhost:5432
2. Load the knowledge base (fast path — no scraping)
The repo ships a prebuilt Chroma store (~4,075 FOMC chunks) but no source PDFs,
so the quickest way to a working knowledge base is to copy those vectors into
pgvector:
1python scripts/ask.py "How did the Committee view inflation risks in 2022?"2python scripts/ask.py "What was decided about the federal funds rate?" --k 83python scripts/ask.py "Balance sheet plans" --provider ollama
The LLM provider is set by LLM_PROVIDER (default auto): OpenAI if
OPENAI_API_KEY is set, else a local Ollama server if reachable, else none
(prints the most relevant excerpts — works fully offline, no keys required).
5. Chat UI (Streamlit)
streamlit run app.py
Then open the URL it prints (usually http://localhost:8501). Pick the backend,
LLM provider, and number of retrieved chunks in the sidebar.
PDF parsing: PyMuPDF (column-aware), with pypdf / pdfplumber as fallbacks
Embeddings: sentence-transformers (all-MiniLM-L6-v2, auto CUDA)
Vector DB: pgvector (PostgreSQL, default via docker compose) or ChromaDB (server-less)
LLM: OpenAI, local Ollama, or offline excerpts-only fallback
UI: Streamlit (deployed to Hugging Face Spaces via Docker)
Logging: loguru
Development philosophy
We follow a working skeleton first approach:
Build a minimal but complete end-to-end pipeline (scrape → parse → chunk → embed → store → retrieve → generate)
Then iteratively improve quality, features, and robustness.
Roadmap
See plan.md for the detailed, prioritized next steps. In short:
retrieval-quality evaluation, hybrid search + reranking, richer metadata
filtering in the UI, additional document types (Beige Book, speeches, SEP),
and a test suite.
Contributing
Issues and pull requests are welcome at
github.com/murai1998/FedInsight.
Keep everything in English; lint with python -m ruff check src scripts app.py.
Made with love for monetary policy and clean code ❤️
A full RAG + Vector DB system over Federal Reserve communications.
