This dataset contains csv's from the Lex Fridman podcast transcripts provided by Whispering-GPT.
I split the episode transcripts into parent and child chunks for use with RAG.
The parent chunks is size 500 and the child is size 50. The children come with embeddings using OpenAI text-embedding-3-small with 1024 dimensionality.
This was designed for use with ParentDocumentRetriever or LlamaIndex.
It should provide better retrievals for queries on… See the full description on the dataset page:
https://huggingface.co/datasets/jaiw/lex_fridman_podcast_embeddings.
