Skip to content
All tags

#information-retrieval

3 posts
aiguideReading NTHU Hung-Yu Kao Natural Language Processing

Why Language Is Hard, and How Text Became Numbers Before LLMs | NTHU Hung-Yu Kao Natural Language Processing Week 1

Week 1 of Hung-Yu Kao's NLP course at NTHU is a 91-page deck, W1_NLP_brief. It opens with 'Watch for kids', five readings of the telescope sentence, and a Chinese tongue-twister about eleven uncles to show why language is hard. Then, from an information-retrieval angle, it builds one pipeline: inverted index, tokenization, stemming, TF-IDF, BM25. The second half hits that pipeline's dead ends (synonyms, polysemy, vocabulary mismatch), moves to SVD-based LSA, and closes with a preview of dense vectors through Skip-gram, GloVe, and FastText.

#nthu-nlp#nthu#ai-course#nlp#information-retrieval#tf-idf#bm25#word2vec
aideep-diveReading Stanford CS124 From Languages to Information

Information Retrieval: The Indexing and Ranking Layer Beneath RAG | Stanford CS124 Week 4

Week 4 teaches search: turn the query and the documents into vectors, weight them with tf-idf, rank by cosine similarity, speed it up with an inverted index, evaluate with precision, recall and MAP, and finally connect to dense retrieval and RAG. This post works the slides' four-document example in full.

#cs124#stanford#information-retrieval#rag#nlp
aideep-divePrivate Corpus Pipeline

Private Corpus Search Boundaries: Decide Where Data May Go First

The first private-corpus decision is not which vector database to buy. Define data classes, trust zones, policy enforcement points, and freshness SLAs so every index, model, and observability system receives only the minimum data it is allowed to process.

#rag#information-retrieval#data-security#access-control#qdrant#meilisearch