Skip to content
All tags

#gpt-2

3 posts
aiguideReading NTHU Hung-Yu Kao Natural Language Processing

NTHU NLP Guide 11: GPT-2 vs. T5 for Chinese Summarization — Left Padding, −100, and ROUGE | NTHU Hung-Yu Kao Natural Language Processing Topic Guide

A guide to the GPT-2 / T5 tutorial in NTHU Prof. Hung-Yu Kao's NLP course (Fall 2025). One task, LCSTS Chinese summarization, is solved twice. Decoder-only GPT-2 is written in native PyTorch: you join article and summary into one sequence, switch to left padding, and set padding labels to −100. Encoder-decoder mT5 uses Seq2SeqTrainer: no left padding needed, and DataCollatorForSeq2Seq handles the −100 for you. Both segment with jieba and score word-level ROUGE.

#nthu-nlp#ai-course#course-guide#taiwan#gpt-2#hugging-face#fine-tuning#chinese-nlp#pytorch#nlp
aideep-diveReading CMU 07-280 AI & ML I

From Position Encoding to Causal Self-Attention | CMU 07-280 Lecture 20

Lecture 20 expands one-token embeddings into sequences, adds positional information, derives Q/K/V scaled dot-product attention and causal masking, and assembles multi-head blocks into a GPT-2 skeleton.

#cmu#ai-course#transformer#attention#gpt-2
aideep-diveReading CMU 07-280 AI & ML I

Stage Review II: Building AlexNet and GPT-2 as Working Systems | CMU 07-280 Topic Guide

Stage II uses HW8 and HW11 to test whether representation, computation graphs, training, transfer, and generation actually connect, rather than treating CNNs and Transformers as diagrams to memorize.

#cmu#ai-course#alexnet#gpt-2#deep-learning