Deep Learning Interview Guide: Core Intuitions from CNN to Transformer

Deep learning interviews shifted noticeably after 2025: interviewers increasingly ask you to explain "why it's designed this way" and "when it breaks" rather than having you hand-derive backpropagation or write out LSTM gate equations. This post covers the design intuitions needed in interviews, not textbook knowledge.

CNN: Locality and Translation Invariance

The core assumption of convolutional neural networks is locality — relationships between adjacent pixels matter more than distant ones. A 3×3 kernel only looks at a local region, keeping parameters far fewer than a fully connected layer.

The most common intuition question in interviews is: "Why are CNNs effective for images?" A good answer hits two points. First, locality — meaningful patterns in images (edges, textures, object parts) tend to be local. Second, translation invariance — the same kernel slides across the entire image, detecting objects regardless of where they appear.

Pooling does more than just dimensionality reduction. Max pooling extracts the strongest signal in a local region and provides tolerance to small shifts. But pooling also discards precise spatial position information — this is why semantic segmentation (requiring pixel-level localization) later replaced pooling with dilated convolution or encoder-decoder architectures.

Architecture evolution: LeNet → AlexNet (depth + ReLU + Dropout) → VGGNet (uniform 3×3 kernels) → GoogLeNet/Inception (multi-scale parallel convolutions) → ResNet (skip connections solving degradation) → EfficientNet (compound scaling). Interviews don't require memorizing every architecture's details, but you need to explain why ResNet's skip connections work — they allow gradients to flow directly to shallow layers, solving the degradation problem in deep networks (note: degradation, not overfitting — deep networks had higher training loss than shallow ones).

RNN and LSTM: First-Generation Sequence Modeling

RNN's design motivation is straightforward: the input is a sequence, and each step's hidden state depends on both the current input and the previous state, forming an information chain along the time axis. The problem is that this chain breaks when it gets too long.

Vanishing gradients: When backpropagation unrolls along the time axis, gradients are multiplied by many weight matrices. If the spectral radius is less than 1, gradients decay exponentially, making early inputs virtually invisible to the loss. Exploding gradients are the opposite — spectral radius greater than 1 causes exponential gradient growth. Exploding gradients can be brutally fixed with gradient clipping; vanishing gradients require architectural changes.

LSTM's solution introduces three gates (forget, input, output) and a cell state channel. Cell state updates use addition (not multiplication), allowing gradients to flow long distances along the cell state without decaying. Common interview follow-up "Why do gates use sigmoid?" — because sigmoid outputs range from 0 to 1, corresponding to a continuous control from "completely forget" to "completely retain."

GRU is a simplified LSTM — merging the forget and input gates into an update gate, meaning fewer parameters. When asked "How do you choose between LSTM and GRU?" in an interview, the practical answer is: differences are usually small, GRU trains slightly faster, LSTM has a slight edge on extremely long sequences, but these differences are typically smaller than hyperparameter tuning effects.

Transformer: Why It Replaced RNN

The 2017 "Attention Is All You Need" paper proposed Transformer to solve two RNN problems: inability to parallelize (each step depends on the previous, preventing parallel computation along the sequence dimension) and long-range dependencies (even with LSTM, information transfer across very long sequences is still insufficient).

Self-attention computation flow, as you'd explain it in an interview:

1. Each token gets three vectors — Query, Key, and Value — through three linear projections
2. Query dot-products with all Keys to get attention scores (measuring "how much should this token attend to each other token")
3. Scores are divided by √d_k (square root of key dimension) for scaling, preventing large dot products from saturating softmax
4. Softmax produces weights, which are used for a weighted sum over Values

Multi-head attention splits Q, K, V into multiple groups (heads), each attending to different subspaces, then concatenated. When asked "Why multiple heads?" — a single head can only learn one attention pattern; multiple heads let the model simultaneously capture different types of dependencies (syntactic, semantic, positional, etc.).

Positional encoding exists because self-attention is inherently permutation invariant — it only sees pairwise token relationships without knowing ordering. The original Transformer used fixed sinusoidal functions; later models mostly switched to learnable positional encoding or RoPE (rotary position embedding). Interview bonus: mentioning that RoPE encodes relative position information directly in the attention computation, generalizing better to different sequence lengths than absolute positional encoding.

Self-attention's computational complexity is O(n²d), where n is sequence length and d is dimension. This is why long sequences (beyond tens of thousands of tokens) need special handling — FlashAttention uses tiled computation to reduce memory access without changing complexity but dramatically improving actual speed.

Normalization: BatchNorm vs LayerNorm vs RMSNorm

BatchNorm normalizes across the batch dimension — computing the mean and variance of all samples at the same feature within a batch. Problems: statistics become unstable with small batch sizes; inference requires running statistics accumulated during training, risking train/eval behavioral inconsistency.

LayerNorm normalizes across the feature dimension — computing mean and variance of all features within a single sample. Independent of batch size, making it the standard in Transformers (where batch size can be small or dynamic).

RMSNorm is a simplified LayerNorm — using only RMS (root mean square) for scaling, skipping the mean computation. LLaMA uses RMSNorm; in practice, performance is comparable to LayerNorm but computationally faster.

Must-know interview follow-up: "When would you use BatchNorm? When LayerNorm?" — CNN + large batch → BatchNorm; Transformer or RNN → LayerNorm; large language models optimizing for efficiency → RMSNorm.

Training Tricks

When asked "What training tricks do you use?", don't just list terms. Pick 3-4 you've actually used, explain why, and when you wouldn't use them.

Learning rate schedule: Warmup + cosine decay is the most common combination. Warmup lets the model use a small learning rate early to stabilize update directions, avoiding large learning rates pushing randomly initialized parameters into bad regions. Cosine decay is smoother than step decay, typically converging more stably.

Gradient clipping: Sets a maximum gradient norm (usually 1.0); if exceeded, scales proportionally. Primarily prevents gradient explosion — nearly mandatory for training RNNs and large Transformers.

Mixed precision training: Uses FP16 for forward and backward passes to accelerate computation, but maintains FP32 master weights for parameter updates to preserve precision. Combined with loss scaling to prevent FP16 underflow. PyTorch's torch.amp makes this nearly automatic.

Data augmentation: In CV, random crop, flip, and color jitter are practically free lunch. In NLP, augmentation is more nuanced — back-translation, token dropout, and synonym replacement vary in effectiveness by task and can't be blindly applied.

Common Follow-ups and Answer Strategy

When answering technical follow-ups, avoid giving conclusions only. Interviewers want to hear your thinking process: first state the tradeoffs you'd consider, then give your judgment, and finally add when you'd change that judgment.

What's Next

The next post covers NLP & LLM — from tokenization, fine-tuning to RLHF and LLM evaluation, organizing answer frameworks for large language model interview questions.

Practice Question

Question

"You need to design a document classification system. Input is business documents of 500-5000 words, output is one of 20 categories. Would you choose CNN, RNN, or Transformer? Why?"

Source: Meta MLE onsite　Difficulty: Advanced　Round: onsite ML deep dive

Solution Framework

1. Clarify the problem: Ask the interviewer — how large is the dataset? What are the inference latency requirements? Any budget constraints? Do you need to explain classification results?
2. Build a framework: Compare across three dimensions — sequence modeling capability (long-range dependencies), training efficiency, inference cost.
3. Go deep: Document length of 500-5000 words is key — RNNs have vanishing gradient issues on long sequences, CNNs can handle them via pooling but lose sequential information, Transformers have the O(n²) bottleneck but can leverage pretrained models (BERT/RoBERTa) for fine-tuning.
4. Close: Give a concrete recommendation with tradeoffs — if you have enough data and GPUs, fine-tuning a pretrained Transformer is strongest; if resources are limited, a distilled model or CNN + attention is a practical compromise.

Sample Answer (as you'd say it in an interview)

I'd choose Transformer — specifically, fine-tuning a pretrained RoBERTa-base. Three reasons. First, document lengths up to 5000 words require capturing long-range dependencies — keywords introduced at the beginning may only connect to decisive statements at the end. RNNs suffer gradient decay at this length; LSTM improves this but still can't match self-attention directly. Second, a pretrained model brings built-in language understanding — 20-category classification can achieve good results with limited labeled data. Third, RoBERTa-base has only 125M parameters, making fine-tuning cost manageable.

The main tradeoff is sequence length limitation. RoBERTa's max length is 512 tokens; a 5000-word document tokenizes to roughly 1500-2000 tokens, exceeding the limit. Two solutions: truncation with a sliding window approach, classifying each window then aggregating; or switching to Longformer, which uses sliding window attention to reduce complexity to O(n) with a 4096 max length. I'd choose Longformer if latency isn't a bottleneck; truncation strategy if latency needs to be compressed.

If the interviewer follows up with "Would CNN work?" — TextCNN is fastest with lowest inference cost and performs reasonably on short text (<500 words), but on long documents the pooling layer loses global sequential information, typically 5-10 percentage points below Transformer accuracy. If latency is a hard constraint (<5ms), CNN is worth considering.

Self-Check Rubric

References

• Attention Is All You Need (Vaswani et al., 2017) — The original Transformer architecture paper; the basis for self-attention computation and multi-head attention design
• Deep Residual Learning for Image Recognition (He et al., 2016) — ResNet and skip connections; explains the source and solution of the deep network degradation problem
• FlashAttention: Fast and Memory-Efficient Exact Attention (Dao et al., 2022) — Reduces attention's practical memory and compute costs through tiled computation and IO-aware design
• Dive into Deep Learning — CNN/RNN/Transformer Chapters — Interactive deep learning textbook covering CNN, RNN, and Transformer architectures with attention mechanism computation flows
• Layer Normalization (Ba et al., 2016) — The original LayerNorm paper; theoretical basis for the BatchNorm vs LayerNorm selection logic in deep learning interviews