Modern CNN Architectures — ResNet, EfficientNet, ConvNeXt | MIT 6.7960 Topic Guide

🌏 中文版

Video status: Related supplementary video included; the original lecture recording has not been verified. Source details

Material Version: Based on MIT 6.7960 Fall 2024 OCW. Videos, slides, assignments fully public on MIT OCW.

TL;R

ResNet skip connections solve degradation; EfficientNet compound scales; ConvNeXt absorbs Transformer design to reclaim CV crown.

Background / Course Context

This topic guide follows Lecture 4 (CNN architectures); it was once labelled L06, which is unrelated to OCW Lec 06, Generalization Theory. Lecture 4 established basic CNN; this post enters modern architectures, exploring "how to stack networks deeper, more efficient, more accurate" — directly affecting which backbone you choose when training your own models.

Video: OCW has no "Modern CNN" lecture. The embedded Lec 04 Architectures: Grids covers ResNet residual connections only in its later part (about 60–70 min) and says nothing about MobileNet, EfficientNet or ConvNeXt. Slides: OCW has no slide deck called "Lecture 6: Modern CNN"; the old link was removed.

Core Concept Comparison

Method Architecture

1. ResNet Residual Connection

x → [Conv-BN-ReLU] → [Conv-BN] → + (skip x) → ReLU
                              ↑
                          identity shortcut

• Learns residual $F(x) = H(x) - x$, optimizing "increment" instead of "absolute mapping"
• Degradation problem: plain nets deepen → training error rises; ResNet lets worst case degrade to identity (copy previous layer)

2. Depthwise Separable Convolution

Standard conv:  C_in × C_out × k × k params
Separable:      (C_in × k × k) + (C_in × C_out × 1 × 1) params
               compute reduced ~ k² × C_out / (1 + k²)

• MobileNet: 3×3 depthwise + 1×1 pointwise
• Example: 64 in-channels, 128 out, kernel 3×3
  • Standard: $64 \times 128 \times 9 = 73728$
  • Separable: $64 \times 9 + 64 \times 128 = 8768$ (8.4× savings)

3. EfficientNet Compound Scaling

$$\text{depth}: d=\alpha^\phi, \quad \text{width}: w=\beta^\phi, \quad \text{resolution}: r=\gamma^\phi$$ Constraint $\alpha \cdot \beta^2 \cdot \gamma^2 \approx 2$ (fixed total compute)

• B0→B7: $\phi$ from 0 to 7, systematically scaled
• Key: scaling any single dimension alone is suboptimal; compound scaling is optimal

4. ConvNeXt Design Choices

Implementation Insights

PyTorch Implementation: ResNet Residual Block

import torch
import torch.nn as nn
import torch.nn.functional as F

class ResidualBlock(nn.Module):
    def __init__(self, in_ch, out_ch, stride=1):
        super().__init__()
        self.conv1 = nn.Conv2d(in_ch, out_ch, 3, stride, 1, bias=False)
        self.bn1 = nn.BatchNorm2d(out_ch)
        self.conv2 = nn.Conv2d(out_ch, out_ch, 3, 1, 1, bias=False)
        self.bn2 = nn.BatchNorm2d(out_ch)
        
        # shortcut: 1×1 conv to align dims (when in_ch ≠ out_ch or stride>1)
        self.shortcut = nn.Sequential()
        if in_ch != out_ch or stride != 1:
            self.shortcut = nn.Sequential(
                nn.Conv2d(in_ch, out_ch, 1, stride, bias=False),
                nn.BatchNorm2d(out_ch)
            )
    
    def forward(self, x):
        out = F.relu(self.bn1(self.conv1(x)))
        out = self.bn2(self.conv2(out))
        out += self.shortcut(x)   # residual connection
        return F.relu(out)

# Stack ResNet-18 style
class ResNet18(nn.Module):
    def __init__(self, num_classes=10):
        super().__init__()
        self.conv1 = nn.Conv2d(3, 64, 7, 2, 3, bias=False)
        self.bn1 = nn.BatchNorm2d(64)
        self.pool = nn.MaxPool2d(3, 2, 1)
        self.layer1 = self._make_layer(64, 64, 2)
        self.layer2 = self._make_layer(64, 128, 2, stride=2)
        self.layer3 = self._make_layer(128, 256, 2, stride=2)
        self.layer4 = self._make_layer(256, 512, 2, stride=2)
        self.gap = nn.AdaptiveAvgPool2d(1)
        self.fc = nn.Linear(512, num_classes)
    
    def _make_layer(self, in_ch, out_ch, blocks, stride=1):
        layers = [ResidualBlock(in_ch, out_ch, stride)]
        for _ in range(1, blocks):
            layers.append(ResidualBlock(out_ch, out_ch))
        return nn.Sequential(*layers)
    
    def forward(self, x):
        x = self.pool(F.relu(self.bn1(self.conv1(x))))
        x = self.layer1(x); x = self.layer2(x)
        x = self.layer3(x); x = self.layer4(x)
        x = self.gap(x).flatten(1)
        return self.fc(x)

Depthwise Separable Convolution

class DepthwiseSeparableConv(nn.Module):
    def __init__(self, in_ch, out_ch, kernel=3):
        super().__init__()
        self.depthwise = nn.Conv2d(in_ch, in_ch, kernel, padding=kernel//2, groups=in_ch, bias=False)
        self.pointwise = nn.Conv2d(in_ch, out_ch, 1, bias=False)
        self.bn = nn.BatchNorm2d(out_ch)
    
    def forward(self, x):
        return F.relu(self.bn(self.pointwise(self.depthwise(x))))

Use Pretrained Models (Recommended in Practice)

import torchvision.models as models

# ResNet-50 pretrained (ImageNet weights)
resnet = models.resnet50(weights=models.ResNet50_Weights.IMAGENET1K_V2)
# EfficientNet-B0
effnet = models.efficientnet_b0(weights=models.EfficientNet_B0_Weights.IMAGENET1K_V1)
# ConvNeXt-Tiny
convnext = models.convnext_tiny(weights=models.ConvNeXt_Tiny_Weights.IMAGENET1K_V1)

Critical Reflection

Course Emphasis: Choose backbone by three factors — (1) data volume (small data → CNN/ConvNeXt, large data → try ViT); (2) deployment constraints (mobile → MobileNet/EfficientNet); (3) accuracy needs (extreme → EfficientNetV2/ConvNeXt-XXL). Don't blindly chase novelty; match constraints.

Course video sources

Source correction: Fall 2024 Lecture 6 is Generalization Theory, not the claimed “L06 Modern CNN.” The supplementary video below is Lecture 4, Architectures: Grids, covering CNN architectures and ResNet. It is related background and does not verify the article’s EfficientNet/ConvNeXt material as Lecture 6 content. The original video vidCX_dMCu0 is Lecture 2; the unverified 2:30:00 timestamp is omitted.

Original videos: MIT 6.7960 Fall 2024 Lec 04: Architectures: Grids — related CNN/ResNet lecture

Official sources:

• Official OCW Lec 04 video and transcript
• Official Fall 2024 lecture video index

Checked on 2026-10-10.

Numbering note: the L03–L06 numbers of these four posts were planned by this site early on from a generic deep-learning teaching order (optimization, regularization, CNN, modern CNN) without matching each one to the OCW lecture list. OCW Fall 2024 Lec 03–06 are actually Approximation Theory, Architectures: Grids, Architectures: Graphs and Generalization Theory, so these four L numbers differ from OCW lecture numbers. Since 2026-10-13 the titles follow OCW: the CNN architectures post matches Lec 04 and is labelled Lecture 4, while the optimization, regularization and modern CNN posts have no single matching lecture and are labelled Topic Guide. URLs are unchanged, so the modern CNN post still has l06 in its URL; from Lec 07 on the posts already matched OCW.

Content check: verified against the video transcript (2026-10-10): read the transcript of the embedded Lec 04 Architectures: Grids (Sara Beery): only at about 60–70 min does it cover ResNet residual connections (close to this article's ResNet section); MobileNet, depthwise separable convolution, EfficientNet and ConvNeXt do not appear in the transcript, so those parts are entirely this site's supplement. The reference entry listing "0:00–15:00 ResNet; 15:00–30:00 MobileNet/EfficientNet; 30:00–45:00 ConvNeXt" and the link to a non-existent slide deck were removed.

Update Log

• 2026-10-10: Added explicit video status and checked recording sources and access notes.
• 2026-10-10: Rechecked video status. OCW Lecture 6 is still Generalization Theory with no “Modern CNN” lecture; the supplementary video stays Lecture 4 and the status is unchanged.
• 2026-10-10: Checked the video content against its transcript. Removed the segment timetable that matched no video and the link to a non-existent slide deck, and noted that MobileNet / EfficientNet / ConvNeXt are not in the embedded video.

References

• Lec 04 Architectures: Grids (MIT OCW 6.7960 Fall 2024) — the embedded supplementary video; its later part covers ResNet residual connections, while MobileNet / EfficientNet / ConvNeXt are not in it
• He et al., "Deep Residual Learning for Image Recognition" (CVPR 2016) — ResNet
• Howard et al., "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications" (2017)
• Tan & Le, "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks" (ICML 2019)
• Liu et al., "A ConvNet for the 2020s" (CVPR 2022) — ConvNeXt
• PyTorch Official: torchvision.models