Schedule

Date

Presenters

Topic

Papers

Length

Slides

Jan 10

Gedas [1]

Course Overview;

75min

pdf

Introduction to Transformers

[1] Attention is All You Need

Jan 15

No Class (MLK Day)

Jan 17

Gedas [2]

[2] An image is worth 16x16 words: Transformers for image recognition at scale

Detailed Paper Discussion #1

30min

45min

pdf

Vision Transformers

Jan 22

Gedas [3,4]

[3] Training data-efficient image transformers & distillation through attention

[4] Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet

20min

pdf

Data-efficient ViT Training

Paper Battle #1

35min

pdf

Jan 24

Gedas [5]

[5] Scaling Vision Transformers to 22 Billion Parameters

Detailed Paper Discussion #2

30min

45min

pdf

Scaling ViTs

Jan 29

[6] Early Convolutions Help Transformers See Better

[7] ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases

20min

pdf

Mixing Convolutions with Self-Attention

Paper Battle #2

35min

pdf

Andrea [6]

Noah [6]

Ronit [6]

Charlie [7]

Sabiq [7]

Michael [7]

Jan 31

Li Hui [8]

[8] Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Detailed Paper Discussion #3

30min

45min

pdf

Hierarchical Transformers

Liujie [8]

Feb 5

Louie [9]

Efficient ViTs

[9] Token Merging: Your ViT But Faster

pdf

[10] XCiT: Cross-Covariance Image Transformers

30min+QA

Michael [9]

Chongyi [10]

Sabiq [10]

Feb 7

[11] End-to-End Object Detection with Transformers

[12] Pix2seq: A Language Modeling Framework for Object Detection

20min

pdf

Object Detection

Paper Battle #3

35min

pdf

Xinyu [11]

Tarik [11]

Liujie [11]

Nathan [12]

Dohhyun [12]

Rodrigo [12]

Feb 12

No Class (Well-being Day)

Feb 14

[13] Masked-attention Mask Transformer for Universal Image Segmentation

[14] SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers

20min

pdf

Image Segmentation

Paper Battle #4

35min

pdf

Mingcheng [13]

Louie [13]

Chongyi [13]

Alex [14]

Pan [14]

David Z. [14]

Feb 19

Project Proposal

(Part 1)

Report due Feb 25, 11:59 PM

Feb 21

Project Proposal

(Part 2)

Report due Feb 25, 11:59 PM

Feb 26

Gedas [15]

[15] Is Space-Time Attention All You Need for Video Understanding?

Detailed Paper Discussion #4

30min

45min

pdf

Video Transformers

Feb 28

Sizhe [16]

Tracking

[16] TrackFormer: Multi-Object Tracking with Transformers

pdf

[17] End-to-End Video Instance Segmentation with Transformers

30min+QA

Kaan [16]

Charlie [17]

Connor [17]

Mar 4

[18] Point Transformer

[19] An End-to-End Transformer Model for 3D Object Detection

20min

pdf

Transformers for 3D Peception

Paper Battle #5

35min

pdf

Connor [18]

Daniel [18]

Han [18]

Sizhe [19]

Kaan [19]

Andrew [19]

Ananya [19]

Mar 6

Pan [20]

[20] RT-1: Robotics Transformer for Real-World Control at Scale

Detailed Paper Discussion #5

30min

45min

pdf

Transformers in Robotics

Nathan [20]

Mar 11

No Class (Spring Break)

Mar 13

No Class (Spring Break)

Mar 18

Junjie [21]

Multimodal Transformers

[21] Perceiver: General Perception with Iterative Attention

pdf

[22] Flamingo: a Visual Language Model for Few-Shot Learning

30min+QA

David K. [21]

Ananya [22]

Mingcheng [22]

Mar 20

Andrew [23]

[23] ViperGPT: Visual Inference via Python Execution for Reasoning

Detailed Paper Discussion #6

30min

45min

pdf

Multimodal Transformers

Dohhyun [23]

Project Milestone

(Part 1)

Report due Mar 31, 11:59 PM

Mar 25

Project Milestone

(Part 2)

Report due Mar 31, 11:59 PM

Mar 27

Apr 1

[24] VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text

[25] Emerging Properties in Self-Supervised Vision Transformers

20min

pdf

Self-Supervised Learning

Paper Battle #6

35min

pdf

David K. [24]

Li Hui [24]

Junjie [24]

Nicholas [25]

Soumitri [25]

Levi [25]

Apr 3

Han [26]

[26] Masked Autoencoders Are Scalable Vision Learners

Detailed Paper Discussion #7

30min

45min

pdf

Self-Supervised Learning

Apr 8

Andrea [27]

Image and Video Generation

[27] MaskGIT: Masked Generative Image Transformer

pdf

[28] Phenaki: Variable Length Video Generation From Open Domain Textual Description

30min+QA

Rodrigo [27]

Noah [28]

Daniel [28]

Apr 10

Xinyu [29]

[29] Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action

Detailed Paper Discussion #8

30min

45min

pdf

Unified Multimodal Generation

Levi [29]

Apr 15

Soumitri [30]

Beyond Transformers

[30] MLP-Mixer: An all-MLP Architecture for Vision

pdf

[31] MetaFormer Is Actually What You Need for Vision

30min+QA

Tarik [31]

Nicholas [31]

Apr 17

Alex [32]

[32] A ConvNet for the 2020s

Detailed Paper Discussion #9

30min

45min

pdf

Back to Convolutions

David Z. [32]

Ronit [30]

Apr 22

Apr 24

Final Presentation

(Part 1)

Report due Apr 29, 11:59 PM

Final Presentation

(Part 2)

Report due Apr 29 10, 11:59 PM

Apr 29

No Class (Tentative)

About

Project Details

Paper Critiques

Paper Battles