Aug 15

Course Overview; 

pdf

Intro to Self-Supervised Learning (SSL)

Aug 17

Gedas [1,2]

Pretext Tasks for Images

[1] Unsupervised Visual Representation Learning by Context Prediction

pdf

pdf

[2] Learning Features by Watching Objects Move

30min+QA

30min+QA

Aug 22

Gedas [3]

[3] Deep Clustering for Unsupervised Learning of Visual Features

pdf

pdf

Clustering-based SSL for Images

Aug 24

Gedas [4,5]

[4] A Simple Framework for Contrastive Learning of Visual Representations

[5] Momentum Contrast for Unsupervised Visual Representation Learning

20min

20min

pdf

pdf

Contrastive SSL for Images

Paper Battle #1

35min

pdf

Sep 5

No Class (Labor Day)

Aug 31

Michael T. [7]

[7] Large Scale Adversarial Representation Learning

pdf

[8] Generative Pretraining from Pixels

20min

20min

pdf

Paper Battle #2

Generative SSL for Images

35min

pdf

Jack [7]

Yunkui [7]

Bilen [8]

Jiaqi [8]

Michael W. [8]

Aug 29

Avinash [6]

[6] Exploring Simple Siamese Representation Learning

Detailed Paper Discussion #2

45min

pdf

30min

pdf

Distillation-based SSL for Images

Robert [6]

Sep 7

pdf

pdf

Scaling SSL Models & Data

[9] Scaling and Benchmarking Self-Supervised Visual Representation Learning

Detailed Paper Discussion #3

Gedas [9]

Sep 12

Lin [10]

[10] Emerging Properties in Self-Supervised Vision Transformers

[11] An Empirical Study of Training Self-Supervised Vision Transformers

30min+QA

pdf

30min+QA

pdf

Vision Transformers for SSL

Jiaqi [11]

Sep 19

Project Proposal

(Part 1)

Report due Sep 25, 11:59 PM

Sep 21

Project Proposal

(Part 2)

Report due Sep 25, 11:59 PM

Sep 14

pdf

pdf

[12] Masked Autoencoders Are Scalable Vision Learners

Detailed Paper Discussion #4

Jiang [12]

Vision Transformers for SSL

Shreyas [12]

SSL on Uncurated          Data (Fairness &        Robustness)

[13] Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision 

Detailed Paper Discussion #5

pdf

pdf

Sep 28

Jack [13]

Sep 26

No Class (Well-being Day)

Oct 3

Avinash [14]

[14] Anticipating the Future by Watching Unlabeled Video

pdf

[15] Shuffle and Learn: Unsupervised Learning using Temporal Order Verification

20min

20min

pdf

Paper Battle #3

35min

pdf

Pretext Tasks for Video

Robert [14]

David [15]

Lin [15]

Tony [15]

Oct 5

Nick A. [16]

[16] Self-supervised Tracking by Colorization

[17] Learning Correspondence from the Cycle-consistency of Time

30min+QA

pdf

30min+QA

pdf

Pretext Tasks for Video

Michael T. [17]

Oct 10

Video Transformers for SSL

[18] Masked Autoencoders As Spatiotemporal Learners

Detailed Paper Discussion #6

30min

pdf

pdf

Yunkui [18]

Oct 12

No Class (University Day)

Oct 17

Gedas [19]

[19] Self-Supervised Learning by Cross-Modal Audio-Video Clustering

[20] Audio-Visual Instance Discrimination with Cross-Modal Agreement

30min+QA

pdf

30min+QA

pdf

SSL with Video & Audio

Yan-Bo [20]

Yan-Bo [21]

SSL with Video & Audio

[21] Learning to Separate Object Sounds by Watching Unlabeled Video

20min

pdf

[22] The Sound of Pixels

20min

pdf

Paper Battle #4

35min

pdf

Oct 19

Nick A. [21]

Nick H.-T. [22]

Jiang [22]

Project Milestone

(Part 1)

Report due Oct 30, 11:59 PM

Oct 24

Report due Oct 30, 11:59 PM

Oct 26

Oct 31

SSL with Video & Text

[23] End-to-End Learning of Visual Representations from Uncurated Instructional Videos

Detailed Paper Discussion #7

30min

pdf

pdf

Vaidehi [23]

Ziyang [23]

Nov 2

Pierre [24]

SSL with Video, Audio & Text

[24] Self-Supervised MultiModal Versatile Networks

20min

pdf

[25] VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text

20min

pdf

Paper Battle #5

35min

pdf

Shoubin [24]

Vaidehi [25]

Ziyang [25]

Nov 7

SSL with Video, Audio & Text

[26] MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound

Detailed Paper Discussion #8

30min

pdf

pdf

Bilen [26]

Shoubin [26]

Nov 9

No Class

Nick H.-T. [27]

SSL for 3D

[27] Self-Supervised Pretraining of 3D Features on any Point-Cloud

Detailed Paper Discussion #9

30min

pdf

pdf

Nov 14

Pierre [27]

Nov 21

SSL for Robotics

[30] Masked Visual Pre-training for Motor Control

Detailed Paper Discussion #10

30min

pdf

pdf

Gedas [30]

Nov 23

No Class (Thanksgiving Break)

Nov 28

Nov 30

Final Presentation

(Part 1)

Report due Dec 2, 11:59 PM

Final Presentation

(Part 2)

Report due Dec 2, 11:59 PM

Tony [28]

SSL for Robotics

[28] Curiosity-driven Exploration by Self-supervised Prediction

30min+QA

pdf

[29] Learning to Poke by Poking: Experiential Learning of Intuitive Physics

30min+QA

pdf

Nov 16

David [29]