Aug 15

Course Overview; 

pdf

Intro to Self-Supervised Learning (SSL)

Aug 17

Pretext Tasks for Images

[1] Unsupervised Visual Representation Learning by Context Prediction

pdf

pdf

[2] Learning Features by Watching Objects Move

30min+QA

30min+QA

Aug 22

Gedas

[3] Deep Clustering for Unsupervised Learning of Visual Features

pdf

pdf

Clustering-based SSL for Images

Aug 24

Gedas

[4] A Simple Framework for Contrastive Learning of Visual Representations

[5] Momentum Contrast for Unsupervised Visual Representation Learning 

20min

20min

pdf

pdf

Contrastive SSL for Images

Paper Battle #1

35min

pdf

Aug 31

TBD

[7] Large Scale Adversarial Representation Learning

pdf

[8] Generative Pretraining from Pixels

20min

20min

pdf

Paper Battle #2

35min

pdf

Generative SSL for Images

Aug 29

TBD

[6] Exploring Simple Siamese Representation Learning

Detailed Paper Discussion #2

45min

pdf

30min

pdf

Distillation-based SSL for Images

Sep 5

No Class (Labor Day)

Sep 7

pdf

pdf

Scaling SSL Models & Data

[9] Scaling and Benchmarking Self-Supervised Visual Representation Learning

Detailed Paper Discussion #3

TBD

Sep 12

TBD

[10] Emerging Properties in Self-Supervised Vision Transformers

[11] An Empirical Study of Training Self-Supervised Vision Transformers

30min+QA

pdf

30min+QA

pdf

Vision Transformers for SSL

Sep 14

pdf

pdf

[12] Masked Autoencoders Are Scalable Vision Learners

Detailed Paper Discussion #4

TBD

Vision Transformers for SSL

Sep 19

Project Proposal

(Part 1)

Report due Sep 25, 11:59 PM

Sep 21

Project Proposal

(Part 2)

Report due Sep 25, 11:59 PM

Oct 3

TBD

[14] Anticipating the Future by Watching Unlabeled Video

pdf

[15] Shuffle and Learn: Unsupervised Learning using Temporal Order Verification

20min

20min

pdf

Paper Battle #3

35min

pdf

Pretext Tasks for Video

SSL on Uncurated Data (Fairness & Robustness)

[13] Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision 

Detailed Paper Discussion #5

pdf

pdf

Sep 28

TBD

Sep 26

No Class (Well-being Day)

Oct 10

Video Transformers for SSL

[18] Masked Autoencoders As Spatiotemporal Learners

Detailed Paper Discussion #6

30min

pdf

pdf

TBD

Oct 5

TBD

[16] Self-supervised Tracking by Colorization

[17] Learning Correspondence from the Cycle-consistency of Time

30min+QA

pdf

30min+QA

pdf

Pretext Tasks for Video

Oct 12

No Class (University Day)

Oct 17

TBD

[19] Self-Supervised Learning by Cross-Modal Audio-Video Clustering

[20] Audio-Visual Instance Discrimination with Cross-Modal Agreement

30min+QA

pdf

30min+QA

pdf

SSL with Video & Audio

TBD

SSL with Video & Audio

[21] Learning to Separate Object Sounds by Watching Unlabeled Video

20min

pdf

[22] The Sound of Pixels

20min

pdf

Paper Battle #4

35min

pdf

Oct 19

Oct 31

SSL with Video & Text

[23] End-to-End Learning of Visual Representations from Uncurated Instructional Videos

Detailed Paper Discussion #7

30min

pdf

pdf

TBD

Project Milestone

(Part 1)

Report due Oct 30, 11:59 PM

Oct 24

Report due Oct 30, 11:59 PM

Oct 26

Nov 2

TBD

SSL with Video, Audio & Text

[24] Self-Supervised MultiModal Versatile Networks

20min

pdf

[25] VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text

20min

pdf

Paper Battle #5

35min

pdf

Nov 7

SSL with Video, Audio & Text

[26] MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound

Detailed Paper Discussion #8

30min

pdf

pdf

TBD

Nov 9

No Class

TBD

SSL for 3D

[27] Self-Supervised Pretraining of 3D Features on any Point-Cloud

Detailed Paper Discussion #9

30min

pdf

pdf

Nov 14

TBD

SSL for Robotics

[28] Curiosity-driven Exploration by Self-supervised Prediction

30min+QA

pdf

[29] Learning to Poke by Poking: Experiential Learning of Intuitive Physics

30min+QA

pdf

Nov 16

Nov 21

SSL for Robotics

[30] Masked Visual Pre-training for Motor Control

Detailed Paper Discussion #10

30min

pdf

pdf

TBD

Nov 23

No Class (Thanksgiving Break)

Nov 28

Nov 30

Final Presentation

(Part 1)

Report due Dec 2, 11:59 PM

Final Presentation

(Part 2)

Report due Dec 2, 11:59 PM