Schedule

Aug 21

Gedas

Course Overview;

75min

pdf

Intro to Deep Learning and 2D CNNs

Date

Presenters

Topic

Papers

Length

Slides

Aug 23

Gedas [1,2]

Video Modeling

[1] Long-term Recurrent Convolutional Networks for Visual Recognition and Description

pdf

[2] Learning Spatiotemporal Features with 3D Convolutional Networks

30min+QA

Sep 6

Gedas [3,4]

[3] Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset

[4] A Closer Look at Spatiotemporal Convolutions for Action Recognition

20min

pdf

Video Modeling

Paper Battle #1

35min

pdf

Sep 4

No Class (Labor Day)

Sep 11

Gedas [5]

[5] SlowFast Networks for Video Recognition

Detailed Paper Discussion #1

45min

30min

pdf

Video Modeling

Sep 13

Soumitri [6]

[6] Tracking Everything Everywhere All at Once

30min+QA

pdf

30min+QA

pdf

Optical Flow &

Tracking

Nick [7]

[7] Tracking Objects as Points

Lorry [7]

Sep 18

[8] Fast Online Object Tracking and Segmentation: A Unifying Approach

[9] Video Instance Segmentation

20min

pdf

Video Object Segmentation

Paper Battle #2

35min

pdf

Alessandro [8]

Vish [8]

Mel [8]

Jun [9]

Amit [9]

Michael [9]

Sep 20

Gedas [10]

[10] Is Space-Time Attention All You Need for Video Understanding?

Detailed Paper Discussion #2

45min

30min

pdf

Video Transformers

Sep 25

No Class (Well-being Day)

Oct 2

Project Proposal

(Part 1)

Report due Oct 8, 11:59 PM

Oct 4

Project Proposal

(Part 2)

Report due Oct 8, 11:59 PM

Sep 27

Bang [11]

[11] The One Where They Reconstructed 3D Humans and Environments in TV Shows

Detailed Paper Discussion #3

45min

30min

pdf

3D Video Understanding

Xinyu [11]

Oct 9

Jiunjie [12]

[12] VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training

Detailed Paper Discussion #4

45min

30min

pdf

Self-Supervised Learning in Video

Liujie [12]

Oct 11

Soumitri [13]

[13] Audio-Visual Scene Analysis with Self-Supervised Multisensory Features

[14] The Sound of Pixels

20min

pdf

Video+Audio Modeling

Paper Battle #3

35min

pdf

Oct 16

[15] End-to-End Learning of Visual Representations from Uncurated Instructional Videos

[16] Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval

20min

pdf

Video+Text Modeling

Paper Battle #4

35min

pdf

Chih-Yao [13]

Yulu [13]

Liujie [13]

Doki [14]

Yue [14]

Ce [14]

Feihong [14]

Bang [15]

Xinyu [15]

Luchao [15]

Lorry [15]

Ziyang [16]

Han W. [16]

Han L. [16]

Oct 18

[17] VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text

Detailed Paper Discussion #5

45min

30min

pdf

Multimodal Video Representation Learning

Myles [17]

Luchao [17]

Oct 23

Ziyang [18]

[18] VideoChat: Chat-Centric Video Understanding

Detailed Paper Discussion #6

45min

30min

pdf

Video and Large Language Models

Oct 25

Wei [19]

[19] Prompting Visual-Language Models for Efficient Video Understanding

[20] AIM: Adapting Image Models for Efficient Video Action Recognition

20min

pdf

Efficient Video Modeling

Paper Battle #5

35min

pdf

Feihong [18]

Jeff [19]

Myles [19]

Taixi [19]

Annie [20]

Shoubin [20]

Nick [20]

Junjie [20]

Project Milestone

(Part 1)

Report due Nov 5, 11:59 PM

Oct 30

Project Milestone

(Part 2)

Report due Nov 5, 11:59 PM

Nov 1

Nov 6

Han L. [21]

[21] Phenaki: Variable Length Video Generation From Open Domain Textual Description

Detailed Paper Discussion #7

45min

30min

pdf

Video Generation

Nov 8

Liujie [22]

[23] MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptions

[22] Long Movie Clip Classification with State-Space Video Models

30min+QA

pdf

30min+QA

pdf

Long Video & Movie

Understanding

Mel [27]

Shoubin [21]

Vish [22]

Han W. [27]

Nov 13

Wei [24]

[24] Ego4D: Around the World in 3,000 Hours of Egocentric Video

Detailed Paper Discussion #8

45min

30min

pdf

Egocentric Video Analysis

Jeff [24]

Nov 15

[26] Recognition in Terra Incognita

30min+QA

pdf

Optical Flow & Conservation

Amit [26]

Taixi [26]

Soumitri [6]

[6] Tracking Everything Everywhere All at Once

pdf

Nov 20

Jun [27]

[27] GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Driving

Detailed Paper Discussion #9

45min

30min

pdf

Autonomous Driving

Annie [27]

Nov 22

No Class (Thanksgiving Break)

Nov 27

Doki [28]

[28] End-to-End High-Risk Tackle Detection System for Rugby

[29] Pose-based Tremor Classification for Parkinson’s Disease Diagnosis from Video

20min+QA

pdf

20min+QA

pdf

Tracking, Sport & Health

Yulu [28]

Alessandro [29]

Michael [29]

[7] Tracking Objects as Points

20min+QA

pdf

Nick [7]

Lorry [7]

Nov 29

Ce [30]

[30] Human-to-Robot Imitation in the Wild

Detailed Paper Discussion #10

45min

30min

pdf

Robotics

Dec 4

Dec 6

Final Presentation

(Part 1)

Report due Dec 10, 11:59 PM

Final Presentation

(Part 2)

Report due Dec 10, 11:59 PM

Yue [30]

About

Project Details

Paper Critiques

Paper Battles