Long-form video representation learning (Part 2: Video as sparse transformers)

We explore novel video representations methods that are equipped with long-form reasoning capability. This is part II focusing on sparse video-text transformers. See Part I on video as graphs. And Part III provides a sneak peek into our latest and greatest explorations.

The first blog in this series was about learning explicit sparse graph-based video representation methods for “long-form” video representation learning. They are effective methods; however, they were not end-to-end trainable. We needed to rely on other CNN or …

blog capability cvpr cvpr-2024 explore form graphs iii latest novel part reasoning representation representation learning series text transformers video