Publications

2024

1. 

   Online Temporal Action Localization with Memory-Augmented Transoformer

   Youngkil Song*, Dongkeun Kim*, Minsu Cho,  and Suha Kwak (*equal contribution)

   European Conference on Computer Vision (ECCV), 2024

   Abstract Code Website

   Online temporal action localization (On-TAL) is the task of identifying multiple action instances given a streaming video. Since existing methods take as input only a video segment of fixed size per iteration, they are limited in considering long-term context and require tuning the segment size carefully. To overcome these limitations, we propose memory-augmented transformer (MATR). MATR utilizes the memory queue that selectively preserves the past segment features, allowing to leverage long-term context for inference. We also propose a novel action localization method that observes the current input segment to predict the end time of the ongoing action and accesses the memory queue to estimate the start time of the action. Our method outperformed existing methods on two datasets, THUMOS14 and MUSES, surpassing not only TAL methods in the online setting but also some offline TAL methods.

2. 

   Towards More Practical Group Activity Detection: A New Benchmark and Model

   Dongkeun Kim, Youngkil Song, Minsu Cho,  and Suha Kwak

   European Conference on Computer Vision (ECCV), 2024

   Abstract Code Website

   Group activity detection (GAD) is the task of identifying members of each group and classifying the activity of the group at the same time in a video. While GAD has been studied recently, there is still much room for improvement in both dataset and methodology due to their limited capability to address practical GAD scenarios. To resolve these issues, we first present a new dataset, dubbed Café. Unlike existing datasets, Café is constructed primarily for GAD and presents more practical scenarios and metrics, as well as being large-scale and providing rich annotations. Along with the dataset, we propose a new GAD model that deals with an unknown number of groups and latent group members efficiently and effectively. We evaluated our model on three datasets including Café, where it outperformed previous work in terms of both accuracy and inference speed.