2 February 2023 — NUS Computing professor and researchers clinched the Best BNI (Brave New Ideas) Paper award for their work on narrative generation at the ACM Multimedia Conference held in Lisbon, Portugal between 10-14 October 2022.
The NUS team consists of Provost Chair Professor Mohan Kankanhalli, Senior Research Fellows Yongkang Wong, and Shaojing Fan, Research Fellow Yangyang Guo, Ph.D. Candidate Ziwei Xu, and lastly, Graduate Student Researcher Rishabh Sheoran.
It was an outcome of their collaboration with NEC Corporation researchers Karen Stephen, Anusha Bhamidipati, Vivek Barsopia, and Head of Media Analysis Platform Research Group Jianquan Liu.
Their paper titled, “Compute to Tell the Tale: Goal-Driven Narrative Generation,” tackles the task of narrative generation by combining perspectives from social science and computer science. It models the problem of narrative generation from the machine learning perspective and proposes a general narrative generation framework that can be applied to different domains. It further discusses the key challenges and research problems of narrative generation, such as the factual consistency of generated narrative, narrative structure modeling, personalization, and requirements of the dataset for machine learning.
Yongkang revealed that the paper was inspired by the collaboration between NUS Computing and NEC Corporation’s Visual Intelligence Research Laboratory. He shared: “While the collaboration focuses on a specific real-world application, the intensive research discussions led us to a fundamental question—can an artificial computation system generate narratives?” To address this fundamental question, the team identified the need of bridging the social science findings on narratives to the computational narrative generation task.
“In addition, we acknowledge that a trusted narrative generation model should be highly explainable, controllable, and scale to various domains and preferences. As a result, our proposed framework brings new perspectives into the mainstream deep learning-based approach, which highlights the need for modularization, adaption to arbitrary domain knowledge without re-training, understanding of the narrative structure, factual narrative generation, and personalization. The identified challenges and research problems can be studied either singly or jointly in future work,” Yongkang explained.
Besides the paper, the group also organized the “1st Workshop on User-Centric Narrative Summarisation of Long Videos” in conjunction with ACM Multimedia Conference. The workshop had two keynote speakers, Prof. Manmohan Chandraker of UCSD and Prof. Ioannis Patras of QMUL, and featured Professor Shin’ichi Satoh of NII Japan in a panel discussion on emerging topics on video summarization. “This brought together experts with common research interests to have stimulating discussions on potential research opportunities,” concluded Yongkang.
