Property-Constrained Dual Learning for Video Summarization | |
Zhao, Bin1,2; Li, Xuelong1,2; Lu, Xiaoqiang3 | |
作者部门 | 光谱成像技术研究室 |
2020-10 | |
发表期刊 | IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS |
ISSN | 2162-237X;2162-2388 |
卷号 | 31期号:10页码:3989-4000 |
产权排序 | 3 |
摘要 | Video summarization is the technique to condense large-scale videos into summaries composed of key-frames or key-shots so that the viewers can browse the video content efficiently. Recently, supervised approaches have achieved great success by taking advantages of recurrent neural networks (RNNs). Most of them focus on generating summaries by maximizing the overlap between the generated summary and the ground truth. However, they neglect the most critical principle, i.e., whether the viewer can infer the original video content from the summary. As a result, existing approaches cannot preserve the summary quality well and usually demand large amounts of training data to reduce overfitting. In our view, video summarization has two tasks, i.e., generating summaries from videos and inferring the original content from summaries. Motivated by this, we propose a dual learning framework by integrating the summary generation (primal task) and video reconstruction (dual task) together, which targets to reward the summary generator under the assistance of the video reconstructor. Moreover, to provide more guidance to the summary generator, two property models are developed to measure the representativeness and diversity of the generated summary. Practically, experiments on four popular data sets (SumMe, TVsum, OVP, and YouTube) have demonstrated that our approach, with compact RNNs as the summary generator, using less training data, and even in the unsupervised setting, can get comparable performance with those supervised ones adopting more complex summary generators and trained on more annotated data. |
关键词 | Generators Task analysis Recurrent neural networks Machine learning Training Semantics Learning systems Dual learning property model recurrent neural network (RNN) video summarization |
DOI | 10.1109/TNNLS.2019.2951680 |
收录类别 | SCI ; EI |
语种 | 英语 |
WOS记录号 | WOS:000576436600017 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
EI入藏号 | 20204509445393 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.opt.ac.cn/handle/181661/93737 |
专题 | 光谱成像技术研究室 |
通讯作者 | Lu, Xiaoqiang |
作者单位 | 1.Northwestern Polytech Univ, Sch Comp Sci, Xian 710072, Peoples R China 2.Northwestern Polytech Univ, Ctr OPT Imagery Anal & Learning OPTIMAL, Xian 710072, Peoples R China 3.Chinese Acad Sci, Key Lab Spectral Imaging Technol CAS, Xian Inst Opt & Precis Mech, Xian 710119, Peoples R China |
推荐引用方式 GB/T 7714 | Zhao, Bin,Li, Xuelong,Lu, Xiaoqiang. Property-Constrained Dual Learning for Video Summarization[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2020,31(10):3989-4000. |
APA | Zhao, Bin,Li, Xuelong,&Lu, Xiaoqiang.(2020).Property-Constrained Dual Learning for Video Summarization.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,31(10),3989-4000. |
MLA | Zhao, Bin,et al."Property-Constrained Dual Learning for Video Summarization".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 31.10(2020):3989-4000. |
条目包含的文件 | ||||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
Property-Constrained(2537KB) | 期刊论文 | 出版稿 | 限制开放 | CC BY-NC-SA | 请求全文 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论