OPT OpenIR  > 光谱成像技术研究室
Multi-modal gated recurrent units for image description
Li, Xuelong1,2; Yuan, Aihong1,2; Lu, Xiaoqiang1
作者部门光学影像学习与分析中心
2018-11-01
发表期刊Multimedia Tools and Applications
ISSN13807501;15737721
卷号77期号:22页码:29847-29869
产权排序1
摘要

Using a natural language sentence to describe the content of an image is a challenging but very important task. It is challenging because a description must not only capture objects contained in the image and the relationships among them, but also be relevant and grammatically correct. In this paper a multi-modal embedding model based on gated recurrent units (GRU) which can generate variable-length description for a given image. In the training step, we apply the convolutional neural network (CNN) to extract the image feature. Then the feature is imported into the multi-modal GRU as well as the corresponding sentence representations. The multi-modal GRU learns the inter-modal relations between image and sentence. And in the testing step, when an image is imported to our multi-modal GRU model, a sentence which describes the image content is generated. The experimental results demonstrate that our multi-modal GRU model obtains the state-of-the-art performance on Flickr8K, Flickr30K and MS COCO datasets. © 2018, Springer Science+Business Media, LLC, part of Springer Nature.

关键词Image Description Gated Recurrent Unit Convolutional Neural Network Multi-modal Embedding
DOI10.1007/s11042-018-5856-1
收录类别SCI ; EI
语种英语
WOS记录号WOS:000451780800038
出版者Springer New York LLC
EI入藏号20181204916306
引用统计
被引频次:15[WOS]   [WOS记录]     [WOS相关记录]
文献类型期刊论文
条目标识符http://ir.opt.ac.cn/handle/181661/30849
专题光谱成像技术研究室
通讯作者Lu, Xiaoqiang
作者单位1.Center for OPTical IMagery Analysis and Learning (OPTIMAL), Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an; Shaanxi; 710119, China;
2.University of Chinese Academy of Sciences, 19A Yuquanlu, Beijing; 100049, China
推荐引用方式
GB/T 7714
Li, Xuelong,Yuan, Aihong,Lu, Xiaoqiang. Multi-modal gated recurrent units for image description[J]. Multimedia Tools and Applications,2018,77(22):29847-29869.
APA Li, Xuelong,Yuan, Aihong,&Lu, Xiaoqiang.(2018).Multi-modal gated recurrent units for image description.Multimedia Tools and Applications,77(22),29847-29869.
MLA Li, Xuelong,et al."Multi-modal gated recurrent units for image description".Multimedia Tools and Applications 77.22(2018):29847-29869.
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
Multi-modal gated re(2037KB)期刊论文出版稿限制开放CC BY-NC-SA请求全文
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[Li, Xuelong]的文章
[Yuan, Aihong]的文章
[Lu, Xiaoqiang]的文章
百度学术
百度学术中相似的文章
[Li, Xuelong]的文章
[Yuan, Aihong]的文章
[Lu, Xiaoqiang]的文章
必应学术
必应学术中相似的文章
[Li, Xuelong]的文章
[Yuan, Aihong]的文章
[Lu, Xiaoqiang]的文章
相关权益政策
暂无数据
收藏/分享
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。