OPT OpenIR  > 光学影像学习与分析中心
Image2song: Song Retrieval via Bridging Image Content and Lyric Words
Li, Xuelong1; Hu, Di2; Lu, Xiaoqiang1
2017-12-22
Conference Name16th IEEE International Conference on Computer Vision, ICCV 2017
Source PublicationProceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017
Volume2017-October
Pages5650-5659
Conference Date2017-10-22
Conference PlaceVenice, Italy
PublisherInstitute of Electrical and Electronics Engineers Inc.
Contribution Rank1
Abstract

Image is usually taken for expressing some kinds of emotions or purposes, such as love, celebrating Christmas. There is another better way that combines the image and relevant song to amplify the expression, which has drawn much attention in the social network recently. Hence, the automatic selection of songs should be expected. In this paper, we propose to retrieve semantic relevant songs just by an image query, which is named as the image2song problem. Motivated by the requirements of establishing correlation in semantic/content, we build a semantic-based song retrieval framework, which learns the correlation between image content and lyric words. This model uses a convolutional neural network to generate rich tags from image regions, a recurrent neural network to model lyric, and then establishes correlation via a multi-layer perceptron. To reduce the content gap between image and lyric, we propose to make the lyric modeling focus on the main image content via a tag attention. We collect a dataset from the social-sharing multimodal data to study the proposed problem, which consists of (image, music clip, lyric) triplets. We demonstrate that our proposed model shows noticeable results in the image2song retrieval task and provides suitable songs. Besides, the song2image task is also performed. © 2017 IEEE.

Department光学影像学习与分析中心
DOI10.1109/ICCV.2017.602
Indexed ByEI ; ISTP
ISBN9781538610329
Language英语
ISSN15505499
Citation statistics
Document Type会议论文
Identifierhttp://ir.opt.ac.cn/handle/181661/29942
Collection光学影像学习与分析中心
Affiliation1.Xi'An Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an, 710119, China
2.School of Computer Science, Center for OPTical IMagery Analysis and Learning (OPTIMAL), Northwestern Polytechnical University, Xi'an, 710072, China
Recommended Citation
GB/T 7714
Li, Xuelong,Hu, Di,Lu, Xiaoqiang. Image2song: Song Retrieval via Bridging Image Content and Lyric Words[C]:Institute of Electrical and Electronics Engineers Inc.,2017:5650-5659.
Files in This Item:
File Name/Size DocType Version Access License
Image2song Song Retr(1295KB)会议论文 限制开放CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Li, Xuelong]'s Articles
[Hu, Di]'s Articles
[Lu, Xiaoqiang]'s Articles
Baidu academic
Similar articles in Baidu academic
[Li, Xuelong]'s Articles
[Hu, Di]'s Articles
[Lu, Xiaoqiang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Li, Xuelong]'s Articles
[Hu, Di]'s Articles
[Lu, Xiaoqiang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.