Learning to Predict Eye Fixations via Multiresolution Convolutional Neural Networks | |
Liu, Nian1; Han, Junwei1; Liu, Tianming2; Li, Xuelong3 | |
作者部门 | 光学影像学习与分析中心 |
2018-02-01 | |
发表期刊 | IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS |
ISSN | 2162-237X |
卷号 | 29期号:2页码:392-404 |
产权排序 | 3 |
摘要 | Eye movements in the case of freely viewing natural scenes are believed to be guided by local contrast, global contrast, and top-down visual factors. Although a lot of previous works have explored these three saliency cues for several years, there still exists much room for improvement on how to model them and integrate them effectively. This paper proposes a novel computation model to predict eye fixations, which adopts a multiresolution convolutional neural network (Mr-CNN) to infer these three types of saliency cues from raw image data simultaneously. The proposed Mr-CNN is trained directly from fixation and nonfixation pixels with multiresolution input image regions with different contexts. It utilizes image pixels as inputs and eye fixation points as labels. Then, both the local and global contrasts are learned by fusing information in multiple contexts. Meanwhile, various top-down factors are learned in higher layers. Finally, optimal combination of top-down factors and bottom-up contrasts can be learned to predict eye fixations. The proposed approach significantly outperforms the state-of-the-art methods on several publically available benchmark databases, demonstrating the superiority of Mr-CNN. We also apply our method to the RGB-D image saliency detection problem. Through learning saliency cues induced by depth and RGB information on pixel level jointly and their interactions, our model achieves better performance on predicting eye fixations in RGB-D images.
|
关键词 | Contrast Convolutional Neural Network (Cnn) Eye Fixation Prediction Rgb-d Saliency Detection |
DOI | 10.1109/TNNLS.2016.2628878 |
收录类别 | SCI ; EI ; CPCI |
语种 | 英语 |
WOS记录号 | WOS:000422952400013 |
EI入藏号 | 20164903093468 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.opt.ac.cn/handle/181661/30792 |
专题 | 光谱成像技术研究室 |
作者单位 | 1.Northwestern Polytech Univ, Sch Automat, Xian 710072, Shaanxi, Peoples R China; 2.Univ Georgia, Dept Comp Sci, Athens, GA 30602 USA; 3.Chinese Acad Sci, Xian Inst Opt & Precis Mech, Ctr Opt IMagery Anal & Learning, State Key Lab Transient Opt & Photon, Xian 710119, Shaanxi, Peoples R China |
推荐引用方式 GB/T 7714 | Liu, Nian,Han, Junwei,Liu, Tianming,et al. Learning to Predict Eye Fixations via Multiresolution Convolutional Neural Networks[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2018,29(2):392-404. |
APA | Liu, Nian,Han, Junwei,Liu, Tianming,&Li, Xuelong.(2018).Learning to Predict Eye Fixations via Multiresolution Convolutional Neural Networks.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,29(2),392-404. |
MLA | Liu, Nian,et al."Learning to Predict Eye Fixations via Multiresolution Convolutional Neural Networks".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 29.2(2018):392-404. |
条目包含的文件 | ||||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
Learning to Predict (2296KB) | 期刊论文 | 出版稿 | 限制开放 | CC BY-NC-SA | 请求全文 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论