OPT OpenIR  > 光学影像学习与分析中心
Learning to Predict Eye Fixations via Multiresolution Convolutional Neural Networks
Liu, Nian1; Han, Junwei1; Liu, Tianming2; Li, Xuelong3
Department光学影像学习与分析中心
2018-02-01
Source PublicationIEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
ISSN2162-237X
Volume29Issue:2Pages:392-404
Contribution Rank3
Abstract

Eye movements in the case of freely viewing natural scenes are believed to be guided by local contrast, global contrast, and top-down visual factors. Although a lot of previous works have explored these three saliency cues for several years, there still exists much room for improvement on how to model them and integrate them effectively. This paper proposes a novel computation model to predict eye fixations, which adopts a multiresolution convolutional neural network (Mr-CNN) to infer these three types of saliency cues from raw image data simultaneously. The proposed Mr-CNN is trained directly from fixation and nonfixation pixels with multiresolution input image regions with different contexts. It utilizes image pixels as inputs and eye fixation points as labels. Then, both the local and global contrasts are learned by fusing information in multiple contexts. Meanwhile, various top-down factors are learned in higher layers. Finally, optimal combination of top-down factors and bottom-up contrasts can be learned to predict eye fixations. The proposed approach significantly outperforms the state-of-the-art methods on several publically available benchmark databases, demonstrating the superiority of Mr-CNN. We also apply our method to the RGB-D image saliency detection problem. Through learning saliency cues induced by depth and RGB information on pixel level jointly and their interactions, our model achieves better performance on predicting eye fixations in RGB-D images.

 

KeywordContrast Convolutional Neural Network (Cnn) Eye Fixation Prediction Rgb-d Saliency Detection
DOI10.1109/TNNLS.2016.2628878
Indexed BySCI ; EI ; CPCI
Language英语
WOS IDWOS:000422952400013
EI Accession Number20164903093468
Citation statistics
Cited Times:7[WOS]   [WOS Record]     [Related Records in WOS]
Document Type期刊论文
Identifierhttp://ir.opt.ac.cn/handle/181661/30792
Collection光学影像学习与分析中心
Affiliation1.Northwestern Polytech Univ, Sch Automat, Xian 710072, Shaanxi, Peoples R China;
2.Univ Georgia, Dept Comp Sci, Athens, GA 30602 USA;
3.Chinese Acad Sci, Xian Inst Opt & Precis Mech, Ctr Opt IMagery Anal & Learning, State Key Lab Transient Opt & Photon, Xian 710119, Shaanxi, Peoples R China
Recommended Citation
GB/T 7714
Liu, Nian,Han, Junwei,Liu, Tianming,et al. Learning to Predict Eye Fixations via Multiresolution Convolutional Neural Networks[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2018,29(2):392-404.
APA Liu, Nian,Han, Junwei,Liu, Tianming,&Li, Xuelong.(2018).Learning to Predict Eye Fixations via Multiresolution Convolutional Neural Networks.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,29(2),392-404.
MLA Liu, Nian,et al."Learning to Predict Eye Fixations via Multiresolution Convolutional Neural Networks".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 29.2(2018):392-404.
Files in This Item:
File Name/Size DocType Version Access License
Learning to Predict (2296KB)期刊论文出版稿开放获取CC BY-NC-SAView Application Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Liu, Nian]'s Articles
[Han, Junwei]'s Articles
[Liu, Tianming]'s Articles
Baidu academic
Similar articles in Baidu academic
[Liu, Nian]'s Articles
[Han, Junwei]'s Articles
[Liu, Tianming]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Liu, Nian]'s Articles
[Han, Junwei]'s Articles
[Liu, Tianming]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: Learning to Predict Eye Fixations via Multiresolution Convolutional Neural Networks.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.