OPT OpenIR  > 光学影像学习与分析中心
Deep neural networks with Elastic Rectified Linear Units for object recognition
Jiang, Xiaoheng1; Pang, Yanwei1; Li, Xuelong2; Pan, Jing1,3; Xie, Yinghong1,4
Department光学影像学习与分析中心
2018-01-31
Source PublicationNEUROCOMPUTING
ISSN0925-2312
Volume275Pages:1132-1139
Contribution Rank2
Abstract

Rectified Linear Unit (ReLU) is crucial to the recent success of deep neural networks (DNNs). In this paper, we propose a novel Elastic Rectified Linear Unit (EReLU) that focuses on processing the positive part of input. Unlike previous variants of ReLU that typically adopt linear or piecewise linear functions to represent the positive part, EReLU is characterized by that each positive value scales within a moderate range like a spring during training stage. On test time, EReLU becomes standard ReLU. EReLU improves model fitting with no extra parameters and little overfitting risk. Furthermore, we propose Elastic Parametric Rectified Linear Unit (EPReLU) by taking advantage of EReLU and parametric ReLU (PReLU). EPReLU is able to further improve the performance of networks. In addition, we present a new training strategy to train DNNs with EPReLU. Experiments on four benchmarks including CIFAR10, CIFAR10, SVHN and ImageNet 2012 demonstrate the effectiveness of both EReLU and EPReLU. (C) 2017 Elsevier B.V. All rights reserved.

 

KeywordDeep Neural Networks Elastic Rectified Linear Unit (Erelu) Elastic Parametric Rectified Linear Unit (Eprelu) Non-saturating Nonlinear Activation Function
DOI10.1016/j.neucom.2017.09.056
Indexed BySCI ; EI
Language英语
WOS IDWOS:000418370200108
EI Accession Number20174104249355
Citation statistics
Cited Times:5[WOS]   [WOS Record]     [Related Records in WOS]
Document Type期刊论文
Identifierhttp://ir.opt.ac.cn/handle/181661/30822
Collection光学影像学习与分析中心
Corresponding AuthorPang, Yanwei
Affiliation1.Tianjin Univ, Sch Elect & Informat Engn, Tianjin 300072, Peoples R China;
2.Chinese Acad Sci, Xian Inst Opt & Precis Mech, State Key Lab Transient Opt & Photon, Ctr OPT IMagery Anal & Learning OPTIMAL, Xian 710119, Shaanxi, Peoples R China;
3.Tianjin Univ Technol & Educ, Sch Elect Engn, Tianjin 300222, Peoples R China;
4.Shenyang Univ, Coll Informat Engn, Shenyang 110044, Liaoning, Peoples R China
Recommended Citation
GB/T 7714
Jiang, Xiaoheng,Pang, Yanwei,Li, Xuelong,et al. Deep neural networks with Elastic Rectified Linear Units for object recognition[J]. NEUROCOMPUTING,2018,275:1132-1139.
APA Jiang, Xiaoheng,Pang, Yanwei,Li, Xuelong,Pan, Jing,&Xie, Yinghong.(2018).Deep neural networks with Elastic Rectified Linear Units for object recognition.NEUROCOMPUTING,275,1132-1139.
MLA Jiang, Xiaoheng,et al."Deep neural networks with Elastic Rectified Linear Units for object recognition".NEUROCOMPUTING 275(2018):1132-1139.
Files in This Item:
File Name/Size DocType Version Access License
Deep neural networks(921KB)期刊论文出版稿开放获取CC BY-NC-SAView Application Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Jiang, Xiaoheng]'s Articles
[Pang, Yanwei]'s Articles
[Li, Xuelong]'s Articles
Baidu academic
Similar articles in Baidu academic
[Jiang, Xiaoheng]'s Articles
[Pang, Yanwei]'s Articles
[Li, Xuelong]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Jiang, Xiaoheng]'s Articles
[Pang, Yanwei]'s Articles
[Li, Xuelong]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: Deep neural networks with Elastic Rectified Linear Units for object recognition.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.