OPT OpenIR  > 光学影像学习与分析中心
Deep neural networks with Elastic Rectified Linear Units for object recognition
Jiang, Xiaoheng1; Pang, Yanwei1; Li, Xuelong2; Pan, Jing1,3; Xie, Yinghong1,4
作者部门光学影像学习与分析中心
2018-01-31
发表期刊NEUROCOMPUTING
ISSN0925-2312
卷号275页码:1132-1139
产权排序2
摘要

Rectified Linear Unit (ReLU) is crucial to the recent success of deep neural networks (DNNs). In this paper, we propose a novel Elastic Rectified Linear Unit (EReLU) that focuses on processing the positive part of input. Unlike previous variants of ReLU that typically adopt linear or piecewise linear functions to represent the positive part, EReLU is characterized by that each positive value scales within a moderate range like a spring during training stage. On test time, EReLU becomes standard ReLU. EReLU improves model fitting with no extra parameters and little overfitting risk. Furthermore, we propose Elastic Parametric Rectified Linear Unit (EPReLU) by taking advantage of EReLU and parametric ReLU (PReLU). EPReLU is able to further improve the performance of networks. In addition, we present a new training strategy to train DNNs with EPReLU. Experiments on four benchmarks including CIFAR10, CIFAR10, SVHN and ImageNet 2012 demonstrate the effectiveness of both EReLU and EPReLU. (C) 2017 Elsevier B.V. All rights reserved.

 

关键词Deep Neural Networks Elastic Rectified Linear Unit (Erelu) Elastic Parametric Rectified Linear Unit (Eprelu) Non-saturating Nonlinear Activation Function
DOI10.1016/j.neucom.2017.09.056
收录类别SCI ; EI
语种英语
WOS记录号WOS:000418370200108
EI入藏号20174104249355
引用统计
被引频次:3[WOS]   [WOS记录]     [WOS相关记录]
文献类型期刊论文
条目标识符http://ir.opt.ac.cn/handle/181661/30822
专题光学影像学习与分析中心
通讯作者Pang, Yanwei
作者单位1.Tianjin Univ, Sch Elect & Informat Engn, Tianjin 300072, Peoples R China;
2.Chinese Acad Sci, Xian Inst Opt & Precis Mech, State Key Lab Transient Opt & Photon, Ctr OPT IMagery Anal & Learning OPTIMAL, Xian 710119, Shaanxi, Peoples R China;
3.Tianjin Univ Technol & Educ, Sch Elect Engn, Tianjin 300222, Peoples R China;
4.Shenyang Univ, Coll Informat Engn, Shenyang 110044, Liaoning, Peoples R China
推荐引用方式
GB/T 7714
Jiang, Xiaoheng,Pang, Yanwei,Li, Xuelong,et al. Deep neural networks with Elastic Rectified Linear Units for object recognition[J]. NEUROCOMPUTING,2018,275:1132-1139.
APA Jiang, Xiaoheng,Pang, Yanwei,Li, Xuelong,Pan, Jing,&Xie, Yinghong.(2018).Deep neural networks with Elastic Rectified Linear Units for object recognition.NEUROCOMPUTING,275,1132-1139.
MLA Jiang, Xiaoheng,et al."Deep neural networks with Elastic Rectified Linear Units for object recognition".NEUROCOMPUTING 275(2018):1132-1139.
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
Deep neural networks(921KB)期刊论文出版稿开放获取CC BY-NC-SA浏览 请求全文
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[Jiang, Xiaoheng]的文章
[Pang, Yanwei]的文章
[Li, Xuelong]的文章
百度学术
百度学术中相似的文章
[Jiang, Xiaoheng]的文章
[Pang, Yanwei]的文章
[Li, Xuelong]的文章
必应学术
必应学术中相似的文章
[Jiang, Xiaoheng]的文章
[Pang, Yanwei]的文章
[Li, Xuelong]的文章
相关权益政策
暂无数据
收藏/分享
文件名: Deep neural networks with Elastic Rectified Linear Units for object recognition.pdf
格式: Adobe PDF
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。