包邮解析深度学习(第2版)

第0 章绪论·············1
0.1 引子·············...1
0.2 人工智能············..2
0.3 深度学习············..5
参考文献··············9
**篇机器学习···········...11
第1 章机器学习基础··········13
1.1 机器学习基本术语··········.13
1.2 模型评估与选择··········...14
1.2.1 经验误差与过拟合·········15
1.2.2 常用的评估方法·········..16
1.2.3 性能度量···········.17
1.2.4 偏差与方差··········...20
1.3 线性模型············..21
1.3.1 基本形式···········.21
1.3.2 线性回归···········.22
1.3.3 线性判别分析··········23
1.3.4 多分类学习··········...25
1.3.5 支持向量机··········...26
1.4 机器学习基本理论··········.29
1.4.1 PAC 学习理论··········30
1.4.2 No Free Lunch 定理········..31
1.4.3 奥卡姆剃刀原理·········..32
1.4.4 归纳偏置···········.32
1.5 总结与扩展阅读··········...33
目录ix
1.6 习题·············...34
参考文献··············35
第二篇深度学习基础··········.37
第2 章深度学习基本概念········...39
2.1 发展历程············..39
2.2 “端到端”思想··········...43
2.3 基本结构············..45
2.4 前馈运算············..47
2.5 反馈运算············..47
2.6 总结与扩展阅读··········...50
2.7 习题·············...51
参考文献··············52
第3 章卷积神经网络··········54
3.1 卷积神经网络基本组件与操作·······.54
3.1.1 符号表示···········.55
3.1.2 卷积层···········...55
3.1.3 汇合层···········...59
3.1.4 激活函数···········.61
3.1.5 全连接层···········.63
3.1.6 目标函数···········.63
3.2 卷积神经网络经典结构·········64
3.2.1 卷积神经网络结构中的重要概念·····...64
3.2.2 经典网络案例分析·········68
3.3 卷积神经网络的压缩·········..81
3.3.1 低秩近似···········.82
3.3.2 剪枝与稀疏约束·········..84
3.3.3 参数量化···········.88
3.3.4 二值网络···········.91
3.3.5 知识蒸馏···········.93
x 目录
3.3.6 紧凑的网络结构·········..95
3.4 总结与扩展阅读··········...97
3.5 习题·············...99
参考文献··············101
第4 章循环神经网络··········106
4.1 循环神经网络基本组件与操作·······.106
4.1.1 符号表示···········.107
4.1.2 记忆模块···········.108
4.1.3 参数学习···········.110
4.1.4 长程依赖问题··········113
4.2 循环神经网络经典结构·········115
4.2.1 长短时记忆网络·········..115
4.2.2 门控循环单元网络·········117
4.2.3 堆叠循环神经网络·········118
4.2.4 双向循环神经网络·········119
4.3 循环神经网络拓展··········.120
4.3.1 递归神经网络··········121
4.3.2 图神经网络··········...122
4.4 循环神经网络训练··········.123
4.5 总结与扩展阅读··········...124
4.6 习题·············...126
参考文献··············128
第5 章Transformer 网络········.130
5.1 Transformer 网络基本组件与操作······.130
5.1.1 符号表示···········.131
5.1.2 位置编码···········.132
5.1.3 多头注意力机制·········..133
5.1.4 编码器···········...137
5.1.5 解码器···········...138
目录xi
5.2 Transformer 网络经典结构········141
5.2.1 Transformer-XL········...141
5.2.2 Longformer··········...145
5.2.3 Reformer···········.146
5.2.4 Universal Transformer·······.149
5.3 Transformer 网络训练·········.151
5.4 总结与扩展阅读··········...154
5.5 习题·············...156
参考文献··············157
第三篇深度学习实践··········.159
第6 章数据扩充与数据预处理·······.161
6.1 简单的数据扩充方式·········..161
6.2 特殊的数据扩充方式·········..162
6.2.1 Fancy PCA·········...162
6.2.2 监督式数据扩充·········..163
6.2.3 mixup 法···········.164
6.2.4 自动化数据扩充·········..169
6.3 深度学习数据预处理·········..171
6.4 总结与扩展阅读··········...173
6.5 习题·············...174
参考文献··············177
第7 章网络参数初始化·········..179
7.1 全零初始化············179
7.2 随机初始化············181
7.3 其他初始化方法··········...194
7.4 总结与扩展阅读··········...194
7.5 习题·············...195
参考文献··············197
xii 目录
第8 章激活函数···········..198
8.1 Sigmoid 函数···········..198
8.2 tanh(x) 函数···········..199
8.3 修正线性单元（ReLU）········...200
8.4 Leaky ReLU···········201
8.5 参数化ReLU··········...201
8.6 随机化ReLU··········...203
8.7 指数化线性单元（ELU）········..204
8.8 激活函数实践···········..204
8.9 总结与扩展阅读··········...206
8.10 习题·············..207
参考文献··············209
第9 章目标函数···········..210
9.1 分类任务的目标函数·········..210
9.1.1 交叉熵损失函数·········..210
9.1.2 合页损失函数··········216
9.1.3 坡道损失函数··········219
9.1.4 大间隔交叉熵损失函数········221
9.1.5 中心损失函数··········228
9.2 回归任务的目标函数·········..231
9.2.1 ?1 损失函数··········..232
9.2.2 ?2 损失函数··········..235
9.2.3 Tukey’s biweight 损失函数·······238
9.3 其他任务的目标函数·········..239
9.4 总结与扩展阅读··········...241
9.5 习题·············...242
参考文献··············244
第10 章网络正则化··········..245
10.1 ?2 正则化············.246
目录xiii
10.2 ?1 正则化············.246
10.3 *大范数约束···········247
10.4 随机失活············.247
10.5 验证集的使用···········249
10.6 总结与扩展阅读··········..250
10.7 习题·············..251
参考文献··············253
第11 章超参数设定和网络训练······...254
11.1 网络超参数设定··········..254
11.1.1 输入图像像素大小········...254
11.1.2 卷积层超参数的设定········.255
11.1.3 汇合层超参数的设定········.256
11.2 训练技巧············.256
11.2.1 训练数据随机打乱········...256
11.2.2 学习率的设定·········...256
11.2.3 批规范化操作·········...258
11.2.4 网络模型优化算法选择·······...271
11.2.5 微调神经网络·········...282
11.3 总结与扩展阅读··········..286
11.4 习题·············..288
参考文献··············289
第12 章不平衡样本的处理········.291
12.1 数据层面处理方法··········292
12.1.1 数据重采样··········..292
12.1.2 类别平衡采样·········...292
12.2 算法层面处理方法··········294
12.2.1 代价敏感方法·········...294
12.2.2 代价敏感法中权重的指定方式······295
12.3 总结与扩展阅读··········..297
12.4 习题···x