Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
OpenDocCN
d2l-zh
提交
c6307085
D
d2l-zh
项目概览
OpenDocCN
/
d2l-zh
通知
2
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
D
d2l-zh
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode，发现更多精彩内容 >>
提交
c6307085
编写于
10月 04, 2018
作者:
A
Aston Zhang
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
2pass opt intro
上级
95f96269
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
50 addition
and
10 deletion
+50
-10
chapter_optimization/index.md
chapter_optimization/index.md
+2
-3
chapter_optimization/optimization-intro.md
chapter_optimization/optimization-intro.md
+48
-7
未找到文件。
chapter_optimization/index.md
浏览文件 @
c6307085
# 优化算法
如果你一直按照本书的顺序读到这里，
很可能已经使用了优化算法来训练深度学习模型。具体来说，在训练模型时，我们会使用优化算法不断迭代模型参数以最小化模型的损失函数
。当迭代终止时，模型的训练随之终止。此时的模型参数就是模型通过训练所学习到的参数。
如果你一直按照本书的顺序读到这里，
那么你已经使用了优化算法来训练深度学习模型。具体来说，在训练模型时，我们会使用优化算法不断迭代模型参数以降低模型损失函数的值
。当迭代终止时，模型的训练随之终止。此时的模型参数就是模型通过训练所学习到的参数。
优化算法对于深度学习十分重要。一方面，训练一个复杂的深度学习模型可能需要数小时、数日、甚至数周时间。而优化算法的表现直接影响模型
训练效率。另一方面，理解各种优化算法的原理以及其中各
参数的意义将有助于我们更有针对性地调参，从而使深度学习模型表现更好。
优化算法对于深度学习十分重要。一方面，训练一个复杂的深度学习模型可能需要数小时、数日、甚至数周时间。而优化算法的表现直接影响模型
的训练效率。另一方面，理解各种优化算法的原理以及其中超
参数的意义将有助于我们更有针对性地调参，从而使深度学习模型表现更好。
本章将详细介绍深度学习中的常用优化算法。
```
eval_rst
.. toctree::
...
...
chapter_optimization/optimization-intro.md
浏览文件 @
c6307085
# 优化与深度学习
本节将讨论优化与深度学习的关系以及优化在深度学习中的挑战。在一个深度学习问题中，通常我们会预先定义一个损失函数。有了损失函数以后，我们就可以使用优化算法试图
使其最小化。在优化中，这样的损失函数通常被称作优化问题的目标函数（objective function）。依据惯例，优化算法通常只考虑最小化目标函数。其实，任何最大化问题都可以很容易地转化为最小化问题：我们只需把目标函数前面的正号或负号取相反
。
本节将讨论优化与深度学习的关系以及优化在深度学习中的挑战。在一个深度学习问题中，通常我们会预先定义一个损失函数。有了损失函数以后，我们就可以使用优化算法试图
将其最小化。在优化中，这样的损失函数通常被称作优化问题的目标函数（objective function）。依据惯例，优化算法通常只考虑最小化目标函数。其实，任何最大化问题都可以很容易地转化为最小化问题：我们只需令目标函数的相反数为新的目标函数
。
虽然优化为深度学习提供了最小化损失函数的方法，但本质上，这两者之间的目标是有区别的。
## 优化与深度学习的关系
虽然优化为深度学习提供了最小化损失函数的方法，但本质上，优化与深度学习之间的目标是有区别的。
在
[
“模型选择、欠拟合和过拟合”
](
../chapter_deep-learning-basics/underfit-overfit.md
)
一节中，我们区分了训练误差和泛化误差。
由于优化算法的目标函数通常是一个基于训练数据集的损失函数，优化的目标在于降低训练误差。
而深度学习的目标在于降低泛化误差。
...
...
@@ -13,7 +15,7 @@
## 优化在深度学习中的挑战
绝大多数深度学习中的目标函数都很复杂。因此，很多优化问题并不存在解析解，而需要使用基于数值方法的优化算法找到近似解。这类优化算法一般通过不断迭代更新解的数值来找到近似解。我们讨论的优化算法都是这类基于数值方法的算法
。
我们在
[
“线性回归”
](
../chapter_deep-learning-basics/linear-regression.md
)
一节中对优化问题的解析解和数值解做了区分。深度学习中绝大多数的目标函数都很复杂。因此，很多优化问题并不存在解析解，而需要使用基于数值方法的优化算法找到近似解，即数值解。我们讨论的优化算法都是这类基于数值方法的算法。为了求得最小化目标函数的数值解，我们将通过优化算法有限次迭代模型参数来尽可能降低损失函数的值
。
优化在深度学习中有很多挑战。以下描述了其中的两个挑战：局部最小值和鞍点。为了更好地描述问题，我们先导入本节中实验需要的包或模块。
...
...
@@ -52,6 +54,19 @@ gb.plt.xlabel('x')
gb.plt.ylabel('f(x)');
```
```
{.json .output n=2}
[
{
"data": {
"image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<!-- Created with matplotlib (http://matplotlib.org/) -->\n<svg height=\"184.15625pt\" version=\"1.1\" viewBox=\"0 0 305.329712 184.15625\" width=\"305.329712pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n <defs>\n <style type=\"text/css\">\n*{stroke-linecap:butt;stroke-linejoin:round;}\n </style>\n </defs>\n <g id=\"figure_1\">\n <g id=\"patch_1\">\n <path d=\"M 0 184.15625 \nL 305.329712 184.15625 \nL 305.329712 -0 \nL 0 -0 \nz\n\" style=\"fill:none;\"/>\n </g>\n <g id=\"axes_1\">\n <g id=\"patch_2\">\n <path d=\"M 42.620313 146.6 \nL 293.720313 146.6 \nL 293.720313 10.7 \nL 42.620313 10.7 \nz\n\" style=\"fill:#ffffff;\"/>\n </g>\n <g id=\"matplotlib.axis_1\">\n <g id=\"xtick_1\">\n <g id=\"line2d_1\">\n <defs>\n <path d=\"M 0 0 \nL 0 3.5 \n\" id=\"ma486107ba3\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n </defs>\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"54.033949\" xlink:href=\"#ma486107ba3\" y=\"146.6\"/>\n </g>\n </g>\n <g id=\"text_1\">\n <!-- \u22121.0 -->\n <defs>\n <path d=\"M 10.59375 35.5 \nL 73.1875 35.5 \nL 73.1875 27.203125 \nL 10.59375 27.203125 \nz\n\" id=\"DejaVuSans-2212\"/>\n <path d=\"M 12.40625 8.296875 \nL 28.515625 8.296875 \nL 28.515625 63.921875 \nL 10.984375 60.40625 \nL 10.984375 69.390625 \nL 28.421875 72.90625 \nL 38.28125 72.90625 \nL 38.28125 8.296875 \nL 54.390625 8.296875 \nL 54.390625 0 \nL 12.40625 0 \nz\n\" id=\"DejaVuSans-31\"/>\n <path d=\"M 10.6875 12.40625 \nL 21 12.40625 \nL 21 0 \nL 10.6875 0 \nz\n\" id=\"DejaVuSans-2e\"/>\n <path d=\"M 31.78125 66.40625 \nQ 24.171875 66.40625 20.328125 58.90625 \nQ 16.5 51.421875 16.5 36.375 \nQ 16.5 21.390625 20.328125 13.890625 \nQ 24.171875 6.390625 31.78125 6.390625 \nQ 39.453125 6.390625 43.28125 13.890625 \nQ 47.125 21.390625 47.125 36.375 \nQ 47.125 51.421875 43.28125 58.90625 \nQ 39.453125 66.40625 31.78125 66.40625 \nz\nM 31.78125 74.21875 \nQ 44.046875 74.21875 50.515625 64.515625 \nQ 56.984375 54.828125 56.984375 36.375 \nQ 56.984375 17.96875 50.515625 8.265625 \nQ 44.046875 -1.421875 31.78125 -1.421875 \nQ 19.53125 -1.421875 13.0625 8.265625 \nQ 6.59375 17.96875 6.59375 36.375 \nQ 6.59375 54.828125 13.0625 64.515625 \nQ 19.53125 74.21875 31.78125 74.21875 \nz\n\" id=\"DejaVuSans-30\"/>\n </defs>\n <g transform=\"translate(41.892543 161.198437)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-2212\"/>\n <use x=\"83.789062\" xlink:href=\"#DejaVuSans-31\"/>\n <use x=\"147.412109\" xlink:href=\"#DejaVuSans-2e\"/>\n <use x=\"179.199219\" xlink:href=\"#DejaVuSans-30\"/>\n </g>\n </g>\n </g>\n <g id=\"xtick_2\">\n <g id=\"line2d_2\">\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"93.391316\" xlink:href=\"#ma486107ba3\" y=\"146.6\"/>\n </g>\n </g>\n <g id=\"text_2\">\n <!-- \u22120.5 -->\n <defs>\n <path d=\"M 10.796875 72.90625 \nL 49.515625 72.90625 \nL 49.515625 64.59375 \nL 19.828125 64.59375 \nL 19.828125 46.734375 \nQ 21.96875 47.46875 24.109375 47.828125 \nQ 26.265625 48.1875 28.421875 48.1875 \nQ 40.625 48.1875 47.75 41.5 \nQ 54.890625 34.8125 54.890625 23.390625 \nQ 54.890625 11.625 47.5625 5.09375 \nQ 40.234375 -1.421875 26.90625 -1.421875 \nQ 22.3125 -1.421875 17.546875 -0.640625 \nQ 12.796875 0.140625 7.71875 1.703125 \nL 7.71875 11.625 \nQ 12.109375 9.234375 16.796875 8.0625 \nQ 21.484375 6.890625 26.703125 6.890625 \nQ 35.15625 6.890625 40.078125 11.328125 \nQ 45.015625 15.765625 45.015625 23.390625 \nQ 45.015625 31 40.078125 35.4375 \nQ 35.15625 39.890625 26.703125 39.890625 \nQ 22.75 39.890625 18.8125 39.015625 \nQ 14.890625 38.140625 10.796875 36.28125 \nz\n\" id=\"DejaVuSans-35\"/>\n </defs>\n <g transform=\"translate(81.249909 161.198437)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-2212\"/>\n <use x=\"83.789062\" xlink:href=\"#DejaVuSans-30\"/>\n <use x=\"147.412109\" xlink:href=\"#DejaVuSans-2e\"/>\n <use x=\"179.199219\" xlink:href=\"#DejaVuSans-35\"/>\n </g>\n </g>\n </g>\n <g id=\"xtick_3\">\n <g id=\"line2d_3\">\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"132.748682\" xlink:href=\"#ma486107ba3\" y=\"146.6\"/>\n </g>\n </g>\n <g id=\"text_3\">\n <!-- 0.0 -->\n <g transform=\"translate(124.79712 161.198437)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-30\"/>\n <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n </g>\n </g>\n </g>\n <g id=\"xtick_4\">\n <g id=\"line2d_4\">\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"172.106049\" xlink:href=\"#ma486107ba3\" y=\"146.6\"/>\n </g>\n </g>\n <g id=\"text_4\">\n <!-- 0.5 -->\n <g transform=\"translate(164.154487 161.198437)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-30\"/>\n <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n <use x=\"95.410156\" xlink:href=\"#DejaVuSans-35\"/>\n </g>\n </g>\n </g>\n <g id=\"xtick_5\">\n <g id=\"line2d_5\">\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"211.463416\" xlink:href=\"#ma486107ba3\" y=\"146.6\"/>\n </g>\n </g>\n <g id=\"text_5\">\n <!-- 1.0 -->\n <g transform=\"translate(203.511853 161.198437)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-31\"/>\n <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n </g>\n </g>\n </g>\n <g id=\"xtick_6\">\n <g id=\"line2d_6\">\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"250.820783\" xlink:href=\"#ma486107ba3\" y=\"146.6\"/>\n </g>\n </g>\n <g id=\"text_6\">\n <!-- 1.5 -->\n <g transform=\"translate(242.86922 161.198437)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-31\"/>\n <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n <use x=\"95.410156\" xlink:href=\"#DejaVuSans-35\"/>\n </g>\n </g>\n </g>\n <g id=\"xtick_7\">\n <g id=\"line2d_7\">\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"290.178149\" xlink:href=\"#ma486107ba3\" y=\"146.6\"/>\n </g>\n </g>\n <g id=\"text_7\">\n <!-- 2.0 -->\n <defs>\n <path d=\"M 19.1875 8.296875 \nL 53.609375 8.296875 \nL 53.609375 0 \nL 7.328125 0 \nL 7.328125 8.296875 \nQ 12.9375 14.109375 22.625 23.890625 \nQ 32.328125 33.6875 34.8125 36.53125 \nQ 39.546875 41.84375 41.421875 45.53125 \nQ 43.3125 49.21875 43.3125 52.78125 \nQ 43.3125 58.59375 39.234375 62.25 \nQ 35.15625 65.921875 28.609375 65.921875 \nQ 23.96875 65.921875 18.8125 64.3125 \nQ 13.671875 62.703125 7.8125 59.421875 \nL 7.8125 69.390625 \nQ 13.765625 71.78125 18.9375 73 \nQ 24.125 74.21875 28.421875 74.21875 \nQ 39.75 74.21875 46.484375 68.546875 \nQ 53.21875 62.890625 53.21875 53.421875 \nQ 53.21875 48.921875 51.53125 44.890625 \nQ 49.859375 40.875 45.40625 35.40625 \nQ 44.1875 33.984375 37.640625 27.21875 \nQ 31.109375 20.453125 19.1875 8.296875 \nz\n\" id=\"DejaVuSans-32\"/>\n </defs>\n <g transform=\"translate(282.226587 161.198437)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-32\"/>\n <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n </g>\n </g>\n </g>\n <g id=\"text_8\">\n <!-- x -->\n <defs>\n <path d=\"M 54.890625 54.6875 \nL 35.109375 28.078125 \nL 55.90625 0 \nL 45.3125 0 \nL 29.390625 21.484375 \nL 13.484375 0 \nL 2.875 0 \nL 24.125 28.609375 \nL 4.6875 54.6875 \nL 15.28125 54.6875 \nL 29.78125 35.203125 \nL 44.28125 54.6875 \nz\n\" id=\"DejaVuSans-78\"/>\n </defs>\n <g transform=\"translate(165.210938 174.876562)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-78\"/>\n </g>\n </g>\n </g>\n <g id=\"matplotlib.axis_2\">\n <g id=\"ytick_1\">\n <g id=\"line2d_8\">\n <defs>\n <path d=\"M 0 0 \nL -3.5 0 \n\" id=\"mff8792995b\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n </defs>\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#mff8792995b\" y=\"138.423853\"/>\n </g>\n </g>\n <g id=\"text_9\">\n <!-- \u22121 -->\n <g transform=\"translate(20.878125 142.223072)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-2212\"/>\n <use x=\"83.789062\" xlink:href=\"#DejaVuSans-31\"/>\n </g>\n </g>\n </g>\n <g id=\"ytick_2\">\n <g id=\"line2d_9\">\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#mff8792995b\" y=\"95.122727\"/>\n </g>\n </g>\n <g id=\"text_10\">\n <!-- 0 -->\n <g transform=\"translate(29.257813 98.921946)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-30\"/>\n </g>\n </g>\n </g>\n <g id=\"ytick_3\">\n <g id=\"line2d_10\">\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#mff8792995b\" y=\"51.821601\"/>\n </g>\n </g>\n <g id=\"text_11\">\n <!-- 1 -->\n <g transform=\"translate(29.257813 55.62082)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-31\"/>\n </g>\n </g>\n </g>\n <g id=\"text_12\">\n <!-- f(x) -->\n <defs>\n <path d=\"M 37.109375 75.984375 \nL 37.109375 68.5 \nL 28.515625 68.5 \nQ 23.6875 68.5 21.796875 66.546875 \nQ 19.921875 64.59375 19.921875 59.515625 \nL 19.921875 54.6875 \nL 34.71875 54.6875 \nL 34.71875 47.703125 \nL 19.921875 47.703125 \nL 19.921875 0 \nL 10.890625 0 \nL 10.890625 47.703125 \nL 2.296875 47.703125 \nL 2.296875 54.6875 \nL 10.890625 54.6875 \nL 10.890625 58.5 \nQ 10.890625 67.625 15.140625 71.796875 \nQ 19.390625 75.984375 28.609375 75.984375 \nz\n\" id=\"DejaVuSans-66\"/>\n <path d=\"M 31 75.875 \nQ 24.46875 64.65625 21.28125 53.65625 \nQ 18.109375 42.671875 18.109375 31.390625 \nQ 18.109375 20.125 21.3125 9.0625 \nQ 24.515625 -2 31 -13.1875 \nL 23.1875 -13.1875 \nQ 15.875 -1.703125 12.234375 9.375 \nQ 8.59375 20.453125 8.59375 31.390625 \nQ 8.59375 42.28125 12.203125 53.3125 \nQ 15.828125 64.359375 23.1875 75.875 \nz\n\" id=\"DejaVuSans-28\"/>\n <path d=\"M 8.015625 75.875 \nL 15.828125 75.875 \nQ 23.140625 64.359375 26.78125 53.3125 \nQ 30.421875 42.28125 30.421875 31.390625 \nQ 30.421875 20.453125 26.78125 9.375 \nQ 23.140625 -1.703125 15.828125 -13.1875 \nL 8.015625 -13.1875 \nQ 14.5 -2 17.703125 9.0625 \nQ 20.90625 20.125 20.90625 31.390625 \nQ 20.90625 42.671875 17.703125 53.65625 \nQ 14.5 64.65625 8.015625 75.875 \nz\n\" id=\"DejaVuSans-29\"/>\n </defs>\n <g transform=\"translate(14.798438 87.271094)rotate(-90)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-66\"/>\n <use x=\"35.205078\" xlink:href=\"#DejaVuSans-28\"/>\n <use x=\"74.21875\" xlink:href=\"#DejaVuSans-78\"/>\n <use x=\"133.398438\" xlink:href=\"#DejaVuSans-29\"/>\n </g>\n </g>\n </g>\n <g id=\"line2d_11\">\n <path clip-path=\"url(#pea42bcdc15)\" d=\"M 54.033949 51.821601 \nL 61.905422 58.059091 \nL 69.776896 67.09765 \nL 77.648369 77.306493 \nL 85.519842 87.094257 \nL 93.391316 95.122727 \nL 101.262789 100.475041 \nL 109.134262 102.758256 \nL 117.005736 102.128997 \nL 124.877209 99.240909 \nL 132.748682 95.122727 \nL 140.620156 91.004545 \nL 148.491629 88.116458 \nL 156.363102 87.487198 \nL 164.234576 89.770414 \nL 172.106049 95.122727 \nL 179.977523 103.151198 \nL 187.848996 112.938962 \nL 195.720469 123.147805 \nL 203.591943 132.186364 \nL 211.463416 138.423853 \nL 219.334889 140.422727 \nL 227.206363 137.160344 \nL 235.077836 128.21002 \nL 242.949309 113.855825 \nL 250.820783 95.122727 \nL 258.692256 73.713473 \nL 266.563729 51.85473 \nL 274.435203 32.066303 \nL 282.306676 16.877273 \n\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n </g>\n <g id=\"patch_3\">\n <path d=\"M 42.620313 146.6 \nL 42.620313 10.7 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n </g>\n <g id=\"patch_4\">\n <path d=\"M 293.720313 146.6 \nL 293.720313 10.7 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n </g>\n <g id=\"patch_5\">\n <path d=\"M 42.620313 146.6 \nL 293.720313 146.6 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n </g>\n <g id=\"patch_6\">\n <path d=\"M 42.620313 10.7 \nL 293.720313 10.7 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n </g>\n <g id=\"patch_7\">\n <path d=\"M 109.113525 126.824204 \nQ 109.1229 117.386339 109.131165 109.066508 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\n <path d=\"M 107.127192 113.064519 \nL 109.131165 109.066508 \nL 111.12719 113.068492 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\n </g>\n <g id=\"text_13\">\n <!-- local minimum -->\n <defs>\n <path d=\"M 9.421875 75.984375 \nL 18.40625 75.984375 \nL 18.40625 0 \nL 9.421875 0 \nz\n\" id=\"DejaVuSans-6c\"/>\n <path d=\"M 30.609375 48.390625 \nQ 23.390625 48.390625 19.1875 42.75 \nQ 14.984375 37.109375 14.984375 27.296875 \nQ 14.984375 17.484375 19.15625 11.84375 \nQ 23.34375 6.203125 30.609375 6.203125 \nQ 37.796875 6.203125 41.984375 11.859375 \nQ 46.1875 17.53125 46.1875 27.296875 \nQ 46.1875 37.015625 41.984375 42.703125 \nQ 37.796875 48.390625 30.609375 48.390625 \nz\nM 30.609375 56 \nQ 42.328125 56 49.015625 48.375 \nQ 55.71875 40.765625 55.71875 27.296875 \nQ 55.71875 13.875 49.015625 6.21875 \nQ 42.328125 -1.421875 30.609375 -1.421875 \nQ 18.84375 -1.421875 12.171875 6.21875 \nQ 5.515625 13.875 5.515625 27.296875 \nQ 5.515625 40.765625 12.171875 48.375 \nQ 18.84375 56 30.609375 56 \nz\n\" id=\"DejaVuSans-6f\"/>\n <path d=\"M 48.78125 52.59375 \nL 48.78125 44.1875 \nQ 44.96875 46.296875 41.140625 47.34375 \nQ 37.3125 48.390625 33.40625 48.390625 \nQ 24.65625 48.390625 19.8125 42.84375 \nQ 14.984375 37.3125 14.984375 27.296875 \nQ 14.984375 17.28125 19.8125 11.734375 \nQ 24.65625 6.203125 33.40625 6.203125 \nQ 37.3125 6.203125 41.140625 7.25 \nQ 44.96875 8.296875 48.78125 10.40625 \nL 48.78125 2.09375 \nQ 45.015625 0.34375 40.984375 -0.53125 \nQ 36.96875 -1.421875 32.421875 -1.421875 \nQ 20.0625 -1.421875 12.78125 6.34375 \nQ 5.515625 14.109375 5.515625 27.296875 \nQ 5.515625 40.671875 12.859375 48.328125 \nQ 20.21875 56 33.015625 56 \nQ 37.15625 56 41.109375 55.140625 \nQ 45.0625 54.296875 48.78125 52.59375 \nz\n\" id=\"DejaVuSans-63\"/>\n <path d=\"M 34.28125 27.484375 \nQ 23.390625 27.484375 19.1875 25 \nQ 14.984375 22.515625 14.984375 16.5 \nQ 14.984375 11.71875 18.140625 8.90625 \nQ 21.296875 6.109375 26.703125 6.109375 \nQ 34.1875 6.109375 38.703125 11.40625 \nQ 43.21875 16.703125 43.21875 25.484375 \nL 43.21875 27.484375 \nz\nM 52.203125 31.203125 \nL 52.203125 0 \nL 43.21875 0 \nL 43.21875 8.296875 \nQ 40.140625 3.328125 35.546875 0.953125 \nQ 30.953125 -1.421875 24.3125 -1.421875 \nQ 15.921875 -1.421875 10.953125 3.296875 \nQ 6 8.015625 6 15.921875 \nQ 6 25.140625 12.171875 29.828125 \nQ 18.359375 34.515625 30.609375 34.515625 \nL 43.21875 34.515625 \nL 43.21875 35.40625 \nQ 43.21875 41.609375 39.140625 45 \nQ 35.0625 48.390625 27.6875 48.390625 \nQ 23 48.390625 18.546875 47.265625 \nQ 14.109375 46.140625 10.015625 43.890625 \nL 10.015625 52.203125 \nQ 14.9375 54.109375 19.578125 55.046875 \nQ 24.21875 56 28.609375 56 \nQ 40.484375 56 46.34375 49.84375 \nQ 52.203125 43.703125 52.203125 31.203125 \nz\n\" id=\"DejaVuSans-61\"/>\n <path id=\"DejaVuSans-20\"/>\n <path d=\"M 52 44.1875 \nQ 55.375 50.25 60.0625 53.125 \nQ 64.75 56 71.09375 56 \nQ 79.640625 56 84.28125 50.015625 \nQ 88.921875 44.046875 88.921875 33.015625 \nL 88.921875 0 \nL 79.890625 0 \nL 79.890625 32.71875 \nQ 79.890625 40.578125 77.09375 44.375 \nQ 74.3125 48.1875 68.609375 48.1875 \nQ 61.625 48.1875 57.5625 43.546875 \nQ 53.515625 38.921875 53.515625 30.90625 \nL 53.515625 0 \nL 44.484375 0 \nL 44.484375 32.71875 \nQ 44.484375 40.625 41.703125 44.40625 \nQ 38.921875 48.1875 33.109375 48.1875 \nQ 26.21875 48.1875 22.15625 43.53125 \nQ 18.109375 38.875 18.109375 30.90625 \nL 18.109375 0 \nL 9.078125 0 \nL 9.078125 54.6875 \nL 18.109375 54.6875 \nL 18.109375 46.1875 \nQ 21.1875 51.21875 25.484375 53.609375 \nQ 29.78125 56 35.6875 56 \nQ 41.65625 56 45.828125 52.96875 \nQ 50 49.953125 52 44.1875 \nz\n\" id=\"DejaVuSans-6d\"/>\n <path d=\"M 9.421875 54.6875 \nL 18.40625 54.6875 \nL 18.40625 0 \nL 9.421875 0 \nz\nM 9.421875 75.984375 \nL 18.40625 75.984375 \nL 18.40625 64.59375 \nL 9.421875 64.59375 \nz\n\" id=\"DejaVuSans-69\"/>\n <path d=\"M 54.890625 33.015625 \nL 54.890625 0 \nL 45.90625 0 \nL 45.90625 32.71875 \nQ 45.90625 40.484375 42.875 44.328125 \nQ 39.84375 48.1875 33.796875 48.1875 \nQ 26.515625 48.1875 22.3125 43.546875 \nQ 18.109375 38.921875 18.109375 30.90625 \nL 18.109375 0 \nL 9.078125 0 \nL 9.078125 54.6875 \nL 18.109375 54.6875 \nL 18.109375 46.1875 \nQ 21.34375 51.125 25.703125 53.5625 \nQ 30.078125 56 35.796875 56 \nQ 45.21875 56 50.046875 50.171875 \nQ 54.890625 44.34375 54.890625 33.015625 \nz\n\" id=\"DejaVuSans-6e\"/>\n <path d=\"M 8.5 21.578125 \nL 8.5 54.6875 \nL 17.484375 54.6875 \nL 17.484375 21.921875 \nQ 17.484375 14.15625 20.5 10.265625 \nQ 23.53125 6.390625 29.59375 6.390625 \nQ 36.859375 6.390625 41.078125 11.03125 \nQ 45.3125 15.671875 45.3125 23.6875 \nL 45.3125 54.6875 \nL 54.296875 54.6875 \nL 54.296875 0 \nL 45.3125 0 \nL 45.3125 8.40625 \nQ 42.046875 3.421875 37.71875 1 \nQ 33.40625 -1.421875 27.6875 -1.421875 \nQ 18.265625 -1.421875 13.375 4.4375 \nQ 8.5 10.296875 8.5 21.578125 \nz\nM 31.109375 56 \nz\n\" id=\"DejaVuSans-75\"/>\n </defs>\n <g transform=\"translate(72.138338 138.423853)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-6c\"/>\n <use x=\"27.783203\" xlink:href=\"#DejaVuSans-6f\"/>\n <use x=\"88.964844\" xlink:href=\"#DejaVuSans-63\"/>\n <use x=\"143.945312\" xlink:href=\"#DejaVuSans-61\"/>\n <use x=\"205.224609\" xlink:href=\"#DejaVuSans-6c\"/>\n <use x=\"233.007812\" xlink:href=\"#DejaVuSans-20\"/>\n <use x=\"264.794922\" xlink:href=\"#DejaVuSans-6d\"/>\n <use x=\"362.207031\" xlink:href=\"#DejaVuSans-69\"/>\n <use x=\"389.990234\" xlink:href=\"#DejaVuSans-6e\"/>\n <use x=\"453.369141\" xlink:href=\"#DejaVuSans-69\"/>\n <use x=\"481.152344\" xlink:href=\"#DejaVuSans-6d\"/>\n <use x=\"578.564453\" xlink:href=\"#DejaVuSans-75\"/>\n <use x=\"641.943359\" xlink:href=\"#DejaVuSans-6d\"/>\n </g>\n </g>\n <g id=\"patch_8\">\n <path d=\"M 220.407145 66.562567 \nQ 219.88643 100.408884 219.382913 133.137299 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\n <path d=\"M 221.444207 129.168538 \nL 219.382913 133.137299 \nL 217.444681 129.107006 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\n </g>\n <g id=\"text_14\">\n <!-- global minimum -->\n <defs>\n <path d=\"M 45.40625 27.984375 \nQ 45.40625 37.75 41.375 43.109375 \nQ 37.359375 48.484375 30.078125 48.484375 \nQ 22.859375 48.484375 18.828125 43.109375 \nQ 14.796875 37.75 14.796875 27.984375 \nQ 14.796875 18.265625 18.828125 12.890625 \nQ 22.859375 7.515625 30.078125 7.515625 \nQ 37.359375 7.515625 41.375 12.890625 \nQ 45.40625 18.265625 45.40625 27.984375 \nz\nM 54.390625 6.78125 \nQ 54.390625 -7.171875 48.1875 -13.984375 \nQ 42 -20.796875 29.203125 -20.796875 \nQ 24.46875 -20.796875 20.265625 -20.09375 \nQ 16.0625 -19.390625 12.109375 -17.921875 \nL 12.109375 -9.1875 \nQ 16.0625 -11.328125 19.921875 -12.34375 \nQ 23.78125 -13.375 27.78125 -13.375 \nQ 36.625 -13.375 41.015625 -8.765625 \nQ 45.40625 -4.15625 45.40625 5.171875 \nL 45.40625 9.625 \nQ 42.625 4.78125 38.28125 2.390625 \nQ 33.9375 0 27.875 0 \nQ 17.828125 0 11.671875 7.65625 \nQ 5.515625 15.328125 5.515625 27.984375 \nQ 5.515625 40.671875 11.671875 48.328125 \nQ 17.828125 56 27.875 56 \nQ 33.9375 56 38.28125 53.609375 \nQ 42.625 51.21875 45.40625 46.390625 \nL 45.40625 54.6875 \nL 54.390625 54.6875 \nz\n\" id=\"DejaVuSans-67\"/>\n <path d=\"M 48.6875 27.296875 \nQ 48.6875 37.203125 44.609375 42.84375 \nQ 40.53125 48.484375 33.40625 48.484375 \nQ 26.265625 48.484375 22.1875 42.84375 \nQ 18.109375 37.203125 18.109375 27.296875 \nQ 18.109375 17.390625 22.1875 11.75 \nQ 26.265625 6.109375 33.40625 6.109375 \nQ 40.53125 6.109375 44.609375 11.75 \nQ 48.6875 17.390625 48.6875 27.296875 \nz\nM 18.109375 46.390625 \nQ 20.953125 51.265625 25.265625 53.625 \nQ 29.59375 56 35.59375 56 \nQ 45.5625 56 51.78125 48.09375 \nQ 58.015625 40.1875 58.015625 27.296875 \nQ 58.015625 14.40625 51.78125 6.484375 \nQ 45.5625 -1.421875 35.59375 -1.421875 \nQ 29.59375 -1.421875 25.265625 0.953125 \nQ 20.953125 3.328125 18.109375 8.203125 \nL 18.109375 0 \nL 9.078125 0 \nL 9.078125 75.984375 \nL 18.109375 75.984375 \nz\n\" id=\"DejaVuSans-62\"/>\n </defs>\n <g transform=\"translate(179.977523 60.481826)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-67\"/>\n <use x=\"63.476562\" xlink:href=\"#DejaVuSans-6c\"/>\n <use x=\"91.259766\" xlink:href=\"#DejaVuSans-6f\"/>\n <use x=\"152.441406\" xlink:href=\"#DejaVuSans-62\"/>\n <use x=\"215.917969\" xlink:href=\"#DejaVuSans-61\"/>\n <use x=\"277.197266\" xlink:href=\"#DejaVuSans-6c\"/>\n <use x=\"304.980469\" xlink:href=\"#DejaVuSans-20\"/>\n <use x=\"336.767578\" xlink:href=\"#DejaVuSans-6d\"/>\n <use x=\"434.179688\" xlink:href=\"#DejaVuSans-69\"/>\n <use x=\"461.962891\" xlink:href=\"#DejaVuSans-6e\"/>\n <use x=\"525.341797\" xlink:href=\"#DejaVuSans-69\"/>\n <use x=\"553.125\" xlink:href=\"#DejaVuSans-6d\"/>\n <use x=\"650.537109\" xlink:href=\"#DejaVuSans-75\"/>\n <use x=\"713.916016\" xlink:href=\"#DejaVuSans-6d\"/>\n </g>\n </g>\n </g>\n </g>\n <defs>\n <clipPath id=\"pea42bcdc15\">\n <rect height=\"135.9\" width=\"251.1\" x=\"42.620313\" y=\"10.7\"/>\n </clipPath>\n </defs>\n</svg>\n",
"text/plain": "<Figure size 324x180 with 1 Axes>"
},
"metadata": {},
"output_type": "display_data"
}
]
```
深度学习模型的目标函数可能有若干局部最优值。当一个优化问题的数值解在局部最优解附近时，由于目标函数有关解的梯度接近或变成零，最终迭代求得的数值解可能只令目标函数局部最小化而非全局最小化。
### 鞍点
...
...
@@ -71,6 +86,19 @@ gb.plt.xlabel('x')
gb.plt.ylabel('f(x)');
```
```
{.json .output n=3}
[
{
"data": {
"image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<!-- Created with matplotlib (http://matplotlib.org/) -->\n<svg height=\"184.15625pt\" version=\"1.1\" viewBox=\"0 0 304.420313 184.15625\" width=\"304.420313pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n <defs>\n <style type=\"text/css\">\n*{stroke-linecap:butt;stroke-linejoin:round;}\n </style>\n </defs>\n <g id=\"figure_1\">\n <g id=\"patch_1\">\n <path d=\"M 0 184.15625 \nL 304.420313 184.15625 \nL 304.420313 -0 \nL 0 -0 \nz\n\" style=\"fill:none;\"/>\n </g>\n <g id=\"axes_1\">\n <g id=\"patch_2\">\n <path d=\"M 42.620313 146.6 \nL 293.720313 146.6 \nL 293.720313 10.7 \nL 42.620313 10.7 \nz\n\" style=\"fill:#ffffff;\"/>\n </g>\n <g id=\"matplotlib.axis_1\">\n <g id=\"xtick_1\">\n <g id=\"line2d_1\">\n <defs>\n <path d=\"M 0 0 \nL 0 3.5 \n\" id=\"m10f676b948\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n </defs>\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"54.033949\" xlink:href=\"#m10f676b948\" y=\"146.6\"/>\n </g>\n </g>\n <g id=\"text_1\">\n <!-- \u22122 -->\n <defs>\n <path d=\"M 10.59375 35.5 \nL 73.1875 35.5 \nL 73.1875 27.203125 \nL 10.59375 27.203125 \nz\n\" id=\"DejaVuSans-2212\"/>\n <path d=\"M 19.1875 8.296875 \nL 53.609375 8.296875 \nL 53.609375 0 \nL 7.328125 0 \nL 7.328125 8.296875 \nQ 12.9375 14.109375 22.625 23.890625 \nQ 32.328125 33.6875 34.8125 36.53125 \nQ 39.546875 41.84375 41.421875 45.53125 \nQ 43.3125 49.21875 43.3125 52.78125 \nQ 43.3125 58.59375 39.234375 62.25 \nQ 35.15625 65.921875 28.609375 65.921875 \nQ 23.96875 65.921875 18.8125 64.3125 \nQ 13.671875 62.703125 7.8125 59.421875 \nL 7.8125 69.390625 \nQ 13.765625 71.78125 18.9375 73 \nQ 24.125 74.21875 28.421875 74.21875 \nQ 39.75 74.21875 46.484375 68.546875 \nQ 53.21875 62.890625 53.21875 53.421875 \nQ 53.21875 48.921875 51.53125 44.890625 \nQ 49.859375 40.875 45.40625 35.40625 \nQ 44.1875 33.984375 37.640625 27.21875 \nQ 31.109375 20.453125 19.1875 8.296875 \nz\n\" id=\"DejaVuSans-32\"/>\n </defs>\n <g transform=\"translate(46.662855 161.198437)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-2212\"/>\n <use x=\"83.789062\" xlink:href=\"#DejaVuSans-32\"/>\n </g>\n </g>\n </g>\n <g id=\"xtick_2\">\n <g id=\"line2d_2\">\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"112.565417\" xlink:href=\"#m10f676b948\" y=\"146.6\"/>\n </g>\n </g>\n <g id=\"text_2\">\n <!-- \u22121 -->\n <defs>\n <path d=\"M 12.40625 8.296875 \nL 28.515625 8.296875 \nL 28.515625 63.921875 \nL 10.984375 60.40625 \nL 10.984375 69.390625 \nL 28.421875 72.90625 \nL 38.28125 72.90625 \nL 38.28125 8.296875 \nL 54.390625 8.296875 \nL 54.390625 0 \nL 12.40625 0 \nz\n\" id=\"DejaVuSans-31\"/>\n </defs>\n <g transform=\"translate(105.194324 161.198437)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-2212\"/>\n <use x=\"83.789062\" xlink:href=\"#DejaVuSans-31\"/>\n </g>\n </g>\n </g>\n <g id=\"xtick_3\">\n <g id=\"line2d_3\">\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"171.096886\" xlink:href=\"#m10f676b948\" y=\"146.6\"/>\n </g>\n </g>\n <g id=\"text_3\">\n <!-- 0 -->\n <defs>\n <path d=\"M 31.78125 66.40625 \nQ 24.171875 66.40625 20.328125 58.90625 \nQ 16.5 51.421875 16.5 36.375 \nQ 16.5 21.390625 20.328125 13.890625 \nQ 24.171875 6.390625 31.78125 6.390625 \nQ 39.453125 6.390625 43.28125 13.890625 \nQ 47.125 21.390625 47.125 36.375 \nQ 47.125 51.421875 43.28125 58.90625 \nQ 39.453125 66.40625 31.78125 66.40625 \nz\nM 31.78125 74.21875 \nQ 44.046875 74.21875 50.515625 64.515625 \nQ 56.984375 54.828125 56.984375 36.375 \nQ 56.984375 17.96875 50.515625 8.265625 \nQ 44.046875 -1.421875 31.78125 -1.421875 \nQ 19.53125 -1.421875 13.0625 8.265625 \nQ 6.59375 17.96875 6.59375 36.375 \nQ 6.59375 54.828125 13.0625 64.515625 \nQ 19.53125 74.21875 31.78125 74.21875 \nz\n\" id=\"DejaVuSans-30\"/>\n </defs>\n <g transform=\"translate(167.915636 161.198437)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-30\"/>\n </g>\n </g>\n </g>\n <g id=\"xtick_4\">\n <g id=\"line2d_4\">\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"229.628354\" xlink:href=\"#m10f676b948\" y=\"146.6\"/>\n </g>\n </g>\n <g id=\"text_4\">\n <!-- 1 -->\n <g transform=\"translate(226.447104 161.198437)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-31\"/>\n </g>\n </g>\n </g>\n <g id=\"xtick_5\">\n <g id=\"line2d_5\">\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"288.159823\" xlink:href=\"#m10f676b948\" y=\"146.6\"/>\n </g>\n </g>\n <g id=\"text_5\">\n <!-- 2 -->\n <g transform=\"translate(284.978573 161.198437)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-32\"/>\n </g>\n </g>\n </g>\n <g id=\"text_6\">\n <!-- x -->\n <defs>\n <path d=\"M 54.890625 54.6875 \nL 35.109375 28.078125 \nL 55.90625 0 \nL 45.3125 0 \nL 29.390625 21.484375 \nL 13.484375 0 \nL 2.875 0 \nL 24.125 28.609375 \nL 4.6875 54.6875 \nL 15.28125 54.6875 \nL 29.78125 35.203125 \nL 44.28125 54.6875 \nz\n\" id=\"DejaVuSans-78\"/>\n </defs>\n <g transform=\"translate(165.210938 174.876562)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-78\"/>\n </g>\n </g>\n </g>\n <g id=\"matplotlib.axis_2\">\n <g id=\"ytick_1\">\n <g id=\"line2d_6\">\n <defs>\n <path d=\"M 0 0 \nL -3.5 0 \n\" id=\"mfeebf05174\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n </defs>\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#mfeebf05174\" y=\"115.479167\"/>\n </g>\n </g>\n <g id=\"text_7\">\n <!-- \u22125 -->\n <defs>\n <path d=\"M 10.796875 72.90625 \nL 49.515625 72.90625 \nL 49.515625 64.59375 \nL 19.828125 64.59375 \nL 19.828125 46.734375 \nQ 21.96875 47.46875 24.109375 47.828125 \nQ 26.265625 48.1875 28.421875 48.1875 \nQ 40.625 48.1875 47.75 41.5 \nQ 54.890625 34.8125 54.890625 23.390625 \nQ 54.890625 11.625 47.5625 5.09375 \nQ 40.234375 -1.421875 26.90625 -1.421875 \nQ 22.3125 -1.421875 17.546875 -0.640625 \nQ 12.796875 0.140625 7.71875 1.703125 \nL 7.71875 11.625 \nQ 12.109375 9.234375 16.796875 8.0625 \nQ 21.484375 6.890625 26.703125 6.890625 \nQ 35.15625 6.890625 40.078125 11.328125 \nQ 45.015625 15.765625 45.015625 23.390625 \nQ 45.015625 31 40.078125 35.4375 \nQ 35.15625 39.890625 26.703125 39.890625 \nQ 22.75 39.890625 18.8125 39.015625 \nQ 14.890625 38.140625 10.796875 36.28125 \nz\n\" id=\"DejaVuSans-35\"/>\n </defs>\n <g transform=\"translate(20.878125 119.278386)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-2212\"/>\n <use x=\"83.789062\" xlink:href=\"#DejaVuSans-35\"/>\n </g>\n </g>\n </g>\n <g id=\"ytick_2\">\n <g id=\"line2d_7\">\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#mfeebf05174\" y=\"73.906566\"/>\n </g>\n </g>\n <g id=\"text_8\">\n <!-- 0 -->\n <g transform=\"translate(29.257813 77.705785)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-30\"/>\n </g>\n </g>\n </g>\n <g id=\"ytick_3\">\n <g id=\"line2d_8\">\n <g>\n <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#mfeebf05174\" y=\"32.333966\"/>\n </g>\n </g>\n <g id=\"text_9\">\n <!-- 5 -->\n <g transform=\"translate(29.257813 36.133184)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-35\"/>\n </g>\n </g>\n </g>\n <g id=\"text_10\">\n <!-- f(x) -->\n <defs>\n <path d=\"M 37.109375 75.984375 \nL 37.109375 68.5 \nL 28.515625 68.5 \nQ 23.6875 68.5 21.796875 66.546875 \nQ 19.921875 64.59375 19.921875 59.515625 \nL 19.921875 54.6875 \nL 34.71875 54.6875 \nL 34.71875 47.703125 \nL 19.921875 47.703125 \nL 19.921875 0 \nL 10.890625 0 \nL 10.890625 47.703125 \nL 2.296875 47.703125 \nL 2.296875 54.6875 \nL 10.890625 54.6875 \nL 10.890625 58.5 \nQ 10.890625 67.625 15.140625 71.796875 \nQ 19.390625 75.984375 28.609375 75.984375 \nz\n\" id=\"DejaVuSans-66\"/>\n <path d=\"M 31 75.875 \nQ 24.46875 64.65625 21.28125 53.65625 \nQ 18.109375 42.671875 18.109375 31.390625 \nQ 18.109375 20.125 21.3125 9.0625 \nQ 24.515625 -2 31 -13.1875 \nL 23.1875 -13.1875 \nQ 15.875 -1.703125 12.234375 9.375 \nQ 8.59375 20.453125 8.59375 31.390625 \nQ 8.59375 42.28125 12.203125 53.3125 \nQ 15.828125 64.359375 23.1875 75.875 \nz\n\" id=\"DejaVuSans-28\"/>\n <path d=\"M 8.015625 75.875 \nL 15.828125 75.875 \nQ 23.140625 64.359375 26.78125 53.3125 \nQ 30.421875 42.28125 30.421875 31.390625 \nQ 30.421875 20.453125 26.78125 9.375 \nQ 23.140625 -1.703125 15.828125 -13.1875 \nL 8.015625 -13.1875 \nQ 14.5 -2 17.703125 9.0625 \nQ 20.90625 20.125 20.90625 31.390625 \nQ 20.90625 42.671875 17.703125 53.65625 \nQ 14.5 64.65625 8.015625 75.875 \nz\n\" id=\"DejaVuSans-29\"/>\n </defs>\n <g transform=\"translate(14.798438 87.271094)rotate(-90)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-66\"/>\n <use x=\"35.205078\" xlink:href=\"#DejaVuSans-28\"/>\n <use x=\"74.21875\" xlink:href=\"#DejaVuSans-78\"/>\n <use x=\"133.398438\" xlink:href=\"#DejaVuSans-29\"/>\n </g>\n </g>\n </g>\n <g id=\"line2d_9\">\n <path clip-path=\"url(#p12d7e6b615)\" d=\"M 54.033949 140.422727 \nL 59.887096 130.93586 \nL 65.740243 122.396848 \nL 71.593389 114.755804 \nL 77.446536 107.962841 \nL 83.299683 101.968072 \nL 89.15283 96.721609 \nL 95.005977 92.173567 \nL 100.859124 88.274057 \nL 106.712271 84.973193 \nL 112.565417 82.221086 \nL 118.418564 79.967851 \nL 124.271711 78.163601 \nL 130.124858 76.758447 \nL 135.978005 75.702503 \nL 141.831152 74.945881 \nL 147.684299 74.438696 \nL 153.537445 74.131058 \nL 159.390592 73.973082 \nL 165.243739 73.914881 \nL 171.096886 73.906566 \nL 176.950033 73.898252 \nL 182.80318 73.84005 \nL 188.656326 73.682074 \nL 194.509473 73.374437 \nL 200.36262 72.867251 \nL 206.215767 72.11063 \nL 212.068914 71.054686 \nL 217.922061 69.649532 \nL 223.775208 67.845281 \nL 229.628354 65.592046 \nL 235.481501 62.83994 \nL 241.334648 59.539075 \nL 247.187795 55.639566 \nL 253.040942 51.091523 \nL 258.894089 45.845061 \nL 264.747236 39.850292 \nL 270.600382 33.057329 \nL 276.453529 25.416285 \nL 282.306676 16.877273 \n\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n </g>\n <g id=\"patch_3\">\n <path d=\"M 42.620313 146.6 \nL 42.620313 10.7 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n </g>\n <g id=\"patch_4\">\n <path d=\"M 293.720313 146.6 \nL 293.720313 10.7 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n </g>\n <g id=\"patch_5\">\n <path d=\"M 42.620313 146.6 \nL 293.720313 146.6 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n </g>\n <g id=\"patch_6\">\n <path d=\"M 42.620313 10.7 \nL 293.720313 10.7 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n </g>\n <g id=\"patch_7\">\n <path d=\"M 171.391806 103.87507 \nQ 171.254768 90.722572 171.129379 78.688047 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\n <path d=\"M 169.171162 82.708667 \nL 171.129379 78.688047 \nL 173.170944 82.666993 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\n </g>\n <g id=\"text_11\">\n <!-- saddle point -->\n <defs>\n <path d=\"M 44.28125 53.078125 \nL 44.28125 44.578125 \nQ 40.484375 46.53125 36.375 47.5 \nQ 32.28125 48.484375 27.875 48.484375 \nQ 21.1875 48.484375 17.84375 46.4375 \nQ 14.5 44.390625 14.5 40.28125 \nQ 14.5 37.15625 16.890625 35.375 \nQ 19.28125 33.59375 26.515625 31.984375 \nL 29.59375 31.296875 \nQ 39.15625 29.25 43.1875 25.515625 \nQ 47.21875 21.78125 47.21875 15.09375 \nQ 47.21875 7.46875 41.1875 3.015625 \nQ 35.15625 -1.421875 24.609375 -1.421875 \nQ 20.21875 -1.421875 15.453125 -0.5625 \nQ 10.6875 0.296875 5.421875 2 \nL 5.421875 11.28125 \nQ 10.40625 8.6875 15.234375 7.390625 \nQ 20.0625 6.109375 24.8125 6.109375 \nQ 31.15625 6.109375 34.5625 8.28125 \nQ 37.984375 10.453125 37.984375 14.40625 \nQ 37.984375 18.0625 35.515625 20.015625 \nQ 33.0625 21.96875 24.703125 23.78125 \nL 21.578125 24.515625 \nQ 13.234375 26.265625 9.515625 29.90625 \nQ 5.8125 33.546875 5.8125 39.890625 \nQ 5.8125 47.609375 11.28125 51.796875 \nQ 16.75 56 26.8125 56 \nQ 31.78125 56 36.171875 55.265625 \nQ 40.578125 54.546875 44.28125 53.078125 \nz\n\" id=\"DejaVuSans-73\"/>\n <path d=\"M 34.28125 27.484375 \nQ 23.390625 27.484375 19.1875 25 \nQ 14.984375 22.515625 14.984375 16.5 \nQ 14.984375 11.71875 18.140625 8.90625 \nQ 21.296875 6.109375 26.703125 6.109375 \nQ 34.1875 6.109375 38.703125 11.40625 \nQ 43.21875 16.703125 43.21875 25.484375 \nL 43.21875 27.484375 \nz\nM 52.203125 31.203125 \nL 52.203125 0 \nL 43.21875 0 \nL 43.21875 8.296875 \nQ 40.140625 3.328125 35.546875 0.953125 \nQ 30.953125 -1.421875 24.3125 -1.421875 \nQ 15.921875 -1.421875 10.953125 3.296875 \nQ 6 8.015625 6 15.921875 \nQ 6 25.140625 12.171875 29.828125 \nQ 18.359375 34.515625 30.609375 34.515625 \nL 43.21875 34.515625 \nL 43.21875 35.40625 \nQ 43.21875 41.609375 39.140625 45 \nQ 35.0625 48.390625 27.6875 48.390625 \nQ 23 48.390625 18.546875 47.265625 \nQ 14.109375 46.140625 10.015625 43.890625 \nL 10.015625 52.203125 \nQ 14.9375 54.109375 19.578125 55.046875 \nQ 24.21875 56 28.609375 56 \nQ 40.484375 56 46.34375 49.84375 \nQ 52.203125 43.703125 52.203125 31.203125 \nz\n\" id=\"DejaVuSans-61\"/>\n <path d=\"M 45.40625 46.390625 \nL 45.40625 75.984375 \nL 54.390625 75.984375 \nL 54.390625 0 \nL 45.40625 0 \nL 45.40625 8.203125 \nQ 42.578125 3.328125 38.25 0.953125 \nQ 33.9375 -1.421875 27.875 -1.421875 \nQ 17.96875 -1.421875 11.734375 6.484375 \nQ 5.515625 14.40625 5.515625 27.296875 \nQ 5.515625 40.1875 11.734375 48.09375 \nQ 17.96875 56 27.875 56 \nQ 33.9375 56 38.25 53.625 \nQ 42.578125 51.265625 45.40625 46.390625 \nz\nM 14.796875 27.296875 \nQ 14.796875 17.390625 18.875 11.75 \nQ 22.953125 6.109375 30.078125 6.109375 \nQ 37.203125 6.109375 41.296875 11.75 \nQ 45.40625 17.390625 45.40625 27.296875 \nQ 45.40625 37.203125 41.296875 42.84375 \nQ 37.203125 48.484375 30.078125 48.484375 \nQ 22.953125 48.484375 18.875 42.84375 \nQ 14.796875 37.203125 14.796875 27.296875 \nz\n\" id=\"DejaVuSans-64\"/>\n <path d=\"M 9.421875 75.984375 \nL 18.40625 75.984375 \nL 18.40625 0 \nL 9.421875 0 \nz\n\" id=\"DejaVuSans-6c\"/>\n <path d=\"M 56.203125 29.59375 \nL 56.203125 25.203125 \nL 14.890625 25.203125 \nQ 15.484375 15.921875 20.484375 11.0625 \nQ 25.484375 6.203125 34.421875 6.203125 \nQ 39.59375 6.203125 44.453125 7.46875 \nQ 49.3125 8.734375 54.109375 11.28125 \nL 54.109375 2.78125 \nQ 49.265625 0.734375 44.1875 -0.34375 \nQ 39.109375 -1.421875 33.890625 -1.421875 \nQ 20.796875 -1.421875 13.15625 6.1875 \nQ 5.515625 13.8125 5.515625 26.8125 \nQ 5.515625 40.234375 12.765625 48.109375 \nQ 20.015625 56 32.328125 56 \nQ 43.359375 56 49.78125 48.890625 \nQ 56.203125 41.796875 56.203125 29.59375 \nz\nM 47.21875 32.234375 \nQ 47.125 39.59375 43.09375 43.984375 \nQ 39.0625 48.390625 32.421875 48.390625 \nQ 24.90625 48.390625 20.390625 44.140625 \nQ 15.875 39.890625 15.1875 32.171875 \nz\n\" id=\"DejaVuSans-65\"/>\n <path id=\"DejaVuSans-20\"/>\n <path d=\"M 18.109375 8.203125 \nL 18.109375 -20.796875 \nL 9.078125 -20.796875 \nL 9.078125 54.6875 \nL 18.109375 54.6875 \nL 18.109375 46.390625 \nQ 20.953125 51.265625 25.265625 53.625 \nQ 29.59375 56 35.59375 56 \nQ 45.5625 56 51.78125 48.09375 \nQ 58.015625 40.1875 58.015625 27.296875 \nQ 58.015625 14.40625 51.78125 6.484375 \nQ 45.5625 -1.421875 35.59375 -1.421875 \nQ 29.59375 -1.421875 25.265625 0.953125 \nQ 20.953125 3.328125 18.109375 8.203125 \nz\nM 48.6875 27.296875 \nQ 48.6875 37.203125 44.609375 42.84375 \nQ 40.53125 48.484375 33.40625 48.484375 \nQ 26.265625 48.484375 22.1875 42.84375 \nQ 18.109375 37.203125 18.109375 27.296875 \nQ 18.109375 17.390625 22.1875 11.75 \nQ 26.265625 6.109375 33.40625 6.109375 \nQ 40.53125 6.109375 44.609375 11.75 \nQ 48.6875 17.390625 48.6875 27.296875 \nz\n\" id=\"DejaVuSans-70\"/>\n <path d=\"M 30.609375 48.390625 \nQ 23.390625 48.390625 19.1875 42.75 \nQ 14.984375 37.109375 14.984375 27.296875 \nQ 14.984375 17.484375 19.15625 11.84375 \nQ 23.34375 6.203125 30.609375 6.203125 \nQ 37.796875 6.203125 41.984375 11.859375 \nQ 46.1875 17.53125 46.1875 27.296875 \nQ 46.1875 37.015625 41.984375 42.703125 \nQ 37.796875 48.390625 30.609375 48.390625 \nz\nM 30.609375 56 \nQ 42.328125 56 49.015625 48.375 \nQ 55.71875 40.765625 55.71875 27.296875 \nQ 55.71875 13.875 49.015625 6.21875 \nQ 42.328125 -1.421875 30.609375 -1.421875 \nQ 18.84375 -1.421875 12.171875 6.21875 \nQ 5.515625 13.875 5.515625 27.296875 \nQ 5.515625 40.765625 12.171875 48.375 \nQ 18.84375 56 30.609375 56 \nz\n\" id=\"DejaVuSans-6f\"/>\n <path d=\"M 9.421875 54.6875 \nL 18.40625 54.6875 \nL 18.40625 0 \nL 9.421875 0 \nz\nM 9.421875 75.984375 \nL 18.40625 75.984375 \nL 18.40625 64.59375 \nL 9.421875 64.59375 \nz\n\" id=\"DejaVuSans-69\"/>\n <path d=\"M 54.890625 33.015625 \nL 54.890625 0 \nL 45.90625 0 \nL 45.90625 32.71875 \nQ 45.90625 40.484375 42.875 44.328125 \nQ 39.84375 48.1875 33.796875 48.1875 \nQ 26.515625 48.1875 22.3125 43.546875 \nQ 18.109375 38.921875 18.109375 30.90625 \nL 18.109375 0 \nL 9.078125 0 \nL 9.078125 54.6875 \nL 18.109375 54.6875 \nL 18.109375 46.1875 \nQ 21.34375 51.125 25.703125 53.5625 \nQ 30.078125 56 35.796875 56 \nQ 45.21875 56 50.046875 50.171875 \nQ 54.890625 44.34375 54.890625 33.015625 \nz\n\" id=\"DejaVuSans-6e\"/>\n <path d=\"M 18.3125 70.21875 \nL 18.3125 54.6875 \nL 36.8125 54.6875 \nL 36.8125 47.703125 \nL 18.3125 47.703125 \nL 18.3125 18.015625 \nQ 18.3125 11.328125 20.140625 9.421875 \nQ 21.96875 7.515625 27.59375 7.515625 \nL 36.8125 7.515625 \nL 36.8125 0 \nL 27.59375 0 \nQ 17.1875 0 13.234375 3.875 \nQ 9.28125 7.765625 9.28125 18.015625 \nL 9.28125 47.703125 \nL 2.6875 47.703125 \nL 2.6875 54.6875 \nL 9.28125 54.6875 \nL 9.28125 70.21875 \nz\n\" id=\"DejaVuSans-74\"/>\n </defs>\n <g transform=\"translate(140.660522 115.479167)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-73\"/>\n <use x=\"52.099609\" xlink:href=\"#DejaVuSans-61\"/>\n <use x=\"113.378906\" xlink:href=\"#DejaVuSans-64\"/>\n <use x=\"176.855469\" xlink:href=\"#DejaVuSans-64\"/>\n <use x=\"240.332031\" xlink:href=\"#DejaVuSans-6c\"/>\n <use x=\"268.115234\" xlink:href=\"#DejaVuSans-65\"/>\n <use x=\"329.638672\" xlink:href=\"#DejaVuSans-20\"/>\n <use x=\"361.425781\" xlink:href=\"#DejaVuSans-70\"/>\n <use x=\"424.902344\" xlink:href=\"#DejaVuSans-6f\"/>\n <use x=\"486.083984\" xlink:href=\"#DejaVuSans-69\"/>\n <use x=\"513.867188\" xlink:href=\"#DejaVuSans-6e\"/>\n <use x=\"577.246094\" xlink:href=\"#DejaVuSans-74\"/>\n </g>\n </g>\n </g>\n </g>\n <defs>\n <clipPath id=\"p12d7e6b615\">\n <rect height=\"135.9\" width=\"251.1\" x=\"42.620313\" y=\"10.7\"/>\n </clipPath>\n </defs>\n</svg>\n",
"text/plain": "<Figure size 324x180 with 1 Axes>"
},
"metadata": {},
"output_type": "display_data"
}
]
```
再举个定义在二维空间的函数的例子，例如
$$f(x, y) = x^2 - y^2.$$
...
...
@@ -92,9 +120,22 @@ gb.plt.xlabel('x')
gb.plt.ylabel('y');
```
```
{.json .output n=4}
[
{
"data": {
"image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<!-- Created with matplotlib (http://matplotlib.org/) -->\n<svg height=\"157.3pt\" version=\"1.1\" viewBox=\"0 0 272.5 157.3\" width=\"272.5pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n <defs>\n <style type=\"text/css\">\n*{stroke-linecap:butt;stroke-linejoin:round;}\n </style>\n </defs>\n <g id=\"figure_1\">\n <g id=\"patch_1\">\n <path d=\"M 0 157.3 \nL 272.5 157.3 \nL 272.5 0 \nL 0 0 \nz\n\" style=\"fill:none;\"/>\n </g>\n <g id=\"patch_2\">\n <path d=\"M 10.7 146.6 \nL 261.8 146.6 \nL 261.8 10.7 \nL 10.7 10.7 \nz\n\" style=\"fill:#ffffff;\"/>\n </g>\n <g id=\"pane3d_1\">\n <g id=\"patch_3\">\n <path d=\"M 43.987185 116.570889 \nL 115.853319 83.251289 \nL 114.656824 19.583 \nL 38.783682 49.519976 \n\" style=\"fill:#f2f2f2;opacity:0.5;stroke:#f2f2f2;stroke-linejoin:miter;\"/>\n </g>\n </g>\n <g id=\"pane3d_2\">\n <g id=\"patch_4\">\n <path d=\"M 115.853319 83.251289 \nL 232.269699 101.879096 \nL 237.140421 36.290916 \nL 114.656824 19.583 \n\" style=\"fill:#e6e6e6;opacity:0.5;stroke:#e6e6e6;stroke-linejoin:miter;\"/>\n </g>\n </g>\n <g id=\"pane3d_3\">\n <g id=\"patch_5\">\n <path d=\"M 43.987185 116.570889 \nL 166.472947 138.266006 \nL 232.269699 101.879096 \nL 115.853319 83.251289 \n\" style=\"fill:#ececec;opacity:0.5;stroke:#ececec;stroke-linejoin:miter;\"/>\n </g>\n </g>\n <g id=\"axis3d_1\">\n <g id=\"line2d_1\">\n <path d=\"M 43.987185 116.570889 \nL 166.472947 138.266006 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n </g>\n <g id=\"text_1\">\n <!-- x -->\n <defs>\n <path d=\"M 54.890625 54.6875 \nL 35.109375 28.078125 \nL 55.90625 0 \nL 45.3125 0 \nL 29.390625 21.484375 \nL 13.484375 0 \nL 2.875 0 \nL 24.125 28.609375 \nL 4.6875 54.6875 \nL 15.28125 54.6875 \nL 29.78125 35.203125 \nL 44.28125 54.6875 \nz\n\" id=\"DejaVuSans-78\"/>\n </defs>\n <g transform=\"translate(82.750785 155.409667)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-78\"/>\n </g>\n </g>\n <g id=\"Line3DCollection_1\">\n <path d=\"M 51.454708 117.893564 \nL 122.973198 84.390542 \nL 122.132462 20.602748 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n <path d=\"M 103.83065 127.170578 \nL 172.828996 92.367978 \nL 174.535205 27.750975 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n <path d=\"M 158.338065 136.825126 \nL 224.562087 100.645799 \nL 229.014604 35.182478 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n </g>\n <g id=\"xtick_1\">\n <g id=\"line2d_2\">\n <path d=\"M 52.070877 117.604918 \nL 50.220073 118.471932 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n </g>\n <g id=\"text_2\">\n <!-- \u22121 -->\n <defs>\n <path d=\"M 10.59375 35.5 \nL 73.1875 35.5 \nL 73.1875 27.203125 \nL 10.59375 27.203125 \nz\n\" id=\"DejaVuSans-2212\"/>\n <path d=\"M 12.40625 8.296875 \nL 28.515625 8.296875 \nL 28.515625 63.921875 \nL 10.984375 60.40625 \nL 10.984375 69.390625 \nL 28.421875 72.90625 \nL 38.28125 72.90625 \nL 38.28125 8.296875 \nL 54.390625 8.296875 \nL 54.390625 0 \nL 12.40625 0 \nz\n\" id=\"DejaVuSans-31\"/>\n </defs>\n <g transform=\"translate(35.785551 136.914626)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-2212\"/>\n <use x=\"83.789062\" xlink:href=\"#DejaVuSans-31\"/>\n </g>\n </g>\n </g>\n <g id=\"xtick_2\">\n <g id=\"line2d_3\">\n <path d=\"M 104.425952 126.870309 \nL 102.63778 127.772259 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n </g>\n <g id=\"text_3\">\n <!-- 0 -->\n <defs>\n <path d=\"M 31.78125 66.40625 \nQ 24.171875 66.40625 20.328125 58.90625 \nQ 16.5 51.421875 16.5 36.375 \nQ 16.5 21.390625 20.328125 13.890625 \nQ 24.171875 6.390625 31.78125 6.390625 \nQ 39.453125 6.390625 43.28125 13.890625 \nQ 47.125 21.390625 47.125 36.375 \nQ 47.125 51.421875 43.28125 58.90625 \nQ 39.453125 66.40625 31.78125 66.40625 \nz\nM 31.78125 74.21875 \nQ 44.046875 74.21875 50.515625 64.515625 \nQ 56.984375 54.828125 56.984375 36.375 \nQ 56.984375 17.96875 50.515625 8.265625 \nQ 44.046875 -1.421875 31.78125 -1.421875 \nQ 19.53125 -1.421875 13.0625 8.265625 \nQ 6.59375 17.96875 6.59375 36.375 \nQ 6.59375 54.828125 13.0625 64.515625 \nQ 19.53125 74.21875 31.78125 74.21875 \nz\n\" id=\"DejaVuSans-30\"/>\n </defs>\n <g transform=\"translate(92.343524 146.448134)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-30\"/>\n </g>\n </g>\n </g>\n <g id=\"xtick_3\">\n <g id=\"line2d_4\">\n <path d=\"M 158.910277 136.512517 \nL 157.19142 137.451558 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n </g>\n <g id=\"text_4\">\n <!-- 1 -->\n <g transform=\"translate(146.849377 156.37081)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-31\"/>\n </g>\n </g>\n </g>\n </g>\n <g id=\"axis3d_2\">\n <g id=\"line2d_5\">\n <path d=\"M 232.269699 101.879096 \nL 166.472947 138.266006 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n </g>\n <g id=\"text_5\">\n <!-- y -->\n <defs>\n <path d=\"M 32.171875 -5.078125 \nQ 28.375 -14.84375 24.75 -17.8125 \nQ 21.140625 -20.796875 15.09375 -20.796875 \nL 7.90625 -20.796875 \nL 7.90625 -13.28125 \nL 13.1875 -13.28125 \nQ 16.890625 -13.28125 18.9375 -11.515625 \nQ 21 -9.765625 23.484375 -3.21875 \nL 25.09375 0.875 \nL 2.984375 54.6875 \nL 12.5 54.6875 \nL 29.59375 11.921875 \nL 46.6875 54.6875 \nL 56.203125 54.6875 \nz\n\" id=\"DejaVuSans-79\"/>\n </defs>\n <g transform=\"translate(227.803284 143.124581)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-79\"/>\n </g>\n </g>\n <g id=\"Line3DCollection_2\">\n <path d=\"M 43.978971 47.470091 \nL 48.890254 114.297659 \nL 170.975981 135.775739 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n <path d=\"M 78.207339 33.964735 \nL 81.25884 99.290474 \nL 200.651948 119.36434 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n <path d=\"M 110.171251 21.352856 \nL 111.589022 85.228363 \nL 228.377837 104.031372 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n </g>\n <g id=\"xtick_4\">\n <g id=\"line2d_6\">\n <path d=\"M 169.954231 135.595986 \nL 173.021765 136.135646 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n </g>\n <g id=\"text_6\">\n <!-- \u22121 -->\n <g transform=\"translate(177.98021 153.369862)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-2212\"/>\n <use x=\"83.789062\" xlink:href=\"#DejaVuSans-31\"/>\n </g>\n </g>\n </g>\n <g id=\"xtick_5\">\n <g id=\"line2d_7\">\n <path d=\"M 199.654229 119.196591 \nL 202.649539 119.7002 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n </g>\n <g id=\"text_7\">\n <!-- 0 -->\n <g transform=\"translate(211.334516 136.632167)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-30\"/>\n </g>\n </g>\n </g>\n <g id=\"xtick_6\">\n <g id=\"line2d_8\">\n <path d=\"M 227.40325 103.874464 \nL 230.329048 104.345517 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n </g>\n <g id=\"text_8\">\n <!-- 1 -->\n <g transform=\"translate(238.583983 120.993501)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-31\"/>\n </g>\n </g>\n </g>\n </g>\n <g id=\"axis3d_3\">\n <g id=\"line2d_9\">\n <path d=\"M 232.269699 101.879096 \nL 237.140421 36.290916 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n </g>\n <g id=\"Line3DCollection_3\">\n <path d=\"M 232.362339 100.631618 \nL 115.830513 82.037737 \nL 43.888382 115.297743 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n <path d=\"M 234.642669 69.925148 \nL 115.269747 52.198043 \nL 41.454325 83.933148 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n <path d=\"M 237.037992 37.670208 \nL 114.681932 20.919065 \nL 38.893294 50.932407 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n </g>\n <g id=\"xtick_7\">\n <g id=\"line2d_10\">\n <path d=\"M 231.390048 100.476479 \nL 234.308945 100.94222 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n </g>\n <g id=\"text_9\">\n <!-- \u22121 -->\n <g transform=\"translate(245.740836 106.615424)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-2212\"/>\n <use x=\"83.789062\" xlink:href=\"#DejaVuSans-31\"/>\n </g>\n </g>\n </g>\n <g id=\"xtick_8\">\n <g id=\"line2d_11\">\n <path d=\"M 233.645664 69.777091 \nL 236.638804 70.221578 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n </g>\n <g id=\"text_10\">\n <!-- 0 -->\n <g transform=\"translate(252.711018 75.982787)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-30\"/>\n </g>\n </g>\n </g>\n <g id=\"xtick_9\">\n <g id=\"line2d_12\">\n <path d=\"M 236.014986 37.530153 \nL 239.086238 37.950623 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n </g>\n <g id=\"text_11\">\n <!-- 1 -->\n <g transform=\"translate(255.631033 43.811251)scale(0.1 -0.1)\">\n <use xlink:href=\"#DejaVuSans-31\"/>\n </g>\n </g>\n </g>\n </g>\n <g id=\"axes_1\">\n <g id=\"Line3DCollection_4\">\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 54.133789 83.013765 \nL 56.139919 77.877046 \nL 58.168328 73.018701 \nL 60.21769 68.442641 \nL 62.286647 64.152404 \nL 64.373813 60.151146 \nL 66.477777 56.441634 \nL 68.597104 53.026238 \nL 70.730341 49.906927 \nL 72.876017 47.085267 \nL 75.032652 44.562414 \nL 77.198755 42.339118 \nL 79.372827 40.415722 \nL 81.55337 38.792162 \nL 83.738887 37.467973 \nL 85.927882 36.44229 \nL 88.118869 35.713859 \nL 90.310374 35.281039 \nL 92.500935 35.141811 \nL 94.689107 35.293791 \nL 96.873466 35.734237 \nL 99.052611 36.460061 \nL 101.225166 37.467842 \nL 103.389782 38.753839 \nL 105.545142 40.314003 \nL 107.689962 42.143996 \nL 109.822991 44.239201 \nL 111.943017 46.594743 \nL 114.048865 49.205501 \nL 116.1394 52.066124 \nL 118.213531 55.171053 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 61.646625 92.119798 \nL 63.65267 87.010529 \nL 65.679222 82.176611 \nL 67.724956 77.621918 \nL 69.788521 73.349956 \nL 71.868537 69.363853 \nL 73.963603 65.666357 \nL 76.072295 62.259822 \nL 78.193176 59.146208 \nL 80.324791 56.327075 \nL 82.46568 53.803584 \nL 84.614371 51.576492 \nL 86.769392 49.646154 \nL 88.929268 48.012527 \nL 91.09253 46.675169 \nL 93.257713 45.633247 \nL 95.423361 44.885539 \nL 97.588033 44.430447 \nL 99.750301 44.265997 \nL 101.908758 44.389854 \nL 104.062017 44.799331 \nL 106.208714 45.491399 \nL 108.347515 46.462699 \nL 110.47711 47.709557 \nL 112.596226 49.227995 \nL 114.70362 51.013747 \nL 116.798086 53.062274 \nL 118.878454 55.368778 \nL 120.943595 57.928221 \nL 122.992419 60.735335 \nL 125.023878 63.784647 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 69.041795 100.041427 \nL 71.049442 94.953716 \nL 73.075882 90.138857 \nL 75.119791 85.600694 \nL 77.17982 81.342708 \nL 79.254594 77.36801 \nL 81.342717 73.679329 \nL 83.442778 70.279009 \nL 85.553348 67.169004 \nL 87.672989 64.350872 \nL 89.800255 61.825776 \nL 91.933694 59.59448 \nL 94.071854 57.657352 \nL 96.213282 56.014365 \nL 98.356534 54.665097 \nL 100.50017 53.60874 \nL 102.642764 52.844103 \nL 104.782905 52.369619 \nL 106.919195 52.183353 \nL 109.050261 52.283009 \nL 111.17475 52.665947 \nL 113.291334 53.329183 \nL 115.398714 54.269412 \nL 117.495623 55.483013 \nL 119.580822 56.966064 \nL 121.653111 58.714361 \nL 123.711322 60.723425 \nL 125.75433 62.988523 \nL 127.781044 65.504683 \nL 129.790419 68.266705 \nL 131.781449 71.269186 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 76.340106 106.809598 \nL 78.351052 101.737687 \nL 80.379127 96.936635 \nL 82.423004 92.410265 \nL 84.481334 88.162041 \nL 86.552745 84.195059 \nL 88.635845 80.512037 \nL 90.729228 77.115313 \nL 92.831477 74.006837 \nL 94.941164 71.188167 \nL 97.056856 68.660469 \nL 99.177117 66.424515 \nL 101.300512 64.480683 \nL 103.425609 62.828958 \nL 105.550983 61.468936 \nL 107.675221 60.399831 \nL 109.79692 59.620473 \nL 111.914695 59.129323 \nL 114.027179 58.924474 \nL 116.133028 59.003666 \nL 118.230921 59.36429 \nL 120.319563 60.003402 \nL 122.39769 60.917736 \nL 124.464068 62.103713 \nL 126.517497 63.557458 \nL 128.556813 65.274811 \nL 130.580887 67.251343 \nL 132.588633 69.48237 \nL 134.579001 71.962971 \nL 136.550983 74.688 \nL 138.503618 77.652106 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 83.561656 112.448787 \nL 85.577618 107.387018 \nL 87.609084 102.594606 \nL 89.654723 98.075362 \nL 91.713182 93.83274 \nL 93.78309 89.869828 \nL 95.863058 86.189338 \nL 97.951683 82.793606 \nL 100.047554 79.684579 \nL 102.149252 76.863819 \nL 104.255356 74.332495 \nL 106.36444 72.091386 \nL 108.475087 70.140878 \nL 110.585879 68.480969 \nL 112.695413 67.111268 \nL 114.802294 66.031003 \nL 116.905143 65.239025 \nL 119.002599 64.733812 \nL 121.093321 64.513482 \nL 123.175992 64.575797 \nL 125.24932 64.918175 \nL 127.312043 65.537701 \nL 129.362926 66.431136 \nL 131.400769 67.594933 \nL 133.424407 69.02525 \nL 135.43271 70.71796 \nL 137.424588 72.668669 \nL 139.398988 74.872729 \nL 141.3549 77.325258 \nL 143.291356 80.021146 \nL 145.207431 82.955082 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 90.726022 116.977284 \nL 92.748743 111.920057 \nL 94.785375 107.13117 \nL 96.834578 102.614427 \nL 98.894996 98.37328 \nL 100.965254 94.410814 \nL 103.043963 90.729742 \nL 105.129723 87.332399 \nL 107.221125 84.220736 \nL 109.316757 81.396316 \nL 111.415205 78.860316 \nL 113.515055 76.613519 \nL 115.6149 74.65632 \nL 117.713339 72.988724 \nL 119.808983 71.610353 \nL 121.900456 70.520445 \nL 123.9864 69.717864 \nL 126.065474 69.201102 \nL 128.136362 68.968291 \nL 130.197772 69.017211 \nL 132.248438 69.345295 \nL 134.287126 69.949646 \nL 136.312633 70.827047 \nL 138.323788 71.973968 \nL 140.319459 73.386589 \nL 142.298549 75.060803 \nL 144.260003 76.99224 \nL 146.202803 79.176274 \nL 148.125975 81.608045 \nL 150.028589 84.282469 \nL 151.909757 87.194258 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 97.852434 120.407381 \nL 99.883695 115.349123 \nL 101.927294 110.558663 \nL 103.981884 106.039811 \nL 106.0461 101.796019 \nL 108.118564 97.830379 \nL 110.197881 94.145606 \nL 112.282653 90.744042 \nL 114.371471 87.627641 \nL 116.462927 84.797973 \nL 118.555611 82.25622 \nL 120.648119 80.003172 \nL 122.739052 78.039229 \nL 124.827022 76.364404 \nL 126.910653 74.978325 \nL 128.988586 73.880239 \nL 131.059478 73.069014 \nL 133.12201 72.543154 \nL 135.174887 72.300797 \nL 137.216839 72.33973 \nL 139.246627 72.657396 \nL 141.263041 73.250906 \nL 143.264906 74.117051 \nL 145.251081 75.252312 \nL 147.220464 76.652876 \nL 149.171991 78.314647 \nL 151.104637 80.233261 \nL 153.017421 82.404106 \nL 154.909403 84.822328 \nL 156.779688 87.482853 \nL 158.627425 90.380404 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 104.959949 122.745487 \nL 107.001576 117.680612 \nL 109.053982 112.883471 \nL 111.11581 108.357882 \nL 113.185685 104.107311 \nL 115.26222 100.134858 \nL 117.344017 96.443248 \nL 119.429672 93.034831 \nL 121.517777 89.91157 \nL 123.606924 87.075042 \nL 125.695706 84.526435 \nL 127.782725 82.266545 \nL 129.866588 80.295779 \nL 131.945918 78.614153 \nL 134.019348 77.2213 \nL 136.085535 76.116467 \nL 138.14315 75.298528 \nL 140.190893 74.765986 \nL 142.227487 74.516981 \nL 144.251683 74.549301 \nL 146.262265 74.860388 \nL 148.258047 75.447353 \nL 150.237881 76.306983 \nL 152.200654 77.435759 \nL 154.145292 78.829865 \nL 156.070761 80.485201 \nL 157.976069 82.397402 \nL 159.860267 84.561849 \nL 161.722449 86.973686 \nL 163.561754 89.627835 \nL 165.377367 92.519012 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 112.067624 123.992157 \nL 114.121497 118.915034 \nL 116.184598 114.106056 \nL 118.255554 109.569064 \nL 120.332979 105.307539 \nL 122.415474 101.324596 \nL 124.501635 97.622977 \nL 126.59005 94.205043 \nL 128.679309 91.07277 \nL 130.768002 88.227742 \nL 132.854722 85.671156 \nL 134.938074 83.403814 \nL 137.016672 81.426125 \nL 139.089142 79.73811 \nL 141.15413 78.3394 \nL 143.210302 77.229243 \nL 145.256344 76.40651 \nL 147.29097 75.869698 \nL 149.312921 75.616943 \nL 151.320968 75.646022 \nL 153.313915 75.954371 \nL 155.290599 76.539089 \nL 157.249897 77.396953 \nL 159.190722 78.524428 \nL 161.112027 79.917686 \nL 163.012808 81.57261 \nL 164.892104 83.484821 \nL 166.748997 85.64968 \nL 168.582613 88.062315 \nL 170.392128 90.717629 \nL 172.17676 93.610317 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 119.194682 124.142048 \nL 121.262751 119.046961 \nL 123.338492 114.220916 \nL 125.420516 109.667781 \nL 127.507422 105.391061 \nL 129.597799 101.393896 \nL 131.690231 97.679044 \nL 133.783298 94.248885 \nL 135.875585 91.105408 \nL 137.965675 88.25021 \nL 140.052162 85.684495 \nL 142.13365 83.409071 \nL 144.208756 81.424351 \nL 146.276111 79.730354 \nL 148.334368 78.326711 \nL 150.382201 77.212662 \nL 152.418309 76.38707 \nL 154.44142 75.848423 \nL 156.450288 75.594842 \nL 158.443704 75.62409 \nL 160.42049 75.933584 \nL 162.379507 76.520402 \nL 164.319652 77.381299 \nL 166.239864 78.512717 \nL 168.139124 79.9108 \nL 170.016455 81.571405 \nL 171.870926 83.490121 \nL 173.701649 85.66228 \nL 175.507786 88.082978 \nL 177.288542 90.747083 \nL 179.043173 93.649259 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 126.360695 123.18379 \nL 128.444986 118.064903 \nL 130.535383 113.216449 \nL 132.630476 108.64233 \nL 134.728848 104.346086 \nL 136.829071 100.330883 \nL 138.929715 96.599507 \nL 141.029352 93.154356 \nL 143.126555 89.997438 \nL 145.219902 87.130364 \nL 147.307982 84.554346 \nL 149.389397 82.270198 \nL 151.462764 80.278336 \nL 153.526717 78.578775 \nL 155.579915 77.17114 \nL 157.621038 76.054662 \nL 159.648796 75.228191 \nL 161.661926 74.690196 \nL 163.659198 74.438777 \nL 165.639419 74.471675 \nL 167.601428 74.786278 \nL 169.544107 75.379635 \nL 171.466375 76.248467 \nL 173.367196 77.38918 \nL 175.245576 78.797879 \nL 177.100565 80.470382 \nL 178.931261 82.402233 \nL 180.736809 84.588721 \nL 182.5164 87.024894 \nL 184.269275 89.705572 \nL 185.994723 92.62537 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 133.585763 121.099775 \nL 135.688394 115.951091 \nL 137.795545 111.074737 \nL 139.905782 106.474663 \nL 142.017668 102.154447 \nL 144.129756 98.117291 \nL 146.240601 94.36601 \nL 148.34876 90.903031 \nL 150.452794 87.730379 \nL 152.551274 84.849682 \nL 154.64278 82.262164 \nL 156.72591 79.968642 \nL 158.799279 77.969533 \nL 160.861523 76.264849 \nL 162.911302 74.854203 \nL 164.947302 73.736813 \nL 166.96824 72.911508 \nL 168.972864 72.376735 \nL 170.959958 72.130565 \nL 172.92834 72.170706 \nL 174.876869 72.494508 \nL 176.804445 73.098981 \nL 178.710008 73.9808 \nL 180.592546 75.136324 \nL 182.451089 76.561606 \nL 184.284715 78.25241 \nL 186.092552 80.204223 \nL 187.873772 82.412275 \nL 189.627602 84.871551 \nL 191.353314 87.576809 \nL 193.050234 90.522597 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 140.890712 117.865856 \nL 143.013908 112.681172 \nL 145.140006 107.771245 \nL 147.26755 103.140077 \nL 149.395074 98.791295 \nL 151.521114 94.728143 \nL 153.644205 90.953473 \nL 155.762886 87.469741 \nL 157.875704 84.278998 \nL 159.981218 81.382888 \nL 162.077999 78.782647 \nL 164.164638 76.479098 \nL 166.239746 74.472657 \nL 168.301957 72.763327 \nL 170.349932 71.350708 \nL 172.382361 70.234001 \nL 174.397966 69.412007 \nL 176.395504 68.883142 \nL 178.373769 68.645442 \nL 180.331595 68.696571 \nL 182.267855 69.033834 \nL 184.181467 69.654188 \nL 186.071394 70.554251 \nL 187.936643 71.730323 \nL 189.776272 73.17839 \nL 191.589385 74.894149 \nL 193.375137 76.873016 \nL 195.132734 79.110144 \nL 196.861432 81.600442 \nL 198.56054 84.338588 \nL 200.229416 87.319047 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 148.297308 113.45094 \nL 150.443414 108.223805 \nL 152.590766 103.274404 \nL 154.737878 98.606802 \nL 156.883258 94.224683 \nL 159.025417 90.13134 \nL 161.162868 86.329669 \nL 163.294129 82.822159 \nL 165.417731 79.610891 \nL 167.532218 76.697529 \nL 169.636149 74.08332 \nL 171.728106 71.769093 \nL 173.806693 69.755261 \nL 175.870539 68.041817 \nL 177.918306 66.628344 \nL 179.948684 65.514017 \nL 181.960401 64.697607 \nL 183.952221 64.17749 \nL 185.922946 63.951657 \nL 187.871423 64.01772 \nL 189.796542 64.372926 \nL 191.697236 65.014168 \nL 193.572489 65.937997 \nL 195.421331 67.140635 \nL 197.242844 68.617993 \nL 199.036159 70.365682 \nL 200.800461 72.379031 \nL 202.534985 74.653103 \nL 204.239022 77.182711 \nL 205.911914 79.962436 \nL 207.553058 82.986644 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 155.828489 107.816478 \nL 157.999989 102.54014 \nL 160.171028 97.545093 \nL 162.340088 92.835474 \nL 164.505647 88.415035 \nL 166.666187 84.287127 \nL 168.820195 80.454693 \nL 170.966167 76.920264 \nL 173.102613 73.685952 \nL 175.228059 70.753443 \nL 177.34105 68.123999 \nL 179.440157 65.798453 \nL 181.523973 63.777212 \nL 183.591123 62.060258 \nL 185.640264 60.647152 \nL 187.670087 59.537038 \nL 189.679323 58.728647 \nL 191.666741 58.22031 \nL 193.631153 58.009962 \nL 195.571417 58.095153 \nL 197.486435 58.473062 \nL 199.375161 59.140503 \nL 201.236596 60.093946 \nL 203.069794 61.329524 \nL 204.873861 62.843053 \nL 206.647955 64.630044 \nL 208.391291 66.685723 \nL 210.103136 69.005043 \nL 211.782814 71.582706 \nL 213.429704 74.413177 \nL 215.043239 77.490704 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 163.508622 100.915821 \nL 165.708158 95.583173 \nL 167.905466 90.535985 \nL 170.09899 85.778484 \nL 172.287174 81.314494 \nL 174.468467 77.147433 \nL 176.641328 73.280302 \nL 178.804226 69.715676 \nL 180.955647 66.455703 \nL 183.094098 63.502094 \nL 185.218106 60.856125 \nL 187.326225 58.518635 \nL 189.41704 56.490024 \nL 191.489166 54.770257 \nL 193.541256 53.358869 \nL 195.571999 52.254966 \nL 197.580125 51.457234 \nL 199.564411 50.963947 \nL 201.523674 50.772975 \nL 203.456785 50.881796 \nL 205.362659 51.287504 \nL 207.240265 51.986825 \nL 209.088627 52.97613 \nL 210.906819 54.251449 \nL 212.693973 55.808487 \nL 214.449276 57.642636 \nL 216.171972 59.749001 \nL 217.861362 62.122408 \nL 219.516805 64.757427 \nL 221.137716 67.648386 \nL 222.72357 70.789397 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 54.133789 83.013765 \nL 57.906249 87.717028 \nL 61.646625 92.119798 \nL 65.357599 96.226574 \nL 69.041795 100.041427 \nL 72.701787 103.568017 \nL 76.340106 106.809598 \nL 79.959242 109.769029 \nL 83.561656 112.448787 \nL 87.149779 114.850964 \nL 90.726022 116.977284 \nL 94.292779 118.829095 \nL 97.852434 120.407381 \nL 101.407365 121.712761 \nL 104.959949 122.745487 \nL 108.512571 123.505446 \nL 112.067624 123.992157 \nL 115.627517 124.204767 \nL 119.194682 124.142048 \nL 122.771578 123.802388 \nL 126.360695 123.18379 \nL 129.964565 122.283854 \nL 133.585763 121.099775 \nL 137.226916 119.628329 \nL 140.890712 117.865856 \nL 144.579901 115.808251 \nL 148.297308 113.45094 \nL 152.04584 110.78887 \nL 155.828489 107.816478 \nL 159.64835 104.527676 \nL 163.508622 100.915821 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 58.168328 73.018701 \nL 61.939387 77.749329 \nL 65.679222 82.176611 \nL 69.390511 86.305085 \nL 73.075882 90.138857 \nL 76.737909 93.681614 \nL 80.379127 96.936635 \nL 84.00203 99.906802 \nL 87.609084 102.594606 \nL 91.202726 105.002152 \nL 94.785375 107.13117 \nL 98.359433 108.983012 \nL 101.927294 110.558663 \nL 105.491347 111.858735 \nL 109.053982 112.883471 \nL 112.617596 113.632744 \nL 116.184598 114.106056 \nL 119.757414 114.302532 \nL 123.338492 114.220916 \nL 126.930311 113.859567 \nL 130.535383 113.216449 \nL 134.156261 112.289123 \nL 137.795545 111.074737 \nL 141.455888 109.570017 \nL 145.140006 107.771245 \nL 148.85068 105.674255 \nL 152.590766 103.274404 \nL 156.363205 100.566564 \nL 160.171028 97.545093 \nL 164.017368 94.203812 \nL 167.905466 90.535985 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 62.286647 64.152404 \nL 66.052737 68.904008 \nL 69.788521 73.349956 \nL 73.496675 77.494812 \nL 77.17982 81.342708 \nL 80.840529 84.897352 \nL 84.481334 88.162041 \nL 88.10473 91.13967 \nL 91.713182 93.83274 \nL 95.309131 96.243366 \nL 98.894996 98.37328 \nL 102.473187 100.223838 \nL 106.0461 101.796019 \nL 109.616134 103.090432 \nL 113.185685 104.107311 \nL 116.75716 104.846518 \nL 120.332979 105.307539 \nL 123.915579 105.489479 \nL 127.507422 105.391061 \nL 131.111002 105.010619 \nL 134.728848 104.346086 \nL 138.36353 103.394992 \nL 142.017668 102.154447 \nL 145.693937 100.621135 \nL 149.395074 98.791295 \nL 153.123887 96.660709 \nL 156.883258 94.224683 \nL 160.676157 91.478027 \nL 164.505647 88.415035 \nL 168.374893 85.029462 \nL 172.287174 81.314494 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 66.477777 56.441634 \nL 70.235355 61.207701 \nL 73.963603 65.666357 \nL 77.665186 69.822185 \nL 81.342717 73.679329 \nL 84.998762 77.241511 \nL 88.635845 80.512037 \nL 92.256456 83.493811 \nL 95.863058 86.189338 \nL 99.458087 88.600738 \nL 103.043963 90.729742 \nL 106.623095 92.577706 \nL 110.197881 94.145606 \nL 113.770721 95.434045 \nL 117.344017 96.443248 \nL 120.92018 97.173068 \nL 124.501635 97.622977 \nL 128.090828 97.792068 \nL 131.690231 97.679044 \nL 135.302346 97.28222 \nL 138.929715 96.599507 \nL 142.574922 95.628409 \nL 146.240601 94.36601 \nL 149.929444 92.808963 \nL 153.644205 90.953473 \nL 157.387711 88.795286 \nL 161.162868 86.329669 \nL 164.972666 83.551388 \nL 168.820195 80.454693 \nL 172.708646 77.033286 \nL 176.641328 73.280302 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 70.730341 49.906927 \nL 74.475909 54.680868 \nL 78.193176 59.146208 \nL 81.884789 63.307536 \nL 85.553348 67.169004 \nL 89.201406 70.734334 \nL 92.831477 74.006837 \nL 96.446042 76.989416 \nL 100.047554 79.684579 \nL 103.638444 82.094442 \nL 107.221125 84.220736 \nL 110.798002 86.064811 \nL 114.371471 87.627641 \nL 117.943929 88.90982 \nL 121.517777 89.91157 \nL 125.095428 90.632732 \nL 128.679309 91.07277 \nL 132.271869 91.230764 \nL 135.875585 91.105408 \nL 139.492964 90.695 \nL 143.126555 89.997438 \nL 146.77895 89.010209 \nL 150.452794 87.730379 \nL 154.15079 86.15458 \nL 157.875704 84.278998 \nL 161.630378 82.099354 \nL 165.417731 79.610891 \nL 169.240774 76.808351 \nL 173.102613 73.685952 \nL 177.006461 70.237369 \nL 180.955647 66.455703 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 75.032652 44.562414 \nL 78.762781 49.337608 \nL 82.46568 53.803584 \nL 86.143978 57.964925 \nL 89.800255 61.825776 \nL 93.437049 65.389856 \nL 97.056856 68.660469 \nL 100.662145 71.640515 \nL 104.255356 74.332495 \nL 107.838907 76.73852 \nL 111.415205 78.860316 \nL 114.986643 80.699227 \nL 118.555611 82.25622 \nL 122.1245 83.531885 \nL 125.695706 84.526435 \nL 129.271639 85.239707 \nL 132.854722 85.671156 \nL 136.447405 85.819857 \nL 140.052162 85.684495 \nL 143.671505 85.263361 \nL 147.307982 84.554346 \nL 150.964191 83.554928 \nL 154.64278 82.262164 \nL 158.346458 80.672678 \nL 162.077999 78.782647 \nL 165.840253 76.587783 \nL 169.636149 74.08332 \nL 173.468709 71.263988 \nL 177.34105 68.123999 \nL 181.256401 64.657014 \nL 185.218106 60.856125 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 79.372827 40.415722 \nL 83.084169 45.185572 \nL 86.769392 49.646154 \nL 90.431101 53.802038 \nL 94.071854 57.657352 \nL 97.694164 61.215803 \nL 101.300512 64.480683 \nL 104.893345 67.454877 \nL 108.475087 70.140878 \nL 112.048142 72.540787 \nL 115.6149 74.65632 \nL 119.177744 76.488814 \nL 122.739052 78.039229 \nL 126.301204 79.308148 \nL 129.866588 80.295779 \nL 133.437605 81.001954 \nL 137.016672 81.426125 \nL 140.606231 81.567363 \nL 144.208756 81.424351 \nL 147.82675 80.995379 \nL 151.462764 80.278336 \nL 155.119391 79.270701 \nL 158.799279 77.969533 \nL 162.505139 76.371459 \nL 166.239746 74.472657 \nL 170.005953 72.268845 \nL 173.806693 69.755261 \nL 177.644991 66.926644 \nL 181.523973 63.777212 \nL 185.446872 60.300638 \nL 189.41704 56.490024 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 83.738887 37.467973 \nL 87.428195 42.225945 \nL 91.09253 46.675169 \nL 94.734466 50.820184 \nL 98.356534 54.665097 \nL 101.961221 58.213591 \nL 105.550983 61.468936 \nL 109.128246 64.434004 \nL 112.695413 67.111268 \nL 116.254868 69.502816 \nL 119.808983 71.610353 \nL 123.360124 73.435206 \nL 126.910653 74.978325 \nL 130.462937 76.240289 \nL 134.019348 77.2213 \nL 137.582277 77.921186 \nL 141.15413 78.3394 \nL 144.73734 78.475013 \nL 148.334368 78.326711 \nL 151.947713 77.892788 \nL 155.579915 77.17114 \nL 159.233564 76.159255 \nL 162.911302 74.854203 \nL 166.615834 73.252622 \nL 170.349932 71.350708 \nL 174.116445 69.144196 \nL 177.918306 66.628344 \nL 181.758538 63.797915 \nL 185.640264 60.647152 \nL 189.566719 57.169759 \nL 193.541256 53.358869 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 88.118869 35.713859 \nL 91.783017 40.453536 \nL 95.423361 44.885539 \nL 99.042444 49.014371 \nL 102.642764 52.844103 \nL 106.226781 56.378389 \nL 109.79692 59.620473 \nL 113.355581 62.5732 \nL 116.905143 65.239025 \nL 120.447965 67.620016 \nL 123.9864 69.717864 \nL 127.522789 71.533883 \nL 131.059478 73.069014 \nL 134.598812 74.32383 \nL 138.14315 75.298528 \nL 141.694864 75.992937 \nL 145.256344 76.40651 \nL 148.83001 76.538323 \nL 152.418309 76.38707 \nL 156.023729 75.951056 \nL 159.648796 75.228191 \nL 163.296089 74.215978 \nL 166.96824 72.911508 \nL 170.667945 71.311444 \nL 174.397966 69.412007 \nL 178.161143 67.208963 \nL 181.960401 64.697607 \nL 185.798756 61.872739 \nL 189.679323 58.728647 \nL 193.605331 55.259082 \nL 197.580125 51.457234 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 92.500935 35.141811 \nL 96.136924 39.856932 \nL 99.750301 44.265997 \nL 103.343571 48.373461 \nL 106.919195 52.183353 \nL 110.4796 55.699287 \nL 114.027179 58.924474 \nL 117.564303 61.86173 \nL 121.093321 64.513482 \nL 124.616566 66.881779 \nL 128.136362 68.968291 \nL 131.655029 70.77432 \nL 135.174887 72.300797 \nL 138.698261 73.548285 \nL 142.227487 74.516981 \nL 145.764916 75.206715 \nL 149.312921 75.616943 \nL 152.873902 75.74675 \nL 156.450288 75.594842 \nL 160.04455 75.159541 \nL 163.659198 74.438777 \nL 167.296795 73.430079 \nL 170.959958 72.130565 \nL 174.651366 70.536933 \nL 178.373769 68.645442 \nL 182.129993 66.451902 \nL 185.922946 63.951657 \nL 189.755632 61.13956 \nL 193.631153 58.009962 \nL 197.552723 54.55668 \nL 201.523674 50.772975 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 96.873466 35.734237 \nL 100.478446 40.41874 \nL 104.062017 44.799331 \nL 107.626644 48.880407 \nL 111.17475 52.665947 \nL 114.708722 56.159517 \nL 118.230921 59.36429 \nL 121.743681 62.283043 \nL 125.24932 64.918175 \nL 128.750141 67.271709 \nL 132.248438 69.345295 \nL 135.746502 71.140217 \nL 139.246627 72.657396 \nL 142.751111 73.897388 \nL 146.262265 74.860388 \nL 149.782416 75.546227 \nL 153.313915 75.954371 \nL 156.859136 76.083916 \nL 160.42049 75.933584 \nL 164.000424 75.501719 \nL 167.601428 74.786278 \nL 171.226044 73.784822 \nL 174.876869 72.494508 \nL 178.556563 70.912076 \nL 182.267855 69.033834 \nL 186.013551 66.85565 \nL 189.796542 64.372926 \nL 193.619809 61.580588 \nL 197.486435 58.473062 \nL 201.399615 55.044249 \nL 205.362659 51.287504 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 101.225166 37.467842 \nL 104.796438 42.115903 \nL 108.347515 46.462699 \nL 111.880813 50.512563 \nL 115.398714 54.269412 \nL 118.903567 57.736762 \nL 122.39769 60.917736 \nL 125.883382 63.815073 \nL 129.362926 66.431136 \nL 132.838588 68.767918 \nL 136.312633 70.827047 \nL 139.787318 72.609789 \nL 143.264906 74.117051 \nL 146.747666 75.349385 \nL 150.237881 76.306983 \nL 153.737851 76.989681 \nL 157.249897 77.396953 \nL 160.77637 77.527911 \nL 164.319652 77.381299 \nL 167.882165 76.955488 \nL 171.466375 76.248467 \nL 175.074799 75.257837 \nL 178.710008 73.9808 \nL 182.374638 72.414148 \nL 186.071394 70.554251 \nL 189.803056 68.397041 \nL 193.572489 65.937997 \nL 197.38265 63.172126 \nL 201.236596 60.093946 \nL 205.137494 56.697459 \nL 209.088627 52.97613 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 105.545142 40.314003 \nL 109.080176 44.920067 \nL 112.596226 49.227995 \nL 116.095663 53.242044 \nL 119.580822 56.966064 \nL 123.054007 60.403512 \nL 126.517497 63.557458 \nL 129.973549 66.430596 \nL 133.424407 69.02525 \nL 136.872302 71.34338 \nL 140.319459 73.386589 \nL 143.768104 75.156122 \nL 147.220464 76.652876 \nL 150.678777 77.877393 \nL 154.145292 78.829865 \nL 157.622278 79.510133 \nL 161.112027 79.917686 \nL 164.616858 80.051651 \nL 168.139124 79.9108 \nL 171.681217 79.493533 \nL 175.245576 78.797879 \nL 178.834685 77.821485 \nL 182.451089 76.561606 \nL 186.097393 75.015095 \nL 189.776272 73.17839 \nL 193.490478 71.047501 \nL 197.242844 68.617993 \nL 201.036297 65.88497 \nL 204.873861 62.843053 \nL 208.758669 59.486361 \nL 212.693973 55.808487 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 109.822991 44.239201 \nL 113.319427 48.798016 \nL 116.798086 53.062274 \nL 120.261289 57.036151 \nL 123.711322 60.723425 \nL 127.150444 64.127484 \nL 130.580887 67.251343 \nL 134.004868 70.097644 \nL 137.424588 72.668669 \nL 140.842237 74.966343 \nL 144.260003 76.99224 \nL 147.680072 78.747586 \nL 151.104637 80.233261 \nL 154.535898 81.449803 \nL 157.976069 82.397402 \nL 161.427386 83.075908 \nL 164.892104 83.484821 \nL 168.372511 83.623292 \nL 171.870926 83.490121 \nL 175.389708 83.083743 \nL 178.931261 82.402233 \nL 182.498039 81.443288 \nL 186.092552 80.204223 \nL 189.717371 78.681961 \nL 193.375137 76.873016 \nL 197.068568 74.773485 \nL 200.800461 72.379031 \nL 204.573706 69.684864 \nL 208.391291 66.685723 \nL 212.256309 63.375857 \nL 216.171972 59.749001 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 114.048865 49.205501 \nL 117.50452 53.712137 \nL 120.943595 57.928221 \nL 124.368358 61.857837 \nL 127.781044 65.504683 \nL 131.183863 68.872077 \nL 134.579001 71.962971 \nL 137.968627 74.779952 \nL 141.3549 77.325258 \nL 144.739968 79.600774 \nL 148.125975 81.608045 \nL 151.51507 83.348274 \nL 154.909403 84.822328 \nL 158.311137 86.030734 \nL 161.722449 86.973686 \nL 165.145534 87.651041 \nL 168.582613 88.062315 \nL 172.035936 88.206684 \nL 175.507786 88.082978 \nL 179.000485 87.689674 \nL 182.5164 87.024894 \nL 186.057948 86.086392 \nL 189.627602 84.871551 \nL 193.227896 83.377367 \nL 196.861432 81.600442 \nL 200.530888 79.536967 \nL 204.239022 77.182711 \nL 207.988683 74.533 \nL 211.782814 71.582706 \nL 215.624467 68.326218 \nL 219.516805 64.757427 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n <path clip-path=\"url(#p6efe1666f2)\" d=\"M 118.213531 55.171053 \nL 121.626404 59.620929 \nL 125.023878 63.784647 \nL 128.408166 67.666195 \nL 131.781449 71.269186 \nL 135.145887 74.596862 \nL 138.503618 77.652106 \nL 141.856762 80.437451 \nL 145.207431 82.955082 \nL 148.557729 85.206847 \nL 151.909757 87.194258 \nL 155.265619 88.918493 \nL 158.627425 90.380404 \nL 161.997296 91.580512 \nL 165.377367 92.519012 \nL 168.769795 93.195769 \nL 172.17676 93.610317 \nL 175.600471 93.761859 \nL 179.043173 93.649259 \nL 182.507148 93.271038 \nL 185.994723 92.62537 \nL 189.508274 91.710072 \nL 193.050234 90.522597 \nL 196.623094 89.060025 \nL 200.229416 87.319047 \nL 203.871832 85.295959 \nL 207.553058 82.986644 \nL 211.275894 80.386556 \nL 215.043239 77.490704 \nL 218.858093 74.293632 \nL 222.72357 70.789397 \n\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\n </g>\n <g id=\"line2d_13\">\n <defs>\n <path d=\"M -3 3 \nL 3 -3 \nM -3 -3 \nL 3 3 \n\" id=\"m51d18eb0e3\" style=\"stroke:#ff0000;\"/>\n </defs>\n <g clip-path=\"url(#p6efe1666f2)\">\n <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"139.643243\" xlink:href=\"#m51d18eb0e3\" y=\"76.813514\"/>\n </g>\n </g>\n </g>\n </g>\n <defs>\n <clipPath id=\"p6efe1666f2\">\n <rect height=\"135.9\" width=\"251.1\" x=\"10.7\" y=\"10.7\"/>\n </clipPath>\n </defs>\n</svg>\n",
"text/plain": "<Figure size 324x180 with 1 Axes>"
},
"metadata": {},
"output_type": "display_data"
}
]
```
在上图的鞍点位置，目标函数在$x$轴方向上是局部最小值，而在$y$轴方向上是局部最大值。
假设一个函数的输入为$k$维向量，输出为标量，那么它的黑塞矩阵（Hessian matrix）有$k$个特征值。
需要注意的是，
该函数在梯度为零的位置上可能是局部最小值、局部最大值或者鞍点：
假设一个函数的输入为$k$维向量，输出为标量，那么它的黑塞矩阵（Hessian matrix）有$k$个特征值。该函数在梯度为零的位置上可能是局部最小值、局部最大值或者鞍点：
*
当函数的黑塞矩阵在梯度为零的位置上的特征值全为正时，该函数得到局部最小值。
*
当函数的黑塞矩阵在梯度为零的位置上的特征值全为负时，该函数得到局部最大值。
...
...
@@ -102,7 +143,7 @@ gb.plt.ylabel('y');
随机矩阵理论告诉我们，对于一个大的高斯随机矩阵来说，任一特征值是正或者是负的概率都是0.5 [1]。那么，以上第一种情况的概率为 $0.5^k$。由于深度学习模型参数通常都是高维的（$k$很大），目标函数的鞍点通常比局部最小值更常见。
深度学习中，虽然找到目标函数的全局最优解很难，但这并非必要。我们将在
接下来的章
节中逐一介绍深度学习中常用的优化算法，它们在很多实际问题中都训练出了十分有效的深度学习模型。
深度学习中，虽然找到目标函数的全局最优解很难，但这并非必要。我们将在
本章接下来的小
节中逐一介绍深度学习中常用的优化算法，它们在很多实际问题中都训练出了十分有效的深度学习模型。
## 小结
...
...
@@ -113,14 +154,14 @@ gb.plt.ylabel('y');
## 练习
*
你还能想到哪些深度学习中的优化问题
的挑战？
*
对于深度学习中的优化问题，你还能想到哪些其他
的挑战？
## 扫码直达[讨论区](https://discuss.gluon.ai/t/topic/1876)
## 参考文献
[1] Wigner, E. P. (1958). On the distribution of the roots of certain symmetric matrices. Annals of Mathematics, 325-327.
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑！
取消
想要评论请
注册
或
登录
