Skip to content

B
book

PaddlePaddle / book

通知 17
Star 4
Fork 0
B
book
“c3158527717014f7875d61a4151ebdfaf3d30bd0”上不存在“paddle/phi/kernels/selected_rows/cpu/add_n_kernel.cc”
提交 fcce28ed 编写于 作者: D dingjiaweiww
浏览文件
操作

Merge branch 'develop' of https://github.com/PaddlePaddle/book into course

上级 4ccb6e7e 1ede63d1
隐藏空白更改
内联 并排
Showing with 435 addition and 697 deletion
paddle2.0_docs/dynamic_graph/dynamic_graph.ipynb
浏览文件 @ fcce28ed
...@@ -22,7 +22,7 @@ ...@@ -22,7 +22,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 6, "execution_count": 2,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -30,7 +30,7 @@ ...@@ -30,7 +30,7 @@
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"0.0.0\n", "0.0.0\n",
"7f2aa2db3c69cb9ebb8bae9e19280e75f964e1d0\n" "89af2088b6e74bdfeef2d4d78e08461ed2aafee5\n"
] ]
} }
], ],
...@@ -55,23 +55,23 @@ ...@@ -55,23 +55,23 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 2, "execution_count": 3,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"[[ 0.40741017 0.2083312 ]\n", "[[-0.49341336 -0.8112665 ]\n",
" [-1.7567089 0.72117436]\n", " [ 0.8929015 0.24661176]\n",
" [ 0.8870686 -1.1389219 ]\n", " [-0.64440054 -0.7945008 ]\n",
" [ 1.1233491 0.34348443]]\n", " [-0.07345356 1.3641853 ]]\n",
"[1. 2.]\n", "[1. 2.]\n",
"[[ 1.4074101 2.208331 ]\n", "[[0.5065867 1.1887336 ]\n",
" [-0.75670886 2.7211742 ]\n", " [1.8929014 2.2466118 ]\n",
" [ 1.8870686 0.86107814]\n", " [0.35559946 1.2054992 ]\n",
" [ 2.1233492 2.3434844 ]]\n", " [0.92654645 3.3641853 ]]\n",
"[ 0.8240726 -0.31436014 -1.3907751 1.810318 ]\n" "[-2.1159463 1.386125 -2.2334023 2.654917 ]\n"
] ]
} }
], ],
...@@ -100,7 +100,7 @@ ...@@ -100,7 +100,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 3, "execution_count": 4,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -111,12 +111,12 @@ ...@@ -111,12 +111,12 @@
"1 +> [5 7 9]\n", "1 +> [5 7 9]\n",
"2 +> [ 5 9 15]\n", "2 +> [ 5 9 15]\n",
"3 -> [-3 3 21]\n", "3 -> [-3 3 21]\n",
"4 +> [ 5 21 87]\n", "4 -> [-3 11 75]\n",
"5 +> [ 5 37 249]\n", "5 +> [ 5 37 249]\n",
"6 -> [ -3 59 723]\n", "6 +> [ 5 69 735]\n",
"7 +> [ 5 133 2193]\n", "7 -> [ -3 123 2181]\n",
"8 -> [ -3 251 6555]\n", "8 +> [ 5 261 6567]\n",
"9 -> [ -3 507 19677]\n" "9 +> [ 5 517 19689]\n"
] ]
} }
], ],
...@@ -138,15 +138,15 @@ ...@@ -138,15 +138,15 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 构建更加灵活的网络\n", "# 构建更加灵活的网络：控制流\n",
"\n", "\n",
"- 使用动态图可以用来创建更加灵活的网络，比如根据控制流选择不同的分支网络，和方便的构建权重共享的网络。接下来我们来看一个具体的例子，在这个例子中，第二个线性变换只有0.5的可能性会运行。\n", "- 使用动态图可以用来创建更加灵活的网络，比如根据控制流选择不同的分支网络，和方便的构建权重共享的网络。接下来我们来看一个具体的例子，在这个例子中，第二个线性变换只有0.5的可能性会运行。\n",
"\n" "- 在sequence to sequence with attention的机器翻译的示例中，你会看到更实际的使用动态图构建RNN类的网络带来的灵活性。\n"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 4, "execution_count": 5,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
...@@ -224,6 +224,56 @@ ...@@ -224,6 +224,56 @@
" model.clear_gradients()" " model.clear_gradients()"
] ]
}, },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 构建更加灵活的网络：共享权重\n",
"\n",
"- 使用动态图还可以更加方便的创建共享权重的网络，下面的示例展示了一个共享了权重的简单的AutoEncoder的示例。\n",
"- 你也可以参考图像搜索的示例看到共享参数权重的更实际的使用。"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"step: 0, loss: [0.37666085]\n",
"step: 1, loss: [0.3063845]\n",
"step: 2, loss: [0.2647248]\n",
"step: 3, loss: [0.23831272]\n",
"step: 4, loss: [0.21714918]\n",
"step: 5, loss: [0.1955545]\n",
"step: 6, loss: [0.17261818]\n",
"step: 7, loss: [0.15009595]\n",
"step: 8, loss: [0.13051331]\n",
"step: 9, loss: [0.11537809]\n"
]
}
],
"source": [
"inputs = paddle.rand((256, 64))\n",
"\n",
"linear = paddle.nn.Linear(64, 8, bias_attr=False)\n",
"loss_fn = paddle.nn.MSELoss()\n",
"optimizer = paddle.optimizer.Adam(0.01, parameters=linear.parameters())\n",
"\n",
"for i in range(10):\n",
" hidden = linear(inputs)\n",
" # weight from input to hidden is shared with the linear mapping from hidden to output\n",
" outputs = paddle.matmul(hidden, linear.weight, transpose_y=True) \n",
" loss = loss_fn(outputs, inputs)\n",
" loss.backward()\n",
" print(\"step: {}, loss: {}\".format(i, loss.numpy()))\n",
" optimizer.minimize(loss)\n",
" linear.clear_gradients()"
]
},
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
......
paddle2.0_docs/hello_paddle/hello_paddle.ipynb
浏览文件 @ fcce28ed
...@@ -37,7 +37,7 @@ ...@@ -37,7 +37,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 1, "execution_count": 24,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -54,8 +54,8 @@ ...@@ -54,8 +54,8 @@
} }
], ],
"source": [ "source": [
"def calculate_fee(minutes_dialed):\n", "def calculate_fee(distance_travelled):\n",
" return 10 + 2 * minutes_dialed\n", " return 10 + 2 * distance_travelled\n",
"\n", "\n",
"for x in [1.0, 3.0, 5.0, 9.0, 10.0, 20.0]:\n", "for x in [1.0, 3.0, 5.0, 9.0, 10.0, 20.0]:\n",
" print(calculate_fee(x))" " print(calculate_fee(x))"
...@@ -90,21 +90,20 @@ ...@@ -90,21 +90,20 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 2, "execution_count": 25,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"paddle version 2.0.0-alpha0\n" "paddle version 0.0.0\n"
] ]
} }
], ],
"source": [ "source": [
"import numpy as np\n",
"import paddle\n", "import paddle\n",
"\n", "paddle.disable_static()\n",
"print(\"paddle version \" + paddle.__version__)" "print(\"paddle version \" + paddle.__version__)"
] ]
}, },
...@@ -117,34 +116,17 @@ ...@@ -117,34 +116,17 @@
"在这个机器学习任务中，我们已经知道了乘客的行驶里程`distance_travelled`，和对应的，这些乘客的总费用`total_fee`。\n", "在这个机器学习任务中，我们已经知道了乘客的行驶里程`distance_travelled`，和对应的，这些乘客的总费用`total_fee`。\n",
"通常情况下，在机器学习任务中，像`distance_travelled`这样的输入值，一般被称为`x`（或者特征`feature`），像`total_fee`这样的输出值，一般被称为`y`（或者标签`label`)。\n", "通常情况下，在机器学习任务中，像`distance_travelled`这样的输入值，一般被称为`x`（或者特征`feature`），像`total_fee`这样的输出值，一般被称为`y`（或者标签`label`)。\n",
"\n", "\n",
"我们用numpy当中的数组来表示这两组数据，然后用`np.hstack`把这两组数据拼成最终的数据。" "我们用`paddle.to_tensor`把示例数据转换为paddle的Tensor数据。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 3, "execution_count": 26,
"metadata": { "metadata": {},
"scrolled": true "outputs": [],
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[ 1. 12.]\n",
" [ 3. 16.]\n",
" [ 5. 20.]\n",
" [ 9. 28.]\n",
" [10. 30.]\n",
" [20. 50.]]\n"
]
}
],
"source": [ "source": [
"x_data = np.array([[1.], [3.0], [5.0], [9.0], [10.0], [20.0]]).astype('float32')\n", "x_data = paddle.to_tensor([[1.], [3.0], [5.0], [9.0], [10.0], [20.0]])\n",
"y_data = np.array( [[12.], [16.0], [20.0], [28.0], [30.0], [50.0]]).astype('float32')\n", "y_data = paddle.to_tensor([[12.], [16.0], [20.0], [28.0], [30.0], [50.0]])"
"data = np.hstack((x_data, y_data))\n",
"print(data)"
] ]
}, },
{ {
...@@ -168,13 +150,11 @@ ...@@ -168,13 +150,11 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 4, "execution_count": 27,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"# this line will be removed after 2.0beta's official release\n", "linear = paddle.nn.Linear(in_features=1, out_features=1)"
"paddle.enable_imperative()\n",
"linear = paddle.nn.Linear(input_dim=1, output_dim=1)"
] ]
}, },
{ {
...@@ -188,24 +168,24 @@ ...@@ -188,24 +168,24 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 5, "execution_count": 28,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"w before optimize: 1.7059897184371948\n", "w before optimize: -1.7107375860214233\n",
"b before optimize: 0.0\n" "b before optimize: 0.0\n"
] ]
} }
], ],
"source": [ "source": [
"w_before_opt = linear.weight.numpy()[0][0]\n", "w_before_opt = linear.weight.numpy().item()\n",
"b_before_opt = linear.bias.numpy()[0]\n", "b_before_opt = linear.bias.numpy().item()\n",
"\n", "\n",
"print(\"w before optimize: {}\".format(w_before_opt))\n", "print(\"w before optimize: {}\".format(w_before_opt))\n",
"print(\"b before optimize: {}\".format(b_before_opt))\n" "print(\"b before optimize: {}\".format(b_before_opt))"
] ]
}, },
{ {
...@@ -220,17 +200,17 @@ ...@@ -220,17 +200,17 @@
"\n", "\n",
"用更加技术的语言来说，衡量**差距**的函数（一个公式）就是损失函数，用来**调整**参数的方法就是优化算法。\n", "用更加技术的语言来说，衡量**差距**的函数（一个公式）就是损失函数，用来**调整**参数的方法就是优化算法。\n",
"\n", "\n",
"在本示例当中，我们用最简单的均方误差(mean square error)作为损失函数(`paddle.nn.MSELoss`)；和最常见的优化算法SGD（stocastic gradient descent)作为优化算法（传给`paddle.optimizer.SGDOptimizer`的参数`learning_rate`，你可以理解为控制每次调整的步子大小的参数）。" "在本示例当中，我们用最简单的均方误差(mean square error)作为损失函数(`paddle.nn.MSELoss`)；和最常见的优化算法SGD（stocastic gradient descent)作为优化算法（传给`paddle.optimizer.SGD`的参数`learning_rate`，你可以理解为控制每次调整的步子大小的参数）。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 6, "execution_count": 29,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"mse_loss = paddle.nn.MSELoss()\n", "mse_loss = paddle.nn.MSELoss()\n",
"sgd_optimizer = paddle.optimizer.SGDOptimizer(learning_rate=0.001, parameter_list = linear.parameters())" "sgd_optimizer = paddle.optimizer.SGD(learning_rate=0.001, parameters = linear.parameters())"
] ]
}, },
{ {
...@@ -244,41 +224,35 @@ ...@@ -244,41 +224,35 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 7, "execution_count": 30,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"epoch 0 average loss [155.91637]\n", "epoch 0 loss [2107.3943]\n",
"epoch 1000 average loss [8.280919]\n", "epoch 1000 loss [7.8432994]\n",
"epoch 2000 average loss [1.851555]\n", "epoch 2000 loss [1.7537074]\n",
"epoch 3000 average loss [0.4139988]\n", "epoch 3000 loss [0.39211753]\n",
"epoch 4000 average loss [0.09256991]\n", "epoch 4000 loss [0.08767726]\n",
"finished training， average loss [0.02072961]\n" "finished training， loss [0.01963376]\n"
] ]
} }
], ],
"source": [ "source": [
"TOTAL_EPOCH = 5000\n", "total_epoch = 5000\n",
"for i in range(TOTAL_EPOCH):\n", "for i in range(total_epoch):\n",
" # 每个epoch对数据进行随机打乱，会让优化算法更好的前进。\n", " y_predict = linear(x_data)\n",
" np.random.shuffle(data)\n", " loss = mse_loss(y_predict, y_data)\n",
" # 把numpy数据转换为paddle的tensor数据\n",
" x = paddle.imperative.to_variable(data[:,:1])\n",
" y = paddle.imperative.to_variable(data[:,1:])\n",
" \n",
" y_predict = linear(x)\n",
" loss = mse_loss(y_predict, y)\n",
" loss.backward()\n", " loss.backward()\n",
" sgd_optimizer.minimize(loss)\n", " sgd_optimizer.minimize(loss)\n",
" linear.clear_gradients()\n", " linear.clear_gradients()\n",
" \n", " \n",
" if i%1000 == 0:\n", " if i%1000 == 0:\n",
" print(\"epoch {} average loss {}\".format(i, loss.numpy()))\n", " print(\"epoch {} loss {}\".format(i, loss.numpy()))\n",
" \n", " \n",
"print(\"finished training， average loss {}\".format(loss.numpy()))" "print(\"finished training， loss {}\".format(loss.numpy()))"
] ]
}, },
{ {
...@@ -292,21 +266,21 @@ ...@@ -292,21 +266,21 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 8, "execution_count": 31,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"w after optimize: 2.018333911895752\n", "w after optimize: 2.017843246459961\n",
"b after optimize: 9.765570640563965\n" "b after optimize: 9.771851539611816\n"
] ]
} }
], ],
"source": [ "source": [
"w_after_opt = linear.weight.numpy()[0][0]\n", "w_after_opt = linear.weight.numpy().item()\n",
"b_after_opt = linear.bias.numpy()[0]\n", "b_after_opt = linear.bias.numpy().item()\n",
"\n", "\n",
"print(\"w after optimize: {}\".format(w_after_opt))\n", "print(\"w after optimize: {}\".format(w_after_opt))\n",
"print(\"b after optimize: {}\".format(b_after_opt))\n" "print(\"b after optimize: {}\".format(b_after_opt))\n"
...@@ -323,7 +297,7 @@ ...@@ -323,7 +297,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 9, "execution_count": 32,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
......
paddle2.0_docs/image_classification/mnist_lenet_classification.ipynb
浏览文件 @ fcce28ed
...@@ -13,13 +13,13 @@ ...@@ -13,13 +13,13 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 环境\n", "## 环境\n",
"本教程基于paddle-develop编写，如果您的环境不是本版本，请先安装paddle-develop版本。" "本教程基于paddle-develop编写，如果您的环境不是本版本，请先安装paddle-develop版本。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 35, "execution_count": 13,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -40,13 +40,13 @@ ...@@ -40,13 +40,13 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 加载数据集\n", "## 加载数据集\n",
"我们使用飞桨自带的paddle.dataset完成mnist数据集的加载。" "我们使用飞桨自带的paddle.dataset完成mnist数据集的加载。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 36, "execution_count": 14,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -74,7 +74,7 @@ ...@@ -74,7 +74,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 37, "execution_count": 15,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -111,13 +111,13 @@ ...@@ -111,13 +111,13 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 2.组网\n", "## 组网\n",
"用paddle.nn下的API，如`Conv2d`、`Pool2D`、`Linead`完成LeNet的构建。" "用paddle.nn下的API，如`Conv2d`、`MaxPool2d`、`Linear`完成LeNet的构建。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 38, "execution_count": 18,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
...@@ -141,7 +141,7 @@ ...@@ -141,7 +141,7 @@
" x = F.relu(x)\n", " x = F.relu(x)\n",
" x = self.conv2(x)\n", " x = self.conv2(x)\n",
" x = self.max_pool2(x)\n", " x = self.max_pool2(x)\n",
" x = paddle.reshape(x, shape=[-1, 16*5*5])\n", " x = paddle.flatten(x, start_axis=1,stop_axis=-1)\n",
" x = self.linear1(x)\n", " x = self.linear1(x)\n",
" x = F.relu(x)\n", " x = F.relu(x)\n",
" x = self.linear2(x)\n", " x = self.linear2(x)\n",
...@@ -155,45 +155,45 @@ ...@@ -155,45 +155,45 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 3.训练方式一\n", "## 训练方式一\n",
"组网后，开始对模型进行训练，先构建`train_loader`，加载训练数据，然后定义`train`函数，设置好损失函数后，按batch加载数据，完成模型的训练。" "组网后，开始对模型进行训练，先构建`train_loader`，加载训练数据，然后定义`train`函数，设置好损失函数后，按batch加载数据，完成模型的训练。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 39, "execution_count": 19,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"epoch: 0, batch_id: 0, loss is: [2.3064885], acc is: [0.109375]\n", "epoch: 0, batch_id: 0, loss is: [2.3079572], acc is: [0.125]\n",
"epoch: 0, batch_id: 100, loss is: [1.5477252], acc is: [1.]\n", "epoch: 0, batch_id: 100, loss is: [1.7078608], acc is: [0.828125]\n",
"epoch: 0, batch_id: 200, loss is: [1.5201148], acc is: [1.]\n", "epoch: 0, batch_id: 200, loss is: [1.5642334], acc is: [0.90625]\n",
"epoch: 0, batch_id: 300, loss is: [1.525354], acc is: [0.953125]\n", "epoch: 0, batch_id: 300, loss is: [1.7024238], acc is: [0.78125]\n",
"epoch: 0, batch_id: 400, loss is: [1.5201038], acc is: [1.]\n", "epoch: 0, batch_id: 400, loss is: [1.5536337], acc is: [0.921875]\n",
"epoch: 0, batch_id: 500, loss is: [1.4901408], acc is: [1.]\n", "epoch: 0, batch_id: 500, loss is: [1.6908336], acc is: [0.828125]\n",
"epoch: 0, batch_id: 600, loss is: [1.4925538], acc is: [0.984375]\n", "epoch: 0, batch_id: 600, loss is: [1.5622432], acc is: [0.921875]\n",
"epoch: 0, batch_id: 700, loss is: [1.5247533], acc is: [0.96875]\n", "epoch: 0, batch_id: 700, loss is: [1.5251796], acc is: [0.953125]\n",
"epoch: 0, batch_id: 800, loss is: [1.5365943], acc is: [1.]\n", "epoch: 0, batch_id: 800, loss is: [1.5698484], acc is: [0.890625]\n",
"epoch: 0, batch_id: 900, loss is: [1.5154861], acc is: [0.984375]\n", "epoch: 0, batch_id: 900, loss is: [1.5524453], acc is: [0.9375]\n",
"epoch: 1, batch_id: 0, loss is: [1.4988302], acc is: [0.984375]\n", "epoch: 1, batch_id: 0, loss is: [1.6443151], acc is: [0.84375]\n",
"epoch: 1, batch_id: 100, loss is: [1.493154], acc is: [0.984375]\n", "epoch: 1, batch_id: 100, loss is: [1.5547533], acc is: [0.90625]\n",
"epoch: 1, batch_id: 200, loss is: [1.4974915], acc is: [1.]\n", "epoch: 1, batch_id: 200, loss is: [1.5019028], acc is: [1.]\n",
"epoch: 1, batch_id: 300, loss is: [1.5089471], acc is: [0.984375]\n", "epoch: 1, batch_id: 300, loss is: [1.4820204], acc is: [1.]\n",
"epoch: 1, batch_id: 400, loss is: [1.5041347], acc is: [1.]\n", "epoch: 1, batch_id: 400, loss is: [1.5215418], acc is: [0.984375]\n",
"epoch: 1, batch_id: 500, loss is: [1.5145375], acc is: [1.]\n", "epoch: 1, batch_id: 500, loss is: [1.4972374], acc is: [1.]\n",
"epoch: 1, batch_id: 600, loss is: [1.4904011], acc is: [0.984375]\n", "epoch: 1, batch_id: 600, loss is: [1.4930981], acc is: [0.984375]\n",
"epoch: 1, batch_id: 700, loss is: [1.5121607], acc is: [0.96875]\n", "epoch: 1, batch_id: 700, loss is: [1.4971689], acc is: [0.984375]\n",
"epoch: 1, batch_id: 800, loss is: [1.5078678], acc is: [1.]\n", "epoch: 1, batch_id: 800, loss is: [1.4611597], acc is: [1.]\n",
"epoch: 1, batch_id: 900, loss is: [1.500349], acc is: [0.984375]\n" "epoch: 1, batch_id: 900, loss is: [1.4903957], acc is: [0.984375]\n"
] ]
} }
], ],
"source": [ "source": [
"import paddle\n", "import paddle\n",
"train_loader = paddle.io.DataLoader(train_dataset, places=paddle.CPUPlace(), batch_size=64)\n", "train_loader = paddle.io.DataLoader(train_dataset, places=paddle.CPUPlace(), batch_size=64, shuffle=True)\n",
"# 加载训练集 batch_size 设为 64\n", "# 加载训练集 batch_size 设为 64\n",
"def train(model):\n", "def train(model):\n",
" model.train()\n", " model.train()\n",
...@@ -229,23 +229,21 @@ ...@@ -229,23 +229,21 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 40, "execution_count": 20,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"batch_id: 0, loss is: [1.4659549], acc is: [1.]\n", "batch_id: 0, loss is: [1.4767745], acc is: [1.]\n",
"batch_id: 100, loss is: [1.4933192], acc is: [0.984375]\n", "batch_id: 20, loss is: [1.4841802], acc is: [0.984375]\n",
"batch_id: 200, loss is: [1.4779761], acc is: [1.]\n", "batch_id: 40, loss is: [1.4997194], acc is: [1.]\n",
"batch_id: 300, loss is: [1.4919193], acc is: [0.984375]\n", "batch_id: 60, loss is: [1.4895413], acc is: [1.]\n",
"batch_id: 400, loss is: [1.5036212], acc is: [1.]\n", "batch_id: 80, loss is: [1.4668798], acc is: [1.]\n",
"batch_id: 500, loss is: [1.4922347], acc is: [0.984375]\n", "batch_id: 100, loss is: [1.4611752], acc is: [1.]\n",
"batch_id: 600, loss is: [1.4765416], acc is: [0.984375]\n", "batch_id: 120, loss is: [1.4613602], acc is: [1.]\n",
"batch_id: 700, loss is: [1.4997746], acc is: [0.984375]\n", "batch_id: 140, loss is: [1.4923686], acc is: [1.]\n"
"batch_id: 800, loss is: [1.4831288], acc is: [1.]\n",
"batch_id: 900, loss is: [1.498342], acc is: [0.984375]\n"
] ]
} }
], ],
...@@ -256,7 +254,7 @@ ...@@ -256,7 +254,7 @@
"def test(model):\n", "def test(model):\n",
" model.eval()\n", " model.eval()\n",
" batch_size = 64\n", " batch_size = 64\n",
" for batch_id, data in enumerate(train_loader()):\n", " for batch_id, data in enumerate(test_loader()):\n",
" x_data = data[0]\n", " x_data = data[0]\n",
" y_data = data[1]\n", " y_data = data[1]\n",
" predicts = model(x_data)\n", " predicts = model(x_data)\n",
...@@ -266,7 +264,7 @@ ...@@ -266,7 +264,7 @@
" avg_loss = paddle.mean(loss)\n", " avg_loss = paddle.mean(loss)\n",
" avg_acc = paddle.mean(acc)\n", " avg_acc = paddle.mean(acc)\n",
" avg_loss.backward()\n", " avg_loss.backward()\n",
" if batch_id % 100 == 0:\n", " if batch_id % 20 == 0:\n",
" print(\"batch_id: {}, loss is: {}, acc is: {}\".format(batch_id, avg_loss.numpy(), avg_acc.numpy()))\n", " print(\"batch_id: {}, loss is: {}, acc is: {}\".format(batch_id, avg_loss.numpy(), avg_acc.numpy()))\n",
"test(model)" "test(model)"
] ]
...@@ -289,7 +287,7 @@ ...@@ -289,7 +287,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 41, "execution_count": 21,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
...@@ -317,7 +315,7 @@ ...@@ -317,7 +315,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 42, "execution_count": 22,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -325,202 +323,28 @@ ...@@ -325,202 +323,28 @@
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"Epoch 1/2\n", "Epoch 1/2\n",
"step 10/938 - loss: 2.2434 - acc_top1: 0.1344 - acc_top2: 0.3719 - 14ms/step\n", "step 100/938 - loss: 1.5644 - acc_top1: 0.6281 - acc_top2: 0.7145 - 14ms/step\n",
"step 20/938 - loss: 2.0292 - acc_top1: 0.2836 - acc_top2: 0.4633 - 14ms/step\n", "step 200/938 - loss: 1.6221 - acc_top1: 0.7634 - acc_top2: 0.8380 - 13ms/step\n",
"step 30/938 - loss: 1.9341 - acc_top1: 0.3755 - acc_top2: 0.5214 - 14ms/step\n", "step 300/938 - loss: 1.5123 - acc_top1: 0.8215 - acc_top2: 0.8835 - 13ms/step\n",
"step 40/938 - loss: 1.8009 - acc_top1: 0.4469 - acc_top2: 0.5727 - 14ms/step\n", "step 400/938 - loss: 1.4791 - acc_top1: 0.8530 - acc_top2: 0.9084 - 13ms/step\n",
"step 50/938 - loss: 1.8000 - acc_top1: 0.4975 - acc_top2: 0.6125 - 13ms/step\n", "step 500/938 - loss: 1.4904 - acc_top1: 0.8733 - acc_top2: 0.9235 - 13ms/step\n",
"step 60/938 - loss: 1.6335 - acc_top1: 0.5417 - acc_top2: 0.6438 - 14ms/step\n", "step 600/938 - loss: 1.5101 - acc_top1: 0.8875 - acc_top2: 0.9341 - 13ms/step\n",
"step 70/938 - loss: 1.7931 - acc_top1: 0.5708 - acc_top2: 0.6643 - 13ms/step\n", "step 700/938 - loss: 1.4642 - acc_top1: 0.8983 - acc_top2: 0.9417 - 13ms/step\n",
"step 80/938 - loss: 1.6699 - acc_top1: 0.5961 - acc_top2: 0.6846 - 13ms/step\n", "step 800/938 - loss: 1.4789 - acc_top1: 0.9069 - acc_top2: 0.9477 - 13ms/step\n",
"step 90/938 - loss: 1.6832 - acc_top1: 0.6189 - acc_top2: 0.7069 - 13ms/step\n", "step 900/938 - loss: 1.4773 - acc_top1: 0.9135 - acc_top2: 0.9523 - 13ms/step\n",
"step 100/938 - loss: 1.6336 - acc_top1: 0.6409 - acc_top2: 0.7245 - 14ms/step\n", "step 938/938 - loss: 1.4714 - acc_top1: 0.9157 - acc_top2: 0.9538 - 13ms/step\n",
"step 110/938 - loss: 1.6598 - acc_top1: 0.6557 - acc_top2: 0.7376 - 13ms/step\n",
"step 120/938 - loss: 1.6348 - acc_top1: 0.6708 - acc_top2: 0.7488 - 13ms/step\n",
"step 130/938 - loss: 1.6223 - acc_top1: 0.6851 - acc_top2: 0.7601 - 13ms/step\n",
"step 140/938 - loss: 1.5622 - acc_top1: 0.6970 - acc_top2: 0.7694 - 13ms/step\n",
"step 150/938 - loss: 1.6455 - acc_top1: 0.7065 - acc_top2: 0.7767 - 14ms/step\n",
"step 160/938 - loss: 1.6127 - acc_top1: 0.7154 - acc_top2: 0.7837 - 14ms/step\n",
"step 170/938 - loss: 1.5963 - acc_top1: 0.7242 - acc_top2: 0.7898 - 14ms/step\n",
"step 180/938 - loss: 1.6485 - acc_top1: 0.7310 - acc_top2: 0.7948 - 14ms/step\n",
"step 190/938 - loss: 1.6666 - acc_top1: 0.7368 - acc_top2: 0.7992 - 14ms/step\n",
"step 200/938 - loss: 1.7862 - acc_top1: 0.7419 - acc_top2: 0.8030 - 14ms/step\n",
"step 210/938 - loss: 1.5479 - acc_top1: 0.7464 - acc_top2: 0.8064 - 14ms/step\n",
"step 220/938 - loss: 1.5650 - acc_top1: 0.7515 - acc_top2: 0.8106 - 14ms/step\n",
"step 230/938 - loss: 1.5822 - acc_top1: 0.7562 - acc_top2: 0.8141 - 14ms/step\n",
"step 240/938 - loss: 1.5966 - acc_top1: 0.7608 - acc_top2: 0.8179 - 14ms/step\n",
"step 250/938 - loss: 1.5551 - acc_top1: 0.7650 - acc_top2: 0.8213 - 14ms/step\n",
"step 260/938 - loss: 1.5584 - acc_top1: 0.7699 - acc_top2: 0.8249 - 14ms/step\n",
"step 270/938 - loss: 1.5933 - acc_top1: 0.7730 - acc_top2: 0.8273 - 14ms/step\n",
"step 280/938 - loss: 1.5589 - acc_top1: 0.7769 - acc_top2: 0.8301 - 14ms/step\n",
"step 290/938 - loss: 1.6513 - acc_top1: 0.7793 - acc_top2: 0.8315 - 14ms/step\n",
"step 300/938 - loss: 1.5929 - acc_top1: 0.7821 - acc_top2: 0.8337 - 14ms/step\n",
"step 310/938 - loss: 1.5672 - acc_top1: 0.7849 - acc_top2: 0.8360 - 14ms/step\n",
"step 320/938 - loss: 1.5147 - acc_top1: 0.7879 - acc_top2: 0.8381 - 14ms/step\n",
"step 330/938 - loss: 1.5697 - acc_top1: 0.7902 - acc_top2: 0.8397 - 14ms/step\n",
"step 340/938 - loss: 1.5697 - acc_top1: 0.7919 - acc_top2: 0.8406 - 14ms/step\n",
"step 350/938 - loss: 1.6122 - acc_top1: 0.7941 - acc_top2: 0.8423 - 14ms/step\n",
"step 360/938 - loss: 1.5934 - acc_top1: 0.7960 - acc_top2: 0.8435 - 14ms/step\n",
"step 370/938 - loss: 1.6258 - acc_top1: 0.7982 - acc_top2: 0.8451 - 14ms/step\n",
"step 380/938 - loss: 1.6805 - acc_top1: 0.7996 - acc_top2: 0.8463 - 14ms/step\n",
"step 390/938 - loss: 1.5997 - acc_top1: 0.8011 - acc_top2: 0.8475 - 14ms/step\n",
"step 400/938 - loss: 1.6151 - acc_top1: 0.8029 - acc_top2: 0.8488 - 14ms/step\n",
"step 410/938 - loss: 1.5800 - acc_top1: 0.8047 - acc_top2: 0.8499 - 14ms/step\n",
"step 420/938 - loss: 1.5950 - acc_top1: 0.8060 - acc_top2: 0.8508 - 14ms/step\n",
"step 430/938 - loss: 1.5533 - acc_top1: 0.8075 - acc_top2: 0.8517 - 14ms/step\n",
"step 440/938 - loss: 1.6171 - acc_top1: 0.8086 - acc_top2: 0.8521 - 14ms/step\n",
"step 450/938 - loss: 1.5756 - acc_top1: 0.8103 - acc_top2: 0.8533 - 14ms/step\n",
"step 460/938 - loss: 1.5655 - acc_top1: 0.8121 - acc_top2: 0.8544 - 14ms/step\n",
"step 470/938 - loss: 1.5816 - acc_top1: 0.8139 - acc_top2: 0.8555 - 14ms/step\n",
"step 480/938 - loss: 1.6202 - acc_top1: 0.8148 - acc_top2: 0.8562 - 14ms/step\n",
"step 490/938 - loss: 1.6223 - acc_top1: 0.8157 - acc_top2: 0.8567 - 14ms/step\n",
"step 500/938 - loss: 1.5198 - acc_top1: 0.8167 - acc_top2: 0.8574 - 14ms/step\n",
"step 510/938 - loss: 1.5853 - acc_top1: 0.8181 - acc_top2: 0.8583 - 14ms/step\n",
"step 520/938 - loss: 1.5252 - acc_top1: 0.8196 - acc_top2: 0.8593 - 14ms/step\n",
"step 530/938 - loss: 1.5265 - acc_top1: 0.8207 - acc_top2: 0.8601 - 14ms/step\n",
"step 540/938 - loss: 1.5297 - acc_top1: 0.8217 - acc_top2: 0.8608 - 14ms/step\n",
"step 550/938 - loss: 1.5743 - acc_top1: 0.8226 - acc_top2: 0.8613 - 13ms/step\n",
"step 560/938 - loss: 1.6419 - acc_top1: 0.8237 - acc_top2: 0.8622 - 13ms/step\n",
"step 570/938 - loss: 1.5556 - acc_top1: 0.8247 - acc_top2: 0.8630 - 13ms/step\n",
"step 580/938 - loss: 1.5349 - acc_top1: 0.8254 - acc_top2: 0.8635 - 13ms/step\n",
"step 590/938 - loss: 1.4915 - acc_top1: 0.8263 - acc_top2: 0.8640 - 13ms/step\n",
"step 600/938 - loss: 1.5672 - acc_top1: 0.8277 - acc_top2: 0.8651 - 13ms/step\n",
"step 610/938 - loss: 1.5464 - acc_top1: 0.8288 - acc_top2: 0.8659 - 13ms/step\n",
"step 620/938 - loss: 1.6329 - acc_top1: 0.8292 - acc_top2: 0.8661 - 13ms/step\n",
"step 630/938 - loss: 1.6121 - acc_top1: 0.8296 - acc_top2: 0.8662 - 13ms/step\n",
"step 640/938 - loss: 1.5636 - acc_top1: 0.8305 - acc_top2: 0.8668 - 13ms/step\n",
"step 650/938 - loss: 1.6227 - acc_top1: 0.8311 - acc_top2: 0.8672 - 13ms/step\n",
"step 660/938 - loss: 1.5646 - acc_top1: 0.8319 - acc_top2: 0.8678 - 13ms/step\n",
"step 670/938 - loss: 1.5620 - acc_top1: 0.8325 - acc_top2: 0.8681 - 13ms/step\n",
"step 680/938 - loss: 1.4908 - acc_top1: 0.8333 - acc_top2: 0.8688 - 13ms/step\n",
"step 690/938 - loss: 1.6010 - acc_top1: 0.8339 - acc_top2: 0.8691 - 13ms/step\n",
"step 700/938 - loss: 1.5592 - acc_top1: 0.8346 - acc_top2: 0.8695 - 13ms/step\n",
"step 710/938 - loss: 1.6226 - acc_top1: 0.8352 - acc_top2: 0.8699 - 13ms/step\n",
"step 720/938 - loss: 1.5642 - acc_top1: 0.8362 - acc_top2: 0.8705 - 13ms/step\n",
"step 730/938 - loss: 1.5807 - acc_top1: 0.8367 - acc_top2: 0.8707 - 13ms/step\n",
"step 740/938 - loss: 1.5721 - acc_top1: 0.8371 - acc_top2: 0.8708 - 13ms/step\n",
"step 750/938 - loss: 1.6542 - acc_top1: 0.8377 - acc_top2: 0.8711 - 13ms/step\n",
"step 760/938 - loss: 1.5128 - acc_top1: 0.8385 - acc_top2: 0.8716 - 13ms/step\n",
"step 770/938 - loss: 1.5711 - acc_top1: 0.8391 - acc_top2: 0.8721 - 14ms/step\n",
"step 780/938 - loss: 1.6095 - acc_top1: 0.8395 - acc_top2: 0.8725 - 14ms/step\n",
"step 790/938 - loss: 1.5348 - acc_top1: 0.8402 - acc_top2: 0.8730 - 14ms/step\n",
"step 800/938 - loss: 1.5715 - acc_top1: 0.8407 - acc_top2: 0.8732 - 14ms/step\n",
"step 810/938 - loss: 1.5880 - acc_top1: 0.8413 - acc_top2: 0.8737 - 14ms/step\n",
"step 820/938 - loss: 1.6160 - acc_top1: 0.8418 - acc_top2: 0.8740 - 14ms/step\n",
"step 830/938 - loss: 1.5585 - acc_top1: 0.8426 - acc_top2: 0.8746 - 14ms/step\n",
"step 840/938 - loss: 1.5829 - acc_top1: 0.8429 - acc_top2: 0.8748 - 14ms/step\n",
"step 850/938 - loss: 1.5348 - acc_top1: 0.8435 - acc_top2: 0.8753 - 14ms/step\n",
"step 860/938 - loss: 1.5448 - acc_top1: 0.8438 - acc_top2: 0.8754 - 14ms/step\n",
"step 870/938 - loss: 1.5463 - acc_top1: 0.8443 - acc_top2: 0.8759 - 14ms/step\n",
"step 880/938 - loss: 1.5763 - acc_top1: 0.8449 - acc_top2: 0.8762 - 14ms/step\n",
"step 890/938 - loss: 1.5699 - acc_top1: 0.8453 - acc_top2: 0.8764 - 14ms/step\n",
"step 900/938 - loss: 1.5616 - acc_top1: 0.8456 - acc_top2: 0.8766 - 14ms/step\n",
"step 910/938 - loss: 1.5026 - acc_top1: 0.8461 - acc_top2: 0.8771 - 14ms/step\n",
"step 920/938 - loss: 1.5380 - acc_top1: 0.8467 - acc_top2: 0.8774 - 14ms/step\n",
"step 930/938 - loss: 1.5993 - acc_top1: 0.8470 - acc_top2: 0.8777 - 14ms/step\n",
"step 938/938 - loss: 1.4942 - acc_top1: 0.8473 - acc_top2: 0.8778 - 14ms/step\n",
"save checkpoint at /Users/chenlong/online_repo/book/paddle2.0_docs/image_classification/mnist_checkpoint/0\n", "save checkpoint at /Users/chenlong/online_repo/book/paddle2.0_docs/image_classification/mnist_checkpoint/0\n",
"Epoch 2/2\n", "Epoch 2/2\n",
"step 10/938 - loss: 1.5919 - acc_top1: 0.8875 - acc_top2: 0.9047 - 14ms/step\n", "step 100/938 - loss: 1.4863 - acc_top1: 0.9695 - acc_top2: 0.9897 - 13ms/step\n",
"step 20/938 - loss: 1.5900 - acc_top1: 0.8875 - acc_top2: 0.9062 - 14ms/step\n", "step 200/938 - loss: 1.4883 - acc_top1: 0.9707 - acc_top2: 0.9912 - 13ms/step\n",
"step 30/938 - loss: 1.5929 - acc_top1: 0.8891 - acc_top2: 0.9036 - 13ms/step\n", "step 300/938 - loss: 1.4695 - acc_top1: 0.9720 - acc_top2: 0.9910 - 13ms/step\n",
"step 40/938 - loss: 1.5855 - acc_top1: 0.8883 - acc_top2: 0.9027 - 13ms/step\n", "step 400/938 - loss: 1.4628 - acc_top1: 0.9720 - acc_top2: 0.9915 - 13ms/step\n",
"step 50/938 - loss: 1.6197 - acc_top1: 0.8916 - acc_top2: 0.9072 - 13ms/step\n", "step 500/938 - loss: 1.5079 - acc_top1: 0.9727 - acc_top2: 0.9918 - 13ms/step\n",
"step 60/938 - loss: 1.5084 - acc_top1: 0.8914 - acc_top2: 0.9078 - 13ms/step\n", "step 600/938 - loss: 1.4803 - acc_top1: 0.9727 - acc_top2: 0.9919 - 13ms/step\n",
"step 70/938 - loss: 1.5552 - acc_top1: 0.8904 - acc_top2: 0.9067 - 13ms/step\n", "step 700/938 - loss: 1.4612 - acc_top1: 0.9732 - acc_top2: 0.9923 - 13ms/step\n",
"step 80/938 - loss: 1.5700 - acc_top1: 0.8887 - acc_top2: 0.9049 - 13ms/step\n", "step 800/938 - loss: 1.4755 - acc_top1: 0.9732 - acc_top2: 0.9923 - 13ms/step\n",
"step 90/938 - loss: 1.6073 - acc_top1: 0.8866 - acc_top2: 0.9030 - 13ms/step\n", "step 900/938 - loss: 1.4698 - acc_top1: 0.9732 - acc_top2: 0.9922 - 13ms/step\n",
"step 100/938 - loss: 1.5754 - acc_top1: 0.8859 - acc_top2: 0.9022 - 13ms/step\n" "step 938/938 - loss: 1.4764 - acc_top1: 0.9734 - acc_top2: 0.9923 - 13ms/step\n",
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"step 110/938 - loss: 1.5484 - acc_top1: 0.8848 - acc_top2: 0.9017 - 14ms/step\n",
"step 120/938 - loss: 1.5904 - acc_top1: 0.8840 - acc_top2: 0.9020 - 14ms/step\n",
"step 130/938 - loss: 1.5108 - acc_top1: 0.8852 - acc_top2: 0.9025 - 14ms/step\n",
"step 140/938 - loss: 1.6199 - acc_top1: 0.8840 - acc_top2: 0.9016 - 14ms/step\n",
"step 150/938 - loss: 1.5337 - acc_top1: 0.8842 - acc_top2: 0.9019 - 13ms/step\n",
"step 160/938 - loss: 1.6094 - acc_top1: 0.8846 - acc_top2: 0.9023 - 13ms/step\n",
"step 170/938 - loss: 1.5653 - acc_top1: 0.8843 - acc_top2: 0.9019 - 13ms/step\n",
"step 180/938 - loss: 1.5978 - acc_top1: 0.8835 - acc_top2: 0.9011 - 13ms/step\n",
"step 190/938 - loss: 1.5950 - acc_top1: 0.8833 - acc_top2: 0.9012 - 13ms/step\n",
"step 200/938 - loss: 1.6422 - acc_top1: 0.8828 - acc_top2: 0.9002 - 13ms/step\n",
"step 210/938 - loss: 1.5752 - acc_top1: 0.8831 - acc_top2: 0.9004 - 13ms/step\n",
"step 220/938 - loss: 1.6635 - acc_top1: 0.8832 - acc_top2: 0.9001 - 13ms/step\n",
"step 230/938 - loss: 1.5726 - acc_top1: 0.8823 - acc_top2: 0.8991 - 13ms/step\n",
"step 240/938 - loss: 1.5702 - acc_top1: 0.8814 - acc_top2: 0.8981 - 13ms/step\n",
"step 250/938 - loss: 1.5748 - acc_top1: 0.8814 - acc_top2: 0.8981 - 14ms/step\n",
"step 260/938 - loss: 1.5589 - acc_top1: 0.8822 - acc_top2: 0.8988 - 14ms/step\n",
"step 270/938 - loss: 1.5902 - acc_top1: 0.8823 - acc_top2: 0.8988 - 14ms/step\n",
"step 280/938 - loss: 1.5646 - acc_top1: 0.8817 - acc_top2: 0.8982 - 14ms/step\n",
"step 290/938 - loss: 1.6280 - acc_top1: 0.8819 - acc_top2: 0.8985 - 14ms/step\n",
"step 300/938 - loss: 1.5697 - acc_top1: 0.8815 - acc_top2: 0.8982 - 14ms/step\n",
"step 310/938 - loss: 1.5540 - acc_top1: 0.8814 - acc_top2: 0.8981 - 14ms/step\n",
"step 320/938 - loss: 1.5598 - acc_top1: 0.8821 - acc_top2: 0.8988 - 14ms/step\n",
"step 330/938 - loss: 1.5498 - acc_top1: 0.8824 - acc_top2: 0.8991 - 14ms/step\n",
"step 340/938 - loss: 1.6276 - acc_top1: 0.8818 - acc_top2: 0.8984 - 14ms/step\n",
"step 350/938 - loss: 1.5129 - acc_top1: 0.8821 - acc_top2: 0.8988 - 14ms/step\n",
"step 360/938 - loss: 1.6158 - acc_top1: 0.8818 - acc_top2: 0.8984 - 14ms/step\n",
"step 370/938 - loss: 1.5300 - acc_top1: 0.8820 - acc_top2: 0.8986 - 14ms/step\n",
"step 380/938 - loss: 1.5718 - acc_top1: 0.8822 - acc_top2: 0.8988 - 14ms/step\n",
"step 390/938 - loss: 1.5898 - acc_top1: 0.8825 - acc_top2: 0.8990 - 14ms/step\n",
"step 400/938 - loss: 1.5177 - acc_top1: 0.8834 - acc_top2: 0.9000 - 14ms/step\n",
"step 410/938 - loss: 1.6493 - acc_top1: 0.8831 - acc_top2: 0.8997 - 14ms/step\n",
"step 420/938 - loss: 1.5071 - acc_top1: 0.8838 - acc_top2: 0.9002 - 14ms/step\n",
"step 430/938 - loss: 1.5982 - acc_top1: 0.8840 - acc_top2: 0.9002 - 14ms/step\n",
"step 440/938 - loss: 1.5649 - acc_top1: 0.8841 - acc_top2: 0.9003 - 14ms/step\n",
"step 450/938 - loss: 1.5555 - acc_top1: 0.8844 - acc_top2: 0.9005 - 14ms/step\n",
"step 460/938 - loss: 1.5536 - acc_top1: 0.8845 - acc_top2: 0.9005 - 14ms/step\n",
"step 470/938 - loss: 1.5401 - acc_top1: 0.8851 - acc_top2: 0.9011 - 14ms/step\n",
"step 480/938 - loss: 1.5549 - acc_top1: 0.8854 - acc_top2: 0.9013 - 14ms/step\n",
"step 490/938 - loss: 1.5596 - acc_top1: 0.8858 - acc_top2: 0.9017 - 14ms/step\n",
"step 500/938 - loss: 1.5059 - acc_top1: 0.8860 - acc_top2: 0.9018 - 14ms/step\n",
"step 510/938 - loss: 1.6073 - acc_top1: 0.8858 - acc_top2: 0.9017 - 14ms/step\n",
"step 520/938 - loss: 1.5588 - acc_top1: 0.8857 - acc_top2: 0.9016 - 14ms/step\n",
"step 530/938 - loss: 1.6165 - acc_top1: 0.8859 - acc_top2: 0.9019 - 14ms/step\n",
"step 540/938 - loss: 1.5884 - acc_top1: 0.8862 - acc_top2: 0.9023 - 14ms/step\n",
"step 550/938 - loss: 1.6552 - acc_top1: 0.8863 - acc_top2: 0.9027 - 14ms/step\n",
"step 560/938 - loss: 1.5529 - acc_top1: 0.8867 - acc_top2: 0.9030 - 14ms/step\n",
"step 570/938 - loss: 1.5441 - acc_top1: 0.8866 - acc_top2: 0.9029 - 14ms/step\n",
"step 580/938 - loss: 1.5438 - acc_top1: 0.8867 - acc_top2: 0.9029 - 14ms/step\n",
"step 590/938 - loss: 1.5761 - acc_top1: 0.8868 - acc_top2: 0.9029 - 14ms/step\n",
"step 600/938 - loss: 1.5384 - acc_top1: 0.8867 - acc_top2: 0.9029 - 14ms/step\n",
"step 610/938 - loss: 1.5858 - acc_top1: 0.8871 - acc_top2: 0.9032 - 14ms/step\n",
"step 620/938 - loss: 1.5524 - acc_top1: 0.8872 - acc_top2: 0.9034 - 14ms/step\n",
"step 630/938 - loss: 1.6182 - acc_top1: 0.8875 - acc_top2: 0.9035 - 14ms/step\n",
"step 640/938 - loss: 1.6326 - acc_top1: 0.8877 - acc_top2: 0.9037 - 14ms/step\n",
"step 650/938 - loss: 1.5871 - acc_top1: 0.8877 - acc_top2: 0.9035 - 14ms/step\n",
"step 660/938 - loss: 1.5403 - acc_top1: 0.8877 - acc_top2: 0.9034 - 14ms/step\n",
"step 670/938 - loss: 1.5539 - acc_top1: 0.8879 - acc_top2: 0.9035 - 14ms/step\n",
"step 680/938 - loss: 1.4918 - acc_top1: 0.8881 - acc_top2: 0.9036 - 14ms/step\n",
"step 690/938 - loss: 1.6007 - acc_top1: 0.8882 - acc_top2: 0.9036 - 14ms/step\n",
"step 700/938 - loss: 1.5539 - acc_top1: 0.8883 - acc_top2: 0.9037 - 14ms/step\n",
"step 710/938 - loss: 1.6036 - acc_top1: 0.8882 - acc_top2: 0.9035 - 14ms/step\n",
"step 720/938 - loss: 1.5943 - acc_top1: 0.8881 - acc_top2: 0.9035 - 14ms/step\n",
"step 730/938 - loss: 1.5714 - acc_top1: 0.8881 - acc_top2: 0.9035 - 14ms/step\n",
"step 740/938 - loss: 1.5095 - acc_top1: 0.8881 - acc_top2: 0.9035 - 14ms/step\n",
"step 750/938 - loss: 1.5069 - acc_top1: 0.8882 - acc_top2: 0.9035 - 14ms/step\n",
"step 760/938 - loss: 1.5816 - acc_top1: 0.8882 - acc_top2: 0.9035 - 14ms/step\n",
"step 770/938 - loss: 1.5855 - acc_top1: 0.8880 - acc_top2: 0.9033 - 14ms/step\n",
"step 780/938 - loss: 1.5599 - acc_top1: 0.8881 - acc_top2: 0.9034 - 14ms/step\n",
"step 790/938 - loss: 1.6029 - acc_top1: 0.8879 - acc_top2: 0.9032 - 14ms/step\n",
"step 800/938 - loss: 1.5839 - acc_top1: 0.8880 - acc_top2: 0.9033 - 14ms/step\n",
"step 810/938 - loss: 1.5545 - acc_top1: 0.8882 - acc_top2: 0.9035 - 14ms/step\n",
"step 820/938 - loss: 1.5458 - acc_top1: 0.8881 - acc_top2: 0.9036 - 14ms/step\n",
"step 830/938 - loss: 1.5911 - acc_top1: 0.8879 - acc_top2: 0.9033 - 14ms/step\n",
"step 840/938 - loss: 1.5845 - acc_top1: 0.8881 - acc_top2: 0.9035 - 14ms/step\n",
"step 850/938 - loss: 1.5628 - acc_top1: 0.8880 - acc_top2: 0.9035 - 14ms/step\n",
"step 860/938 - loss: 1.5596 - acc_top1: 0.8880 - acc_top2: 0.9035 - 14ms/step\n",
"step 870/938 - loss: 1.5843 - acc_top1: 0.8882 - acc_top2: 0.9036 - 14ms/step\n",
"step 880/938 - loss: 1.5393 - acc_top1: 0.8883 - acc_top2: 0.9036 - 14ms/step\n",
"step 890/938 - loss: 1.5382 - acc_top1: 0.8882 - acc_top2: 0.9035 - 14ms/step\n",
"step 900/938 - loss: 1.5910 - acc_top1: 0.8884 - acc_top2: 0.9036 - 14ms/step\n",
"step 910/938 - loss: 1.5682 - acc_top1: 0.8886 - acc_top2: 0.9038 - 14ms/step\n",
"step 920/938 - loss: 1.5736 - acc_top1: 0.8889 - acc_top2: 0.9039 - 14ms/step\n",
"step 930/938 - loss: 1.5283 - acc_top1: 0.8888 - acc_top2: 0.9038 - 14ms/step\n",
"step 938/938 - loss: 1.5582 - acc_top1: 0.8888 - acc_top2: 0.9038 - 14ms/step\n",
"save checkpoint at /Users/chenlong/online_repo/book/paddle2.0_docs/image_classification/mnist_checkpoint/1\n", "save checkpoint at /Users/chenlong/online_repo/book/paddle2.0_docs/image_classification/mnist_checkpoint/1\n",
"save checkpoint at /Users/chenlong/online_repo/book/paddle2.0_docs/image_classification/mnist_checkpoint/final\n" "save checkpoint at /Users/chenlong/online_repo/book/paddle2.0_docs/image_classification/mnist_checkpoint/final\n"
] ]
...@@ -530,6 +354,7 @@ ...@@ -530,6 +354,7 @@
"model.fit(train_dataset,\n", "model.fit(train_dataset,\n",
" epochs=2,\n", " epochs=2,\n",
" batch_size=64,\n", " batch_size=64,\n",
" log_freq=100,\n",
" save_dir='mnist_checkpoint')" " save_dir='mnist_checkpoint')"
] ]
}, },
...@@ -542,7 +367,7 @@ ...@@ -542,7 +367,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 43, "execution_count": 23,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -550,32 +375,32 @@ ...@@ -550,32 +375,32 @@
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"Eval begin...\n", "Eval begin...\n",
"step 10/157 - loss: 1.5447 - acc_top1: 0.8953 - acc_top2: 0.9078 - 5ms/step\n", "step 10/157 - loss: 1.5238 - acc_top1: 0.9750 - acc_top2: 0.9938 - 7ms/step\n",
"step 20/157 - loss: 1.6185 - acc_top1: 0.8930 - acc_top2: 0.9078 - 5ms/step\n", "step 20/157 - loss: 1.5143 - acc_top1: 0.9727 - acc_top2: 0.9922 - 7ms/step\n",
"step 30/157 - loss: 1.6497 - acc_top1: 0.8917 - acc_top2: 0.9057 - 5ms/step\n", "step 30/157 - loss: 1.5290 - acc_top1: 0.9698 - acc_top2: 0.9932 - 7ms/step\n",
"step 40/157 - loss: 1.6318 - acc_top1: 0.8902 - acc_top2: 0.9055 - 5ms/step\n", "step 40/157 - loss: 1.4624 - acc_top1: 0.9684 - acc_top2: 0.9930 - 7ms/step\n",
"step 50/157 - loss: 1.5533 - acc_top1: 0.8856 - acc_top2: 0.9012 - 5ms/step\n", "step 50/157 - loss: 1.4771 - acc_top1: 0.9697 - acc_top2: 0.9925 - 7ms/step\n",
"step 60/157 - loss: 1.6212 - acc_top1: 0.8878 - acc_top2: 0.9036 - 5ms/step\n", "step 60/157 - loss: 1.5066 - acc_top1: 0.9701 - acc_top2: 0.9922 - 6ms/step\n",
"step 70/157 - loss: 1.5674 - acc_top1: 0.8839 - acc_top2: 0.9002 - 5ms/step\n", "step 70/157 - loss: 1.4804 - acc_top1: 0.9699 - acc_top2: 0.9920 - 6ms/step\n",
"step 80/157 - loss: 1.5409 - acc_top1: 0.8891 - acc_top2: 0.9043 - 5ms/step\n", "step 80/157 - loss: 1.4718 - acc_top1: 0.9707 - acc_top2: 0.9930 - 6ms/step\n",
"step 90/157 - loss: 1.6133 - acc_top1: 0.8903 - acc_top2: 0.9045 - 5ms/step\n", "step 90/157 - loss: 1.4874 - acc_top1: 0.9726 - acc_top2: 0.9934 - 6ms/step\n",
"step 100/157 - loss: 1.5535 - acc_top1: 0.8909 - acc_top2: 0.9044 - 5ms/step\n", "step 100/157 - loss: 1.4612 - acc_top1: 0.9736 - acc_top2: 0.9936 - 6ms/step\n",
"step 110/157 - loss: 1.5690 - acc_top1: 0.8916 - acc_top2: 0.9054 - 5ms/step\n", "step 110/157 - loss: 1.4612 - acc_top1: 0.9746 - acc_top2: 0.9938 - 6ms/step\n",
"step 120/157 - loss: 1.6147 - acc_top1: 0.8926 - acc_top2: 0.9055 - 5ms/step\n", "step 120/157 - loss: 1.4763 - acc_top1: 0.9763 - acc_top2: 0.9941 - 6ms/step\n",
"step 130/157 - loss: 1.5203 - acc_top1: 0.8944 - acc_top2: 0.9066 - 5ms/step\n", "step 130/157 - loss: 1.4786 - acc_top1: 0.9764 - acc_top2: 0.9935 - 6ms/step\n",
"step 140/157 - loss: 1.5066 - acc_top1: 0.8952 - acc_top2: 0.9068 - 5ms/step\n", "step 140/157 - loss: 1.4612 - acc_top1: 0.9775 - acc_top2: 0.9939 - 6ms/step\n",
"step 150/157 - loss: 1.5536 - acc_top1: 0.8958 - acc_top2: 0.9072 - 5ms/step\n", "step 150/157 - loss: 1.4894 - acc_top1: 0.9785 - acc_top2: 0.9943 - 6ms/step\n",
"step 157/157 - loss: 1.5855 - acc_top1: 0.8956 - acc_top2: 0.9076 - 5ms/step\n", "step 157/157 - loss: 1.4612 - acc_top1: 0.9777 - acc_top2: 0.9941 - 6ms/step\n",
"Eval samples: 10000\n" "Eval samples: 10000\n"
] ]
}, },
{ {
"data": { "data": {
"text/plain": [ "text/plain": [
"{'loss': [1.585474], 'acc_top1': 0.8956, 'acc_top2': 0.9076}" "{'loss': [1.4611504], 'acc_top1': 0.9777, 'acc_top2': 0.9941}"
] ]
}, },
"execution_count": 43, "execution_count": 23,
"metadata": {}, "metadata": {},
"output_type": "execute_result" "output_type": "execute_result"
} }
...@@ -605,13 +430,6 @@ ...@@ -605,13 +430,6 @@
"source": [ "source": [
"以上就是用LeNet对手写数字数据及MNIST进行分类。本示例提供了两种训练模型的方式，一种可以快速完成模型的组建与预测，非常适合新手用户上手。另一种则需要多个步骤来完成模型的训练，适合进阶用户使用。" "以上就是用LeNet对手写数字数据及MNIST进行分类。本示例提供了两种训练模型的方式，一种可以快速完成模型的组建与预测，非常适合新手用户上手。另一种则需要多个步骤来完成模型的训练，适合进阶用户使用。"
] ]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
} }
], ],
"metadata": { "metadata": {
......
paddle2.0_docs/image_search/image_search.ipynb
浏览文件 @ fcce28ed
因为 它太大了无法显示 source diff 。你可以改为 查看blob。
paddle2.0_docs/imdb_bow_classification/imdb_bow_classification.ipynb
浏览文件 @ fcce28ed
...@@ -25,14 +25,15 @@ ...@@ -25,14 +25,15 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 1, "execution_count": 4,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"2.0.0-alpha0\n" "0.0.0\n",
"264e76cae6861ad9b1d4bcd8c3212f7a78c01e4d\n"
] ]
} }
], ],
...@@ -40,8 +41,9 @@ ...@@ -40,8 +41,9 @@
"import paddle\n", "import paddle\n",
"import numpy as np\n", "import numpy as np\n",
"\n", "\n",
"paddle.disable_static()\n",
"print(paddle.__version__)\n", "print(paddle.__version__)\n",
"paddle.enable_imperative()" "print(paddle.__git_commit__)\n"
] ]
}, },
{ {
...@@ -55,7 +57,7 @@ ...@@ -55,7 +57,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 2, "execution_count": 5,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -76,7 +78,7 @@ ...@@ -76,7 +78,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 3, "execution_count": 6,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -124,7 +126,7 @@ ...@@ -124,7 +126,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 4, "execution_count": 7,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
...@@ -155,7 +157,7 @@ ...@@ -155,7 +157,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 5, "execution_count": 8,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -188,7 +190,7 @@ ...@@ -188,7 +190,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 6, "execution_count": 9,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -239,16 +241,16 @@ ...@@ -239,16 +241,16 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 7, "execution_count": 11,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"class MyNet(paddle.nn.Layer):\n", "class MyNet(paddle.nn.Layer):\n",
" def __init__(self):\n", " def __init__(self):\n",
" super(MyNet, self).__init__()\n", " super(MyNet, self).__init__()\n",
" self.emb = paddle.nn.Embedding(size=[vocab_size, emb_size],)\n", " self.emb = paddle.nn.Embedding(vocab_size, emb_size)\n",
" self.fc = paddle.nn.Linear(input_dim=emb_size, output_dim=2)\n", " self.fc = paddle.nn.Linear(in_features=emb_size, out_features=2)\n",
" self.dropout = paddle.fluid.dygraph.Dropout(0.5)\n", " self.dropout = paddle.nn.Dropout(0.5)\n",
"\n", "\n",
" def forward(self, x):\n", " def forward(self, x):\n",
" x = self.emb(x)\n", " x = self.emb(x)\n",
...@@ -262,41 +264,32 @@ ...@@ -262,41 +264,32 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 开始模型的训练\n", "# 开始模型的训练\n"
"\n",
"- TODO: make this part shorter by using `fit`"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 9, "execution_count": 13,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"epoch: 0, batch_id: 0, loss is: [0.6932132]\n", "epoch: 0, batch_id: 0, loss is: [0.6926701]\n",
"epoch: 0, batch_id: 500, loss is: [0.40177184]\n", "epoch: 0, batch_id: 500, loss is: [0.41248566]\n",
"[validation] accuracy/loss: 0.8407090306282043/0.3917059302330017\n", "[validation] accuracy/loss: 0.8505121469497681/0.3615057170391083\n",
"epoch: 1, batch_id: 0, loss is: [0.30510694]\n", "epoch: 1, batch_id: 0, loss is: [0.29521096]\n",
"epoch: 1, batch_id: 500, loss is: [0.3284214]\n", "epoch: 1, batch_id: 500, loss is: [0.2916747]\n",
"[validation] accuracy/loss: 0.8573943376541138/0.3381407558917999\n" "[validation] accuracy/loss: 0.86475670337677/0.3259459137916565\n"
] ]
} }
], ],
"source": [ "source": [
"def train(model):\n", "def train(model):\n",
" # 这个函数是把模型设置为训练模式的。\n",
" model.train()\n", " model.train()\n",
"\n", "\n",
"\n", " opt = paddle.optimizer.Adam(learning_rate=0.001, parameters=model.parameters())\n",
" opt = paddle.optimizer.Adam(learning_rate=0.001, \n",
" beta1=0.9, \n",
" beta2=0.999, \n",
" epsilon=1e-07,\n",
" parameter_list=model.parameters())\n",
"\n",
"\n", "\n",
" for epoch in range(epoch_num):\n", " for epoch in range(epoch_num):\n",
" # shuffle data\n", " # shuffle data\n",
...@@ -308,8 +301,8 @@ ...@@ -308,8 +301,8 @@
" x_data = train_sents_shuffled[(batch_id * batch_size):((batch_id+1)*batch_size)]\n", " x_data = train_sents_shuffled[(batch_id * batch_size):((batch_id+1)*batch_size)]\n",
" y_data = train_labels_shuffled[(batch_id * batch_size):((batch_id+1)*batch_size)]\n", " y_data = train_labels_shuffled[(batch_id * batch_size):((batch_id+1)*batch_size)]\n",
" \n", " \n",
" sent = paddle.imperative.to_variable(x_data)\n", " sent = paddle.to_tensor(x_data)\n",
" label = paddle.imperative.to_variable(y_data)\n", " label = paddle.to_tensor(y_data)\n",
" \n", " \n",
" logits = model(sent)\n", " logits = model(sent)\n",
" loss = paddle.nn.functional.softmax_with_cross_entropy(logits, label)\n", " loss = paddle.nn.functional.softmax_with_cross_entropy(logits, label)\n",
...@@ -329,14 +322,12 @@ ...@@ -329,14 +322,12 @@
" x_data = test_sents[(batch_id * batch_size):((batch_id+1)*batch_size)]\n", " x_data = test_sents[(batch_id * batch_size):((batch_id+1)*batch_size)]\n",
" y_data = test_labels[(batch_id * batch_size):((batch_id+1)*batch_size)]\n", " y_data = test_labels[(batch_id * batch_size):((batch_id+1)*batch_size)]\n",
" \n", " \n",
" sent = paddle.imperative.to_variable(x_data)\n", " sent = paddle.to_tensor(x_data)\n",
" label = paddle.imperative.to_variable(y_data)\n", " label = paddle.to_tensor(y_data)\n",
"\n", "\n",
" logits = model(sent)\n", " logits = model(sent)\n",
" pred = paddle.nn.functional.softmax(logits)\n",
"\n",
" loss = paddle.nn.functional.softmax_with_cross_entropy(logits, label)\n", " loss = paddle.nn.functional.softmax_with_cross_entropy(logits, label)\n",
" acc = paddle.metric.accuracy(pred, label)\n", " acc = paddle.metric.accuracy(logits, label)\n",
" \n", " \n",
" accuracies.append(acc.numpy())\n", " accuracies.append(acc.numpy())\n",
" losses.append(loss.numpy())\n", " losses.append(loss.numpy())\n",
...@@ -378,18 +369,6 @@ ...@@ -378,18 +369,6 @@
"display_name": "Python 3", "display_name": "Python 3",
"language": "python", "language": "python",
"name": "python3" "name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.7"
} }
}, },
"nbformat": 4, "nbformat": 4,
......
paddle2.0_docs/n_gram_model/n_gram_model.ipynb
浏览文件 @ fcce28ed
...@@ -5,7 +5,7 @@ ...@@ -5,7 +5,7 @@
"metadata": {}, "metadata": {},
"source": [ "source": [
"\n", "\n",
"## 用N-Gram模型在莎士比亚文集中训练word embedding\n", "# 用N-Gram模型在莎士比亚文集中训练word embedding\n",
"N-gram 是计算机语言学和概率论范畴内的概念，是指给定的一段文本中N个项目的序列。\n", "N-gram 是计算机语言学和概率论范畴内的概念，是指给定的一段文本中N个项目的序列。\n",
"N=1 时 N-gram 又称为 unigram，N=2 称为 bigram，N=3 称为 trigram，以此类推。实际应用通常采用 bigram 和 trigram 进行计算。\n", "N=1 时 N-gram 又称为 unigram，N=2 称为 bigram，N=3 称为 trigram，以此类推。实际应用通常采用 bigram 和 trigram 进行计算。\n",
"本示例在莎士比亚文集上实现了trigram。" "本示例在莎士比亚文集上实现了trigram。"
...@@ -15,13 +15,13 @@ ...@@ -15,13 +15,13 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 环境\n", "## 环境\n",
"本教程基于paddle-develop编写，如果您的环境不是本版本，请先安装paddle-develop。" "本教程基于paddle-develop编写，如果您的环境不是本版本，请先安装paddle-develop。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 17, "execution_count": 23,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -30,7 +30,7 @@ ...@@ -30,7 +30,7 @@
"'0.0.0'" "'0.0.0'"
] ]
}, },
"execution_count": 17, "execution_count": 23,
"metadata": {}, "metadata": {},
"output_type": "execute_result" "output_type": "execute_result"
} }
...@@ -51,23 +51,23 @@ ...@@ -51,23 +51,23 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 18, "execution_count": 24,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"--2020-09-03 08:41:10-- https://ocw.mit.edu/ans7870/6/6.006/s08/lecturenotes/files/t8.shakespeare.txt\n", "--2020-09-09 14:58:26-- https://ocw.mit.edu/ans7870/6/6.006/s08/lecturenotes/files/t8.shakespeare.txt\n",
"正在解析主机 ocw.mit.edu (ocw.mit.edu)... 151.101.230.133\n", "正在解析主机 ocw.mit.edu (ocw.mit.edu)... 151.101.110.133\n",
"正在连接 ocw.mit.edu (ocw.mit.edu)|151.101.230.133|:443... 已连接。\n", "正在连接 ocw.mit.edu (ocw.mit.edu)|151.101.110.133|:443... 已连接。\n",
"已发出 HTTP 请求，正在等待回应... 200 OK\n", "已发出 HTTP 请求，正在等待回应... 200 OK\n",
"长度：5458199 (5.2M) [text/plain]\n", "长度：5458199 (5.2M) [text/plain]\n",
"正在保存至: “t8.shakespeare.txt.1”\n", "正在保存至: “t8.shakespeare.txt”\n",
"\n", "\n",
"t8.shakespeare.txt. 100%[===================>] 5.21M 26.1KB/s 用时 4m 14s \n", "t8.shakespeare.txt 100%[===================>] 5.21M 94.1KB/s 用时 70s \n",
"\n", "\n",
"2020-09-03 08:45:25 (21.0 KB/s) - 已保存 “t8.shakespeare.txt.1” [5458199/5458199])\n", "2020-09-09 14:59:38 (75.7 KB/s) - 已保存 “t8.shakespeare.txt” [5458199/5458199])\n",
"\n" "\n"
] ]
} }
...@@ -78,7 +78,7 @@ ...@@ -78,7 +78,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 19, "execution_count": 4,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
...@@ -88,7 +88,7 @@ ...@@ -88,7 +88,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 20, "execution_count": 5,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -113,12 +113,12 @@ ...@@ -113,12 +113,12 @@
"metadata": {}, "metadata": {},
"source": [ "source": [
"## 去除标点符号\n", "## 去除标点符号\n",
"用`string`库中的punctuation，完成英文符号的替换。" "因为标点符号本身无实际意义，用`string`库中的punctuation，完成英文符号的替换。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 21, "execution_count": 6,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -137,7 +137,7 @@ ...@@ -137,7 +137,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 22, "execution_count": 7,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -166,7 +166,7 @@ ...@@ -166,7 +166,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 23, "execution_count": 8,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -197,7 +197,7 @@ ...@@ -197,7 +197,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 24, "execution_count": 12,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
...@@ -219,7 +219,7 @@ ...@@ -219,7 +219,7 @@
" def __len__(self):\n", " def __len__(self):\n",
" return len(self.tuple_data)\n", " return len(self.tuple_data)\n",
"train_dataset = TrainDataset(trigram)\n", "train_dataset = TrainDataset(trigram)\n",
"train_loader = paddle.io.DataLoader(train_dataset,places=paddle.fluid.cpu_places(), return_list=True,\n", "train_loader = paddle.io.DataLoader(train_dataset,places=paddle.CPUPlace(), return_list=True,\n",
" shuffle=True, batch_size=batch_size, drop_last=True)" " shuffle=True, batch_size=batch_size, drop_last=True)"
] ]
}, },
...@@ -233,7 +233,7 @@ ...@@ -233,7 +233,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 25, "execution_count": 13,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
...@@ -243,7 +243,7 @@ ...@@ -243,7 +243,7 @@
"class NGramModel(paddle.nn.Layer):\n", "class NGramModel(paddle.nn.Layer):\n",
" def __init__(self, vocab_size, embedding_dim, context_size):\n", " def __init__(self, vocab_size, embedding_dim, context_size):\n",
" super(NGramModel, self).__init__()\n", " super(NGramModel, self).__init__()\n",
" self.embedding = paddle.nn.Embedding(size=[vocab_size, embedding_dim])\n", " self.embedding = paddle.nn.Embedding(num_embeddings=vocab_size, embedding_dim=embedding_dim)\n",
" self.linear1 = paddle.nn.Linear(context_size * embedding_dim, hidden_size)\n", " self.linear1 = paddle.nn.Linear(context_size * embedding_dim, hidden_size)\n",
" self.linear2 = paddle.nn.Linear(hidden_size, len(vocab))\n", " self.linear2 = paddle.nn.Linear(hidden_size, len(vocab))\n",
"\n", "\n",
...@@ -260,95 +260,38 @@ ...@@ -260,95 +260,38 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"### 定义`train()`函数，对模型进行训练。" "## 定义`train()`函数，对模型进行训练。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 28, "execution_count": 19,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"epoch: 0, batch_id: 0, loss is: [10.252256]\n", "epoch: 0, batch_id: 0, loss is: [10.252193]\n",
"epoch: 0, batch_id: 100, loss is: [7.0485706]\n", "epoch: 0, batch_id: 500, loss is: [6.894636]\n",
"epoch: 0, batch_id: 200, loss is: [7.282592]\n", "epoch: 0, batch_id: 1000, loss is: [6.849346]\n",
"epoch: 0, batch_id: 300, loss is: [6.9604626]\n", "epoch: 0, batch_id: 1500, loss is: [6.931605]\n",
"epoch: 0, batch_id: 400, loss is: [6.7308316]\n", "epoch: 0, batch_id: 2000, loss is: [6.6860313]\n",
"epoch: 0, batch_id: 500, loss is: [6.7940483]\n", "epoch: 0, batch_id: 2500, loss is: [6.2472367]\n",
"epoch: 0, batch_id: 600, loss is: [6.6574802]\n", "epoch: 0, batch_id: 3000, loss is: [6.8818874]\n",
"epoch: 0, batch_id: 700, loss is: [6.862562]\n", "epoch: 0, batch_id: 3500, loss is: [6.941615]\n",
"epoch: 0, batch_id: 800, loss is: [7.2091002]\n", "epoch: 1, batch_id: 0, loss is: [6.3628616]\n",
"epoch: 0, batch_id: 900, loss is: [7.0172606]\n", "epoch: 1, batch_id: 500, loss is: [6.2065206]\n",
"epoch: 0, batch_id: 1000, loss is: [6.9888105]\n", "epoch: 1, batch_id: 1000, loss is: [6.5334334]\n",
"epoch: 0, batch_id: 1100, loss is: [6.9609995]\n", "epoch: 1, batch_id: 1500, loss is: [6.5788]\n",
"epoch: 0, batch_id: 1200, loss is: [6.550024]\n", "epoch: 1, batch_id: 2000, loss is: [6.352103]\n",
"epoch: 0, batch_id: 1300, loss is: [6.714109]\n", "epoch: 1, batch_id: 2500, loss is: [6.6272373]\n",
"epoch: 0, batch_id: 1400, loss is: [6.995716]\n", "epoch: 1, batch_id: 3000, loss is: [6.801074]\n",
"epoch: 0, batch_id: 1500, loss is: [6.939434]\n", "epoch: 1, batch_id: 3500, loss is: [6.2274427]\n"
"epoch: 0, batch_id: 1600, loss is: [6.5966253]\n",
"epoch: 0, batch_id: 1700, loss is: [6.9880104]\n",
"epoch: 0, batch_id: 1800, loss is: [6.6459093]\n",
"epoch: 0, batch_id: 1900, loss is: [6.8095036]\n",
"epoch: 0, batch_id: 2000, loss is: [6.8447037]\n",
"epoch: 0, batch_id: 2100, loss is: [6.8313]\n",
"epoch: 0, batch_id: 2200, loss is: [6.808483]\n",
"epoch: 0, batch_id: 2300, loss is: [6.502908]\n",
"epoch: 0, batch_id: 2400, loss is: [6.561283]\n",
"epoch: 0, batch_id: 2500, loss is: [7.0093765]\n",
"epoch: 0, batch_id: 2600, loss is: [6.512396]\n",
"epoch: 0, batch_id: 2700, loss is: [6.809763]\n",
"epoch: 0, batch_id: 2800, loss is: [6.806659]\n",
"epoch: 0, batch_id: 2900, loss is: [6.95402]\n",
"epoch: 0, batch_id: 3000, loss is: [6.634927]\n",
"epoch: 0, batch_id: 3100, loss is: [6.644098]\n",
"epoch: 0, batch_id: 3200, loss is: [6.705504]\n",
"epoch: 0, batch_id: 3300, loss is: [6.2121572]\n",
"epoch: 0, batch_id: 3400, loss is: [6.638401]\n",
"epoch: 0, batch_id: 3500, loss is: [6.986831]\n",
"epoch: 1, batch_id: 0, loss is: [6.795429]\n",
"epoch: 1, batch_id: 100, loss is: [6.582568]\n",
"epoch: 1, batch_id: 200, loss is: [6.527663]\n",
"epoch: 1, batch_id: 300, loss is: [6.714637]\n",
"epoch: 1, batch_id: 400, loss is: [6.574902]\n",
"epoch: 1, batch_id: 500, loss is: [6.305031]\n",
"epoch: 1, batch_id: 600, loss is: [6.803609]\n",
"epoch: 1, batch_id: 700, loss is: [6.2429113]\n",
"epoch: 1, batch_id: 800, loss is: [6.7452283]\n",
"epoch: 1, batch_id: 900, loss is: [6.383783]\n",
"epoch: 1, batch_id: 1000, loss is: [6.4906135]\n",
"epoch: 1, batch_id: 1100, loss is: [6.6007314]\n",
"epoch: 1, batch_id: 1200, loss is: [6.63466]\n",
"epoch: 1, batch_id: 1300, loss is: [6.540749]\n",
"epoch: 1, batch_id: 1400, loss is: [6.7752547]\n",
"epoch: 1, batch_id: 1500, loss is: [6.2411666]\n",
"epoch: 1, batch_id: 1600, loss is: [6.540929]\n",
"epoch: 1, batch_id: 1700, loss is: [6.6563463]\n",
"epoch: 1, batch_id: 1800, loss is: [6.4592104]\n",
"epoch: 1, batch_id: 1900, loss is: [7.0268345]\n",
"epoch: 1, batch_id: 2000, loss is: [6.803793]\n",
"epoch: 1, batch_id: 2100, loss is: [6.8454733]\n",
"epoch: 1, batch_id: 2200, loss is: [6.651756]\n",
"epoch: 1, batch_id: 2300, loss is: [6.5876465]\n",
"epoch: 1, batch_id: 2400, loss is: [6.258934]\n",
"epoch: 1, batch_id: 2500, loss is: [6.5422425]\n",
"epoch: 1, batch_id: 2600, loss is: [6.184501]\n",
"epoch: 1, batch_id: 2700, loss is: [6.6847773]\n",
"epoch: 1, batch_id: 2800, loss is: [6.684101]\n",
"epoch: 1, batch_id: 2900, loss is: [6.374978]\n",
"epoch: 1, batch_id: 3000, loss is: [6.8277273]\n",
"epoch: 1, batch_id: 3100, loss is: [6.5195084]\n",
"epoch: 1, batch_id: 3200, loss is: [6.311832]\n",
"epoch: 1, batch_id: 3300, loss is: [6.4282994]\n",
"epoch: 1, batch_id: 3400, loss is: [6.603338]\n",
"epoch: 1, batch_id: 3500, loss is: [6.4541807]\n"
] ]
} }
], ],
"source": [ "source": [
"import time\n",
"vocab_size = len(vocab)\n", "vocab_size = len(vocab)\n",
"epochs = 2\n", "epochs = 2\n",
"losses = []\n", "losses = []\n",
...@@ -360,11 +303,11 @@ ...@@ -360,11 +303,11 @@
" x_data = data[0]\n", " x_data = data[0]\n",
" y_data = data[1]\n", " y_data = data[1]\n",
" predicts = model(x_data)\n", " predicts = model(x_data)\n",
" y_data = paddle.nn.functional.one_hot(y_data, len(vocab))\n", " y_data = paddle.reshape(y_data, ([-1, 1]))\n",
" loss = paddle.nn.functional.softmax_with_cross_entropy(predicts, y_data,soft_label=True)\n", " loss = paddle.nn.functional.softmax_with_cross_entropy(predicts, y_data)\n",
" avg_loss = paddle.mean(loss)\n", " avg_loss = paddle.mean(loss)\n",
" avg_loss.backward()\n", " avg_loss.backward()\n",
" if batch_id % 100 == 0:\n", " if batch_id % 500 == 0:\n",
" losses.append(avg_loss.numpy())\n", " losses.append(avg_loss.numpy())\n",
" print(\"epoch: {}, batch_id: {}, loss is: {}\".format(epoch, batch_id, avg_loss.numpy())) \n", " print(\"epoch: {}, batch_id: {}, loss is: {}\".format(epoch, batch_id, avg_loss.numpy())) \n",
" optim.minimize(avg_loss)\n", " optim.minimize(avg_loss)\n",
...@@ -383,22 +326,22 @@ ...@@ -383,22 +326,22 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 29, "execution_count": 20,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"data": { "data": {
"text/plain": [ "text/plain": [
"[<matplotlib.lines.Line2D at 0x166d69048>]" "[<matplotlib.lines.Line2D at 0x14e27b3c8>]"
] ]
}, },
"execution_count": 29, "execution_count": 20,
"metadata": {}, "metadata": {},
"output_type": "execute_result" "output_type": "execute_result"
}, },
{ {
"data": { "data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXzcVbn48c8zM1mapVknSdukTbM03egaSlvaUihbkQuigOBVEAVE8bpdrxdc4Kf+firqVVG8cMviBQVFkE1k3wtdaLo3NGmbdEnSZm32NOuc3x+zdGYyaZaZNGH6vF+vvDrznW/me5pknjnznOecI8YYlFJKhS/LWDdAKaXU6NJAr5RSYU4DvVJKhTkN9EopFeY00CulVJizjXUD/KWmpprs7OyxboZSSn2sbN26td4YYw/02LgL9NnZ2RQVFY11M5RS6mNFRA4P9JimbpRSKsxpoFdKqTCngV4ppcKcBnqllApzGuiVUirMaaBXSqkwp4FeKaXCXNgE+mPNJ/j1a6WU17WNdVOUUmpcCZtAX9faxe/eOkB5XftYN0UppcaVsAn0kTbnf6W7zzHGLVFKqfElfAK91RXoezXQK6WUt/AJ9DYN9EopFUjYBfouTd0opZSPsAn0UVYroD16pZTyFzaB3tOj7+0b45YopdT4EnaBXnv0SinlK2wCvdUiWC2igV4ppfwMGuhF5BERqRWRPV7HkkXkdRHZ7/o3aYDvvdF1zn4RuTGUDQ8k0mrRQK+UUn6G0qP/X+BSv2N3AG8aY/KBN133fYhIMnA3cA6wBLh7oDeEUIm0WXTClFJK+Rk00Btj3gOO+x2+EnjUdftR4JMBvvUS4HVjzHFjTCPwOv3fMEIqyqY9eqWU8jfSHH26MeaY63Y1kB7gnClAhdf9StexUROpgV4ppfoJejDWGGMAE8xziMitIlIkIkV1dXUjfp5Im0UnTCmllJ+RBvoaEZkE4Pq3NsA5VUCW1/1M17F+jDHrjDGFxphCu90+wibpYKxSSgUy0kD/AuCuorkReD7AOa8CF4tIkmsQ9mLXsVGjOXqllOpvKOWVfwE2AgUiUikiXwJ+DlwkIvuBC133EZFCEXkIwBhzHPgJsMX19WPXsVGjOXqllOrPNtgJxpjrB3hoTYBzi4Cbve4/Ajwy4tYNU6TNQmePBnqllPIWNjNjQXP0SikVSHgFek3dKKVUP2EW6K06M1YppfyEV6DX1I1SSvUTXoHeZqFLA71SSvkIq0DvrKPXjUeUUspbWAV6Xb1SKaX6C69A78rRO5ffUUopBeEW6G0WHAZ6HRrolVLKLewCPei+sUop5S28Ar1VA71SSvkLr0Dv7tHrgKxSSnmEZ6DXHr1SSnmEVaCPcgV6nTSllFInhWWg1x69UkqdFFaBXnP0SinVX1CBXkS+ISJ7RKRYRL4Z4PHVItIsIjtcX3cFc73BRFqtgPbolVLK26A7TA1EROYCtwBLgG7gFRF50RhzwO/U9caYy4No45DpYKxSSvUXTI9+FrDZGNNhjOkF3gU+FZpmjczJ1I0ubKaUUm7BBPo9wEoRSRGRGOAyICvAectEZKeIvCwicwI9kYjcKiJFIlJUV1c34gbphCmllOpvxKkbY8xeEbkHeA1oB3YA/l3pbcA0Y0ybiFwGPAfkB3iudcA6gMLCwhEvVBOp5ZVKKdVPUIOxxpiHjTGLjTGrgEZgn9/jLcaYNtftl4AIEUkN5pqnouWVSinVX7BVN2muf6fizM8/4fd4hoiI6/YS1/UagrnmqWh5pVJK9Tfi1I3L30UkBegBbjfGNInIbQDGmAeAq4GviEgvcAK4zoziYvGao1dKqf6CCvTGmJUBjj3gdfs+4L5grjEcWl6plFL9hefMWA30SinlEVaB3mYRRDRHr5RS3sIq0IuIZ99YpZRSTmEV6MGZvtE6eqWUOinsAn2UBnqllPIRdoFeUzdKKeUr/AK9zaKDsUop5SXsAn2UzUp3r65eqZRSbmEX6CNtmrpRSilv4RnoNXWjlFIe4RfodTBWKaV8hF+g19SNUkr5CMtAr3X0Sil1UlgGes3RK6XUSWEX6KM0R6+UUj6C3WHqGyKyR0SKReSbAR4XEfmdiBwQkV0isiiY6w2F5uiVUsrXiAO9iMwFbgGWAPOBy0Ukz++0tTg3A88HbgXuH+n1hkpTN0op5SuYHv0sYLMxpsMY0wu8i3PfWG9XAo8Zp01AoohMCuKag9LySqWU8hVMoN8DrBSRFBGJAS4DsvzOmQJUeN2vdB3zISK3ikiRiBTV1dUF0SRN3SillL8RB3pjzF7gHuA14BVgBzCiRWaMMeuMMYXGmEK73T7SJgHOQN/rMDgco7YHuVJKfawENRhrjHnYGLPYGLMKaAT2+Z1ShW8vP9N1bNR49o3VPL1SSgHBV92kuf6dijM//4TfKS8AN7iqb5YCzcaYY8FcczCRVud/SSdNKaWUky3I7/+7iKQAPcDtxpgmEbkNwBjzAPASztz9AaADuCnI6w0qyt2j10CvlFJAkIHeGLMywLEHvG4b4PZgrjFc7tRNl65Jr5RSQBjOjI3UHr1SSvkIv0BvtQI6GKuUUm7hF+i1R6+UUj7CLtDrYKxSSvkKu0CvPXqllPIVtoG+S3P0SikFhGOgt2qPXimlvIVdoNccvVJK+Qq7QK85eqWU8hW+gV5z9EopBYRjoNccvVJK+Qi/QK+pG6WU8hG+gV5TN0opBYRjoNf16JVSykfYBXoR0Q3ClVLKS7A7TH1LRIpFZI+I/EVEov0e/4KI1InIDtfXzcE1d2h0g3CllDppxIFeRKYAXwcKjTFzAStwXYBTnzTGLHB9PTTS6w1HpM1Cd59uPKKUUhB86sYGTBARGxADHA2+ScHT1I1SSp004kBvjKkCfgUcAY7h3Pj7tQCnflpEdonI0yKSFei5RORWESkSkaK6urqRNslDUzdKKXVSMKmbJOBKYDowGYgVkc/5nfYPINsYMw94HXg00HMZY9YZYwqNMYV2u32kTfKItFm06kYppVyCSd1cCBw0xtQZY3qAZ4Dl3icYYxqMMV2uuw8Bi4O43pBp6kYppU4KJtAfAZaKSIyICLAG2Ot9gohM8rp7hf/jo8U5GKuBXimlwDmYOiLGmM0i8jSwDegFtgPrROTHQJEx5gXg6yJyhevx48AXgm/y4DR1o5RSJ4040AMYY+4G7vY7fJfX43cCdwZzjZGIsllo7ew93ZdVSqlxKexmxoIz0GuOXimlnMIy0GuOXimlTgrPQK9VN0op5RGegV5TN0op5RG+gV5TN0opBYRroLdatUevlFIu4RnoNXWjlFIe4Rvo+xwYY8a6KUopNebCMtBH6b6xSinlEZaB3r1vrKZvlFIqXAO9TQO9Ukq5hXeg19SNUkqFaaDX1I1SSnmEZ6DX1I1SSnmEdaDXNemVUkoDvVJKhb2gAr2IfEtEikVkj4j8RUSi/R6PEpEnReSAiGwWkexgrjdUUZqjV0opjxEHehGZAnwdKDTGzAWswHV+p30JaDTG5AG/Ae4Z6fWGQ6tulFLqpGBTNzZggojYgBjgqN/jVwKPum4/DaxxbSQ+qqJsVkB79EopBUEEemNMFfAr4AhwDGg2xrzmd9oUoMJ1fi/QDKT4P5eI3CoiRSJSVFdXN9ImeWjVjVJKnRRM6iYJZ499OjAZiBWRz43kuYwx64wxhcaYQrvdPtImeZxM3fQF/VxKKfVxF0zq5kLgoDGmzhjTAzwDLPc7pwrIAnCldxKAhiCuOSTao1dKqZOCCfRHgKUiEuPKu68B9vqd8wJwo+v21cBb5jSsHawzY5VS6qRgcvSbcQ6wbgN2u55rnYj8WESucJ32MJAiIgeAbwN3BNneIdE6eqWUOskWzDcbY+4G7vY7fJfX453ANcFcYyR0PXqllDopPGfGaupGKaU8wjLQWyyCzSIa6JVSijAN9KAbhCullFt4B3rN0SulVBgHeqv26JVSCsI50GvqRimlgDAP9F2aulFKqTAO9Jq6UUopIIwDfZSmbpRSCgjjQB9ps9DVq6tXKqVUWAd67dErpVQ4B3qr1tErpRSEc6DXHr1SSgFhHOijbFYN9EopRRgHeu3RK6WUUzB7xhaIyA6vrxYR+abfOatFpNnrnLsGer5Q07VulFLKacQbjxhjSoEFACJixbk/7LMBTl1vjLl8pNcZqUirRXeYUkopQpe6WQOUGWMOh+j5gqYTppRSyilUgf464C8DPLZMRHaKyMsiMifQCSJyq4gUiUhRXV1dSBrkTt2chr3IlVJqXAs60ItIJHAF8FSAh7cB04wx84HfA88Feg5jzDpjTKExptButwfbJMCZujEGeh0a6JVSZ7ZQ9OjXAtuMMTX+DxhjWowxba7bLwERIpIagmsOKtKm+8YqpRSEJtBfzwBpGxHJEBFx3V7iul5DCK45KA30SinlNOKqGwARiQUuAr7sdew2AGPMA8DVwFdEpBc4AVxnTlPS3BPotcRSKXWGCyrQG2PagRS/Yw943b4PuC+Ya4xUpFV79EopBWE+MxbQWnql1BkvbAN9lObolVIKCONArzl6pZRyCt9Ab7UCvj36zp4+mjt6xqpJSik1JsI30AdI3dzx911c9d8f6GxZpdQZJewDvXvf2Mb2bl7aXU15fTvl9e1j2TSllDqtwjfQ+5VXvrDzqCdfv35faNbTUUqpj4PwDfR+g7FPb61k9qSJZKfEsH5//ahe+/HNh9l6+PioXkMppYYqbAN9lFcdfUl1C7urmrmmMJOV+XY2ljeMWtllW1cvdz1fzB1/341DF1RTSo0DYR/ou3sdPF1USYRVuHLBFFbmp9LR3ce2I42jct0tB4/T5zDsr23jnX21o3INpZQajrAN9O7UzYnuPp7bUcWamekkx0ayLDcFq0VYv3908vQbyuqJtFqYlBDNA++Wj8o1lFJqOMI+0L/2UTX1bd1cvTgTgPjoCBZNTRy1PP3G8gYWTE3k5pU5fHjwONtH6ZODUkoNVfgGelfVzZZDjaTGRXFewckNTVbm29ld1czx9u6QXrO5o4fioy0sy0nhurOzSJgQwbr3tFevlBpbYRvobVYLFnHevmrhZCKsJ/+rK/NTMQbePxDaXv3mgw0YA8tzU4iNsvG5pVN5pbiag1q3r5QaQ2Eb6OFk+uaawiyf4/MyE0mYEBHyevqN5Q1E2SwsmJoIwI3Ls4mwWnhwvfbqlVJjZ8SBXkQKRGSH11eLiHzT7xwRkd+JyAER2SUii4Jv8tBFWi3Mz0xgRnq8z3GrRViRl8r6/fUhXQ5hY1kDhdlJRNmc6+ykxUfz6UVTeHprJXWtXSG7jlJKDceIA70xptQYs8AYswBYDHQAz/qdthbId33dCtw/0uuNxDcvnMH3LpsV8LGV+alUt3RyoLYtJNc63t5NSXUry3J89mHh5pU59PQ5WPdemdbVK+VSfLSZ7zy1U5cRP01ClbpZA5QZYw77Hb8SeMw4bQISRWRSiK45qC+umM45foHXbUW+c4/y90JUfbOp3LkV7rJc3+vl2uO4bO4kHlx/kOU/f4u7n9/DhrJ6ekOwfPLB+nb6PoZvHj19Dq5bt5F3SnWewZnqn7uO8fTWSt7YWzPWTTkjhCrQX0fgDcKnABVe9ytdx3yIyK0iUiQiRXV1p2cdmsykGHLssSGrp99Y1kBMpJV5mYn9Hvuva+fzm8/MZ35WAn/dUsFnH9zMkp++yXef3skbH9XQ2dM37OttP9LI+b96h0/dv4GS6pYhfc++mlaueWADjSGuNhqufTWtbCo/zmsfjf8XefOJHm585EP21bSOdVPCSlmd85P0Xz48MsYtOTMEHehFJBK4AnhqpM9hjFlnjCk0xhTa7fbBvyFEVuXb2VTeMKJA629jeQNnZyf7VPe4RUdYuWphJv/z+UK233UR9//rIlbkpfLy7mpufqyIRT95ndsf30Zta+eQr/fynmoirELl8Q4u/937/PLVkkH/Hy/tPsaWQ428N0qTxYaquMr5xlRybGhvUAOpa+2io7s3FE0a0Hv76nh3Xx2/fWPfqF7nTFNe144IrN9fT8XxjrFuTtgLRY9+LbDNGBOoe1YFeJe8ZLqOjQurC+x09jiCTiHUtjpz/f5pm0BiIm2sPWsSv7t+IVt/eBGPfnEJVy2cwivF1Tzy/qEhXc8Yw6vF1SzPTeWNb5/HlQum8Ie3y1h773r2niJ4bjvSBMCGAw1Dus5o2XO0GYB9NW0jHgw/3t7Nxb95lx+98FEom9bPRldK7uU91RzSMtmQ6O1zcKihnSvmT8Yi8OSWisG/SQUlFIH+egKnbQBeAG5wVd8sBZqNMcdCcM2QWJGXSvrEKP5WVBnU82wqd65U6T8QO5hIm4XzZtj5f1edxZzJE9lRMbRZtPtq2jjc0MHFc9JJio3kv66dz5+/dA5NHd38+vXAPU+Hw7D9sPP5N5SP7uqdg9ld5Qz0bV29VDaeGNFz/PLVEho7enizpHZUN5LZVNbA/KxELZMNoYrGE/T0GVbkpbK6II2ntlaEZMxKDSyoQC8iscBFwDNex24Tkdtcd18CyoEDwIPAV4O5XqjZrBY+vSiTd0prqWkZetrE38ayBuKjbMyZPHHEz7EgK5Hdlc1DGlx9rbgaEbhoVrrn2Ir8VC6Zk8Gm8oaAz7G/to3Wrl7mZyZQcfzEmH1c7u1zsPdYCwtdcw1Kqoef+95R0cRft1QwPTWW+rYu9h4bnfx5TUsn5fXtfOKsDD69KJOntEw2JMpclW65aXFcd3YWNS1dvF2qe0SMpqACvTGm3RiTYoxp9jr2gDHmAddtY4y53RiTa4w5yxhTFGyDQ+3awiwcxrle/UhtKm9gyfRkbAHy80O1ICuR9u4+9tcOHrRe+6iGhVmJpE2M9jm+LDeF1s5ePjraP32z1dWbv/38PMD55jQWyura6exx8KlFzrWHSoc4kOzW5zDc9fweUuOieOjGQoBRG3PwVFLlpHLLyun09Dl4bOOhUbnWmcQ9EJubGscFM9NIi4/SQdlRFtYzY4ciOzWWJdOTeaqoYkQpgKNNJzhY3z6k/PypLMhy9nB3uPLop7re7qpmLp6T0e8xdxs2lPVPzWw70khybCQXzkonNS4q4Dmnwx5X2mbp9GSykiewd5g9+ie3VLCrspnvXzaLXHscMzPieW+UdgzbWNbAxGgbsydPJMcex8Wz03ls42Hau0Z3ADjclde1kxoXSUJMBDarhWsKnZ+qjzWPLI2nBnfGB3qAzxRmcaihgy2Hhr/S5N+KKhCBC73SKCMxPTWWhAkR7Kg4daB/3VWSePHs/tdLi48mPy2ODQF669sON7JoahIWi7A8N4UNZQ2nfGNzOAzv76/n9se3seKet7h+3Sa+/+xuHlpfztultTSf6Bnm/9Bpd1UzEyKs5NjjKEifSOkwAn1jeze/eLWEJdOTuXLBZABWzbBTdKhxVKpvNpY3sGS6c1lrgC+fl0vziR4dPAxSWV0bOfY4z/3PFE7FYeBvW5yfqrt7HfxtSwVr713PbwYYcxprH7cxBdtYN2A8WHtWBne/UMyTWypYMj15yN/X3evgz5uOcH5BGtmpsUG1QUSYn5U4aKB/tbiavLQ4nxeKt+W5KTy1tZLuXodnrZ/j7d2U17dzdWGm55wXdh6lrK6NvDTf5SEa27v5y5Yj/PXDCo4c7yAxJoJzc1M52nyCF3cd8wR4i8DcKQksy03h3NxUluemDCl1VXy0mdmTJ2K1CLMmxfN2aS2dPX1ER1h9zuvpc1ByrBWbVYiOsBIdYeG3r++ntbOXn1w5FxFn8F2Vb2fde+VsKm/ggpnBvdl6O9p0gsMNHdywLNtzbNHUJJZkJ/Pw+wf5/LJpAUtp1eDK6tq4dO7JeZNTU2JYkZfK34oqiIu28dD6co41dxIbaeV/3ivjxuXZJMdGjmGLfe2ubObTD2zgH19bQUFG/ODfMA7oXyrOksd/mT+Jl3Yfo7XTt6f64cHjnnSDv5d2H6O+rYsvLM8OSTsWZCWyr6Z1wNRAU0c3mw8eD9ibd1uW69xBa1flyTcM95r4i6cmAbA81zkr2L/n3+cwXP/gJn7xSimTEqK597oFbLpzDX/410U8+9Vz2XHXRWz9wYU8ccs5fO2CfKJsFh5ef5AbHvmQn71cMuj/z+EwFB9tYa5r0LogI54+hwm4DMWD68v5l/veZ+296zn/V++w7Gdv8WRRBTcuy/Z5cRVmJxEdYeG9faFNRbnHMJbm+L7xf/m8HKqaTvDirqMhvZ6/po5urrjv/VOWy34cHW/vprGjh1y7b8fo+iVTqWo6wU9e/IispBj+eNPZPHv7uXT2OPjTRv8J92NrQ1k93b0OXt4zbgoIB6U9epdrC7P4y4cV/HPXMa5bMpXOnj5+9tJeHt14mKSYCN749nmkxEX5fM8fNxwi1x7LStdyCsFamJWIw8CuyuaAOf+3SmrpcxguCZCfd1uak4yIM4gXZjuD1NbDjdgs4pm1m5U8gSmJE9hwoMGnx/r3bZWUVLdy73ULuHJBvwnMiAgpcVEsj4tyvllcNIP2rl6+9sQ2Xtx1lB98Ypanpx1IeX07Hd19zJ2SAMDMDGfAL61u9Rxze7W4hpkZ8XxjTT6dvX109jiIsFq4fJ7vChrREVbOmZ4S8gHZjeUNJMZEMCvDt5Lq/II08tLi+OMHh7hqYWZIr+lt25FGdlU2s35/HbMmjbyaKxRe/6iGyYnRzJmcMPjJg/AMxKb5fiK9ZE46d6ydSeG0JM/fLcD5BXYe23iIL5+X0+9T31jZ4yp2eKe0jm9eOGOMWzM02qN3WZCVSH5aHE8WVVBa3cqV933AoxsPc83iTNq6evnRP3wn5mw/0sjOiia+sDz7lMFtOOa7B2QHSN+8VlxDxsRozpoy8AsuMSaS2ZMm+gy2bj3cyJzJE5kQ6XyhiAjn5qWwsbzBs9BaZ08fv3l9H/OzErli/uQhtzk2ysbauZOoaRm8zLHYNVHKHdSzU2KItFn6LeFQ19rFzoomLp83ibVnTeKqhZlcv2QqVy/ODPhiXzXDTnldO5WNoSsZ3VTewDnTk7FYfH+3Fotww7Jp7KpsZucgabZguMtOT7XonjFm1BfKaz7Rw+1PbOM3r+8PyfOVe1XceLNZLdx2Xq5PkAe4dVUuDe3d/H1bcHNdQqnY9Ql/Z2VTyDcvGi0a6F1EhGsLs9h+pIl/+f37NLR38783nc0vr5nP7efn8cLOo7zptQDTHz84RHyUzVMmGArJsZFMS4kJOHGqs6ePd/fVcdHs9H7Bx9/y3BS2HWmis6ePnj4HOyubWOhK25w8J5XmEz185EoNPLrhEMeaO7nj0pnDfuNy79719iAzjHdXNhNps5Dn6s3ZrBZmpMf1q6V3P8/5M9OGdv0ZrgXqQpS+qTjeQWXjiQEnwF21cAoxkVb+tGn0UgolrjfNsrqBZ+P+/q0DrL13/ahOGHtx11G6ex0cGELZ71CU1bUTabMwJWnCkM5fmpPMvMwEHlp/cFys/trW1cvBhnbWzEzDGEZt7+lQ00Dv5apFU5gYbePcvBRe+eZKVhc4A81XV+cxIz2OHzy3h9bOHmpaOnlp9zGuPTuL2KjQZr8WDDAgu35/PSd6+rh4zuADjstzU+nudbDtcCMlx1rp7HGweJpvoHenhjaWNdDc0cMf3j7A6gL7iMpE0ydGM2fyxEGXkthztJlZkyb6DGIWpE/sF+jf2ltLxsRoZg8xZZFrj2NyQvSQyixPdPcNGjA2elYiDZySi4+O4KqFU/jHzqNDWiDOGMOfNh0e1k5jpV49+oEC+YayekprWjkyipPfnnLNGj98vCMka0KV1baRkxrrqWQajIhwy8ocDta38/o4WOly77EWjHGOKaTERvJ2ycdjBVYN9F5S46LY8oML+eNNS0j1ysdH2izc8+l5VLd0cs8rJTy+6TB9xnDDsmkhb8OCrERqWrp8aoqNMTz4XjmpcVGcM33wQHz29GSsFmFDWQNbDzuXZ/AP9OkTo8m1x7KhrJ7/fvcArV29fPeSmSNu9/kFaWw70kRzR+CyS4fDUFx1ciDWbWZGPHWtXZ6PwF29fazfX8cFs9KG/MlCRFiZb+eDUyz/3Ocw3P9OGfN+9Cp/HaQ8clNZA8mxkeSnBa5sAvjc0ml09TqGNNHugwMN/PC5PXzlz1vpGUJZXnevg7K6NuKjbTSf6AmYHjDGeN4MNh88PuhzjsSB2lZ2VDSxcGoixpzMrwejrK6N3AEqxgaydm4GmUkTeHCU9l+ub+viUH07Rxo6qDjeQVXTiQFnqLsLM+ZlJrBqhp339tePi08ag9FA78e9O5S/hVOTuGn5dP686QiPfHCINTPTmJYSXEllIO6JU97537dLa/nw0HG+sSbPUzJ5KnFRNuZnJrChrJ6tR5rImBjN5MT+H5WX56ayqfw4//vBIa5aMIXZQSzhcP5MO30Ow/oDgXvVR4530NrV22/QdeYkZwWNO0//4cHjtHf3sWaIaRu3VTPstHb2srOy/6ehIw0dXLduI/e8UkKvw7D54MCzgo0xbCxvYGlO//y8t1mTJnJ2dhJ/3nx40Bf6f79zgAkRVkqqW3lo/cFB/y9ldW30OgwXz3YOugfK09e1dtHoelPdXD46gf7prVVYLcJ/XFwwYDuGo6u3jyPHO8ixD+91Y7NauHnFdIoON3pmeIdKdXMnS3/6Jqt/9Q6rfvk2K3/xNuf+/C1+8NzugOfvqWrBHh9F2sRoVhfYOd7eza4BqvLGEw30w/CdS2aQmTSBtq5evrB8+qhcY/bkiURaLWx3Bfo+h+EXr5SSnRLDdUumDvl5luemsrOymY1lDf168yfPSeFETx/GwLcuCq56YEFWEokxEbxdEjjQu1es9B9IdpdKunPSb+6tJcpm8ZSADtWKvFQsAu+68vTGGI63d/PXD4+w9t73KDnWym8+M5/VM+yeawVyuKGDY82dQ1qg7nNLp3G4oYP1p9hkfkdFExvKGvjWRflcMiede9/cx+GGU6dw3D31T8xzBvpAeXp3uis1LooPD4V+OYvePgfPbKvk/AI7hdnOT4j7a4YW6Dt7+vjes7spOuT7BnSkoQOHYdg9enDu+5wwIYL73tof0s12dlU20eswfLhCvFcAABhRSURBVOfiGfzXNfP55dXzWJaTwqvFNQHfwIuPNns+la7MtyPCx2IDHQ30wxATaeOBzy3m2xfN4Ny84JY8GEiUzcqsyRM9SyE8t72KkupWvnNJwbAm6CzPTaHPYahv62LRAIF+aU4KkTYLNyybRlZyTFDttlqEVfl23t1XG/AFsruqmQirkJ/u+yK3x0WREhtJaXUrxhjeKqnl3LxUT4XQUCXERDA/K5E/fnCQFfe8RcEPXmHRT17njmd2c1ZmAq98axVXLcxk5qSJlNW1DbiF3UA7hQVy6dwMUuMi+dPGQwOec/87B0iYEMFnz5nGj66Yi81i4QfP7TnlAGpJdSsRVuHcvFSiIywBUybuN4PPLsmi4vgJjjYNf/mAutYu/ufdMu58ZhdtfnM31h+op7a1i6sXZxJps5CdEjPkzVd+/OJHPLH5CD/6x0c+/09PaeUIAn1slI3bzsvl7dI6PvvgppAtl+D+Od507nQ+vTiTawqzuG5JVsCeemdPH/tr2zyfSpNjI1mQlfixWJBNA/0wzZ2SwNfX5IespDKQhVmJ7K5qpqO7l1+/vo+zpiRw2dzh7cC4aFqSJ80zUI8+KTaSN799HnesHXlu3tv5M+3Ut3V7eu/eiqtaKMiI75caExEKMuIpqW6hrK6dI8c7uGCYaRu3W1bmcNaUBAqnJXHTimzuunw2D99YyBM3L2WKK3U1MyOeXocZMN+8/UgTCRMihhSMomxWPnN2Fm+W1AZcDfRAbSuvFtdw47JpxEXZyEiI5ruXFrB+fz3P7Rh4W4aS6hZy7XFE2azkpMYFbGtJdStp8VFcMtfZ6/9wiHn63j4Hb3xUwy2PFbH0Z2/ys5dL+OuWCm59rMhnsPXprZUkxUR4ZhvPSI8fUurm2e2VPLH5CDMz4tld1ezTLvcnk+GmbtxuOy+HX10zn91Vzay9dz2vFVeP6Hm8ldS0MjU5xqeoYqCeekl1K30O47NK7eoZaeyqbKKhbfirmo50GZGR0EA/Ds3PSqCju4+7ni+mqukEd6ydOWhJpb/oCCuLpyYRZbOcsnolKzkmqFU3va1yvUD80zfGGPYcbWbuABNuZmZMZF9Nm2cdn5EG+svOmsQTtyzlt9ct5M61s/jiiumsmeVbjuqefDTQ9os7K5uYl5kw5Dfy65dMRQi8Jd7975QzIcLKF849meb713OmsXBqIj95ce+AFTul1a3MdKW08tLiAgbY0hrnG+fMjInER9tOOe7g1t3r4LMPbebmx4rYfqSJm1dM541vn8evr53PhrIGvvbEdnr6HDR1dPN6cQ1XLpji6Szkp8VxqKGdrt6BK2/21bTyvWf2sGR6Mk9/ZTlJMRE86DUmUVbbxqSE6BFXqokIVy/O5MV/W8GUxAnc+qet3P38nqAGQ0uOtfRbxiA5NpL5mYm849dTdw/Eek8cW11gd5VZDq+0d09VM2f/3zd4+P3Bx2xCQQP9OLQgy9kDf3prJSvzUzk3b2Qzb7910Qx+fOWcIQ3ghkJKXBTzMxP71dNXNp6gqaOHOQNM9JqZEc+Jnj7+vOkwsyZNDDhwHCrTU2OJtFoC5ulPdDs/ms8PsO/vQDKTYlgzK52H3j/IH94+4EkJVTWd4PkdVVy3JMtnnRarRfjZp86i5UQPv3i1/7IRzR09HGvuZKbrDSnXHkdV0wlOdJ8MsH0Ow/6aNmZmxGO1CEuyk4dUefOjfxTz4cHj/OSTc9l45wXcedks8tLiuGphJj+5cg5v7K3hP57ayfM7jtLd5+DqxSfniOSlx+MwDFgi2t7Vy1f+vJXYKBv3Xb+QuCgbn186jTdLajyTpMrq20fcm/eWY4/jma8u54Zl03h04+ERz4ru7OnjUEOH503V2+oCe78JUcVHm0mYEEGm1xyAs6YkkBIbOew8/W/f2Ed3n4Nfv1Ya1F4YQxXsxiOJIvK0iJSIyF4RWeb3+GoRaRaRHa6vu4Jr7pkhOyWGxJgIAP7z0pGnVZZMT+YzZw99ADcU3C8Q90fZ2tZO/s8LxQAsGCCAuntUVU0nhl1tM1wRVueErUDLIxcfdW78Mi9zeFP9f3rVWayZmcYvXy3lE79bz4cHj3tKAW9emdPv/JkZE7l6cSbP7zjarza91JUHd/9MctNiMX4B1tmzdlDgWp5hyfRkyuvaT7nn8F8/PMLjm49w23m5fH5p/wXZPr8sm/+4pIDndhzl//1zL7MmTfSpkHKXmu4LMCBrjOGOZ3ZzsL6d31+/0LNPwueWTSPCYuGRDw5ijKG8dvillQOJsln53mWziLRZTtmbbuvqHbDk90BtG30OE3BhstUF/SdE7alqYe6UiT6f9iwWYdUMO+/uqxvyIPGuyibe2FvLtYWZ9PQZfj6EdaKCFWxX717gFWPMTGA+sDfAOeuNMQtcXz8O8npnBBHhM2dnccvK6f3KEce7810vkHf31fHs9kou+vV7rD9Qzw8+MYuzBgigM9Ljcb92Lpg1uoEenJ8gAm14srPS+dHcvRTFUNnjo7j/c4t5+MZCOrr7uPZ/NvLnTYf55MIpnrEBf5fPm0xHt3O2szd3Ssndy3QHRu88vXsA0X3OOa4KoS0HA5cebjvSyF3PF7MyP5X/uKRgwP/H7efncdt5uXT3Obhmse+M7xx7LBaBAwEGZN/dV8c/dh7l3y8u8BnETouP5pMLJ/P01kr21Th3OAtVoAdnenJJdjLvnyLQf+2JbXzx0S0BH/P/OXqbNyWB5NhIT/qmu9fhXJMpQPpxdYGdxo4en4UET+W3b+wnMSaCH14+m1tWTefZ7VVsOTQ6JbJuIw70IpIArAIeBjDGdBtjRm/xjzPMnWtn8f1PzB7rZgyb+6PsXc8X860nd5KXFsfL31gZsGfrNiHSSnZKLCmu3OhomzkpnpqWrn4TkXZVNpE+MYp0v527hmrNrHRe//Yqvrwqh7T4KL66OnfAc8/JSSYxJoKXd/uugFhS3UrChAgyXG2YnhqLiG8Ne0l1KxbBs5TEnMkTiYm0BszT17Z0ctuftpKREM3vr1846IzU/7y0gL9/ZTk3+q3IGmVz/o72BxgveO2jGmIjrdwS4Hf8pRU5dPY4+MmLzrWiQhnoAc7NS6W0pjXgp5mmjm7W769n+5HGgAOfpTWtroqi/ukki0VYlZ/Ke/vqcDgM+2tb6e5zBEw/rsq3YxHnz2EwOyqaeKuklltW5hAfHcHt5+cxKSGau58vDmnZqL9gevTTgTrgjyKyXUQecu0h62+ZiOwUkZdFZE4Q11MfAxaLcOncDHr6HPzgE7P425eXDenF/ZXVuXz30oIhT40PhnvVTP8B2d2VzZ4VPkcqJtLGnZfNYsOdawbcMwCcKaSLZ6fz5t5anwHO0upWCjLiPemB6AgrWUkxfj36FrJTYz0LvEVYLSyeltSv8qart4+vPL6N1s5e1t2wmMSYwdd0FxEWT0sK+HvIS4vrF+iNMbxdUsvKfHvAsaCCjHhWzbDzvmuuQW5aaCcZuleO/SDAXIa3S52rvToM/Wr6wfmGmZ8WN2AxwuqCNBrau9ld1UxxlfNvxX9mNzir1y6Ymc7ftlQMukzEb9/YR1JMhOeNNCbSxvc/MYuPjrXwxChupxhMoLcBi4D7jTELgXbgDr9ztgHTjDHzgd8DzwV6IhG5VUSKRKSorm7816SqU/vh5bP58PsXcvPKnCEH7msLs07beIJnNq7XgGzziR7K69uZP8z8fDDWnjWJ1q5eT5ByL2vgn0rItcf6TJoKdM4505MpqW71VPIYY/jBs3vYeriRX14zz/PmFoz89DgO1bf7zEHYe6yVY82dp6yUumWls+ooJtLq+aQSKrMnTSQpJiJgnv614hrs8VFEWi0BB6sDVdx4WzXDXWZZR/HRZmJdnzwD+cLybBrau/nnroHXqN92pJF3Suu4ZVUOcV6VR584axLLclL41aulo7YaZjCBvhKoNMZsdt1/Gmfg9zDGtBhj2ly3XwIiRKRfCYkxZp0xptAYU2i324NokhoPoiOsJEyIGOtmDMh7kpbbyTVMRj915HZubirx0TZe2u2sB69sPEFbV2+/oJyXFkd5nXPgsKO7l8PHOyhI9z3Hk6d39Vwffv8gT22t5OsX5HH5vKEvO30qM9KdcxAOec3sdVdYrZ458Ot2RV4qMzPifT6phIrFIizPS+WDA/U+k7Pcq71eMied+VkJbC73TWs1tndT29oVMD/vlhwbybzMRN7ZV8ueoy3MmZwwYJnzuXkp5NpjT7l5/G/f2E9ybCQ3eu0BAc5PUT+6cg5tXb386rXSQf/PIzHiQG+MqQYqRMQ9urMG8Fm0XUQyxPWbFZElruuFfr62UsPgPUnLzb1GznArboIRabNw0ax0Xv+ohp4+h+eNx7+XmWuPo6vXwdGmE+yvacOY/ufMy0wgyubsub5dWstPX9rLpXMyQroxhntMwHsphDf31nDWlATS4gfuqYsIj31xCX/47KIBzwnGyrxUalq6fMYxPjhQT0d3HxfPzuCc6SnsOdriM/u3xPOzPvUnndUz7OyoaGJPVTNzpgx8rohw4/JsdlY2e3Z087bl0HHe21fHratyAs4jmJEez7cvmsGKEZZSDybYqpt/Ax4XkV3AAuCnInKbiNzmevxqYI+I7AR+B1xnRnPxbKWGaGbGREprWj0DYLsqmpmWEjOkPHYorT1rEs0nethY1tCvtNLNvRvTgbq2AStFomxWFk5N5NXiar7+xHYKMiby68/MH/ZEu1PJtcchAvtda9Mfb+9me0XTkCa4pQ2wsF4orHDl6b3TN69/VEN8lI2lOSkszXEuB+Kdpy/1q24aiHtCVFevY9Adtj61KJO4KBuPbjjkc7y5o4dvPbmDKYkTTrni7e3n53HZWcObAT9UQQV6Y8wOV8plnjHmk8aYRmPMA8aYB1yP32eMmWOMmW+MWWqM2RCaZisVnJmT4unscXgWGNtV2XRa0zZuK/NTiY208vKeY+w91kJW8gSf/C14lVjWtlFS3Up0hIWpAdYmWjI9hcrGE0RFWHjoxkJiIkO7V0J0hJWpyTGeHv27+2oxZuQzmUMlMymG6amxngHfPofhjb01rJ6ZRqTNwqJpidgs4pOnL61pJSkmgrT4qIGeFnCm8pJcc1rmnqJHD85VY69enMk/dx/zVAEZY/j3p3ZS09LJfZ9dGPLfyVDpzFh1RprltV9tXWsXR5s7T+tArFt0hJULZqXzanENHx1r6Zd7B2euODk2krK6NkprWpiRHh+wp37x7HQmJ0TzwOcWD1i/H6z8tHhPj/7NvbWkxkWdcmvL0+XcvBQ2lTfQ0+dg+5FG6tu6uXi2c52emEgb8zJ98/QlftVNA7FahNUFaUyIsA6peuyGZdPo6TP89UPnngcPrT/IG3tr+N5ls/rt8nY6aaBXZ6T89DgsAnurWz0TXcaiRw9w2dwMjrd3U17XPmAqIdceS1ltO6XVbRSkBz5n7pQENty5pt++q6GUnx7Hwfp2Onv6eG9fHecX2EOaHhqpFXl2Orr72H6kidc+qiHCKqwuODlAfE5OCrsqnQsFOhzu6qahVSLdedlMHr/lnCGtHptjj2PVDDuPbz7MpvIGfv5KCWvnZvAFv3kJp5sGenVGio6wkp0aS8mxFnZWNmORwT+aj5bzCuxERzhfigOV++Xa49hd1Ux9W9cpSwJHW35aHD19hme2VdHS2TvmaRu3ZbkpWATe31/Ha8XVLMtNJT76ZOXXOdOT6XUYth5upLLxBB3dfUP+OabFR7NoGL3xLyyfRk1LFzc88iGZSRO45+p5o7ra7VBooFdnrFkZzv1qd1U2kZ8WP2b505hIG+e79ieeNSlw8MlLi+OEazJOKGriRyo/zdm+B9eXE2EVz0DoWEuY4NyP4MmiCg41dHjSNm7uzVM2lx/3VFuN1hvmeTPSPGMof/jsIiZGj32p8dj8ZSs1DszMiOefu4/R2NHNpXMyxrQt7sll01MD54G988Nj2aPPS3NW3hysb+fcvBSfXvNYW5GXyu/fOgDARX6BPi7KxtwpCWw+2ECUawbvQCmwYFktwoM3FNLR3X/rzLGiPXp1xnIvBdza2cu8YS5kFmqLpyVx32cXDTiT2B3oU2IjsQ9SKTKaJkRaPcv0uj+FjBfuGvT5WYkB1ytaOj2ZnRXN7Kxs6rfZSKgVZMSP6eCrPw306ozlPfA5FhU3wzElaQKRNsuY9ubd3Omb8ZKfd1s4NYnMpAk+6+h7Oycnme4+B2+X1o2Ln+PppKkbdcbKTHLWrHf3OsY07z0UVotw0/Jszw5ZY2nt3AyibJZTLto2FiJtFt7/zwsGfLwwOxmLOOvsB5soFW400Kszlogwd8pEevvMaduFKxh3XjZrrJsAwDWFWVxTmDXWzRi2idERzJ48kT1Vp17MLBxpoFdntN9+ZiEGXZXjTHHO9BT2VLVoj16pM0lGQmiXzVXj2+eXTmNChJWcAaqbwpUGeqXUGSM7NZbvnGI7xXA1/hOTSimlgqKBXimlwpwGeqWUCnMa6JVSKswFFehFJFFEnhaREhHZKyLL/B4XEfmdiBwQkV0iMjp7iSmllBpQsFU39wKvGGOuFpFIwH/bm7VAvuvrHOB+179KKaVOkxH36EUkAVgFPAxgjOk2xjT5nXYl8Jhx2gQkisjobIqolFIqoGBSN9OBOuCPIrJdRB4SkVi/c6YAFV73K13HfIjIrSJSJCJFdXV1QTRJKaWUv2BSNzZgEfBvxpjNInIvcAfww+E+kTFmHbAOQETqRORwEO1KBeoHPWvsfVzaCdrW0aJtHR1nalunDfRAMIG+Eqg0xmx23X8aZ6D3VgV4r36U6To2IGOM/VSPD0ZEiowxhcE8x+nwcWknaFtHi7Z1dGhb+xtx6sYYUw1UiIh7PvEa4CO/014AbnBV3ywFmo0xx0Z6TaWUUsMXbNXNvwGPuypuyoGbROQ2AGPMA8BLwGXAAaADuCnI6ymllBqmoAK9MWYH4P+x4wGvxw1wezDXGIF1p/l6I/VxaSdoW0eLtnV0aFv9iDMWK6WUCle6BIJSSoU5DfRKKRXmwibQi8ilIlLqWlfHv8xzTInIIyJSKyJ7vI4li8jrIrLf9W/SWLbRTUSyRORtEflIRIpF5Buu4+OuvSISLSIfishOV1t/5Do+XUQ2u/4WnnQVC4w5EbG6Jhe+6Lo/Xtt5SER2i8gOESlyHRt3v38IvN7WeGyriBS4fp7urxYR+ebpamtYBHoRsQJ/wLm2zmzgehGZPbat8vG/wKV+x+4A3jTG5ANv0n8OwljpBf7dGDMbWArc7vpZjsf2dgEXGGPmAwuAS11lvPcAvzHG5AGNwJfGsI3evgHs9bo/XtsJcL4xZoFXjfd4/P3DyfW2ZgLzcf58x11bjTGlrp/nAmAxzirEZzldbTXGfOy/gGXAq1737wTuHOt2+bUxG9jjdb8UmOS6PQkoHes2DtDu54GLxnt7cS6otw3nonn1gC3Q38YYti/T9UK+AHgRkPHYTldbDgGpfsfG3e8fSAAO4ioqGc9t9WvfxcAHp7OtYdGjZ4hr6owz6ebk5LFqIH0sGxOIiGQDC4HNjNP2utIhO4Ba4HWgDGgyxvS6Thkvfwu/Bb4LOFz3Uxif7QQwwGsislVEbnUdG4+//4HW2xqPbfV2HfAX1+3T0tZwCfQfa8b5dj6u6lxFJA74O/BNY0yL92Pjqb3GmD7j/DicCSwBZo5xk/oRkcuBWmPM1rFuyxCtMMYswpkKvV1EVnk/OI5+/+71tu43xiwE2vFLfYyjtgLgGoe5AnjK/7HRbGu4BPphr6kzDtS4l2x2/Vs7xu3xEJEInEH+cWPMM67D47a9AMa5RPbbOFMgiSLingw4Hv4WzgWuEJFDwF9xpm/uZfy1EwBjTJXr31qceeQljM/ff6D1thYxPtvqthbYZoypcd0/LW0Nl0C/Bch3VTFE4vxo9MIYt2kwLwA3um7fiDMXPuZERHDuMbDXGPNrr4fGXXtFxC4iia7bE3COJezFGfCvdp025m01xtxpjMk0xmTj/Nt8yxjzr4yzdgKISKyIxLtv48wn72Ec/v7NwOttjbu2ermek2kbOF1tHeuBiRAOcFwG7MOZo/3+WLfHr21/AY4BPTh7IV/CmaN9E9gPvAEkj3U7XW1dgfPj4y5gh+vrsvHYXmAesN3V1j3AXa7jOcCHONdYegqIGuu2erV5NfDieG2nq007XV/F7tfSePz9u9q1AChy/Q08BySN47bGAg1Agtex09JWXQJBKaXCXLikbpRSSg1AA71SSoU5DfRKKRXmNNArpVSY00CvlFJhTgO9UkqFOQ30SikV5v4//7LvMN+FUuoAAAAASUVORK5CYII=\n", "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dd3xV9f3H8dcnBMgAkgCBQIKAgogiIKQoKo46ceHAqq1W/dX5k4qttdUO7c8uOy2Oaqm1rioqLqyjOKqixRH2Hg4gYYWVQPb4/P64NxpiEMjNzb2c+34+Hnnk3nNP7vkw8s7J93zO92vujoiIBFdSrAsQEZHoUtCLiAScgl5EJOAU9CIiAaegFxEJuORYF9BU9+7dvV+/frEuQ0RknzJr1qxN7p7d3GtxF/T9+vWjoKAg1mWIiOxTzGzVrl7T0I2ISMAp6EVEAk5BLyIScAp6EZGAU9CLiAScgl5EJOAU9CIiAReYoC/aVsEfpy9j1eayWJciIhJXAhP0JeU13P3mShatLY11KSIicSUwQZ+blQpA4dbyGFciIhJfAhP0Gant6ZySTOHWiliXIiISVwIT9AB5WWkUKehFRHYSsKBP1Rm9iEgTgQr63MxUCreWowXPRUS+EKigz8tKpay6jpKKmliXIiISNwIW9GkAGr4REWkkYEGvFksRkaZ2G/Rm9qCZbTSzhY22dTWz18xsRfhz1i6+9tLwPivM7NLWLLw5XwS9zuhFRBrsyRn9Q8CpTbbdDLzh7gOBN8LPd2JmXYHbgMOBUcBtu/qB0FoyUtvTqaN66UVEGttt0Lv7O8CWJpvHAQ+HHz8MnN3Ml54CvObuW9x9K/AaX/6B0arMLNx5o6AXEWnQ0jH6nu6+Lvx4PdCzmX1ygTWNnheGt0VVXlYqRdsU9CIiDSK+GOuhpvWIGtfN7CozKzCzguLi4ojqCd00pYuxIiINWhr0G8ysF0D488Zm9ikC+jR6nhfe9iXuPtnd8909Pzs7u4UlheRmpbK9sla99CIiYS0N+mlAQxfNpcALzezzb+BkM8sKX4Q9Obwtqhp66TXnjYhIyJ60Vz4BzAQGmVmhmX0HuAM4ycxWACeGn2Nm+Wb2AIC7bwF+AXwU/rg9vC2q1EsvIrKz5N3t4O4X7eKlE5rZtwC4otHzB4EHW1xdC+RmqpdeRKSxQN0ZC9A1vQOp7dup80ZEJCxwQW9m6rwREWkkcEEPoc4bDd2IiIQEMuh105SIyBcCGvRpbCuvYXuleulFRAIZ9A2dNzqrFxEJaNA39NLrpikRkcAGvVaaEhFpEMig796pAx2TkzR0IyJCQIPezMItluqlFxEJZNBDaPhGQzciIgEO+tzMVF2MFREhwEGfl5XK5rJqyqtrY12KiEhMBTroQS2WIiKBD/pCdd6ISIILcNCrl15EBCIMejObaGYLzWyRmd3QzOvHmVmJmc0Nf9wayfH2RnanjnRol6QWSxFJeLtdYWpXzGwIcCUwCqgGXjWzf7n7yia7znD3MyKosUWSkozemSkaoxeRhBfJGf1g4AN3L3f3WuBt4NzWKat1qJdeRCSyoF8IjDGzbmaWBpwG9Glmv9FmNs/MXjGzQ5p7IzO7yswKzKyguLg4gpJ2lqcFSEREWh707r4E+C0wHXgVmAvUNdltNtDX3YcBdwPP7+K9Jrt7vrvnZ2dnt7SkL8nNTGXTjioqa5qWJSKSOCK6GOvuf3f3ke5+DLAVWN7k9VJ33xF+/DLQ3sy6R3LMvZHXVfPSi4hE2nXTI/x5P0Lj8483eT3HzCz8eFT4eJsjOebeUIuliEgEXTdhz5hZN6AGuM7dt5nZNQDufj8wHrjWzGqBCuBCd/cIj7nHPl9pSkEvIgksoqB39zHNbLu/0eN7gHsiOUYkenZJITnJ1EsvIgktsHfGArRLMnpnqvNGRBJboIMewtMV62KsiCSwwAd9nlaaEpEElwBBn8aG0iqqatVLLyKJKfBBnxuernjdtsoYVyIiEhuBD/rP56XXBVkRSVAJFPQapxeRxBT4oM/pkkK7JFPnjYgkrMAHfXK7JHK6pGjoRkQSVuCDHtRiKSKJLSGCPjcrVfPdiEjCSoigz8tKY31pJdW19bEuRUSkzSVI0KdS77C+RL30IpJ4EiPow9MVF27TOL2IJJ7ECHotQCIiCSzSFaYmmtlCM1tkZjc087qZ2V1mttLM5pvZiEiO11I5GSkkmYJeRBJTi4PezIYAVwKjgGHAGWY2oMluY4GB4Y+rgPtaerxIdEhOomeXFHXeiEhCiuSMfjDwgbuXu3st8DahdWMbGwc84iHvA5lm1iuCY7aYeulFJFFFEvQLgTFm1s3M0oDTgD5N9skF1jR6XhjethMzu8rMCsysoLi4OIKSdi0vK01DNyKSkFoc9O6+BPgtMB14FZgLtGjSd3ef7O757p6fnZ3d0pK+Um5mKutLK6mtUy+9iCSWiC7Guvvf3X2kux8DbAWWN9mliJ3P8vPC29pcXlYqdfXO+lL10otIYom066ZH+PN+hMbnH2+yyzTg2+HumyOAEndfF8kxW0otliKSqJIj/PpnzKwbUANc5+7bzOwaAHe/H3iZ0Nj9SqAcuDzC47VYw0pT6rwRkUQTUdC7+5hmtt3f6LED10VyjNbSOzMF0Bm9iCSehLgzFqBjcjt6dumoFksRSTgJE/QQ6rzRSlMikmgSKujVSy8iiSjBgj6VtdsqqKv3WJciItJmEiroc7NSqa13Nm5XL72IJI6ECnr10otIIkqwoA8vQKLOGxFJIAkV9LmZumlKRBJPQgV9Svt2dO/UUUM3IpJQEirooWFeegW9iCSOhAv63CzdNCUiiSXhgj4vK5WirRXUq5deRBJEAgZ9GtV19RTvqIp1KSIibSLxgj6zocVSwzcikhgSL+jVSy8iCSbhgj43S2f0IpJYIl1K8HtmtsjMFprZE2aW0uT1y8ys2Mzmhj+uiKzcyKV1SKZregd13ohIwmhx0JtZLnA9kO/uQ4B2wIXN7Pqkuw8PfzzQ0uO1JvXSi0giiXToJhlINbNkIA1YG3lJ0RcKeo3Ri0hiaHHQu3sR8AdgNbAOKHH36c3sep6ZzTezqWbWp7n3MrOrzKzAzAqKi4tbWtIey80M9dKHlrQVEQm2SIZusoBxQH+gN5BuZhc32e1FoJ+7DwVeAx5u7r3cfbK757t7fnZ2dktL2mN5WWlU1dazaUd11I8lIhJrkQzdnAh86u7F7l4DPAsc2XgHd9/s7g13Jj0AjIzgeK2mocVSF2RFJBFEEvSrgSPMLM3MDDgBWNJ4BzPr1ejpWU1fj5Vc9dKLSAJJbukXuvsHZjYVmA3UAnOAyWZ2O1Dg7tOA683srPDrW4DLIi85crm6O1ZEEkiLgx7A3W8Dbmuy+dZGr98C3BLJMaKhc0p7MtPaawESEUkICXdnbIPcTLVYikhiSNig101TIpIoEjboczPTKNqmXnoRCb6EDfq8rFTKq+vYWl4T61JERKIqoYMe1GIpIsGXsEHf0EuvzhsRCbqEDfq8rDRAvfQiEnwJG/QZqe3pnJKsoRsRCbyEDXoIz2Kp+W5EJOASOujzstI0dCMigZfgQR+6aUq99CISZAkf9DuqaimtqI11KSIiUZPwQQ+wRhdkRSTAEjzo1WIpIsGX0EHfMC+9Om9EJMgiCnoz+56ZLTKzhWb2hJmlNHm9o5k9aWYrzewDM+sXyfFaW2Zae9I7tFMvvYgEWiSLg+cC1wP57j4EaAdc2GS37wBb3X0AcCfw25YeLxrMTC2WIhJ4kQ7dJAOpZpYMpAFrm7w+Dng4/HgqcEJ4fdm4kZuVqvluRCTQWhz07l4E/IHQIuHrgBJ3n95kt1xgTXj/WqAE6Nb0vczsKjMrMLOC4uLilpbUIqFeeg3diEhwRTJ0k0XojL0/0BtIN7OLW/Je7j7Z3fPdPT87O7ulJbVIXlYqpZW1lFZqXnoRCaZIhm5OBD5192J3rwGeBY5ssk8R0AcgPLyTAWyO4JitLjcz1GKp4RsRCapIgn41cISZpYXH3U8AljTZZxpwafjxeOBNj7P5Br5YgERBLyLBFMkY/QeELrDOBhaE32uymd1uZmeFd/s70M3MVgLfB26OsN5Wp5WmRCTokiP5Yne/DbityeZbG71eCZwfyTGirWt6B1LaJ2noRkQCK6HvjAX10otI8CV80EO4xXKbhm5EJJgU9IRXmtIZvYgElIKe0CyWW8tr2FGleelFJHgU9HzReaOzehEJIgU9ofluAIo0Ti8iAaSgRzdNiUiwKeiB7E4d6ZicpKAXkUBS0BPqpVfnjYgElYI+LFfTFYtIQCnow3R3rIgElYI+LC8rlc1l1VRU18W6FBGRVqWgD8tTi6WIBJSCPqwh6Ndo+EZEAkZBH6aVpkQkqBT0YT06d6R9O9MFWREJnEgWBx9kZnMbfZSa2Q1N9jnOzEoa7XPrrt4v1pKSQr30arEUkaBp8QpT7r4MGA5gZu0ILQT+XDO7znD3M1p6nLaUm5VK0Tad0YtIsLTW0M0JwMfuvqqV3i8m8jLVSy8iwdNaQX8h8MQuXhttZvPM7BUzO6S5HczsKjMrMLOC4uLiVipp7+VlpVK8vYrKGvXSi0hwRBz0ZtYBOAt4upmXZwN93X0YcDfwfHPv4e6T3T3f3fOzs7MjLanFGqYrXqvhGxEJkNY4ox8LzHb3DU1fcPdSd98Rfvwy0N7MurfCMaMiLyvUYqnhGxEJktYI+ovYxbCNmeWYmYUfjwofb3MrHDMqNC+9iARRi7tuAMwsHTgJuLrRtmsA3P1+YDxwrZnVAhXAhe7ukRwzmnp2SSE5yTQNgogESkRB7+5lQLcm2+5v9Pge4J5IjtGW2iUZvTJTdEYvIoGiO2ObUIuliASNgr6J3CytNCUiwaKgbyIvK5UN2yupqlUvvYgEg4K+ibysNNxh3bbKWJciItIqFPRN5GY2LECi4RsRCQYFfRNf9NKrxVJEgkFB30SvjBTaJWleehEJDgV9E8ntksjpkqLOGxEJDAV9M3KzUnVGLyKBoaBvRp4WIBGRAFHQNyMvM5V1JRXU1NXHuhQRkYgp6JuRl5VGvcP6EvXSi8i+T0HfDE1XLCJBoqBvRq566UUkQBT0zeiVkYqZzuhFJBhaHPRmNsjM5jb6KDWzG5rsY2Z2l5mtNLP5ZjYi8pKjr0NyuJdenTciEgAtXnjE3ZcBwwHMrB1QBDzXZLexwMDwx+HAfeHPcS83M1VDNyISCK01dHMC8LG7r2qyfRzwiIe8D2SaWa9WOmZU5emmKREJiNYK+gtpfoHwXGBNo+eF4W07MbOrzKzAzAqKi4tbqaTI5GWlsb6kklr10ovIPi7ioDezDsBZwNMtfQ93n+zu+e6en52dHWlJrSI3K5XaemfD9qpYlyIiEpGIFgcPGwvMdvcNzbxWBPRp9DwvvC3ufd5Lv6X88znq483G0kqKd1RxcK8umFmsy9knVNbU8dnmMj7bVMZnm8v5bFMZG0orueW0wRzYs3OsyxOJitYI+otoftgGYBowwcymELoIW+Lu61rhmFGXl5UGxMcCJO7OhtIqFhSVsKCohIXhj43h3zZG79+N/xt3iIIqrLKmjtVbyvl0U0Ogl/HZpnI+21zGuiZ3O3fv1IHKmnomPD6baROOJqV9uxhVLRI9EQW9maUDJwFXN9p2DYC73w+8DJwGrATKgcsjOV5b6pWRAsAbSzfSNb0DvTJSyclIoUtKclTPnt2dtSWVLCgsYdHaL4J9045qAJIMDsjuxNEDujMkN4N6d+5+cyVjJ83g0tH9uOGkgXRJaR+1+uJFVW0da7aU8+mm0Fn5p+Gz9FWby1lbUoH7F/t2Te9Av25pjD6gG/27pdO3ezr9u6XTr3sanVPa8/byYi598EN+/fISbh83JHZ/KJEoMW/8HREH8vPzvaCgINZlAHD6XTNYtLZ0p21pHdqRk5FCr4wUcrqkhj43PM9IoVdGKllp7ffoh4G7U7i1goXhM/UFRSUsWlvKlrJQqLdLMgb26MSQ3AyG9O7CoXkZDO7VhbQOO/983lJWzR+mL+OJD1fTLb0DPzz1IMaPyCMpKXjDOf9ZtpHbXljEmq3lO4V5Zlp7+nVLp3/3dPqFQzz0OZ2M1N3/4PvlvxbzwLuf8vdL8zlhcM8o/glEosPMZrl7frOvKeh3rbaung3bq1hfUsG6kkrWl1Q2+lzB+pJKNmyvoq5+57/DhhuudvoB0CWFnIxU6ur9i+GXtSVsK68BIDnJOLBnZ4bkduHQ3AyG5IZCfW+GEhYWlXDrCwuZvXobw/tkcvu4Qxial9mqfyextKWsmpP+9DYZae05Y2hv+ofDvH/3dDLTOkT03lW1dZxz739ZX1rJqxPH0KNLSitVLdI2FPRRVFfvbNpRFf4B0MwPhNIKNpRUUd2oTbN9O2NQTufPA31I7wwG5XRulfHh+nrnuTlF/OaVpWwuq+KC/D7cdMogunXqGPF7x9rEKXN4ecE6Xvzu0RyU06XV33/lxu2ccfe7fK1fVx6+fFQgfyOS4PqqoG+Ni7EJrV2S0bNLCj27pECf5s+e3Z0tZdWfXwg8sGdnOiRHZ5qhpCTjvJF5nHxITya9voKH/vsZLy9Yx40nD+Jbh+9Hcrt9c3qj1xdv4IW5a7nhxIFRCXmAAT06c+sZh/Dj5xbw4HufcsWY/aNyHJG2tm9+1+9jzIxunTqGzt5zM6IW8o11TmnPT884mFdvGMPQvExum7aIM+5+lw8+2Rz1Y7e2kooafvL8Ag7K6cz/Hjcgqse6aFQfTjmkJ799dSkLi0qieizZNxVvr+KeN1dw+l0zeHfFpliXs0c0dJMA3J1XF67nly8toWhbBWcN682PTxtMTsa+MQ79o6nzeXrWGp6/7qg2ueawtayasZNmkNaxHf/67tFfuvgticfdmb16K4/MXMXLC9ZRU+d0TkmmY3ISr0w8huzOsR8a/aqhG53RJwAzY+yhvXj9+8dy/QkDeXXRer7+x7f4y1srqaqti3V5X2nGimKeLFjDVccc0GYXlrPSO/Cnbwzj001l/OJfi9vkmBKfKqrrmPLhak6/613Ou28mby7ZyMVH9OWNG4/lmWuPZHtlLTc+PY/6+vg6YW5KZ/QJaPXmcn7x0mJeW7yB/t3TufXMgzl+UI9Yl/UlZVW1nHznO3RMTuLliWPa/Gam3766lPve+pj7Lx7BqUP2ibn4pJV8tqmMR99fxdMFayitrOWgnM5cMrovZw/PJb3jF7/hPfr+Kn72/EJ+ctpgrjwmttd0dDFWdrJftzT+9u183lq2kdtfXMzl//iIEwf34GdnHEzfbumxLu9zv//3MtaWVPD01aNjcsfq9048kPdWbuJHzyxgWJ9MemXE51QY0jrq6p23lm3kkZmreHt5MclJxqlDcvj26H58rV9Ws/fGXHz4fsxYXszv/r2UI/bvxqF5GTGofPd0Rp/gqmvrefC9T7nrjRXU1jtXH7M/1x0/IOZTAXz02RbOv38mlx3Zj5+fdUjM6vh0Uxmn3zWDoXkZ/POKI2inlsvA2VpWzZMFa3js/VUUbq2gR+eOfOvwvlw0qs8e3U/RcE0ntUPomk7jM/62pD562a31JZX85pUlvDB3LWMGdueBS/PpmBybsK+sqeO0STOorqvn3zccE7NvnAZPF6zhpqnz+eGpg6Le9SNtZ37hNh6ZuYpp89ZSXVvP4f278u3R/Tj5kJ6038s25Jkfb+abD7zPeSPy+MP5w6JU8VfT0I3sVk5GCpMuPIyjB3Tnpqnzuf6JOdz7zREx6bu/8/XlfLKpjH9ecXjMQx5g/Mg83lpezJ+mL+fIA7ozfBf3S0j8q6yp46X563jk/VXMW7ONtA7t+EZ+Hpcc0Y9BOS2fFHD0Ad2YcPwA7n5zJcccmM1Zw3q3YtWRi/13kcSV8/P7UFZVy89fXMxNU+fzx/OHtekdovPWbONv73zChV/rw1EDurfZcb+KmfHrsw9l7uptTJwyh5euH0OnOPgBJHtu844q/jbjU54qWMOWsmoOyE7n/846hHNH5NK5lSYBnHjCQN5buYmfPLuAw/pk0qdrWqu8b2tQe6V8yWVH9eemUwbx3Jwibp22kLYa3quuredHz8ynR+cUfnz64DY55p7KSGvPnRcMZ82Wcn4+bVGsy5G98NL8dZx05ztMfudjvtYvi39ecTivf/9YLj2yX6uFPEByuyQmXXgYANdPmUNNHK1Op6CXZv3vcQdwzbEH8Nj7q7nj1aVtEvZ/eWslS9dv51fnDInLqZZH9e/KhK8PZOqsQqbNWxvrcuJKvF3rg9AdrNc+NovrHp9NXlYqr0w8hr9eks9RA7pHbarxPl3T+PW5hzJn9TYmvb4iKsdoCf3+Kc0yM3506iDKqmr569uf0LljMhO+PjBqx1u6vpR73lzJ2cN7x/U0wdd/fQDvrijmJ8/F36/n0VJVW8fG0tDEfQ2ztn4xaV9oMr9NO6o5ekB3bh57EIN7RWcuoj3l7kybt5afT1tEWVUdPzr1IK4c07/NrjedOaw37ywv5t63VnLUgO6MPqBbmxz3q0TUdWNmmcADwBDAgf9x95mNXj8OeAH4NLzpWXe//aveU1038aW+3vnB0/N4dk4Rt515MJcf1b/Vj1FbV8+59/2Xoq0VvPb9Y+maHtmUw9G2Zks5YyfN4KCczky56oh9dqI4CN35ub60mQAvqWR9aWhbw6I3jXXumExOo7UY0jsm8+zsIkorazhvRB7fP+lAesdgCc6N2yv56XMLmb54A8P7ZPL78UMZGIOV18qqajnz7ncpr67jlYljyGqD/9PR7LqZBLzq7uPDi4Q3d3ozw93PiPA4EiNJScbvxg+lvLqO/3txMekdkvnG1/rs/gv3wgPvfsr8whLu/eaIuA95CP16/qtzhjBxylzu/c/HTDwxer/pRMM7y4v5/b+XsWZr+efrITSWmdaenC6hAD80N3OnxXV6ZYRmam1ubPuGEw7kL2+t5B///YwX563l8qP6c+1xB+zRwi+Rcneen1vEz6ctpqKmjh+fdhDfOXr/mN33kN4xmbsuOoxz/vIeP3xmPpMvGRnTdZ1bfEZvZhnAXGB/38WbhM/of7A3Qa8z+vhUVVvHlY/M4t0Vxdx10WGcMbR12sc+Lt7B2EkzOH5QNvdfHNtvhr31vSfn8sLcIp66ejT5/brGupzdqq937nv7Y/4wfRn7d0/nqAHdPw/wnl1Cq6PldEkhtUNk908Ubavgj9OX8dycIjJS2zPh+AFcMrpv1O7L2FBayU+eW8DrSzYysm8Wvxs/lAOyO0XlWHvrgRmf8MuXlvCLs4dwyRF9o3qsqNwwZWbDgcnAYmAYMAuY6O5ljfY5DngGKATWEgr9r2xZUNDHr4rqOi598ENmr97K376dz/EHRTY/Tn29c8HkmSzfsIPXvnfMPreq0/bKGk67awb19fDKDWPi8gJyg+2VNdz41DymL97AuOG9uePcoREH+u4sWlvCHa8sZcaKTeRlpXLTKYM4c2jvVmvXdXeemV3E7S8uoqq2nptOGcTlR/WPq7uX6+udyx76iA8+2cy0CUdH1Ku/O9GavTIZGAHc5+6HAWXAzU32mQ30dfdhwN3A87so8CozKzCzguLi4ghKkmhK7dCOBy7LZ3CvLlzz2CxmfhzZ3PaPvr+Kjz7bys/OOHifC3kIzfk/6cLDWF8aGheOx84TgBUbtjPunvd4c+lGbjvzYP58wfCohzzAIb0zePQ7h/Pod0bRJaU9E6fMZdy97/HfjyOfw31dSQWXP/QRP3h6HoNyOvPqDcdwxZjYDdXsSlKS8cfzh9E5JZnvPjGbyprYzBYbSdAXAoXu/kH4+VRCwf85dy919x3hxy8D7c3sS3fBuPtkd8939/zs7OwISpJo65LSnof/ZxT7dU3jioc/Ys7qrS16nzVbyvntq0s59sBszhuR28pVtp0R+2VxwwkDmTZvLc/NKYp1OV/y0vx1jLv3PUora3n8yiO4/Kj+bT48NmZgNv/67tHcecEwtpRV882/fcDl//iQpetL9/q93J2nPlrDyX96hw8+2cJtZx7Mk1eNpn/3+JmMr6nszh354zeGs3zDDn710pKY1NDioHf39cAaMxsU3nQCoWGcz5lZjoX/V5nZqPDx9r0ljmQnXdM78M8rDqd7545c9o+PWLJu775h3Z1bnl2AAb8+99B9aly+Of97/ABG9evKz55fyKrNZbv/gjZQW1fPr19ewnWPz+agnM68dP3RjOofu+sISUnGOYfl8caNx/KT0wYza9VWxk6awU1Pz2NdScUevUfRtgou/cdH/PCZ+Rzcuwuv3jCGy4/qv0+s7XvsgdlcOaY/j76/iumL1rf58SNtrxxOqL2yA/AJcDlwAYC7329mE4BrgVqgAvi+u//3q95TY/T7jjVbyjn//pnU1tfz1NWj2X8PL4A9+dFqfvTMAn559hAujvIFqrZStK2CsX9+h/7ZnZh6zei9nhSrNW3eUcWEx+cw85PNfHt0X356+sFtsnzl3thWXs1f3vqYh977DDP4ztH9uea4A5q9zuHuTPloDb96aQn17tw89iAuPrzvPhHwjVXX1nPufe9RuLWCVyaOafVprzV7pUTNyo07uOCvM+mYnMRT14wmL+urbyBaX1LJSXe+zSG9u/D4FUfsc9+sX+Wl+eu47vHZTDh+AD84ZdDuvyAK5q3ZxrWPzWJzWTW/PudQzhuZF5M69tSaLeX86bXlPDeniKy09nz36wO5+Ii+n/9gKtxazs3PLODdlZsYvX83fjd+6D59k9onxTs44+53ozLttYJeomrx2lIunDyTrukdeOqa0fTo3PyFVXfnykcKeHflJv59wzFxtchJa/nh1Hk8PauQP18wnNMP7dWmN1M98eFqbnthET26dOT+i0cyJDc+F8FozsKiUIfOuys3sV/XNH5wyiBKK2r4zcuhMe1bThvMN0ftF4gTg4Zpr286ZRDXHd96014r6CXqZq3ayiV//4A+WWk8efURZKZ9+canF+YWMXHKXH56+mCuGBPbZdeipayqlnP+8h7LN+ygR+eOnDMil/NH5jGgR/Ta6ipr6vj5tEVM+WgNxxyYzaQLhrfJnZjR8M7yYjJIGyYAAAdTSURBVH7zytLPr/scPaA7d5x36G5/U9yXuDvXT5nLywvW8dTVoxnZN6tV3ldBL23ivZWbuPyhjxic05nHrjh8p7snN+2o4qQ/vU2/7ulMvebIuGuDa03VtfW8uXQjU2cV8p9lG6mrd4b3yWT8yDzOHNa7Ve8UXbutgmsfm8W8whImHD+A75104D7/d1tf77w4fy317pw9PHefv1jfnNLKGk6bNAOAlye2zj0YCnppM68v3sA1j81iRN8sHr581Of92hMen830RRt46fqjYzL3SKwUb6/ihblFPF1QyLIN2+mQnMQph+QwfmQeRw/oHlEo/3flJiY8MYea2nr++I1hnHxITitWLtE2a9VWvvHXmZx2aC/uunB4xD/QFPTSpqbNW8vEKXM49sBsJl+Sz3+WbeTqR2fxg5MPjOoMmPHM3VlYVMrUWWt4Yd5atpXXkNMlhXNH5DJ+ZN4edyw1vNffZnzCHa8s5YDsTvz1kpF79fUSP+55cwV/mL6c348fyvn5kc0hpaCXNjflw9Xc/OwCTj64J3PWbCO7U0demHBUTNsO40VVbR1vLAkN7by1bCP1DiP7ZjF+ZB6nD+31lb/G76iq5UdT5/PSgnWcfmgvfjd+aFwstygtU1fvfOuB95lfWMK/vnt0RD+wFfQSEw0TOrVLMl647qh9qgukrWwsreS5OUU8PauQlRt3kNI+iVMPyWH8yD4ceUC3nbpMPi7ewdWPzuKT4h3cMnYwV4xp+7tcpfWtK6lg7KQZ5GWl8uy1R7X4ngcFvcTMlA9X0yE5iXNHxHc/d6y5O/MKS5g6aw3T5q6ltLKW3hkpnDcyj/NG5LFsw3ZufGoeHZKTuOeiwzgyTtbTldYxfdF6rnp0FleO6c9PTj+4Re+hoBfZh1TW1PH6kg08XVDIjBXF1Ie/RYf1yeS+b42IyYIeEn2/emkxA3p04oKv7deir1fQi+yj1peEhnbq6uu58pj9ozanu+z7ornClIhEUU5GCtced0Csy5B9nFogREQCTkEvIhJwCnoRkYBT0IuIBJyCXkQk4BT0IiIBp6AXEQk4Bb2ISMDF3Z2xZlYMrIrgLboDm1qpnGiI9/og/muM9/og/muM9/pANe6tvu6e3dwLcRf0kTKzgl3dBhwP4r0+iP8a470+iP8a470+UI2tSUM3IiIBp6AXEQm4IAb95FgXsBvxXh/Ef43xXh/Ef43xXh+oxlYTuDF6ERHZWRDP6EVEpBEFvYhIwAUm6M3sVDNbZmYrzezmWNfTlJn1MbP/mNliM1tkZhNjXVNzzKydmc0xs3/FupbmmFmmmU01s6VmtsTMRse6psbM7Hvhf9+FZvaEmaXEQU0PmtlGM1vYaFtXM3vNzFaEP2fFYY2/D/87zzez58wsM57qa/TajWbmZha3C/kGIujNrB1wLzAWOBi4yMxatsJu9NQCN7r7wcARwHVxWCPARGBJrIv4CpOAV939IGAYcVSrmeUC1wP57j4EaAdcGNuqAHgIOLXJtpuBN9x9IPBG+HksPcSXa3wNGOLuQ4HlwC1tXVQjD/Hl+jCzPsDJwOq2LmhvBCLogVHASnf/xN2rgSnAuBjXtBN3X+fus8OPtxMKqNzYVrUzM8sDTgceiHUtzTGzDOAY4O8A7l7t7ttiW9WXJAOpZpYMpAFrY1wP7v4OsKXJ5nHAw+HHDwNnt2lRTTRXo7tPd/fa8NP3gbw2L+yLWpr7OwS4E/ghENddLUEJ+lxgTaPnhcRZiDZmZv2Aw4APYlvJl/yZ0H/a+lgXsgv9gWLgH+HhpQfMLD3WRTVw9yLgD4TO7tYBJe4+PbZV7VJPd18Xfrwe6BnLYvbA/wCvxLqIxsxsHFDk7vNiXcvuBCXo9xlm1gl4BrjB3UtjXU8DMzsD2Ojus2Jdy1dIBkYA97n7YUAZsR9y+Fx4nHscoR9IvYF0M7s4tlXtnod6rOP2jNTMfkJo6POfsa6lgZmlAT8Gbo11LXsiKEFfBPRp9DwvvC2umFl7QiH/T3d/Ntb1NHEUcJaZfUZo6OvrZvZYbEv6kkKg0N0bfhOaSij448WJwKfuXuzuNcCzwJExrmlXNphZL4Dw540xrqdZZnYZcAbwLY+vm34OIPQDfV74eyYPmG1mOTGtaheCEvQfAQPNrL+ZdSB0AWxajGvaiZkZobHlJe7+p1jX05S73+Luee7ej9Df35vuHldno+6+HlhjZoPCm04AFsewpKZWA0eYWVr43/sE4uhicRPTgEvDjy8FXohhLc0ys1MJDSWe5e7lsa6nMXdf4O493L1f+HumEBgR/j8adwIR9OELNhOAfxP6xnrK3RfFtqovOQq4hNCZ8tzwx2mxLmof9F3gn2Y2HxgO/DrG9Xwu/JvGVGA2sIDQ91fMb5E3syeAmcAgMys0s+8AdwAnmdkKQr+J3BGHNd4DdAZeC3+/3B9n9e0zNAWCiEjABeKMXkREdk1BLyIScAp6EZGAU9CLiAScgl5EJOAU9CIiAaegFxEJuP8HvHiKw1jJ554AAAAASUVORK5CYII=\n",
"text/plain": [ "text/plain": [
"<Figure size 432x288 with 1 Axes>" "<Figure size 432x288 with 1 Axes>"
] ]
...@@ -428,7 +371,7 @@ ...@@ -428,7 +371,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 30, "execution_count": 22,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -439,14 +382,6 @@ ...@@ -439,14 +382,6 @@
"the predict words is: shakespeare\n", "the predict words is: shakespeare\n",
"the true words is: shakespeare\n" "the true words is: shakespeare\n"
] ]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Library/Python/3.7/site-packages/ipykernel/ipkernel.py:287: DeprecationWarning: `should_run_async` will not call `transform_cell` automatically in the future. Please pass the result to `transformed_cell` argument and any exception that happen during thetransform in `preprocessing_exc_tuple` in IPython 7.17 and above.\n",
" and should_run_async(code)\n"
]
} }
], ],
"source": [ "source": [
...@@ -466,13 +401,6 @@ ...@@ -466,13 +401,6 @@
" print('the true words is: ' + y_data)\n", " print('the true words is: ' + y_data)\n",
"test(model)" "test(model)"
] ]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
} }
], ],
"metadata": { "metadata": {
...@@ -480,6 +408,18 @@ ...@@ -480,6 +408,18 @@
"display_name": "Python 3", "display_name": "Python 3",
"language": "python", "language": "python",
"name": "python3" "name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.3"
} }
}, },
"nbformat": 4, "nbformat": 4,
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或