From bf87006ad794ba0a76f67901bbbea417112dbe56 Mon Sep 17 00:00:00 2001 From: Chen Long Date: Sat, 12 Sep 2020 17:51:16 +0800 Subject: [PATCH] fix docs for beta and change docs style (#900) --- .../convnet_image_classification.ipynb | 14 +- .../dynamic_graph/dynamic_graph.ipynb | 14 +- .../hello_paddle/hello_paddle.ipynb | 20 +-- .../mnist_lenet_classification.ipynb | 152 ++++++++---------- .../image_search/image_search.ipynb | 18 +-- .../imdb_bow_classification.ipynb | 16 +- .../n_gram_model/n_gram_model.ipynb | 81 +++++----- .../seq2seq_with_attention.ipynb | 24 +-- 8 files changed, 160 insertions(+), 179 deletions(-) diff --git a/paddle2.0_docs/convnet_image_classification/convnet_image_classification.ipynb b/paddle2.0_docs/convnet_image_classification/convnet_image_classification.ipynb index f386efd..9fe4858 100644 --- a/paddle2.0_docs/convnet_image_classification/convnet_image_classification.ipynb +++ b/paddle2.0_docs/convnet_image_classification/convnet_image_classification.ipynb @@ -13,7 +13,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 设置环境\n", + "## 设置环境\n", "\n", "我们将使用飞桨2.0beta版本。" ] @@ -46,7 +46,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 加载并浏览数据集\n", + "## 加载并浏览数据集\n", "\n", "我们将会使用飞桨提供的API完成数据集的下载并为后续的训练任务准备好数据迭代器。cifar10数据集由60000张大小为32 * 32的彩色图片组成，其中有50000张图片组成了训练集，另外10000张图片组成了测试集。这些图片分为10个类别，我们的任务是训练一个模型能够把图片进行正确的分类。" ] @@ -73,7 +73,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 浏览数据集\n", + "## 浏览数据集\n", "\n", "接下来我们从数据集中随机挑选一些图片并显示，从而对数据集有一个直观的了解。" ] @@ -113,7 +113,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 组建网络\n", + "## 组建网络\n", "\n", "接下来我们使用飞桨定义一个使用了三个二维卷积（`Conv2d`)且每次卷积之后使用`relu`激活函数，两个二维池化层（`MaxPool2d`），和两个线性变换层组成的分类网络，来把一个`(32, 32, 3)`形状的图片通过卷积神经网络映射为10个输出，这对应着10个分类的类别。" ] @@ -164,7 +164,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 模型训练\n", + "## 模型训练\n", "\n", "接下来，我们用一个循环来进行模型的训练，我们将会：\n", "- 使用`paddle.optimizer.Adam`优化器来进行优化。\n", @@ -332,7 +332,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# The End\n", + "## The End\n", "\n", "从上面的示例可以看到，在cifar10数据集上，使用简单的卷积神经网络，用飞桨可以达到71%以上的准确率。你也可以通过调整网络结构和参数，达到更好的效果。" ] @@ -354,7 +354,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.7" + "version": "3.7.3" } }, "nbformat": 4, diff --git a/paddle2.0_docs/dynamic_graph/dynamic_graph.ipynb b/paddle2.0_docs/dynamic_graph/dynamic_graph.ipynb index 2b093b2..26d1119 100644 --- a/paddle2.0_docs/dynamic_graph/dynamic_graph.ipynb +++ b/paddle2.0_docs/dynamic_graph/dynamic_graph.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 设置环境\n", + "## 设置环境\n", "\n", "我们将使用飞桨2.0beta版本，并确认已经开启了动态图模式。" ] @@ -46,7 +46,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 基本用法\n", + "## 基本用法\n", "\n", "在动态图模式下，您可以直接运行一个飞桨提供的API，它会立刻返回结果到python。不再需要首先创建一个计算图，然后再给定数据去运行。" ] @@ -91,7 +91,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 使用python的控制流\n", + "## 使用python的控制流\n", "\n", "动态图模式下，您可以使用python的条件判断和循环，这类控制语句来执行神经网络的计算。（不再需要`cond`, `loop`这类OP)\n" ] @@ -136,7 +136,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 构建更加灵活的网络：控制流\n", + "## 构建更加灵活的网络：控制流\n", "\n", "- 使用动态图可以用来创建更加灵活的网络，比如根据控制流选择不同的分支网络，和方便的构建权重共享的网络。接下来我们来看一个具体的例子，在这个例子中，第二个线性变换只有0.5的可能性会运行。\n", "- 在sequence to sequence with attention的机器翻译的示例中，你会看到更实际的使用动态图构建RNN类的网络带来的灵活性。\n" @@ -226,7 +226,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 构建更加灵活的网络：共享权重\n", + "## 构建更加灵活的网络：共享权重\n", "\n", "- 使用动态图还可以更加方便的创建共享权重的网络，下面的示例展示了一个共享了权重的简单的AutoEncoder。\n", "- 你也可以参考图像搜索的示例看到共享参数权重的更实际的使用。" @@ -276,7 +276,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# The end\n", + "## The end\n", "\n", "可以看到使用动态图带来了更灵活易用的方式来组网和训练。" ] @@ -298,7 +298,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.7" + "version": "3.7.3" } }, "nbformat": 4, diff --git a/paddle2.0_docs/hello_paddle/hello_paddle.ipynb b/paddle2.0_docs/hello_paddle/hello_paddle.ipynb index 4e377ac..c087625 100644 --- a/paddle2.0_docs/hello_paddle/hello_paddle.ipynb +++ b/paddle2.0_docs/hello_paddle/hello_paddle.ipynb @@ -18,7 +18,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 普通程序跟机器学习程序的逻辑区别\n", + "## 普通程序跟机器学习程序的逻辑区别\n", "\n", "作为一名开发者，你最熟悉的开始学习一门编程语言，或者一个深度学习框架的方式，可能是通过一个hello, world程序。\n", "\n", @@ -80,7 +80,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 导入飞桨\n", + "## 导入飞桨\n", "\n", "为了能够使用飞桨，我们需要先用python的`import`语句导入飞桨`paddle`。\n", "同时，为了能够更好的对数组进行计算和处理，我们也还需要导入`numpy`。\n", @@ -111,7 +111,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 准备数据\n", + "## 准备数据\n", "\n", "在这个机器学习任务中，我们已经知道了乘客的行驶里程`distance_travelled`，和对应的，这些乘客的总费用`total_fee`。\n", "通常情况下，在机器学习任务中，像`distance_travelled`这样的输入值，一般被称为`x`（或者特征`feature`），像`total_fee`这样的输出值，一般被称为`y`（或者标签`label`)。\n", @@ -133,7 +133,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 用飞桨定义模型的计算\n", + "## 用飞桨定义模型的计算\n", "\n", "使用飞桨定义模型的计算的过程，本质上，是我们用python，通过飞桨提供的API，来告诉飞桨我们的计算规则的过程。回顾一下，我们想要通过飞桨用机器学习方法，从数据当中学习出来如下公式当中的`w`和`b`。这样在未来，给定`x`时就可以估算出来`y`值（估算出来的`y`记为`y_predict`）\n", "\n", @@ -161,7 +161,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 准备好运行飞桨\n", + "## 准备好运行飞桨\n", "\n", "机器（计算机）在一开始的时候会随便猜`w`和`b`，我们先看看机器猜的怎么样。你应该可以看到，这时候的`w`是一个随机值，`b`是0.0，这是飞桨的初始化策略，也是这个领域常用的初始化策略。（如果你愿意，也可以采用其他的初始化的方式，今后你也会看到，选择不同的初始化策略也是对于做好深度学习任务来说很重要的一点）。" ] @@ -192,7 +192,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 告诉飞桨怎么样学习\n", + "## 告诉飞桨怎么样学习\n", "\n", "前面我们定义好了神经网络（尽管是一个最简单的神经网络），我们还需要告诉飞桨，怎么样去**学习**，从而能得到参数`w`和`b`。\n", "\n", @@ -217,7 +217,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 运行优化算法\n", + "## 运行优化算法\n", "\n", "接下来，我们让飞桨运行一下这个优化算法，这会是一个前面介绍过的逐步调整参数的过程，你应该可以看到loss值（衡量`y`和`y_predict`的差距的`loss`)在不断的降低。" ] @@ -259,7 +259,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 机器学习出来的参数\n", + "## 机器学习出来的参数\n", "\n", "经过了这样的对参数`w`和`b`的调整（**学习**)，我们再通过下面的程序，来看看现在的参数变成了多少。你应该会发现`w`变成了很接近2.0的一个值，`b`变成了接近10.0的一个值。虽然并不是正好的2和10，但却是从数据当中学习出来的还不错的模型的参数，可以在未来的时候，用从这批数据当中学习到的参数来预估了。（如果你愿意，也可以通过让机器多学习一段时间，从而得到更加接近2.0和10.0的参数值。)" ] @@ -290,7 +290,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# hello paddle\n", + "## hello paddle\n", "\n", "通过这个小示例，希望你已经初步了解了飞桨，能在接下来随着对飞桨的更多学习，来解决实际遇到的问题。" ] @@ -335,7 +335,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.7" + "version": "3.7.3" } }, "nbformat": 4, diff --git a/paddle2.0_docs/image_classification/mnist_lenet_classification.ipynb b/paddle2.0_docs/image_classification/mnist_lenet_classification.ipynb index ca3b862..f7012ee 100644 --- a/paddle2.0_docs/image_classification/mnist_lenet_classification.ipynb +++ b/paddle2.0_docs/image_classification/mnist_lenet_classification.ipynb @@ -14,26 +14,27 @@ "metadata": {}, "source": [ "## 环境\n", - "本教程基于paddle-develop编写，如果您的环境不是本版本，请先安装paddle-develop版本。" + "本教程基于paddle-2.0-beta编写，如果您的环境不是本版本，请先安装paddle-2.0-beta版本。" ] }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "0.0.0\n" + "2.0.0-beta0\n" ] } ], "source": [ "import paddle\n", "print(paddle.__version__)\n", - "paddle.disable_static()" + "paddle.disable_static()\n", + "# 开启动态图" ] }, { @@ -46,7 +47,7 @@ }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 4, "metadata": {}, "outputs": [ { @@ -74,7 +75,7 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 5, "metadata": {}, "outputs": [ { @@ -117,7 +118,7 @@ }, { "cell_type": "code", - "execution_count": 18, + "execution_count": 7, "metadata": {}, "outputs": [], "source": [ @@ -161,33 +162,33 @@ }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "epoch: 0, batch_id: 0, loss is: [2.3079572], acc is: [0.125]\n", - "epoch: 0, batch_id: 100, loss is: [1.7078608], acc is: [0.828125]\n", - "epoch: 0, batch_id: 200, loss is: [1.5642334], acc is: [0.90625]\n", - "epoch: 0, batch_id: 300, loss is: [1.7024238], acc is: [0.78125]\n", - "epoch: 0, batch_id: 400, loss is: [1.5536337], acc is: [0.921875]\n", - "epoch: 0, batch_id: 500, loss is: [1.6908336], acc is: [0.828125]\n", - "epoch: 0, batch_id: 600, loss is: [1.5622432], acc is: [0.921875]\n", - "epoch: 0, batch_id: 700, loss is: [1.5251796], acc is: [0.953125]\n", - "epoch: 0, batch_id: 800, loss is: [1.5698484], acc is: [0.890625]\n", - "epoch: 0, batch_id: 900, loss is: [1.5524453], acc is: [0.9375]\n", - "epoch: 1, batch_id: 0, loss is: [1.6443151], acc is: [0.84375]\n", - "epoch: 1, batch_id: 100, loss is: [1.5547533], acc is: [0.90625]\n", - "epoch: 1, batch_id: 200, loss is: [1.5019028], acc is: [1.]\n", - "epoch: 1, batch_id: 300, loss is: [1.4820204], acc is: [1.]\n", - "epoch: 1, batch_id: 400, loss is: [1.5215418], acc is: [0.984375]\n", - "epoch: 1, batch_id: 500, loss is: [1.4972374], acc is: [1.]\n", - "epoch: 1, batch_id: 600, loss is: [1.4930981], acc is: [0.984375]\n", - "epoch: 1, batch_id: 700, loss is: [1.4971689], acc is: [0.984375]\n", - "epoch: 1, batch_id: 800, loss is: [1.4611597], acc is: [1.]\n", - "epoch: 1, batch_id: 900, loss is: [1.4903957], acc is: [0.984375]\n" + "epoch: 0, batch_id: 0, loss is: [2.3037894], acc is: [0.140625]\n", + "epoch: 0, batch_id: 100, loss is: [1.6175328], acc is: [0.9375]\n", + "epoch: 0, batch_id: 200, loss is: [1.5388051], acc is: [0.96875]\n", + "epoch: 0, batch_id: 300, loss is: [1.5251061], acc is: [0.96875]\n", + "epoch: 0, batch_id: 400, loss is: [1.4678856], acc is: [1.]\n", + "epoch: 0, batch_id: 500, loss is: [1.4944503], acc is: [0.984375]\n", + "epoch: 0, batch_id: 600, loss is: [1.5365536], acc is: [0.96875]\n", + "epoch: 0, batch_id: 700, loss is: [1.4885054], acc is: [0.984375]\n", + "epoch: 0, batch_id: 800, loss is: [1.4872254], acc is: [0.984375]\n", + "epoch: 0, batch_id: 900, loss is: [1.4884174], acc is: [0.984375]\n", + "epoch: 1, batch_id: 0, loss is: [1.4776722], acc is: [1.]\n", + "epoch: 1, batch_id: 100, loss is: [1.4751343], acc is: [1.]\n", + "epoch: 1, batch_id: 200, loss is: [1.4772581], acc is: [1.]\n", + "epoch: 1, batch_id: 300, loss is: [1.4918218], acc is: [0.984375]\n", + "epoch: 1, batch_id: 400, loss is: [1.5038397], acc is: [0.96875]\n", + "epoch: 1, batch_id: 500, loss is: [1.5088196], acc is: [0.96875]\n", + "epoch: 1, batch_id: 600, loss is: [1.4961376], acc is: [0.984375]\n", + "epoch: 1, batch_id: 700, loss is: [1.4755756], acc is: [1.]\n", + "epoch: 1, batch_id: 800, loss is: [1.4921497], acc is: [0.984375]\n", + "epoch: 1, batch_id: 900, loss is: [1.4944404], acc is: [1.]\n" ] } ], @@ -213,8 +214,8 @@ " avg_loss.backward()\n", " if batch_id % 100 == 0:\n", " print(\"epoch: {}, batch_id: {}, loss is: {}, acc is: {}\".format(epoch, batch_id, avg_loss.numpy(), avg_acc.numpy()))\n", - " optim.minimize(avg_loss)\n", - " model.clear_gradients()\n", + " optim.step()\n", + " optim.clear_grad()\n", "model = LeNet()\n", "train(model)" ] @@ -229,21 +230,21 @@ }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "batch_id: 0, loss is: [1.4767745], acc is: [1.]\n", - "batch_id: 20, loss is: [1.4841802], acc is: [0.984375]\n", - "batch_id: 40, loss is: [1.4997194], acc is: [1.]\n", - "batch_id: 60, loss is: [1.4895413], acc is: [1.]\n", - "batch_id: 80, loss is: [1.4668798], acc is: [1.]\n", - "batch_id: 100, loss is: [1.4611752], acc is: [1.]\n", - "batch_id: 120, loss is: [1.4613602], acc is: [1.]\n", - "batch_id: 140, loss is: [1.4923686], acc is: [1.]\n" + "batch_id: 0, loss is: [1.4915928], acc is: [1.]\n", + "batch_id: 20, loss is: [1.4818308], acc is: [1.]\n", + "batch_id: 40, loss is: [1.5006062], acc is: [0.984375]\n", + "batch_id: 60, loss is: [1.521233], acc is: [1.]\n", + "batch_id: 80, loss is: [1.4772738], acc is: [1.]\n", + "batch_id: 100, loss is: [1.4755945], acc is: [1.]\n", + "batch_id: 120, loss is: [1.4746133], acc is: [1.]\n", + "batch_id: 140, loss is: [1.4786345], acc is: [1.]\n" ] } ], @@ -287,7 +288,7 @@ }, { "cell_type": "code", - "execution_count": 21, + "execution_count": 12, "metadata": {}, "outputs": [], "source": [ @@ -315,7 +316,7 @@ }, { "cell_type": "code", - "execution_count": 22, + "execution_count": 16, "metadata": {}, "outputs": [ { @@ -323,30 +324,17 @@ "output_type": "stream", "text": [ "Epoch 1/2\n", - "step 100/938 - loss: 1.5644 - acc_top1: 0.6281 - acc_top2: 0.7145 - 14ms/step\n", - "step 200/938 - loss: 1.6221 - acc_top1: 0.7634 - acc_top2: 0.8380 - 13ms/step\n", - "step 300/938 - loss: 1.5123 - acc_top1: 0.8215 - acc_top2: 0.8835 - 13ms/step\n", - "step 400/938 - loss: 1.4791 - acc_top1: 0.8530 - acc_top2: 0.9084 - 13ms/step\n", - "step 500/938 - loss: 1.4904 - acc_top1: 0.8733 - acc_top2: 0.9235 - 13ms/step\n", - "step 600/938 - loss: 1.5101 - acc_top1: 0.8875 - acc_top2: 0.9341 - 13ms/step\n", - "step 700/938 - loss: 1.4642 - acc_top1: 0.8983 - acc_top2: 0.9417 - 13ms/step\n", - "step 800/938 - loss: 1.4789 - acc_top1: 0.9069 - acc_top2: 0.9477 - 13ms/step\n", - "step 900/938 - loss: 1.4773 - acc_top1: 0.9135 - acc_top2: 0.9523 - 13ms/step\n", - "step 938/938 - loss: 1.4714 - acc_top1: 0.9157 - acc_top2: 0.9538 - 13ms/step\n", - "save checkpoint at /Users/chenlong/online_repo/book/paddle2.0_docs/image_classification/mnist_checkpoint/0\n", + "step 200/938 - loss: 1.5219 - acc_top1: 0.9829 - acc_top2: 0.9965 - 14ms/step\n", + "step 400/938 - loss: 1.4765 - acc_top1: 0.9825 - acc_top2: 0.9958 - 13ms/step\n", + "step 600/938 - loss: 1.4624 - acc_top1: 0.9823 - acc_top2: 0.9953 - 13ms/step\n", + "step 800/938 - loss: 1.4768 - acc_top1: 0.9829 - acc_top2: 0.9955 - 13ms/step\n", + "step 938/938 - loss: 1.4612 - acc_top1: 0.9836 - acc_top2: 0.9956 - 13ms/step\n", "Epoch 2/2\n", - "step 100/938 - loss: 1.4863 - acc_top1: 0.9695 - acc_top2: 0.9897 - 13ms/step\n", - "step 200/938 - loss: 1.4883 - acc_top1: 0.9707 - acc_top2: 0.9912 - 13ms/step\n", - "step 300/938 - loss: 1.4695 - acc_top1: 0.9720 - acc_top2: 0.9910 - 13ms/step\n", - "step 400/938 - loss: 1.4628 - acc_top1: 0.9720 - acc_top2: 0.9915 - 13ms/step\n", - "step 500/938 - loss: 1.5079 - acc_top1: 0.9727 - acc_top2: 0.9918 - 13ms/step\n", - "step 600/938 - loss: 1.4803 - acc_top1: 0.9727 - acc_top2: 0.9919 - 13ms/step\n", - "step 700/938 - loss: 1.4612 - acc_top1: 0.9732 - acc_top2: 0.9923 - 13ms/step\n", - "step 800/938 - loss: 1.4755 - acc_top1: 0.9732 - acc_top2: 0.9923 - 13ms/step\n", - "step 900/938 - loss: 1.4698 - acc_top1: 0.9732 - acc_top2: 0.9922 - 13ms/step\n", - "step 938/938 - loss: 1.4764 - acc_top1: 0.9734 - acc_top2: 0.9923 - 13ms/step\n", - "save checkpoint at /Users/chenlong/online_repo/book/paddle2.0_docs/image_classification/mnist_checkpoint/1\n", - "save checkpoint at /Users/chenlong/online_repo/book/paddle2.0_docs/image_classification/mnist_checkpoint/final\n" + "step 200/938 - loss: 1.4705 - acc_top1: 0.9834 - acc_top2: 0.9959 - 13ms/step\n", + "step 400/938 - loss: 1.4620 - acc_top1: 0.9833 - acc_top2: 0.9960 - 13ms/step\n", + "step 600/938 - loss: 1.4613 - acc_top1: 0.9830 - acc_top2: 0.9960 - 13ms/step\n", + "step 800/938 - loss: 1.4763 - acc_top1: 0.9831 - acc_top2: 0.9960 - 13ms/step\n", + "step 938/938 - loss: 1.4924 - acc_top1: 0.9834 - acc_top2: 0.9959 - 13ms/step\n" ] } ], @@ -354,8 +342,8 @@ "model.fit(train_dataset,\n", " epochs=2,\n", " batch_size=64,\n", - " log_freq=100,\n", - " save_dir='mnist_checkpoint')" + " log_freq=200\n", + " )" ] }, { @@ -367,7 +355,7 @@ }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 17, "metadata": {}, "outputs": [ { @@ -375,38 +363,30 @@ "output_type": "stream", "text": [ "Eval begin...\n", - "step 10/157 - loss: 1.5238 - acc_top1: 0.9750 - acc_top2: 0.9938 - 7ms/step\n", - "step 20/157 - loss: 1.5143 - acc_top1: 0.9727 - acc_top2: 0.9922 - 7ms/step\n", - "step 30/157 - loss: 1.5290 - acc_top1: 0.9698 - acc_top2: 0.9932 - 7ms/step\n", - "step 40/157 - loss: 1.4624 - acc_top1: 0.9684 - acc_top2: 0.9930 - 7ms/step\n", - "step 50/157 - loss: 1.4771 - acc_top1: 0.9697 - acc_top2: 0.9925 - 7ms/step\n", - "step 60/157 - loss: 1.5066 - acc_top1: 0.9701 - acc_top2: 0.9922 - 6ms/step\n", - "step 70/157 - loss: 1.4804 - acc_top1: 0.9699 - acc_top2: 0.9920 - 6ms/step\n", - "step 80/157 - loss: 1.4718 - acc_top1: 0.9707 - acc_top2: 0.9930 - 6ms/step\n", - "step 90/157 - loss: 1.4874 - acc_top1: 0.9726 - acc_top2: 0.9934 - 6ms/step\n", - "step 100/157 - loss: 1.4612 - acc_top1: 0.9736 - acc_top2: 0.9936 - 6ms/step\n", - "step 110/157 - loss: 1.4612 - acc_top1: 0.9746 - acc_top2: 0.9938 - 6ms/step\n", - "step 120/157 - loss: 1.4763 - acc_top1: 0.9763 - acc_top2: 0.9941 - 6ms/step\n", - "step 130/157 - loss: 1.4786 - acc_top1: 0.9764 - acc_top2: 0.9935 - 6ms/step\n", - "step 140/157 - loss: 1.4612 - acc_top1: 0.9775 - acc_top2: 0.9939 - 6ms/step\n", - "step 150/157 - loss: 1.4894 - acc_top1: 0.9785 - acc_top2: 0.9943 - 6ms/step\n", - "step 157/157 - loss: 1.4612 - acc_top1: 0.9777 - acc_top2: 0.9941 - 6ms/step\n", + "step 20/157 - loss: 1.5246 - acc_top1: 0.9773 - acc_top2: 0.9969 - 6ms/step\n", + "step 40/157 - loss: 1.4622 - acc_top1: 0.9758 - acc_top2: 0.9961 - 6ms/step\n", + "step 60/157 - loss: 1.5241 - acc_top1: 0.9763 - acc_top2: 0.9951 - 6ms/step\n", + "step 80/157 - loss: 1.4612 - acc_top1: 0.9787 - acc_top2: 0.9959 - 6ms/step\n", + "step 100/157 - loss: 1.4612 - acc_top1: 0.9823 - acc_top2: 0.9967 - 5ms/step\n", + "step 120/157 - loss: 1.4612 - acc_top1: 0.9835 - acc_top2: 0.9966 - 5ms/step\n", + "step 140/157 - loss: 1.4612 - acc_top1: 0.9844 - acc_top2: 0.9969 - 5ms/step\n", + "step 157/157 - loss: 1.4612 - acc_top1: 0.9838 - acc_top2: 0.9966 - 5ms/step\n", "Eval samples: 10000\n" ] }, { "data": { "text/plain": [ - "{'loss': [1.4611504], 'acc_top1': 0.9777, 'acc_top2': 0.9941}" + "{'loss': [1.4611504], 'acc_top1': 0.9838, 'acc_top2': 0.9966}" ] }, - "execution_count": 23, + "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "model.evaluate(test_dataset, batch_size=64)" + "model.evaluate(test_dataset, log_freq=20, batch_size=64)" ] }, { diff --git a/paddle2.0_docs/image_search/image_search.ipynb b/paddle2.0_docs/image_search/image_search.ipynb index dfcbf2a..d9cdaa7 100644 --- a/paddle2.0_docs/image_search/image_search.ipynb +++ b/paddle2.0_docs/image_search/image_search.ipynb @@ -14,7 +14,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 简要介绍\n", + "## 简要介绍\n", "\n", "图片搜索是一种有着广泛的应用场景的深度学习技术的应用，目前，无论是工程图纸的检索，还是互联网上相似图片的搜索，都基于深度学习算法能够实现很好的基于给定图片，检索出跟该图片相似的图片的效果。\n", "\n", @@ -25,7 +25,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 环境设置\n", + "## 环境设置\n", "\n", "本示例基于飞桨开源框架2.0版本。" ] @@ -68,7 +68,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 数据集\n", + "## 数据集\n", "\n", "本示例采用[CIFAR-10](https://www.cs.toronto.edu/~kriz/cifar.html)数据集。这是一个经典的数据集，由50000张图片的训练数据，和10000张图片的测试数据组成，其中每张图片是一个RGB的长和宽都为32的图片。使用`paddle.dataset.cifar`可以方便的完成数据的下载工作，把数据归一化到`(0, 1.0)`区间内，并提供迭代器供按顺序访问数据。我们会把训练数据和测试数据分别存放在两个`numpy`数组中，供后面的训练和评估来使用。" ] @@ -159,7 +159,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 数据探索\n", + "## 数据探索\n", "\n", "接下来我们随机从训练数据里找一些图片，浏览一下这些图片。" ] @@ -221,7 +221,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 构建训练数据\n", + "## 构建训练数据\n", "\n", "图片检索的模型的训练样本跟我们常见的分类任务的训练样本不太一样的地方在于，每个训练样本并不是一个`(image, class)`这样的形式。而是（image0, image1, similary_or_not)的形式，即，每一个训练样本由两张图片组成，而其`label`是这两张图片是否相似的标志位（0或者1）。\n", "\n", @@ -358,7 +358,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 把图片转换为高维的向量表示的网络\n", + "## 把图片转换为高维的向量表示的网络\n", "\n", "我们的目标是首先把图片转换为高维空间的表示，然后计算图片在高维空间表示时的相似度。\n", "下面的网络结构用来把一个形状为`(3, 32, 32)`的图片转换成形状为`(8,)`的向量。在有些资料中也会把这个转换成的向量称为`Embedding`，请注意，这与自然语言处理领域的词向量的区别。\n", @@ -532,7 +532,7 @@ "id": "v2izWWI9PutF" }, "source": [ - "# 模型预测 \n", + "## 模型预测 \n", "\n", "前述的模型训练训练结束之后，我们就可以用该网络结构来计算出任意一张图片的高维向量表示（embedding)，通过计算该图片与图片库中其他图片的高维向量表示之间的相似度，就可以按照相似程度进行排序，排序越靠前，则相似程度越高。\n", "\n", @@ -614,7 +614,7 @@ "outputId": "3f46eb46-fa8e-4b9c-e11f-f9bbb1abf2d5" }, "source": [ - "# The end\n", + "## The end\n", "\n", "上面展示的结果当中，每一行里其余的图片都是跟第一张图片按照相似度进行排序相似的图片。你也可以调整网络结构和超参数，以获得更好的结果。" ] @@ -642,7 +642,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.7" + "version": "3.7.3" } }, "nbformat": 4, diff --git a/paddle2.0_docs/imdb_bow_classification/imdb_bow_classification.ipynb b/paddle2.0_docs/imdb_bow_classification/imdb_bow_classification.ipynb index fb082f9..87dbce9 100644 --- a/paddle2.0_docs/imdb_bow_classification/imdb_bow_classification.ipynb +++ b/paddle2.0_docs/imdb_bow_classification/imdb_bow_classification.ipynb @@ -16,7 +16,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 环境设置\n", + "## 环境设置\n", "\n", "本示例基于飞桨开源框架2.0版本。" ] @@ -46,7 +46,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 加载数据\n", + "## 加载数据\n", "\n", "我们会使用`paddle.dataset`完成数据下载，构建字典和准备数据读取器。在飞桨2.0版本中，推荐使用padding的方式来对同一个batch中长度不一的数据进行补齐，所以在字典中，我们还会添加一个特殊的``词，用来在后续对batch中较短的句子进行填充。" ] @@ -115,7 +115,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 参数设置\n", + "## 参数设置\n", "\n", "在这里我们设置一下词表大小，`embedding`的大小，batch_size，等等" ] @@ -179,7 +179,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 用padding的方式对齐数据\n", + "## 用padding的方式对齐数据\n", "\n", "文本数据中，每一句话的长度都是不一样的，为了方便后续的神经网络的计算，常见的处理方式是把数据集中的数据都统一成同样长度的数据。这包括：对于较长的数据进行截断处理，对于较短的数据用特殊的词``进行填充。接下来的代码会对数据集中的数据进行这样的处理。" ] @@ -230,7 +230,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 组建网络\n", + "## 组建网络\n", "\n", "本示例中，我们将会使用一个不考虑词的顺序的BOW的网络，在查找到每个词对应的embedding后，简单的取平均，作为一个句子的表示。然后用`Linear`进行线性变换。为了防止过拟合，我们还使用了`Dropout`。" ] @@ -260,7 +260,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 开始模型的训练\n" + "## 开始模型的训练\n" ] }, { @@ -341,7 +341,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# The End\n", + "## The End\n", "\n", "可以看到，在这个数据集上，经过两轮的迭代可以得到86%左右的准确率。你也可以通过调整网络结构和超参数，来获得更好的效果。" ] @@ -369,7 +369,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.7" + "version": "3.7.3" } }, "nbformat": 4, diff --git a/paddle2.0_docs/n_gram_model/n_gram_model.ipynb b/paddle2.0_docs/n_gram_model/n_gram_model.ipynb index e26c272..cd8e18f 100644 --- a/paddle2.0_docs/n_gram_model/n_gram_model.ipynb +++ b/paddle2.0_docs/n_gram_model/n_gram_model.ipynb @@ -16,21 +16,21 @@ "metadata": {}, "source": [ "## 环境\n", - "本教程基于paddle-develop编写，如果您的环境不是本版本，请先安装paddle-develop。" + "本教程基于paddle-2.0-beta编写，如果您的环境不是本版本，请先安装paddle-2.0-beta。" ] }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "'0.0.0'" + "'2.0.0-beta0'" ] }, - "execution_count": 23, + "execution_count": 1, "metadata": {}, "output_type": "execute_result" } @@ -51,23 +51,22 @@ }, { "cell_type": "code", - "execution_count": 24, + "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "--2020-09-09 14:58:26-- https://ocw.mit.edu/ans7870/6/6.006/s08/lecturenotes/files/t8.shakespeare.txt\n", - "正在解析主机 ocw.mit.edu (ocw.mit.edu)... 151.101.110.133\n", - "正在连接 ocw.mit.edu (ocw.mit.edu)|151.101.110.133|:443... 已连接。\n", - "已发出 HTTP 请求，正在等待回应... 200 OK\n", + "--2020-09-12 13:49:29-- https://ocw.mit.edu/ans7870/6/6.006/s08/lecturenotes/files/t8.shakespeare.txt\n", + "正在连接 172.19.57.45:3128... 已连接。\n", + "已发出 Proxy 请求，正在等待回应... 200 OK\n", "长度：5458199 (5.2M) [text/plain]\n", "正在保存至: “t8.shakespeare.txt”\n", "\n", - "t8.shakespeare.txt 100%[===================>] 5.21M 94.1KB/s 用时 70s \n", + "t8.shakespeare.txt 100%[===================>] 5.21M 2.01MB/s 用时 2.6s \n", "\n", - "2020-09-09 14:59:38 (75.7 KB/s) - 已保存 “t8.shakespeare.txt” [5458199/5458199])\n", + "2020-09-12 13:49:33 (2.01 MB/s) - 已保存 “t8.shakespeare.txt” [5458199/5458199])\n", "\n" ] } @@ -197,7 +196,7 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": 9, "metadata": {}, "outputs": [], "source": [ @@ -233,12 +232,13 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 30, "metadata": {}, "outputs": [], "source": [ "import paddle\n", "import numpy as np\n", + "import paddle.nn.functional as F\n", "hidden_size = 1024\n", "class NGramModel(paddle.nn.Layer):\n", " def __init__(self, vocab_size, embedding_dim, context_size):\n", @@ -251,7 +251,7 @@ " x = self.embedding(x)\n", " x = paddle.reshape(x, [-1, context_size * embedding_dim])\n", " x = self.linear1(x)\n", - " x = paddle.nn.functional.relu(x)\n", + " x = F.relu(x)\n", " x = self.linear2(x)\n", " return x" ] @@ -265,33 +265,34 @@ }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 31, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "epoch: 0, batch_id: 0, loss is: [10.252193]\n", - "epoch: 0, batch_id: 500, loss is: [6.894636]\n", - "epoch: 0, batch_id: 1000, loss is: [6.849346]\n", - "epoch: 0, batch_id: 1500, loss is: [6.931605]\n", - "epoch: 0, batch_id: 2000, loss is: [6.6860313]\n", - "epoch: 0, batch_id: 2500, loss is: [6.2472367]\n", - "epoch: 0, batch_id: 3000, loss is: [6.8818874]\n", - "epoch: 0, batch_id: 3500, loss is: [6.941615]\n", - "epoch: 1, batch_id: 0, loss is: [6.3628616]\n", - "epoch: 1, batch_id: 500, loss is: [6.2065206]\n", - "epoch: 1, batch_id: 1000, loss is: [6.5334334]\n", - "epoch: 1, batch_id: 1500, loss is: [6.5788]\n", - "epoch: 1, batch_id: 2000, loss is: [6.352103]\n", - "epoch: 1, batch_id: 2500, loss is: [6.6272373]\n", - "epoch: 1, batch_id: 3000, loss is: [6.801074]\n", - "epoch: 1, batch_id: 3500, loss is: [6.2274427]\n" + "epoch: 0, batch_id: 0, loss is: [10.252176]\n", + "epoch: 0, batch_id: 500, loss is: [6.6429553]\n", + "epoch: 0, batch_id: 1000, loss is: [6.801544]\n", + "epoch: 0, batch_id: 1500, loss is: [6.7114644]\n", + "epoch: 0, batch_id: 2000, loss is: [6.628998]\n", + "epoch: 0, batch_id: 2500, loss is: [6.511376]\n", + "epoch: 0, batch_id: 3000, loss is: [6.878798]\n", + "epoch: 0, batch_id: 3500, loss is: [6.8752203]\n", + "epoch: 1, batch_id: 0, loss is: [6.5908413]\n", + "epoch: 1, batch_id: 500, loss is: [6.9765778]\n", + "epoch: 1, batch_id: 1000, loss is: [6.603841]\n", + "epoch: 1, batch_id: 1500, loss is: [6.9935036]\n", + "epoch: 1, batch_id: 2000, loss is: [6.751287]\n", + "epoch: 1, batch_id: 2500, loss is: [7.1222277]\n", + "epoch: 1, batch_id: 3000, loss is: [6.6431484]\n", + "epoch: 1, batch_id: 3500, loss is: [6.6024966]\n" ] } ], "source": [ + "import paddle.nn.functional as F\n", "vocab_size = len(vocab)\n", "epochs = 2\n", "losses = []\n", @@ -303,15 +304,15 @@ " x_data = data[0]\n", " y_data = data[1]\n", " predicts = model(x_data)\n", - " y_data = paddle.reshape(y_data, ([-1, 1]))\n", - " loss = paddle.nn.functional.softmax_with_cross_entropy(predicts, y_data)\n", + " y_data = paddle.reshape(y_data, shape=[-1, 1])\n", + " loss = F.softmax_with_cross_entropy(predicts, y_data)\n", " avg_loss = paddle.mean(loss)\n", " avg_loss.backward()\n", " if batch_id % 500 == 0:\n", " losses.append(avg_loss.numpy())\n", " print(\"epoch: {}, batch_id: {}, loss is: {}\".format(epoch, batch_id, avg_loss.numpy())) \n", - " optim.minimize(avg_loss)\n", - " model.clear_gradients()\n", + " optim.step()\n", + " optim.clear_grad()\n", "model = NGramModel(vocab_size, embedding_dim, context_size)\n", "train(model)" ] @@ -326,22 +327,22 @@ }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "[]" + "[]" ] }, - "execution_count": 20, + "execution_count": 32, "metadata": {}, "output_type": "execute_result" }, { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dd3xV9f3H8dcnBMgAkgCBQIKAgogiIKQoKo46ceHAqq1W/dX5k4qttdUO7c8uOy2Oaqm1rioqLqyjOKqixRH2Hg4gYYWVQPb4/P64NxpiEMjNzb2c+34+Hnnk3nNP7vkw8s7J93zO92vujoiIBFdSrAsQEZHoUtCLiAScgl5EJOAU9CIiAaegFxEJuORYF9BU9+7dvV+/frEuQ0RknzJr1qxN7p7d3GtxF/T9+vWjoKAg1mWIiOxTzGzVrl7T0I2ISMAp6EVEAk5BLyIScAp6EZGAU9CLiAScgl5EJOAU9CIiAReYoC/aVsEfpy9j1eayWJciIhJXAhP0JeU13P3mShatLY11KSIicSUwQZ+blQpA4dbyGFciIhJfAhP0Gant6ZySTOHWiliXIiISVwIT9AB5WWkUKehFRHYSsKBP1Rm9iEgTgQr63MxUCreWowXPRUS+EKigz8tKpay6jpKKmliXIiISNwIW9GkAGr4REWkkYEGvFksRkaZ2G/Rm9qCZbTSzhY22dTWz18xsRfhz1i6+9tLwPivM7NLWLLw5XwS9zuhFRBrsyRn9Q8CpTbbdDLzh7gOBN8LPd2JmXYHbgMOBUcBtu/qB0FoyUtvTqaN66UVEGttt0Lv7O8CWJpvHAQ+HHz8MnN3Ml54CvObuW9x9K/AaX/6B0arMLNx5o6AXEWnQ0jH6nu6+Lvx4PdCzmX1ygTWNnheGt0VVXlYqRdsU9CIiDSK+GOuhpvWIGtfN7CozKzCzguLi4ojqCd00pYuxIiINWhr0G8ysF0D488Zm9ikC+jR6nhfe9iXuPtnd8909Pzs7u4UlheRmpbK9sla99CIiYS0N+mlAQxfNpcALzezzb+BkM8sKX4Q9Obwtqhp66TXnjYhIyJ60Vz4BzAQGmVmhmX0HuAM4ycxWACeGn2Nm+Wb2AIC7bwF+AXwU/rg9vC2q1EsvIrKz5N3t4O4X7eKlE5rZtwC4otHzB4EHW1xdC+RmqpdeRKSxQN0ZC9A1vQOp7dup80ZEJCxwQW9m6rwREWkkcEEPoc4bDd2IiIQEMuh105SIyBcCGvRpbCuvYXuleulFRAIZ9A2dNzqrFxEJaNA39NLrpikRkcAGvVaaEhFpEMig796pAx2TkzR0IyJCQIPezMItluqlFxEJZNBDaPhGQzciIgEO+tzMVF2MFREhwEGfl5XK5rJqyqtrY12KiEhMBTroQS2WIiKBD/pCdd6ISIILcNCrl15EBCIMejObaGYLzWyRmd3QzOvHmVmJmc0Nf9wayfH2RnanjnRol6QWSxFJeLtdYWpXzGwIcCUwCqgGXjWzf7n7yia7znD3MyKosUWSkozemSkaoxeRhBfJGf1g4AN3L3f3WuBt4NzWKat1qJdeRCSyoF8IjDGzbmaWBpwG9Glmv9FmNs/MXjGzQ5p7IzO7yswKzKyguLg4gpJ2lqcFSEREWh707r4E+C0wHXgVmAvUNdltNtDX3YcBdwPP7+K9Jrt7vrvnZ2dnt7SkL8nNTGXTjioqa5qWJSKSOCK6GOvuf3f3ke5+DLAVWN7k9VJ33xF+/DLQ3sy6R3LMvZHXVfPSi4hE2nXTI/x5P0Lj8483eT3HzCz8eFT4eJsjOebeUIuliEgEXTdhz5hZN6AGuM7dt5nZNQDufj8wHrjWzGqBCuBCd/cIj7nHPl9pSkEvIgksoqB39zHNbLu/0eN7gHsiOUYkenZJITnJ1EsvIgktsHfGArRLMnpnqvNGRBJboIMewtMV62KsiCSwwAd9nlaaEpEElwBBn8aG0iqqatVLLyKJKfBBnxuernjdtsoYVyIiEhuBD/rP56XXBVkRSVAJFPQapxeRxBT4oM/pkkK7JFPnjYgkrMAHfXK7JHK6pGjoRkQSVuCDHtRiKSKJLSGCPjcrVfPdiEjCSoigz8tKY31pJdW19bEuRUSkzSVI0KdS77C+RL30IpJ4EiPow9MVF27TOL2IJJ7ECHotQCIiCSzSFaYmmtlCM1tkZjc087qZ2V1mttLM5pvZiEiO11I5GSkkmYJeRBJTi4PezIYAVwKjgGHAGWY2oMluY4GB4Y+rgPtaerxIdEhOomeXFHXeiEhCiuSMfjDwgbuXu3st8DahdWMbGwc84iHvA5lm1iuCY7aYeulFJFFFEvQLgTFm1s3M0oDTgD5N9skF1jR6XhjethMzu8rMCsysoLi4OIKSdi0vK01DNyKSkFoc9O6+BPgtMB14FZgLtGjSd3ef7O757p6fnZ3d0pK+Um5mKutLK6mtUy+9iCSWiC7Guvvf3X2kux8DbAWWN9mliJ3P8vPC29pcXlYqdfXO+lL10otIYom066ZH+PN+hMbnH2+yyzTg2+HumyOAEndfF8kxW0otliKSqJIj/PpnzKwbUANc5+7bzOwaAHe/H3iZ0Nj9SqAcuDzC47VYw0pT6rwRkUQTUdC7+5hmtt3f6LED10VyjNbSOzMF0Bm9iCSehLgzFqBjcjt6dumoFksRSTgJE/QQ6rzRSlMikmgSKujVSy8iiSjBgj6VtdsqqKv3WJciItJmEiroc7NSqa13Nm5XL72IJI6ECnr10otIIkqwoA8vQKLOGxFJIAkV9LmZumlKRBJPQgV9Svt2dO/UUUM3IpJQEirooWFeegW9iCSOhAv63CzdNCUiiSXhgj4vK5WirRXUq5deRBJEAgZ9GtV19RTvqIp1KSIibSLxgj6zocVSwzcikhgSL+jVSy8iCSbhgj43S2f0IpJYIl1K8HtmtsjMFprZE2aW0uT1y8ys2Mzmhj+uiKzcyKV1SKZregd13ohIwmhx0JtZLnA9kO/uQ4B2wIXN7Pqkuw8PfzzQ0uO1JvXSi0giiXToJhlINbNkIA1YG3lJ0RcKeo3Ri0hiaHHQu3sR8AdgNbAOKHH36c3sep6ZzTezqWbWp7n3MrOrzKzAzAqKi4tbWtIey80M9dKHlrQVEQm2SIZusoBxQH+gN5BuZhc32e1FoJ+7DwVeAx5u7r3cfbK757t7fnZ2dktL2mN5WWlU1dazaUd11I8lIhJrkQzdnAh86u7F7l4DPAsc2XgHd9/s7g13Jj0AjIzgeK2mocVSF2RFJBFEEvSrgSPMLM3MDDgBWNJ4BzPr1ejpWU1fj5Vc9dKLSAJJbukXuvsHZjYVmA3UAnOAyWZ2O1Dg7tOA683srPDrW4DLIi85crm6O1ZEEkiLgx7A3W8Dbmuy+dZGr98C3BLJMaKhc0p7MtPaawESEUkICXdnbIPcTLVYikhiSNig101TIpIoEjboczPTKNqmXnoRCb6EDfq8rFTKq+vYWl4T61JERKIqoYMe1GIpIsGXsEHf0EuvzhsRCbqEDfq8rDRAvfQiEnwJG/QZqe3pnJKsoRsRCbyEDXoIz2Kp+W5EJOASOujzstI0dCMigZfgQR+6aUq99CISZAkf9DuqaimtqI11KSIiUZPwQQ+wRhdkRSTAEjzo1WIpIsGX0EHfMC+9Om9EJMgiCnoz+56ZLTKzhWb2hJmlNHm9o5k9aWYrzewDM+sXyfFaW2Zae9I7tFMvvYgEWiSLg+cC1wP57j4EaAdc2GS37wBb3X0AcCfw25YeLxrMTC2WIhJ4kQ7dJAOpZpYMpAFrm7w+Dng4/HgqcEJ4fdm4kZuVqvluRCTQWhz07l4E/IHQIuHrgBJ3n95kt1xgTXj/WqAE6Nb0vczsKjMrMLOC4uLilpbUIqFeeg3diEhwRTJ0k0XojL0/0BtIN7OLW/Je7j7Z3fPdPT87O7ulJbVIXlYqpZW1lFZqXnoRCaZIhm5OBD5192J3rwGeBY5ssk8R0AcgPLyTAWyO4JitLjcz1GKp4RsRCapIgn41cISZpYXH3U8AljTZZxpwafjxeOBNj7P5Br5YgERBLyLBFMkY/QeELrDOBhaE32uymd1uZmeFd/s70M3MVgLfB26OsN5Wp5WmRCTokiP5Yne/DbityeZbG71eCZwfyTGirWt6B1LaJ2noRkQCK6HvjAX10otI8CV80EO4xXKbhm5EJJgU9IRXmtIZvYgElIKe0CyWW8tr2FGleelFJHgU9HzReaOzehEJIgU9ofluAIo0Ti8iAaSgRzdNiUiwKeiB7E4d6ZicpKAXkUBS0BPqpVfnjYgElYI+LFfTFYtIQCnow3R3rIgElYI+LC8rlc1l1VRU18W6FBGRVqWgD8tTi6WIBJSCPqwh6Ndo+EZEAkZBH6aVpkQkqBT0YT06d6R9O9MFWREJnEgWBx9kZnMbfZSa2Q1N9jnOzEoa7XPrrt4v1pKSQr30arEUkaBp8QpT7r4MGA5gZu0ILQT+XDO7znD3M1p6nLaUm5VK0Tad0YtIsLTW0M0JwMfuvqqV3i8m8jLVSy8iwdNaQX8h8MQuXhttZvPM7BUzO6S5HczsKjMrMLOC4uLiVipp7+VlpVK8vYrKGvXSi0hwRBz0ZtYBOAt4upmXZwN93X0YcDfwfHPv4e6T3T3f3fOzs7MjLanFGqYrXqvhGxEJkNY4ox8LzHb3DU1fcPdSd98Rfvwy0N7MurfCMaMiLyvUYqnhGxEJktYI+ovYxbCNmeWYmYUfjwofb3MrHDMqNC+9iARRi7tuAMwsHTgJuLrRtmsA3P1+YDxwrZnVAhXAhe7ukRwzmnp2SSE5yTQNgogESkRB7+5lQLcm2+5v9Pge4J5IjtGW2iUZvTJTdEYvIoGiO2ObUIuliASNgr6J3CytNCUiwaKgbyIvK5UN2yupqlUvvYgEg4K+ibysNNxh3bbKWJciItIqFPRN5GY2LECi4RsRCQYFfRNf9NKrxVJEgkFB30SvjBTaJWleehEJDgV9E8ntksjpkqLOGxEJDAV9M3KzUnVGLyKBoaBvRp4WIBGRAFHQNyMvM5V1JRXU1NXHuhQRkYgp6JuRl5VGvcP6EvXSi8i+T0HfDE1XLCJBoqBvRq566UUkQBT0zeiVkYqZzuhFJBhaHPRmNsjM5jb6KDWzG5rsY2Z2l5mtNLP5ZjYi8pKjr0NyuJdenTciEgAtXnjE3ZcBwwHMrB1QBDzXZLexwMDwx+HAfeHPcS83M1VDNyISCK01dHMC8LG7r2qyfRzwiIe8D2SaWa9WOmZU5emmKREJiNYK+gtpfoHwXGBNo+eF4W07MbOrzKzAzAqKi4tbqaTI5GWlsb6kklr10ovIPi7ioDezDsBZwNMtfQ93n+zu+e6en52dHWlJrSI3K5XaemfD9qpYlyIiEpGIFgcPGwvMdvcNzbxWBPRp9DwvvC3ufd5Lv6X88znq483G0kqKd1RxcK8umFmsy9knVNbU8dnmMj7bVMZnm8v5bFMZG0orueW0wRzYs3OsyxOJitYI+otoftgGYBowwcymELoIW+Lu61rhmFGXl5UGxMcCJO7OhtIqFhSVsKCohIXhj43h3zZG79+N/xt3iIIqrLKmjtVbyvl0U0Ogl/HZpnI+21zGuiZ3O3fv1IHKmnomPD6baROOJqV9uxhVLRI9EQW9maUDJwFXN9p2DYC73w+8DJwGrATKgcsjOV5b6pWRAsAbSzfSNb0DvTJSyclIoUtKclTPnt2dtSWVLCgsYdHaL4J9045qAJIMDsjuxNEDujMkN4N6d+5+cyVjJ83g0tH9uOGkgXRJaR+1+uJFVW0da7aU8+mm0Fn5p+Gz9FWby1lbUoH7F/t2Te9Av25pjD6gG/27pdO3ezr9u6XTr3sanVPa8/byYi598EN+/fISbh83JHZ/KJEoMW/8HREH8vPzvaCgINZlAHD6XTNYtLZ0p21pHdqRk5FCr4wUcrqkhj43PM9IoVdGKllp7ffoh4G7U7i1goXhM/UFRSUsWlvKlrJQqLdLMgb26MSQ3AyG9O7CoXkZDO7VhbQOO/983lJWzR+mL+OJD1fTLb0DPzz1IMaPyCMpKXjDOf9ZtpHbXljEmq3lO4V5Zlp7+nVLp3/3dPqFQzz0OZ2M1N3/4PvlvxbzwLuf8vdL8zlhcM8o/glEosPMZrl7frOvKeh3rbaung3bq1hfUsG6kkrWl1Q2+lzB+pJKNmyvoq5+57/DhhuudvoB0CWFnIxU6ur9i+GXtSVsK68BIDnJOLBnZ4bkduHQ3AyG5IZCfW+GEhYWlXDrCwuZvXobw/tkcvu4Qxial9mqfyextKWsmpP+9DYZae05Y2hv+ofDvH/3dDLTOkT03lW1dZxz739ZX1rJqxPH0KNLSitVLdI2FPRRVFfvbNpRFf4B0MwPhNIKNpRUUd2oTbN9O2NQTufPA31I7wwG5XRulfHh+nrnuTlF/OaVpWwuq+KC/D7cdMogunXqGPF7x9rEKXN4ecE6Xvzu0RyU06XV33/lxu2ccfe7fK1fVx6+fFQgfyOS4PqqoG+Ni7EJrV2S0bNLCj27pECf5s+e3Z0tZdWfXwg8sGdnOiRHZ5qhpCTjvJF5nHxITya9voKH/vsZLy9Yx40nD+Jbh+9Hcrt9c3qj1xdv4IW5a7nhxIFRCXmAAT06c+sZh/Dj5xbw4HufcsWY/aNyHJG2tm9+1+9jzIxunTqGzt5zM6IW8o11TmnPT884mFdvGMPQvExum7aIM+5+lw8+2Rz1Y7e2kooafvL8Ag7K6cz/Hjcgqse6aFQfTjmkJ799dSkLi0qieizZNxVvr+KeN1dw+l0zeHfFpliXs0c0dJMA3J1XF67nly8toWhbBWcN682PTxtMTsa+MQ79o6nzeXrWGp6/7qg2ueawtayasZNmkNaxHf/67tFfuvgticfdmb16K4/MXMXLC9ZRU+d0TkmmY3ISr0w8huzOsR8a/aqhG53RJwAzY+yhvXj9+8dy/QkDeXXRer7+x7f4y1srqaqti3V5X2nGimKeLFjDVccc0GYXlrPSO/Cnbwzj001l/OJfi9vkmBKfKqrrmPLhak6/613Ou28mby7ZyMVH9OWNG4/lmWuPZHtlLTc+PY/6+vg6YW5KZ/QJaPXmcn7x0mJeW7yB/t3TufXMgzl+UI9Yl/UlZVW1nHznO3RMTuLliWPa/Gam3766lPve+pj7Lx7BqUP2ibn4pJV8tqmMR99fxdMFayitrOWgnM5cMrovZw/PJb3jF7/hPfr+Kn72/EJ+ctpgrjwmttd0dDFWdrJftzT+9u183lq2kdtfXMzl//iIEwf34GdnHEzfbumxLu9zv//3MtaWVPD01aNjcsfq9048kPdWbuJHzyxgWJ9MemXE51QY0jrq6p23lm3kkZmreHt5MclJxqlDcvj26H58rV9Ws/fGXHz4fsxYXszv/r2UI/bvxqF5GTGofPd0Rp/gqmvrefC9T7nrjRXU1jtXH7M/1x0/IOZTAXz02RbOv38mlx3Zj5+fdUjM6vh0Uxmn3zWDoXkZ/POKI2inlsvA2VpWzZMFa3js/VUUbq2gR+eOfOvwvlw0qs8e3U/RcE0ntUPomk7jM/62pD562a31JZX85pUlvDB3LWMGdueBS/PpmBybsK+sqeO0STOorqvn3zccE7NvnAZPF6zhpqnz+eGpg6Le9SNtZ37hNh6ZuYpp89ZSXVvP4f278u3R/Tj5kJ6038s25Jkfb+abD7zPeSPy+MP5w6JU8VfT0I3sVk5GCpMuPIyjB3Tnpqnzuf6JOdz7zREx6bu/8/XlfLKpjH9ecXjMQx5g/Mg83lpezJ+mL+fIA7ozfBf3S0j8q6yp46X563jk/VXMW7ONtA7t+EZ+Hpcc0Y9BOS2fFHD0Ad2YcPwA7n5zJcccmM1Zw3q3YtWRi/13kcSV8/P7UFZVy89fXMxNU+fzx/OHtekdovPWbONv73zChV/rw1EDurfZcb+KmfHrsw9l7uptTJwyh5euH0OnOPgBJHtu844q/jbjU54qWMOWsmoOyE7n/846hHNH5NK5lSYBnHjCQN5buYmfPLuAw/pk0qdrWqu8b2tQe6V8yWVH9eemUwbx3Jwibp22kLYa3quuredHz8ynR+cUfnz64DY55p7KSGvPnRcMZ82Wcn4+bVGsy5G98NL8dZx05ztMfudjvtYvi39ecTivf/9YLj2yX6uFPEByuyQmXXgYANdPmUNNHK1Op6CXZv3vcQdwzbEH8Nj7q7nj1aVtEvZ/eWslS9dv51fnDInLqZZH9e/KhK8PZOqsQqbNWxvrcuJKvF3rg9AdrNc+NovrHp9NXlYqr0w8hr9eks9RA7pHbarxPl3T+PW5hzJn9TYmvb4iKsdoCf3+Kc0yM3506iDKqmr569uf0LljMhO+PjBqx1u6vpR73lzJ2cN7x/U0wdd/fQDvrijmJ8/F36/n0VJVW8fG0tDEfQ2ztn4xaV9oMr9NO6o5ekB3bh57EIN7RWcuoj3l7kybt5afT1tEWVUdPzr1IK4c07/NrjedOaw37ywv5t63VnLUgO6MPqBbmxz3q0TUdWNmmcADwBDAgf9x95mNXj8OeAH4NLzpWXe//aveU1038aW+3vnB0/N4dk4Rt515MJcf1b/Vj1FbV8+59/2Xoq0VvPb9Y+maHtmUw9G2Zks5YyfN4KCczky56oh9dqI4CN35ub60mQAvqWR9aWhbw6I3jXXumExOo7UY0jsm8+zsIkorazhvRB7fP+lAesdgCc6N2yv56XMLmb54A8P7ZPL78UMZGIOV18qqajnz7ncpr67jlYljyGqD/9PR7LqZBLzq7uPDi4Q3d3ozw93PiPA4EiNJScbvxg+lvLqO/3txMekdkvnG1/rs/gv3wgPvfsr8whLu/eaIuA95CP16/qtzhjBxylzu/c/HTDwxer/pRMM7y4v5/b+XsWZr+efrITSWmdaenC6hAD80N3OnxXV6ZYRmam1ubPuGEw7kL2+t5B///YwX563l8qP6c+1xB+zRwi+Rcneen1vEz6ctpqKmjh+fdhDfOXr/mN33kN4xmbsuOoxz/vIeP3xmPpMvGRnTdZ1bfEZvZhnAXGB/38WbhM/of7A3Qa8z+vhUVVvHlY/M4t0Vxdx10WGcMbR12sc+Lt7B2EkzOH5QNvdfHNtvhr31vSfn8sLcIp66ejT5/brGupzdqq937nv7Y/4wfRn7d0/nqAHdPw/wnl1Cq6PldEkhtUNk908Ubavgj9OX8dycIjJS2zPh+AFcMrpv1O7L2FBayU+eW8DrSzYysm8Wvxs/lAOyO0XlWHvrgRmf8MuXlvCLs4dwyRF9o3qsqNwwZWbDgcnAYmAYMAuY6O5ljfY5DngGKATWEgr9r2xZUNDHr4rqOi598ENmr97K376dz/EHRTY/Tn29c8HkmSzfsIPXvnfMPreq0/bKGk67awb19fDKDWPi8gJyg+2VNdz41DymL97AuOG9uePcoREH+u4sWlvCHa8sZcaKTeRlpXLTKYM4c2jvVmvXdXeemV3E7S8uoqq2nptOGcTlR/WPq7uX6+udyx76iA8+2cy0CUdH1Ku/O9GavTIZGAHc5+6HAWXAzU32mQ30dfdhwN3A87so8CozKzCzguLi4ghKkmhK7dCOBy7LZ3CvLlzz2CxmfhzZ3PaPvr+Kjz7bys/OOHifC3kIzfk/6cLDWF8aGheOx84TgBUbtjPunvd4c+lGbjvzYP58wfCohzzAIb0zePQ7h/Pod0bRJaU9E6fMZdy97/HfjyOfw31dSQWXP/QRP3h6HoNyOvPqDcdwxZjYDdXsSlKS8cfzh9E5JZnvPjGbyprYzBYbSdAXAoXu/kH4+VRCwf85dy919x3hxy8D7c3sS3fBuPtkd8939/zs7OwISpJo65LSnof/ZxT7dU3jioc/Ys7qrS16nzVbyvntq0s59sBszhuR28pVtp0R+2VxwwkDmTZvLc/NKYp1OV/y0vx1jLv3PUora3n8yiO4/Kj+bT48NmZgNv/67tHcecEwtpRV882/fcDl//iQpetL9/q93J2nPlrDyX96hw8+2cJtZx7Mk1eNpn/3+JmMr6nszh354zeGs3zDDn710pKY1NDioHf39cAaMxsU3nQCoWGcz5lZjoX/V5nZqPDx9r0ljmQnXdM78M8rDqd7545c9o+PWLJu775h3Z1bnl2AAb8+99B9aly+Of97/ABG9evKz55fyKrNZbv/gjZQW1fPr19ewnWPz+agnM68dP3RjOofu+sISUnGOYfl8caNx/KT0wYza9VWxk6awU1Pz2NdScUevUfRtgou/cdH/PCZ+Rzcuwuv3jCGy4/qv0+s7XvsgdlcOaY/j76/iumL1rf58SNtrxxOqL2yA/AJcDlwAYC7329mE4BrgVqgAvi+u//3q95TY/T7jjVbyjn//pnU1tfz1NWj2X8PL4A9+dFqfvTMAn559hAujvIFqrZStK2CsX9+h/7ZnZh6zei9nhSrNW3eUcWEx+cw85PNfHt0X356+sFtsnzl3thWXs1f3vqYh977DDP4ztH9uea4A5q9zuHuTPloDb96aQn17tw89iAuPrzvPhHwjVXX1nPufe9RuLWCVyaOafVprzV7pUTNyo07uOCvM+mYnMRT14wmL+urbyBaX1LJSXe+zSG9u/D4FUfsc9+sX+Wl+eu47vHZTDh+AD84ZdDuvyAK5q3ZxrWPzWJzWTW/PudQzhuZF5M69tSaLeX86bXlPDeniKy09nz36wO5+Ii+n/9gKtxazs3PLODdlZsYvX83fjd+6D59k9onxTs44+53ozLttYJeomrx2lIunDyTrukdeOqa0fTo3PyFVXfnykcKeHflJv59wzFxtchJa/nh1Hk8PauQP18wnNMP7dWmN1M98eFqbnthET26dOT+i0cyJDc+F8FozsKiUIfOuys3sV/XNH5wyiBKK2r4zcuhMe1bThvMN0ftF4gTg4Zpr286ZRDXHd96014r6CXqZq3ayiV//4A+WWk8efURZKZ9+canF+YWMXHKXH56+mCuGBPbZdeipayqlnP+8h7LN+ygR+eOnDMil/NH5jGgR/Ta6ipr6vj5tEVM+WgNxxyYzaQLhrfJnZjR8M7yYjJIGyYAAAdTSURBVH7zytLPr/scPaA7d5x36G5/U9yXuDvXT5nLywvW8dTVoxnZN6tV3ldBL23ivZWbuPyhjxic05nHrjh8p7snN+2o4qQ/vU2/7ulMvebIuGuDa03VtfW8uXQjU2cV8p9lG6mrd4b3yWT8yDzOHNa7Ve8UXbutgmsfm8W8whImHD+A75104D7/d1tf77w4fy317pw9PHefv1jfnNLKGk6bNAOAlye2zj0YCnppM68v3sA1j81iRN8sHr581Of92hMen830RRt46fqjYzL3SKwUb6/ihblFPF1QyLIN2+mQnMQph+QwfmQeRw/oHlEo/3flJiY8MYea2nr++I1hnHxITitWLtE2a9VWvvHXmZx2aC/uunB4xD/QFPTSpqbNW8vEKXM49sBsJl+Sz3+WbeTqR2fxg5MPjOoMmPHM3VlYVMrUWWt4Yd5atpXXkNMlhXNH5DJ+ZN4edyw1vNffZnzCHa8s5YDsTvz1kpF79fUSP+55cwV/mL6c348fyvn5kc0hpaCXNjflw9Xc/OwCTj64J3PWbCO7U0demHBUTNsO40VVbR1vLAkN7by1bCP1DiP7ZjF+ZB6nD+31lb/G76iq5UdT5/PSgnWcfmgvfjd+aFwstygtU1fvfOuB95lfWMK/vnt0RD+wFfQSEw0TOrVLMl647qh9qgukrWwsreS5OUU8PauQlRt3kNI+iVMPyWH8yD4ceUC3nbpMPi7ewdWPzuKT4h3cMnYwV4xp+7tcpfWtK6lg7KQZ5GWl8uy1R7X4ngcFvcTMlA9X0yE5iXNHxHc/d6y5O/MKS5g6aw3T5q6ltLKW3hkpnDcyj/NG5LFsw3ZufGoeHZKTuOeiwzgyTtbTldYxfdF6rnp0FleO6c9PTj+4Re+hoBfZh1TW1PH6kg08XVDIjBXF1Ie/RYf1yeS+b42IyYIeEn2/emkxA3p04oKv7deir1fQi+yj1peEhnbq6uu58pj9ozanu+z7ornClIhEUU5GCtced0Csy5B9nFogREQCTkEvIhJwCnoRkYBT0IuIBJyCXkQk4BT0IiIBp6AXEQk4Bb2ISMDF3Z2xZlYMrIrgLboDm1qpnGiI9/og/muM9/og/muM9/pANe6tvu6e3dwLcRf0kTKzgl3dBhwP4r0+iP8a470+iP8a470+UI2tSUM3IiIBp6AXEQm4IAb95FgXsBvxXh/Ef43xXh/Ef43xXh+oxlYTuDF6ERHZWRDP6EVEpBEFvYhIwAUm6M3sVDNbZmYrzezmWNfTlJn1MbP/mNliM1tkZhNjXVNzzKydmc0xs3/FupbmmFmmmU01s6VmtsTMRse6psbM7Hvhf9+FZvaEmaXEQU0PmtlGM1vYaFtXM3vNzFaEP2fFYY2/D/87zzez58wsM57qa/TajWbmZha3C/kGIujNrB1wLzAWOBi4yMxatsJu9NQCN7r7wcARwHVxWCPARGBJrIv4CpOAV939IGAYcVSrmeUC1wP57j4EaAdcGNuqAHgIOLXJtpuBN9x9IPBG+HksPcSXa3wNGOLuQ4HlwC1tXVQjD/Hl+jCzPsDJwOq2LmhvBCLogVHASnf/xN2rgSnAuBjXtBN3X+fus8OPtxMKqNzYVrUzM8sDTgceiHUtzTGzDOAY4O8A7l7t7ttiW9WXJAOpZpYMpAFrY1wP7v4OsKXJ5nHAw+HHDwNnt2lRTTRXo7tPd/fa8NP3gbw2L+yLWpr7OwS4E/ghENddLUEJ+lxgTaPnhcRZiDZmZv2Aw4APYlvJl/yZ0H/a+lgXsgv9gWLgH+HhpQfMLD3WRTVw9yLgD4TO7tYBJe4+PbZV7VJPd18Xfrwe6BnLYvbA/wCvxLqIxsxsHFDk7vNiXcvuBCXo9xlm1gl4BrjB3UtjXU8DMzsD2Ojus2Jdy1dIBkYA97n7YUAZsR9y+Fx4nHscoR9IvYF0M7s4tlXtnod6rOP2jNTMfkJo6POfsa6lgZmlAT8Gbo11LXsiKEFfBPRp9DwvvC2umFl7QiH/T3d/Ntb1NHEUcJaZfUZo6OvrZvZYbEv6kkKg0N0bfhOaSij448WJwKfuXuzuNcCzwJExrmlXNphZL4Dw540xrqdZZnYZcAbwLY+vm34OIPQDfV74eyYPmG1mOTGtaheCEvQfAQPNrL+ZdSB0AWxajGvaiZkZobHlJe7+p1jX05S73+Luee7ej9Df35vuHldno+6+HlhjZoPCm04AFsewpKZWA0eYWVr43/sE4uhicRPTgEvDjy8FXohhLc0ys1MJDSWe5e7lsa6nMXdf4O493L1f+HumEBgR/j8adwIR9OELNhOAfxP6xnrK3RfFtqovOQq4hNCZ8tzwx2mxLmof9F3gn2Y2HxgO/DrG9Xwu/JvGVGA2sIDQ91fMb5E3syeAmcAgMys0s+8AdwAnmdkKQr+J3BGHNd4DdAZeC3+/3B9n9e0zNAWCiEjABeKMXkREdk1BLyIScAp6EZGAU9CLiAScgl5EJOAU9CIiAaegFxEJuP8HvHiKw1jJ554AAAAASUVORK5CYII=\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXxU9b3/8dcn67AlYQlJ2F0QRVDQCNHq7aK2ar2irW3BfQG0tdVut623rf1da3vb6217a/XqRbAuCNparbZalaqtVmUJm4KIguwSCBASAknI8vn9MRMaYlCYmeQMZ97PxyOPzJxzMufDkvec+ZzvOV9zd0REJLwygi5AREQ6l4JeRCTkFPQiIiGnoBcRCTkFvYhIyGUFXUB7/fr182HDhgVdhojIYWXhwoXb3L2wo3UpF/TDhg2jvLw86DJERA4rZrbuQOvUuhERCTkFvYhIyCnoRURCTkEvIhJyCnoRkZBT0IuIhJyCXkQk5EIT9Jt21vHL51eydtvuoEsREUkpoQn6nXv2cseLq1ixuSboUkREUkpogr44LwLA5ur6gCsREUktoQn6Pj1yyMnMYEuNgl5EpK3QBL2ZUZSfS4WCXkRkP6EJeoi2byrUuhER2U+ogr4oL6IjehGRdkIV9CX50SN6dw+6FBGRlBGqoC/Ki9DQ1EJ1XWPQpYiIpIxQBX1xfnSIpdo3IiL/FK6gj42l1wlZEZF/+sigN7P7zGyrmS1rs6yPmc0xs3dj33sf4GevjG3zrpldmczCO7LviF5BLyKyz8Ec0d8PnNNu2feAF9x9OPBC7Pl+zKwP8CNgPDAO+NGB3hCSpX8vtW5ERNr7yKB395eBHe0WTwAeiD1+ALiwgx/9DDDH3Xe4exUwhw++YSRVTlYG/Xrm6OpYEZE24u3RF7n75tjjCqCog20GAhvaPN8YW9apinTRlIjIfhI+GevRQesJDVw3s6lmVm5m5ZWVlQnVU5If0Y3NRETaiDfot5hZCUDs+9YOttkEDG7zfFBs2Qe4+zR3L3X30sLCwjhLiirKi6h1IyLSRrxB/xTQOormSuDJDrZ5Dvi0mfWOnYT9dGxZpyrOi1C1p5H6xubO3pWIyGHhYIZXzgZeB0aY2UYzuxb4GXC2mb0LnBV7jpmVmtl0AHffAfwYWBD7ujW2rFMVxYZYbq1p6OxdiYgcFrI+agN3n3SAVWd2sG05MLnN8/uA++KuLg4l+a0TkNQxpG/3rty1iEhKCtWVsdDm6lj16UVEgBAGfWvrRidkRUSiQhf0vXKz6J6TSUW1evQiIhDCoDczijXEUkRkn9AFPURvbra5ui7oMkREUkI4gz4vwhYNrxQRAUIa9EX50dZNS4umFBQRCWXQF+dFaGpxtu/eG3QpIiKBC2fQawISEZF9whn0umhKRGSfcAa9JgkXEdknlEHfr2cumRnGFrVuRETCGfSZGUb/XrmagEREhJAGPWgCEhGRVqEN+uK8iHr0IiKEOejzI+rRi4gQ8qDf1dBEbUNT0KWIiAQqvEGfp4umREQgwaA3s5vMbJmZLTezr3ew/hNmVm1mS2JftySyv0NRlKcJSERE4CDmjD0QMxsFTAHGAXuBZ83sz+6+qt2mr7j7+QnUGBfdBkFEJCqRI/rjgHnuvsfdm4C/A59LTlmJ020QRESiEgn6ZcAZZtbXzLoD5wGDO9juVDNbamZ/MbPjO3ohM5tqZuVmVl5ZWZlASf/ULSeT/G7ZOqIXkbQXd+vG3VeY2c+B54HdwBKgud1mi4Ch7l5rZucBfwSGd/Ba04BpAKWlpUm7ibzG0ouIJHgy1t1nuPvJ7v4vQBXwTrv1Ne5eG3v8DJBtZv0S2eehaJ2AREQknSU66qZ/7PsQov35We3WF5uZxR6Pi+1veyL7PBTFeblq3YhI2ou7dRPzBzPrCzQCN7j7TjO7HsDd7wEuBr5sZk1AHTDR3btsfr/i/G5U1jbQ2NxCdmZoLxkQEflQCQW9u5/RwbJ72jy+E7gzkX0kojgvgjtU7mpgQEG3oMoQEQlUqA9zi/NzAQ2xFJH0Fuqg33d1rPr0IpLGQh30JfnRdo0mIBGRdBbqoO/dPZucrAwNsRSRtBbqoDczivJy1aMXkbQW6qCH2NWxat2ISBoLf9Dnd1PrRkTSWviDPi+XzdX1dOF1WiIiKSX0QV+UF6GhqYXqusagSxERCUTog37fBCRq34hImgp90JdopikRSXOhD/oiTRIuImku9EHfv5daNyKS3kIf9DlZGfTrmaMhliKStkIf9BBt36h1IyLpKi2CviQ/ohubiUjaSougL8rT3LEikr4SnTP2JjNbZmbLzezrHaw3M7vDzFaZ2RtmdlIi+4tXcV6Eqj2N1Dc2B7F7EZFAxR30ZjYKmAKMA04Ezjezo9ttdi4wPPY1Fbg73v0loig2ln5rTUMQuxcRCVQiR/THAfPcfY+7NwF/Bz7XbpsJwIMeNRcoMLOSBPYZlxJdHSsiaSyRoF8GnGFmfc2sO3AeMLjdNgOBDW2eb4wt24+ZTTWzcjMrr6ysTKCkjhXHLpraXF2X9NcWEUl1cQe9u68Afg48DzwLLAHiaoK7+zR3L3X30sLCwnhLOqDW1o1OyIpIOkroZKy7z3D3k939X4Aq4J12m2xi/6P8QbFlXapXbhbdczKpqFaPXkTST6KjbvrHvg8h2p+f1W6Tp4ArYqNvyoBqd9+cyD7jYWYU52uIpYikp6wEf/4PZtYXaARucPedZnY9gLvfAzxDtHe/CtgDXJ3g/uJWnBdRj15E0lJCQe/uZ3Sw7J42jx24IZF9JEtxXoR5a3YEXYaISJdLiytjIXpCdktNPS0tmlJQRNJL2gR9SX6EphZn++69QZciItKl0iboNQGJiKSrtAn61oumdHWsiKSb9Al63QZBRNJU2gR9v565ZGYYW9S6EZE0kzZBn5lh9O+VqwlIRCTtpE3QgyYgEZH0lFZBX5wXUY9eRNJOegV9fkQ9ehFJO2kX9LsamqhtaAq6FBGRLpNeQa+LpkQkDaVV0LdeHasTsiKSTtIq6PddNKUjehFJI+kV9LoNgoikobQK+m45meR3y9YRvYiklbQKetBYehFJP2kX9EWaO1ZE0kyik4N/w8yWm9kyM5ttZpF2668ys0ozWxL7mpxYuYkryYuodSMiaSXuoDezgcCNQKm7jwIygYkdbPqou4+JfU2Pd3/JUpQfobK2gcbmlqBLERHpEom2brKAbmaWBXQH3k+8pM5VnBfBHSp3NQRdiohIl4g76N19E/DfwHpgM1Dt7s93sOnnzewNM3vMzAZ39FpmNtXMys2svLKyMt6SDkpxfi6gIZYikj4Sad30BiYARwADgB5mdlm7zf4EDHP3E4A5wAMdvZa7T3P3UncvLSwsjLekg1Kc1w1ANzcTkbSRSOvmLGCNu1e6eyPwOHBa2w3cfbu7t/ZIpgMnJ7C/pNCUgiKSbhIJ+vVAmZl1NzMDzgRWtN3AzEraPL2g/fog9O6eTU5WhkbeiEjayIr3B919npk9BiwCmoDFwDQzuxUod/engBvN7ILY+h3AVYmXnBgzoygvV0f0IpI24g56AHf/EfCjdotvabP+ZuDmRPbRGYo1ll5E0kjaXRkLUJzfTVfHikjaSM+gz8tlc3U97h50KSIinS4tg74oL0JDUwvVdY1BlyIi0unSMug1xFJE0klaBn2JZpoSkTSSlkFfpEnCRSSNpGXQ9++l1o2IpI+0DPqcrAz69czREEsRSQtpGfQQPSGr1o2IpIP0Dfq8CBU1uie9iIRf2gZ9UV6Eiuq6oMsQEel0aRv0xXkRqvY0Ut/YHHQpIiKdKn2DPjaWfqvaNyIScmkf9BpiKSJhl75BH7toarP69CIScmkb9EWxI3qNpReRsEvboO+Vm0WPnEwqqtWjF5FwSyjozewbZrbczJaZ2Wwzi7Rbn2tmj5rZKjObZ2bDEtlfMpkZRfkRHdGLSOjFHfRmNhC4ESh191FAJjCx3WbXAlXufjTwK+Dn8e6vMxTnRdSjF5HQS7R1kwV0M7MsoDvwfrv1E4AHYo8fA840M0twn0lTnBdhi4ZXikjIxR307r4J+G9gPbAZqHb359ttNhDYENu+CagG+rZ/LTObamblZlZeWVkZb0mHrLV109KiKQVFJLwSad30JnrEfgQwAOhhZpfF81ruPs3dS929tLCwMN6SDllJfoSmFmf77r1dtk8Rka6WSOvmLGCNu1e6eyPwOHBau202AYMBYu2dfGB7AvtMKk1AIiLpIJGgXw+UmVn3WN/9TGBFu22eAq6MPb4YeNHdU6ZP0nrRlK6OFZEwS6RHP4/oCdZFwJux15pmZrea2QWxzWYAfc1sFfBN4HsJ1ptUug2CiKSDrER+2N1/BPyo3eJb2qyvB76QyD46U7+euWRmGFvUuhGREEvbK2MBMjOM/r1ydUQvIqGW1kEPrROQKOhFJLzSPuijUwoq6EUkvBT0+RH16EUk1BT0+RF2NTRR29AUdCkiIp1CQa+LpkQk5NI+6FuvjtXtikUkrNI+6EvydUQvIuGW9kGvq2NFJOzSPugj2Znkd8vWEb2IhFbaBz1oLL2IhJuCnthYegW9iISUgp7YEb1aNyISUgp6olMKVtY20NjcEnQpIiJJp6AnekTvDpW7NFG4iISPgh4ozs8FNMRSRMJJQQ8U53UD0M3NRCSUFPTooikRCbe4g97MRpjZkjZfNWb29XbbfMLMqttsc8uBXi9Ivbtnk5OVoZE3IhJKcc8Z6+4rgTEAZpYJbAKe6GDTV9z9/Hj30xXMjKI8TSkoIuGUrNbNmcBqd1+XpNfrciV53XRELyKhlKygnwjMPsC6U81sqZn9xcyO72gDM5tqZuVmVl5ZWZmkkg5Nka6OFZGQSjjozSwHuAD4fQerFwFD3f1E4DfAHzt6DXef5u6l7l5aWFiYaElxKc7LZXN1Pe4eyP5FRDpLMo7ozwUWufuW9ivcvcbda2OPnwGyzaxfEvaZdEV5ERqaWqiuawy6FBGRpEpG0E/iAG0bMys2M4s9Hhfb3/Yk7DPpSvKjY+l1QlZEwiahoDezHsDZwONtll1vZtfHnl4MLDOzpcAdwERP0d7IvqtjdUJWREIm7uGVAO6+G+jbbtk9bR7fCdyZyD66SpEmCReRkNKVsTH9e+nqWBEJJwV9TE5WBv165mqIpYiEjoK+jeL8XLVuRCR0FPRtROeO1T3pRSRcFPRtFOVFqKiuC7oMEZGkUtC3UZwXoWpPI/WNzUGXIiKSNAr6NlrvS79V7RsRCREFfRuagEREwkhB30Zx7KKpzerTi0iIKOjbKIod0WssvYiEiYK+jV65WfTIyaSiWj16EQkPBX0bZqYJSEQkdBT07RTnRdSjFwmhdB42raBvpzgvwhYNrxQJjeYW5+bH3+CkH89hyYadQZcTCAV9O8Wx1k1LS0reNl9EDkFTcwvf+t0SZs/fQIYZ1z+0kMpd6Xcgp6Bvpzg/QlOLs3333qBLEZEE7G1q4cZHFvPHJe/zb58ZwaPXlbGzbi83PLyIvU0tQZfXpRT07bROQKITsiKHr/rGZr7y8EKeebOCH3z2OG745NEcPyCfn3/+BOav3cFtT78VdIldKu6gN7MRZrakzVeNmX293TZmZneY2Soze8PMTkq85M71z4umFPQih6O6vc1MebCcv67Yyo8vHMXkM47ct27CmIFMOeMIHnx9Hb9bsCHAKrtW3FMJuvtKYAyAmWUCm4An2m12LjA89jUeuDv2PWWV6DYIIoet3Q1NXPvAAuat2cF/ff4EvnjK4A9s891zjmXF5l384I/LGF7Uk7FDegdQaddKVuvmTGC1u69rt3wC8KBHzQUKzKwkSfvsFH175pKZYWzREb3IQXvr/Rqq6xoDraGmvpHLZ8xjwdoq/udLYzoMeYCszAx+M2ksRfm5XD9zIVt3hf93PVlBPxGY3cHygUDbz0cbY8v2Y2ZTzazczMorKyuTVFJ8MjOM/r1ydUQvKWNj1R7mvrc96DIO6HcLNnDeHa9w5i/+zlNL38e960es7dyzl8umz+PNTdXcOWksE8Z8IGb207tHDv93WSnVdY18ZWb4T84mHPRmlgNcAPw+3tdw92nuXurupYWFhYmWlLDoBCQKegneqq27uPCu15g4bS4Pvb426HI+4LGFG/nu429w2lF9GVAQ4cbZi7n6/gVs2LGny2rYVtvAxGlzeXvzLu657GTOHX1wTYORA/L4r4tPpHxdFbf+eXknVxmsZBzRnwsscvctHazbBLT9/DQotiylleRHdEQvgVu1dRcTp80D4F+OKeSHTy5n+ivvBVzVPz2xeCP/9thSTjuqL/dddQpPfOVj3HL+SOav2cGnf/Uy9778Hk3NnXukvKWmnonT5rJ2+25mXFXKmccVHdLPX3DiAK77+JHMnLueR+av76Qqg5eMoJ9Ex20bgKeAK2Kjb8qAanffnIR9dqqivIh69BKoVVtr94X8I1PLmHFlKZ8dXcJtT6/grpdWBVwdPLX0fb71u6WUHdGX6VecQiQ7k8wM45rTj2DONz/OaUf15SfPrGDCXa/y5sbqTqlh0846vvR/r7N5Zx33Xz2OM4bH1w34zmeO5Yzh/bjlyeUsWl+V5CpTQ0JBb2Y9gLOBx9ssu97Mro89fQZ4D1gF3At8JZH9dZXi/Ai7GpqobWgKuhRJQ6sra5l071zAeWTqeI7u35PszAx+PXEMF44ZwO3PreRXc94JpBcO8PQbm/nGo0soHdaHGVeV0i0nc7/1Awu6Mf3KUv730pOo3NXAhLv+wa1/eovdSfx9Wr99D1+853W21+7lwWvHU3Zk37hfKzPD+M2ksRTnR7j+oYVsDeGn+biHVwK4+26gb7tl97R57MANiewjCK1j6Suq6zm6f8+Aq5G29ja1sHTjTlpanMwMw8zIsOgva4bFvjIg0zpYlxF9vm+72LqcrAy65yT0q5A0qytrmTRtLu7O7CllHN2/1751WZkZ/OKLY6Kh/8K77G1u4TufGYGZdVl9zy7bzI2PLGbs4AJ+e9UpB/x7MzPOG13C6cP78V/Pvs19r67hueUV3Drh+ENur7S3urKWS++dR31TM7OmlDF6UH5CrwdQ0D2HaVeczEV3vcaXH17E7Cll5GSF53rS1PjfnWLaXh2roE8dtQ1NXHt/dIx0MpnBlacO4+bzjiU3K/Ojf6CTvBcL+eYWZ/bUMoYX9frANpkZxs8/fwI5WRnc/bfVNDS28MPzj+uSsH9+eQVfnbWYEwflc/814+iR+9HxkRfJ5rYLR3PhmIHc/PibXPtAOZ8dXcKP/nUk/WO/Z4diZcUuLp0+b98b4XElefH8UTp0bHEet3/hBL46azH/70/L+elFo5P22kFT0Hdg30VT6tOnjOo9jVz52/m8uamaWyccz1GFPWlxp7nFcWff45bY4+gXtLR88HFz7LnHfmZlxS7uf20t5et2cOekkxjWr0eX//nWbNvNpHv/GfLHdBDyrTIyjNsuHEVOVgb3vbqGvc3N3HrBKDIyOi/sX1ixhRtmLeL4gdGQ73kQId9W6bA+PH3jGUx7eTV3vLiKl9+t5LvnHMsl44YcdN3LNlVz+Yx5ZGdmMGvq/p92kuX8Ewaw/P0a7v7bakYPzGfSuCFJ30cQFPQd0CThqWV7bQOXz5jPqq213H3pSXz6+OKk7+OTx/bnO4+9wfm/+Qc//dxoLjhxQNL3cSBrtu1m4rTXaWp2Zk358JBvZWbccv5IcrMyuefvq2lscn76udFkdkLYv/T2Vr48cxHHleTx4DXjyItkx/U6OVkZfPVTwzlvdAnff2IZP/jjMp5YvIn//Nzoj/wzL15fxZX3zadXJJuHJ4/v1Dfjb396BG+9X8MtTy7jmKJenDz08L9yNjxNqCSKZGeS3y076Tc2q29sTstbpCZiS009X5o2l9WVtdx7ZWmnhDzAZ44v5pmbzuCYop7cOHsxNz/+BnV7O3+iirXbdjNp2lwaYyE/ovjgj1LNjO+eM4IbzxzOo+Ub+PbvlyZ9OOPf36nkupkLGV7Uk4euGU9+t/hCvq0jC3sya8p4br/4BFZX1vLZO17hF8+vPODEIAvW7uDyGfMp6J7Do9eVdfonrswM446JYxlQ0I0vz1wYihscWlBn7g+ktLTUy8vLgy6Dz/zqZYb07c69V5Qm/FqrK2t5eO56Hlu4gZr6Jgb36cYpQ/tQOqwPpwzrzVGFPTv1Y/fhamPVHi6dPo9tuxqYcdUpCY2sOFiNzS38cs473P231RxT1JO7Ljmpw155MqzbvpuJ0+bS0NTCrCnjObY4/n7zXS+t4vbnVvLZ0SX8z8ToCdtE/ePdbVz7wIJoME8eT+8eOQm/Znvbaxu47ekVPLF4E0f068FPLhrFaUf127f+1VXbmPxAOSUFEWZNLtv3absrrKzYxUX/+yojinvxyNSyQM/fHAwzW+juHQaWgv4ArrxvPlV79vLUV0+P6+ebmlv464otPDR3Ha+u2k52pnHuqBJGDcxj8fqdLFi7g2210XveF3TPpnRo733BP2pgfsr/p+psa7bt5tJ751Lb0MQD14zr8htP/f2dSr756BJ2723i1gmj+MLJg5J6wrM15OsboyNHknFScfor73Hb0ys4e2QRd14yNqH/Q6+t3sY19y9gWN8ezJpSRp9OCPm2Xnm3ku8/sYz1O/bwhZMH8e/nHceSjTu57qGFHNG3BzMnj6ewV26n1tCRZ97czFceXsSkcYP5z8+d0OX7PxQK+jh897E3eGnlVuZ//6xD+rktNfXMnr+eR+ZvoKKmnoEF3bhk/BC+WDp4v/+o7s7a7XtYsHYH5Wt3UL62ive27QYgNyuDEwcXcMqwaPifNKR3Uj4yHy7e2RIdWdHc4jx07TiOH5D48Ll4bK2p56ZHlvD6e9u5cMwAbrto9CGfhOzI+u17mDjtdeoam3l4chkjByRv5MiDr6/llieX84kRhdxz2clEsg897Oe+t52rf7uAwX26MXtKGX17dk3A1u1t5o4X32Xay++R3y2bXfWNHFPUi4euHd/pbzQf5vbn3uaul1bzk4tGcen4oYHV8VEU9HH45Zx3+M2L7/LObed+5Mdgd+f11duZOW8dzy3fQnOL8/FjCrm8bCifPLb/QZ8g21bbQPnaKsrX7mDBuiqWb6qmqcUxgxFFvThlWB9Kh/XmlGF9GFDQLRl/zJSz38iKKeM7ZWTFoWhuce58cRW/fuEdhvbtwZ2XjE3ojac15Pc0NjMrySHfavb89fz7E2/ysaP6Me2Kkw/pGoEFa3dw5X3zKcmP8MjUUwM5il6xuYYf/nEZmRnGtCtKAz/IaW5xrn1gAa+u2sbsKWWUDusTaD0HoqCPw6x50V+W1773qQOGanVdI48v2sjMuetYXbmbgu7ZfKl0MJeMH8LQvomfMNqzt4klG3ZSvraKBWt3sGhdFbtjJwgHFnSjNHbEP3ZwASOKeyWlLxukheuquOq388mLZDNryvik/B0my9z3tnPTI4up2t3ID84/jsvLhh5yK2fDjj1MnDaX3XubeHjy+E79pPLYwo1857GllA7rw31XnXJQn0QWrqviihnzKMqL8MjUsrjGuYdVdV0jE+78B7v3NvPnr52+71qbVKKgj8NLb2/l6vsX8PhXTuOkdv3hZZuqmTl3HU8ueZ+6xmbGDingsvFD+ewJJXF9VD5YTc0tvF2xa98R/4I1O9gaG8UTyc5g1IB8xgwuYMyQAsYMLmBgQbcuvWoyEa+tjp50K8qLMHPyeAam4CeW7bUNfPv3S3lpZSXnHF/Mzy8+4aCPNltDvrYhGvKjBnZ+O+qppe/zjUeXcMKgfO6/etyH1rp4fRWXz5hPv545PDL11C496Xm4eGfLLi6661WGF/Xi0etS7+Ssgj4Ob71fw3l3vMLdl57EuaNLqG9s5uk3NvPQ3HUs2bCTSHYGF44ZyGVlQ7vkl7Yj7s7GqjoWb9jJkvU7WbKhimXv1+y7t3a/nrmMGVzA2FjwnzAon15xjoHuTC+t3Mr1Dy1kaN/uzJw8nv69UjdkWlqcGf9Yw8+ffZvi/Ai/mTT2I08UBxHyrZ5dtpmvzV7MscV5PHTtOAq6f7DX/cbGnVw6fR69Y8MXS/JT7002VTy7bDPXz1zEl0oH87PPj06pAykFfRx27N7LST+ew5QzjiDDjN+Vb6BqTyNHFvbgsvFD+fzJgwLvHXZkb1MLb1fUsKQ1/Dfu5L3K6EleMzi6sOd+R/0jinqRFWDLpzWIRhT34sFrgj3pdigWr6/ia7MXU1Fdz3fOGcHk04/scIjsxqpoyNfUNTJrSlkgBwUvrNjCl2cu4qj+PZl57bj9Tq4u21TNJffOJa9bNo9ed2pKfpJKNb94fiW/eXEVP75wFJeXpc7JWQV9HNydET98lr1NLWRmGJ8eWcTlZUM59ai+KfUufjCq9zSydOPOaPjHvnbsjg7t7JadyeiB+YwZUsCJg6JH/111ovePizfxrd8v5cRB+fz2I1oLqai6rpHv/eEN/rKsgk+OKOQXXxyz3xtV25B/eHJybr4Vr5ffqWTKg+UM6dOdh6dEPzUtf7+aS+6dR8/cLB6ZWsbgPt0Dq+9w0tLiTH6wnJffqeT04f0oyY9QlBehOC9CUX70e3FehILu2V2aFQr6ON39t9U0NDUz8ZQhoepZujsbdtSxeEPVvuBf3qblc2RhD84+roizRxYxdkjvTrmsvnVkSNkRfZl+ZelB3SArFbk7M+eu48d/XkHvHtn8euJYyo7sy6addUyc9jrVexqZOXk8JwwqCLpUXlu9jWvvL6ckP8IP/3Uk33x0CZHsTB6deipD+irkD0V1XSP/8dRy3tm6i4rqBrbvbqB9lOZmZbR7A8iNPo+9GRTFvpJ1l0wFvXykvU0trNhcw8J1Vby0citz39tOY7PTt0cOnzq2P2eNLOKM4f2Scjvf+/6xhlv//BafHFHI3XGO9U41y9+v5muzFrN2+26u+/hRPP3GZqr27OXhFAn5VgvW7uDq3y6gtqGJ4tjomiBu4hY2jc0tbN3VQEV1HRXVDVTU1LOlpp6K6vr9Hjd0MDdtv545+94Qzhjej6s+dkRcNSjo5ZDV1Dfy95WV/HXFFl56eys19fCw+7EAAAbPSURBVE3kZmVw+tH9OGtkEWce1z+uk6atl+qfO6qYX08cG6p7ftc2NPHD2I26ekWymHnteE4cnDoh32rJhp3c/bdVfPecYzmyULfh7iruTnVdIxWtbwDt3gQqaho49ci+3PKvI+N6fQW9JKSxuYUFa3YwZ8UW5ry1hY1VdQCMGVzA2SOjLZ7h/Xt+aD/S3bn9uZX8799Wc9HYgdx+8QmBngTuLO7OCyu2MqRv94O6C6VIsnRa0JtZATAdGAU4cI27v95m/SeAJ4E1sUWPu/utH/aaCvrU5u6s3LKLOcu38NcVW1gamw90aN/unHVcEWcdV8Qpw3rvF+Luzn/86S3uf20tk8YN4ScXdu6900XSUWcG/QPAK+4+3cxygO7uvrPN+k8A33b38w/2NRX0h5eK6npeeDt6pP/aqu3sbW4hv1s2nzq2P2ePLOL04f346dMreGTBBq752BFdNhuSSLr5sKCP+8yameUD/wJcBeDue4G98b6eHJ6K8yNcOn4ol44fyu6GJl55t5Ln39rCi29v5YnFmzADd/jap47mm2cfo5AXCUAiQyiOACqB35rZicBC4KbYhOFtnWpmS4H3iR7dL09gn5LCeuRmcc6oEs4ZVUJTcwsL11Xx4sqtHFXYky+WDg66PJG0FXfrxsxKgbnAx9x9npn9Gqhx9x+22SYPaHH3WjM7D/i1uw/v4LWmAlMBhgwZcvK6deviqklEJF19WOsmkWEPG4GN7j4v9vwx4KS2G7h7jbvXxh4/A2SbWT/acfdp7l7q7qWFhYUJlCQiIu3FHfTuXgFsMLMRsUVnAm+13cbMii3WlDWzcbH9bY93nyIicugSvczxa8DDsRE37wFXm9n1AO5+D3Ax8GUzawLqgImeagP3RURCThdMiYiEQGf16EVE5DCgoBcRCTkFvYhIyCnoRURCLuVOxppZJZDIFVP9gG1JKqczpHp9kPo1pnp9kPo1pnp9oBoP1VB37/BCpJQL+kSZWfmBzjynglSvD1K/xlSvD1K/xlSvD1RjMql1IyIScgp6EZGQC2PQTwu6gI+Q6vVB6teY6vVB6teY6vWBakya0PXoRURkf2E8ohcRkTYU9CIiIReaoDezc8xspZmtMrPvBV1Pe2Y22MxeMrO3zGy5md0UdE0dMbNMM1tsZn8OupaOmFmBmT1mZm+b2QozOzXomtoys2/E/n2XmdlsM4ukQE33mdlWM1vWZlkfM5tjZu/GvvdOwRpvj/07v2FmT5hZQSrV12bdt8zMO5prI1WEIujNLBO4CzgXGAlMMrORwVb1AU3At9x9JFAG3JCCNQLcBKwIuogP8WvgWXc/FjiRFKrVzAYCNwKl7j4KyAQmBlsVAPcD57Rb9j3ghdiMby/Engfpfj5Y4xxglLufALwD3NzVRbVxPx+sDzMbDHwaWN/VBR2KUAQ9MA5Y5e7vxSYpfwSYEHBN+3H3ze6+KPZ4F9GAGhhsVfszs0HAZ4HpQdfSkTYT0s+A6IT07r4z2Ko+IAvoZmZZQHeicyUHyt1fBna0WzwBeCD2+AHgwi4tqp2OanT35929KfZ0LjCoywv7Zy0d/R0C/Ar4DpDSo1rCEvQDgQ1tnm8kxUK0LTMbBowF5n34ll3uf4j+p20JupADaDsh/WIzm25mPYIuqpW7bwL+m+jR3Wag2t2fD7aqAypy982xxxVAUZDFHIRrgL8EXURbZjYB2OTuS4Ou5aOEJegPG2bWE/gD8HV3rwm6nlZmdj6w1d0XBl3Lh8giOi/x3e4+FthN8C2HfWJ97glE35AGAD3M7LJgq/posVnfUvaI1My+T7T1+XDQtbQys+7AvwO3BF3LwQhL0G8CBrd5Pii2LKWYWTbRkH/Y3R8Pup52PgZcYGZriba+PmVmM4Mt6QM+ckL6gJ0FrHH3SndvBB4HTgu4pgPZYmYlALHvWwOup0NmdhVwPnBpik1DehTRN/Slsd+ZQcAiMysOtKoDCEvQLwCGm9kRsflrJwJPBVzTfmKTpM8AVrj7L4Oupz13v9ndB7n7MKJ/fy+6e0odjR7MhPQBWw+UmVn32L/3maTQyeJ2ngKujD2+EngywFo6ZGbnEG0lXuDue4Kupy13f9Pd+7v7sNjvzEbgpNj/0ZQTiqCPnbD5KvAc0V+s37n78mCr+oCPAZcTPVJeEvs6L+iiDkOtE9K/AYwBfhpwPfvEPmk8BiwC3iT6+xX4JfJmNht4HRhhZhvN7FrgZ8DZZvYu0U8iP0vBGu8EegFzYr8v96RYfYcN3QJBRCTkQnFELyIiB6agFxEJOQW9iEjIKehFREJOQS8iEnIKehGRkFPQi4iE3P8HF9ly3Z4MnywAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] @@ -371,7 +372,7 @@ }, { "cell_type": "code", - "execution_count": 22, + "execution_count": 36, "metadata": {}, "outputs": [ { diff --git a/paddle2.0_docs/seq2seq_with_attention/seq2seq_with_attention.ipynb b/paddle2.0_docs/seq2seq_with_attention/seq2seq_with_attention.ipynb index 83f28e9..152f0e3 100644 --- a/paddle2.0_docs/seq2seq_with_attention/seq2seq_with_attention.ipynb +++ b/paddle2.0_docs/seq2seq_with_attention/seq2seq_with_attention.ipynb @@ -13,7 +13,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 环境设置\n", + "## 环境设置\n", "\n", "本示例教程基于飞桨2.0-beta版本。" ] @@ -45,7 +45,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 下载数据集\n", + "## 下载数据集\n", "\n", "我们将使用 [http://www.manythings.org/anki/](http://www.manythings.org/anki/) 提供的中英文的英汉句对作为数据集，来完成本任务。该数据集含有23610个中英文双语的句对。" ] @@ -101,7 +101,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 构建双语句对的数据结构\n", + "## 构建双语句对的数据结构\n", "\n", "接下来我们通过处理下载下来的双语句对的文本文件，将双语句对读入到python的数据结构中。这里做了如下的处理。\n", "\n", @@ -167,7 +167,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 创建词表\n", + "## 创建词表\n", "\n", "接下来我们分别创建中英文的词表，这两份词表会用来将英文和中文的句子转换为词的ID构成的序列。词表中还加入了如下三个特殊的词：\n", "- ``: 用来对较短的句子进行填充。\n", @@ -218,7 +218,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 创建padding过的数据集\n", + "## 创建padding过的数据集\n", "\n", "接下来根据词表，我们将会创建一份实际的用于训练的用numpy array组织起来的数据集。\n", "- 所有的句子都通过``补充成为了长度相同的句子。\n", @@ -269,7 +269,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 创建网络\n", + "## 创建网络\n", "\n", "我们将会创建一个Encoder-AttentionDecoder架构的模型结构用来完成机器翻译任务。\n", "首先我们将设置一些必要的网络结构中用到的参数。" @@ -294,7 +294,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Encoder部分\n", + "## Encoder部分\n", "\n", "在编码器的部分，我们通过查找完Embedding之后接一个LSTM的方式构建一个对源语言编码的网络。飞桨的RNN系列的API，除了LSTM之外，还提供了SimleRNN, GRU供使用，同时，还可以使用反向RNN，双向RNN，多层RNN等形式。也可以通过`dropout`参数设置是否对多层RNN的中间层进行`dropout`处理，来防止过拟合。\n", "\n", @@ -326,7 +326,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# AttentionDecoder部分\n", + "## AttentionDecoder部分\n", "\n", "在解码器部分，我们通过一个带有注意力机制的LSTM来完成解码。\n", "\n", @@ -400,7 +400,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 训练模型\n", + "## 训练模型\n", "\n", "接下来我们开始训练模型。\n", "\n", @@ -533,7 +533,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 使用模型进行机器翻译\n", + "## 使用模型进行机器翻译\n", "\n", "根据你所使用的计算设备的不同，上面的训练过程可能需要不等的时间。（在一台Mac笔记本上，大约耗时15~20分钟）\n", "完成上面的模型训练之后，我们可以得到一个能够从英文翻译成中文的机器翻译模型。接下来我们通过一个greedy search来实现使用该模型完成实际的机器翻译。（实际的任务中，你可能需要用beam search算法来提升效果）" @@ -625,7 +625,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# The End\n", + "## The End\n", "\n", "你还可以通过变换网络结构，调整数据集，尝试不同的参数的方式来进一步提升本示例当中的机器翻译的效果。同时，也可以尝试在其他的类似的任务中用飞桨来完成实际的实践。" ] @@ -647,7 +647,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.7" + "version": "3.7.3" } }, "nbformat": 4, -- GitLab