未验证 提交 2870469b 编写于 作者: C Chen Long 提交者: GitHub

fix tutorial test=develop (#2601)

* fix tutorial test=develop

* fix pic test=develop

* update tutorial test=develop
上级 cf666098
...@@ -21,8 +21,8 @@ ...@@ -21,8 +21,8 @@
本文主要介绍飞桨2.0动态图存储载入体系,各接口关系如下图所示: 本文主要介绍飞桨2.0动态图存储载入体系,各接口关系如下图所示:
.. image:: images/save_2.0.png .. image:: https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/paddle/guides/images/save_2.0.png?raw=true
.. image:: images/load_2.0.png .. image:: https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/paddle/guides/images/load_2.0.png?raw=true
1.2 静态图存储载入体系(飞桨1.x) 1.2 静态图存储载入体系(飞桨1.x)
---------------------------- ----------------------------
...@@ -739,4 +739,4 @@ Layer更准确的语义是描述一个具有预测功能的模型对象,接收 ...@@ -739,4 +739,4 @@ Layer更准确的语义是描述一个具有预测功能的模型对象,接收
fluid.io.save_params(exe, model_path) fluid.io.save_params(exe, model_path)
# load # load
state_dict = paddle.io.load_program_state(model_path) state_dict = paddle.io.load_program_state(model_path)
\ No newline at end of file
...@@ -4,7 +4,7 @@ ...@@ -4,7 +4,7 @@
本示例教程将会演示如何使用飞桨的卷积神经网络来完成图像分类任务。这是一个较为简单的示例,将会使用一个由三个卷积层组成的网络完成\ `cifar10 <https://www.cs.toronto.edu/~kriz/cifar.html>`__\ 数据集的图像分类任务。 本示例教程将会演示如何使用飞桨的卷积神经网络来完成图像分类任务。这是一个较为简单的示例,将会使用一个由三个卷积层组成的网络完成\ `cifar10 <https://www.cs.toronto.edu/~kriz/cifar.html>`__\ 数据集的图像分类任务。
设置环境 设置环境
---------- --------
我们将使用飞桨2.0beta版本。 我们将使用飞桨2.0beta版本。
...@@ -18,18 +18,15 @@ ...@@ -18,18 +18,15 @@
paddle.disable_static() paddle.disable_static()
print(paddle.__version__) print(paddle.__version__)
print(paddle.__git_commit__)
.. parsed-literal:: .. parsed-literal::
0.0.0 2.0.0-beta0
264e76cae6861ad9b1d4bcd8c3212f7a78c01e4d
加载并浏览数据集 加载并浏览数据集
------------------- ----------------
我们将会使用飞桨提供的API完成数据集的下载并为后续的训练任务准备好数据迭代器。cifar10数据集由60000张大小为32 我们将会使用飞桨提供的API完成数据集的下载并为后续的训练任务准备好数据迭代器。cifar10数据集由60000张大小为32
\* \*
...@@ -49,7 +46,7 @@ ...@@ -49,7 +46,7 @@
train_labels[i, 0] = train_label train_labels[i, 0] = train_label
浏览数据集 浏览数据集
------------- ----------
接下来我们从数据集中随机挑选一些图片并显示,从而对数据集有一个直观的了解。 接下来我们从数据集中随机挑选一些图片并显示,从而对数据集有一个直观的了解。
...@@ -70,11 +67,11 @@ ...@@ -70,11 +67,11 @@
.. image:: convnet_image_classification_files/convnet_image_classification_6_0.png .. image:: https://github.com/PaddlePaddle/FluidDoc/tree/develop/doc/paddle/tutorial/cv_case/convnet_image_classification/convnet_image_classification_files/convnet_image_classification_001.png?raw=true
组建网络 组建网络
---------- --------
接下来我们使用飞桨定义一个使用了三个二维卷积(\ ``Conv2d``)且每次卷积之后使用\ ``relu``\ 激活函数,两个二维池化层(\ ``MaxPool2d``\ ),和两个线性变换层组成的分类网络,来把一个\ ``(32, 32, 3)``\ 形状的图片通过卷积神经网络映射为10个输出,这对应着10个分类的类别。 接下来我们使用飞桨定义一个使用了三个二维卷积(\ ``Conv2d``)且每次卷积之后使用\ ``relu``\ 激活函数,两个二维池化层(\ ``MaxPool2d``\ ),和两个线性变换层组成的分类网络,来把一个\ ``(32, 32, 3)``\ 形状的图片通过卷积神经网络映射为10个输出,这对应着10个分类的类别。
...@@ -165,8 +162,8 @@ ...@@ -165,8 +162,8 @@
if batch_id % 1000 == 0: if batch_id % 1000 == 0:
print("epoch: {}, batch_id: {}, loss is: {}".format(epoch, batch_id, avg_loss.numpy())) print("epoch: {}, batch_id: {}, loss is: {}".format(epoch, batch_id, avg_loss.numpy()))
avg_loss.backward() avg_loss.backward()
opt.minimize(avg_loss) opt.step()
model.clear_gradients() opt.clear_grad()
# evaluate model after one epoch # evaluate model after one epoch
model.eval() model.eval()
...@@ -198,36 +195,36 @@ ...@@ -198,36 +195,36 @@
.. parsed-literal:: .. parsed-literal::
start training ... start training ...
epoch: 0, batch_id: 0, loss is: [2.3024805] epoch: 0, batch_id: 0, loss is: [2.331658]
epoch: 0, batch_id: 1000, loss is: [1.1422595] epoch: 0, batch_id: 1000, loss is: [1.6067888]
[validation] accuracy/loss: 0.5575079917907715/1.2516425848007202 [validation] accuracy/loss: 0.5676916837692261/1.2106356620788574
epoch: 1, batch_id: 0, loss is: [0.9350736] epoch: 1, batch_id: 0, loss is: [1.1509854]
epoch: 1, batch_id: 1000, loss is: [1.3825703] epoch: 1, batch_id: 1000, loss is: [1.3777964]
[validation] accuracy/loss: 0.5959464907646179/1.1320706605911255 [validation] accuracy/loss: 0.5818690061569214/1.1748384237289429
epoch: 2, batch_id: 0, loss is: [0.979844] epoch: 2, batch_id: 0, loss is: [1.051642]
epoch: 2, batch_id: 1000, loss is: [0.87730503] epoch: 2, batch_id: 1000, loss is: [1.0261706]
[validation] accuracy/loss: 0.6607428193092346/0.9754576086997986 [validation] accuracy/loss: 0.6607428193092346/0.9685573577880859
epoch: 3, batch_id: 0, loss is: [0.7345351] epoch: 3, batch_id: 0, loss is: [0.8457774]
epoch: 3, batch_id: 1000, loss is: [1.0982555] epoch: 3, batch_id: 1000, loss is: [0.6820123]
[validation] accuracy/loss: 0.6671326160430908/0.9667007327079773 [validation] accuracy/loss: 0.6822084784507751/0.9241172075271606
epoch: 4, batch_id: 0, loss is: [0.9291839] epoch: 4, batch_id: 0, loss is: [0.9059805]
epoch: 4, batch_id: 1000, loss is: [1.1812104] epoch: 4, batch_id: 1000, loss is: [0.587117]
[validation] accuracy/loss: 0.6895966529846191/0.9075900316238403 [validation] accuracy/loss: 0.7012779712677002/0.8670551180839539
epoch: 5, batch_id: 0, loss is: [0.5072213] epoch: 5, batch_id: 0, loss is: [1.0894825]
epoch: 5, batch_id: 1000, loss is: [0.60360587] epoch: 5, batch_id: 1000, loss is: [0.9055369]
[validation] accuracy/loss: 0.6944888234138489/0.8740479350090027 [validation] accuracy/loss: 0.6954872012138367/0.8820587992668152
epoch: 6, batch_id: 0, loss is: [0.5917944] epoch: 6, batch_id: 0, loss is: [0.4162583]
epoch: 6, batch_id: 1000, loss is: [0.7963876] epoch: 6, batch_id: 1000, loss is: [0.5274862]
[validation] accuracy/loss: 0.7072683572769165/0.8597638607025146 [validation] accuracy/loss: 0.7074680328369141/0.8538646697998047
epoch: 7, batch_id: 0, loss is: [0.50116754] epoch: 7, batch_id: 0, loss is: [0.52636147]
epoch: 7, batch_id: 1000, loss is: [0.95844793] epoch: 7, batch_id: 1000, loss is: [0.70929015]
[validation] accuracy/loss: 0.700579047203064/0.876727819442749 [validation] accuracy/loss: 0.7107627987861633/0.8633227944374084
epoch: 8, batch_id: 0, loss is: [0.87496114] epoch: 8, batch_id: 0, loss is: [0.57556355]
epoch: 8, batch_id: 1000, loss is: [0.68749857] epoch: 8, batch_id: 1000, loss is: [0.83717]
[validation] accuracy/loss: 0.7198482155799866/0.8403064608573914 [validation] accuracy/loss: 0.69319087266922/0.903077244758606
epoch: 9, batch_id: 0, loss is: [0.8548105] epoch: 9, batch_id: 0, loss is: [0.88774866]
epoch: 9, batch_id: 1000, loss is: [0.6488569] epoch: 9, batch_id: 1000, loss is: [0.91165334]
[validation] accuracy/loss: 0.7106629610061646/0.874437153339386 [validation] accuracy/loss: 0.7194488644599915/0.8668457865715027
.. code:: ipython3 .. code:: ipython3
...@@ -244,15 +241,15 @@ ...@@ -244,15 +241,15 @@
.. parsed-literal:: .. parsed-literal::
<matplotlib.legend.Legend at 0x163d6ec50> <matplotlib.legend.Legend at 0x167d186d0>
.. image:: convnet_image_classification_files/convnet_image_classification_12_1.png .. image:: https://github.com/PaddlePaddle/FluidDoc/tree/develop/doc/paddle/tutorial/cv_case/convnet_image_classification/convnet_image_classification_files/convnet_image_classification_002.png?raw=true
The End The End
------- -------
从上面的示例可以看到,在cifar10数据集上,使用简单的卷积神经网络,用飞桨可以达到71%以上的准确率。 从上面的示例可以看到,在cifar10数据集上,使用简单的卷积神经网络,用飞桨可以达到71%以上的准确率。你也可以通过调整网络结构和参数,达到更好的效果。
因为 它太大了无法显示 source diff 。你可以改为 查看blob
...@@ -25,13 +25,11 @@ ...@@ -25,13 +25,11 @@
paddle.disable_static() paddle.disable_static()
print(paddle.__version__) print(paddle.__version__)
print(paddle.__git_commit__)
.. parsed-literal:: .. parsed-literal::
0.0.0 2.0.0-beta0
89af2088b6e74bdfeef2d4d78e08461ed2aafee5
数据集 数据集
...@@ -127,12 +125,12 @@ ...@@ -127,12 +125,12 @@
.. image:: image_search_files/image_search_8_0.png .. image:: https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/paddle/tutorial/cv_case/image_search/image_search_files/image_search_001.png?raw=true?raw=true
构建训练数据 构建训练数据
-------------- ------------
图片检索的模型的训练样本跟我们常见的分类任务的训练样本不太一样的地方在于,每个训练样本并不是一个\ ``(image, class)``\ 这样的形式。而是(image0, 图片检索的模型的训练样本跟我们常见的分类任务的训练样本不太一样的地方在于,每个训练样本并不是一个\ ``(image, class)``\ 这样的形式。而是(image0,
image1, image1,
...@@ -205,12 +203,12 @@ similary_or_not)的形式,即,每一个训练样本由两张图片组成, ...@@ -205,12 +203,12 @@ similary_or_not)的形式,即,每一个训练样本由两张图片组成,
.. image:: image_search_files/image_search_15_1.png .. image:: https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/paddle/tutorial/cv_case/image_search/image_search_files/image_search_002.png?raw=true
把图片转换为高维的向量表示的网络 把图片转换为高维的向量表示的网络
----------------------------------- --------------------------------
我们的目标是首先把图片转换为高维空间的表示,然后计算图片在高维空间表示时的相似度。 我们的目标是首先把图片转换为高维空间的表示,然后计算图片在高维空间表示时的相似度。
下面的网络结构用来把一个形状为\ ``(3, 32, 32)``\ 的图片转换成形状为\ ``(8,)``\ 的向量。在有些资料中也会把这个转换成的向量称为\ ``Embedding``\ ,请注意,这与自然语言处理领域的词向量的区别。 下面的网络结构用来把一个形状为\ ``(3, 32, 32)``\ 的图片转换成形状为\ ``(8,)``\ 的向量。在有些资料中也会把这个转换成的向量称为\ ``Embedding``\ ,请注意,这与自然语言处理领域的词向量的区别。
...@@ -267,8 +265,6 @@ similary_or_not)的形式,即,每一个训练样本由两张图片组成, ...@@ -267,8 +265,6 @@ similary_or_not)的形式,即,每一个训练样本由两张图片组成,
.. code:: ipython3 .. code:: ipython3
# 定义训练过程
def train(model): def train(model):
print('start training ... ') print('start training ... ')
model.train() model.train()
...@@ -302,8 +298,8 @@ similary_or_not)的形式,即,每一个训练样本由两张图片组成, ...@@ -302,8 +298,8 @@ similary_or_not)的形式,即,每一个训练样本由两张图片组成,
if batch_id % 500 == 0: if batch_id % 500 == 0:
print("epoch: {}, batch_id: {}, loss is: {}".format(epoch, batch_id, avg_loss.numpy())) print("epoch: {}, batch_id: {}, loss is: {}".format(epoch, batch_id, avg_loss.numpy()))
avg_loss.backward() avg_loss.backward()
opt.minimize(avg_loss) opt.step()
model.clear_gradients() opt.clear_grad()
model = MyNet() model = MyNet()
train(model) train(model)
...@@ -312,46 +308,46 @@ similary_or_not)的形式,即,每一个训练样本由两张图片组成, ...@@ -312,46 +308,46 @@ similary_or_not)的形式,即,每一个训练样本由两张图片组成,
.. parsed-literal:: .. parsed-literal::
start training ... start training ...
epoch: 0, batch_id: 0, loss is: [2.3080945] epoch: 0, batch_id: 0, loss is: [2.3078856]
epoch: 0, batch_id: 500, loss is: [2.326215] epoch: 0, batch_id: 500, loss is: [1.9325346]
epoch: 1, batch_id: 0, loss is: [2.0898924] epoch: 1, batch_id: 0, loss is: [1.9889]
epoch: 1, batch_id: 500, loss is: [1.8754089] epoch: 1, batch_id: 500, loss is: [2.0410695]
epoch: 2, batch_id: 0, loss is: [2.2416227] epoch: 2, batch_id: 0, loss is: [2.2465641]
epoch: 2, batch_id: 500, loss is: [1.9024051] epoch: 2, batch_id: 500, loss is: [1.8171736]
epoch: 3, batch_id: 0, loss is: [1.841417] epoch: 3, batch_id: 0, loss is: [1.9939486]
epoch: 3, batch_id: 500, loss is: [2.1239076] epoch: 3, batch_id: 500, loss is: [2.1440036]
epoch: 4, batch_id: 0, loss is: [1.9291763] epoch: 4, batch_id: 0, loss is: [2.1497147]
epoch: 4, batch_id: 500, loss is: [2.2363486] epoch: 4, batch_id: 500, loss is: [2.3686018]
epoch: 5, batch_id: 0, loss is: [2.0078473] epoch: 5, batch_id: 0, loss is: [1.938681]
epoch: 5, batch_id: 500, loss is: [2.0765374] epoch: 5, batch_id: 500, loss is: [1.7729127]
epoch: 6, batch_id: 0, loss is: [2.080376] epoch: 6, batch_id: 0, loss is: [2.0061004]
epoch: 6, batch_id: 500, loss is: [2.1759136] epoch: 6, batch_id: 500, loss is: [1.6132584]
epoch: 7, batch_id: 0, loss is: [1.908263] epoch: 7, batch_id: 0, loss is: [1.8874661]
epoch: 7, batch_id: 500, loss is: [1.7774136] epoch: 7, batch_id: 500, loss is: [1.6153599]
epoch: 8, batch_id: 0, loss is: [1.6335764] epoch: 8, batch_id: 0, loss is: [1.9407685]
epoch: 8, batch_id: 500, loss is: [1.5713912] epoch: 8, batch_id: 500, loss is: [2.1532288]
epoch: 9, batch_id: 0, loss is: [2.287479] epoch: 9, batch_id: 0, loss is: [1.4792883]
epoch: 9, batch_id: 500, loss is: [1.7719988] epoch: 9, batch_id: 500, loss is: [1.857158]
epoch: 10, batch_id: 0, loss is: [1.2894523] epoch: 10, batch_id: 0, loss is: [2.1518302]
epoch: 10, batch_id: 500, loss is: [1.599735] epoch: 10, batch_id: 500, loss is: [1.790559]
epoch: 11, batch_id: 0, loss is: [1.78816] epoch: 11, batch_id: 0, loss is: [1.7292264]
epoch: 11, batch_id: 500, loss is: [1.4773489] epoch: 11, batch_id: 500, loss is: [1.8555079]
epoch: 12, batch_id: 0, loss is: [1.6737808] epoch: 12, batch_id: 0, loss is: [1.6968924]
epoch: 12, batch_id: 500, loss is: [1.8889393] epoch: 12, batch_id: 500, loss is: [1.4554331]
epoch: 13, batch_id: 0, loss is: [1.6156021] epoch: 13, batch_id: 0, loss is: [1.3950458]
epoch: 13, batch_id: 500, loss is: [1.3851049] epoch: 13, batch_id: 500, loss is: [1.7197256]
epoch: 14, batch_id: 0, loss is: [1.3854092] epoch: 14, batch_id: 0, loss is: [1.7336586]
epoch: 14, batch_id: 500, loss is: [2.0325592] epoch: 14, batch_id: 500, loss is: [2.0465684]
epoch: 15, batch_id: 0, loss is: [1.9734558] epoch: 15, batch_id: 0, loss is: [1.7675827]
epoch: 15, batch_id: 500, loss is: [1.8050598] epoch: 15, batch_id: 500, loss is: [2.6443417]
epoch: 16, batch_id: 0, loss is: [1.7084911] epoch: 16, batch_id: 0, loss is: [1.7331158]
epoch: 16, batch_id: 500, loss is: [1.8919995] epoch: 16, batch_id: 500, loss is: [1.6207634]
epoch: 17, batch_id: 0, loss is: [1.3137552] epoch: 17, batch_id: 0, loss is: [2.0908554]
epoch: 17, batch_id: 500, loss is: [1.8817297] epoch: 17, batch_id: 500, loss is: [1.7711265]
epoch: 18, batch_id: 0, loss is: [1.9453808] epoch: 18, batch_id: 0, loss is: [1.8717268]
epoch: 18, batch_id: 500, loss is: [2.1317677] epoch: 18, batch_id: 500, loss is: [1.5269613]
epoch: 19, batch_id: 0, loss is: [1.6051079] epoch: 19, batch_id: 0, loss is: [1.5681677]
epoch: 19, batch_id: 500, loss is: [1.779858] epoch: 19, batch_id: 500, loss is: [1.7821472]
模型预测 模型预测
...@@ -397,7 +393,7 @@ similary_or_not)的形式,即,每一个训练样本由两张图片组成, ...@@ -397,7 +393,7 @@ similary_or_not)的形式,即,每一个训练样本由两张图片组成,
.. image:: image_search_files/image_search_22_0.png .. image:: https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/paddle/tutorial/cv_case/image_search/image_search_files/image_search_003.png?raw=true
......
...@@ -7,7 +7,7 @@ ...@@ -7,7 +7,7 @@
"id": "ueGUN2EQeScw" "id": "ueGUN2EQeScw"
}, },
"source": [ "source": [
"# 基于U型语义分割模型实现的宠物图像分割\n", "# 基于U-Net卷积神经网络实现宠物图像分割\n",
"\n", "\n",
"本示例教程当前是基于2.0-beta版本Paddle做的案例实现,未来会随着2.0的系列版本发布进行升级。" "本示例教程当前是基于2.0-beta版本Paddle做的案例实现,未来会随着2.0的系列版本发布进行升级。"
] ]
...@@ -34,16 +34,18 @@ ...@@ -34,16 +34,18 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 1, "execution_count": 21,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"output_type": "execute_result",
"data": { "data": {
"text/plain": "'0.0.0'" "text/plain": [
"'2.0.0-beta0'"
]
}, },
"execution_count": 21,
"metadata": {}, "metadata": {},
"execution_count": 1 "output_type": "execute_result"
} }
], ],
"source": [ "source": [
...@@ -92,7 +94,7 @@ ...@@ -92,7 +94,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null, "execution_count": 6,
"metadata": { "metadata": {
"colab": { "colab": {
"base_uri": "https://localhost:8080/", "base_uri": "https://localhost:8080/",
...@@ -103,7 +105,20 @@ ...@@ -103,7 +105,20 @@
"outputId": "3985783f-7166-4afa-f511-16427b3e2a71", "outputId": "3985783f-7166-4afa-f511-16427b3e2a71",
"tags": [] "tags": []
}, },
"outputs": [], "outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" % Total % Received % Xferd Average Speed Time Time Time Current\n",
" Dload Upload Total Spent Left Speed\n",
"100 755M 100 755M 0 0 1707k 0 0:07:32 0:07:32 --:--:-- 2865k0 0:12:48 524k 0 0:13:34 0:02:41 0:10:53 668k 0 0:12:45 0:03:06 0:09:39 1702k 0 1221k 0 0:10:33 0:03:25 0:07:08 3108k37 282M 0 0 1243k 0 0:10:21 0:03:52 0:06:29 719k0:05:53 566k0 1237k 0 0:10:25 0:04:43 0:05:42 1593k 0 0:09:46 0:05:28 0:04:18 2952k 1467k 0 0:08:47 0:06:43 0:02:04 1711k\n",
" % Total % Received % Xferd Average Speed Time Time Time Current\n",
" Dload Upload Total Spent Left Speed\n",
"100 18.2M 100 18.2M 0 0 1602k 0 0:00:11 0:00:11 --:--:-- 3226k\n"
]
}
],
"source": [ "source": [
"!curl -O http://www.robots.ox.ac.uk/~vgg/data/pets/data/images.tar.gz\n", "!curl -O http://www.robots.ox.ac.uk/~vgg/data/pets/data/images.tar.gz\n",
"!curl -O http://www.robots.ox.ac.uk/~vgg/data/pets/data/annotations.tar.gz\n", "!curl -O http://www.robots.ox.ac.uk/~vgg/data/pets/data/annotations.tar.gz\n",
...@@ -140,10 +155,10 @@ ...@@ -140,10 +155,10 @@
"├── test.txt\n", "├── test.txt\n",
"├── trainval.txt\n", "├── trainval.txt\n",
"├── trimaps\n", "├── trimaps\n",
"│   ├── Abyssinian_1.png\n", "│ ├── Abyssinian_1.png\n",
"│    ├── Abyssinian_10.png\n", "│ ├── Abyssinian_10.png\n",
"│    ├── ......\n", "│ ├── ......\n",
"│    └── yorkshire_terrier_99.png\n", "│ └── yorkshire_terrier_99.png\n",
"└── xmls\n", "└── xmls\n",
" ├── Abyssinian_1.xml\n", " ├── Abyssinian_1.xml\n",
" ├── Abyssinian_10.xml\n", " ├── Abyssinian_10.xml\n",
...@@ -158,7 +173,7 @@ ...@@ -158,7 +173,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 2, "execution_count": 22,
"metadata": { "metadata": {
"colab": { "colab": {
"base_uri": "https://localhost:8080/", "base_uri": "https://localhost:8080/",
...@@ -171,9 +186,11 @@ ...@@ -171,9 +186,11 @@
}, },
"outputs": [ "outputs": [
{ {
"output_type": "stream",
"name": "stdout", "name": "stdout",
"text": "用于训练的图片样本数量: 7390\n" "output_type": "stream",
"text": [
"用于训练的图片样本数量: 7390\n"
]
} }
], ],
"source": [ "source": [
...@@ -218,7 +235,7 @@ ...@@ -218,7 +235,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 3, "execution_count": 23,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
...@@ -371,7 +388,7 @@ ...@@ -371,7 +388,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 4, "execution_count": 24,
"metadata": { "metadata": {
"colab": { "colab": {
"base_uri": "https://localhost:8080/", "base_uri": "https://localhost:8080/",
...@@ -383,15 +400,16 @@ ...@@ -383,15 +400,16 @@
}, },
"outputs": [ "outputs": [
{ {
"output_type": "display_data",
"data": { "data": {
"text/plain": "<Figure size 432x288 with 2 Axes>", "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAC2CAYAAAB6fF5CAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOy9e9BtyVne93u7e1327bucObe5acRF6MJFAgwi3CxLBklIKiiqcLliBSjKuOxUkn+SGEyRyKbiFGW7SjjGKZKq2KGgHBcxRQVfgGAnNgYHy8SSHYONGMFIMxpp5sx3vtu+rrW6O39091pr729/55xBo3M0c/ZT9Z2z97r2Wmuvt99+3qffV7z37LDDDjvscH+gHnQDdthhhx0eJuyM7g477LDDfcTO6O6www473EfsjO4OO+yww33EzujusMMOO9xH7IzuDjvssMN9xM7o9iAivyQi3/eg27HDDl8oEJF/KiJ/+n7v+1rGq97oisi09+dEZNH7/qdezrG89+/13v/0H7Idz4jIH//D7LvDDvcDu9/oFwbMg27A5wrv/Th9FpFngD/tvf/Hm9uJiPHeN/ezbTvssMMOm3jVe7qXQUTeISLPicgPichngb8tIoci8g9E5JaIHMfPT/T2aYdDIvL9IvLrIvLX4rZ/ICLvvcdzf7+I/IaIfFhETkTk90XkG+PyZ0XkxT6NISLvE5GPishZXP8XN473vSLySRE5EpH/pu+xiIgSkR8WkU/E9T8nIldeiXu4w2sfd3snIr5ERD4Sf5//R//3JSLfICL/Iv7O/42IvOP+XsGrD69ZoxtxE7gCPAX8GcL1/u34/XXAAvjJO+z/duB3gavAXwH+FxGRezz324F/CzwC/B3g7wJfB3wp8EHgJ0Ukeekz4HuBA+B9wJ8Tke8CEJG3AP8j8KeAR4F94PHeef5z4LuAPwo8BhwDf/Me27jDDvfyTnwv8AOE318D/A8AIvI48A+B/47wnv1XwM+LyLX70vJXK7z3r5k/4Bngj8fP7wAqoLzD9m8Djnvf/ymBngD4fuDp3roh4IGb93Du7wd+r7fuK+O+N3rLjoC3XXKsnwA+HD//t8D/ttGOqneufw+8q7f+UaAGzIN+Hru/L6y//m/0Dttseyd+vPf9LfH3p4EfAn5mY/9fAb6vt++fftDX/YX296rndO+CW977ZfoiIkPgw8B7gMO4eCIi2ntvt+z/2fTBez+PTu54y3bb8ELv8yIeY3PZOLbr7cCPA18B5EAB/O9xu8eAZzfacdQ7zlPAL4iI6y2zwA3g0/fY1h0eUtzjO/Fsb5dPAhlh9PcU8D0i8oHe+gz4vz+/rX5147VOL2ymUPsvgTcCb/fe7wHfGpffK2Xw+cLfAX4ReNJ7vw/8FF2bPgP0eecBgbJIeBZ4r/f+oPdXeu93BneHe8G9vBNP9j6/jjCSeonw2/uZjd/eyHv/4/ej4a9WvNaN7iYmBA/zJAYDPvSA25MwAW5775ci8vXAf9xb9/eAD8RAXA78RdZfiJ8C/rKIPAUgItdE5DvvU7t3ePUhE5Ey/RG827u9Ex8UkbdEr/jHgL8XveCfJfw23y0iOh7zHVsCcTv08LAZ3Z8ABoRe+jeBX36wzWnxnwI/JiLnBA7359IK7/1vE4Jlf5fg9U6BF4FV3OSvE7zk/zPu/5uEIN4OO2zDPyIY2fR3wN3fiZ8B/lcC3VYC/wWA9/5Z4DuBHwFuETzf/5qHz668LEgkvHd4lSAqHk6AN3jv/+BBt2eHHXZ4edj1SK8CiMgHRGQoIiPgrwH/HyESvcMOO7zKsDO6rw58J/B8/HsD8Cf9boiyww6vSuzohR122GGH+4idp7vDDjvscB+xM7o77LDDDvcRd5yRlnhD5xzOWc7Pp/yjX/pl/swP/gCP3rxCtVwgKLTKsNbivUdEEIEwecsBChFFkJZ6RPUmTokggCB4FIIH8XgfDiBCWEY3y0GHoyBKAAVe8DQoD0opRHzv8Dp+cqSUCU6F88XrQ6n1fufS1AoS2ysCzvd3QJS0134BLt0TwffbdoHVEUSpcHVeUPEeeu9Jd8nj1vcQwcdF6fzee5SO7en1qUqpNE2z/T9tn9Bf3sel17axX79dIoK1FpMX3HrpJT784b/JH3/3d1CUZbu+vfAHgL/0l/7Sjlfb4fOKD33oQ1t/23c0us6FN/oXf/EXef75T/PP//mv8S9+459xcDBgtZyBV4gIzrnW2HrvCO+gRGMbPgf41tClY4fVwSBbAeWjMWD9bQxbdJsHYxONuO8MiPdu86VeNxgia/PU+gZl2z6bRkmQjXU+Xut227G2bX8T1Z2zPYWA93H44TxOYqfg14+X7ncfqfNIzwHAu+7amqZpt7sbj7/NiPbXbR5j814pCc9XoXBVxfWre/zlv/yjHB0d8Sf/1PdSlOVdDfkOO7xWcUd6QSmFUooPfOADvPOd7+Jf/svfAGcxYhByvO8ZDII9c61nFw7dM1lr3+CiQcP71i7d2bOi9YTBorVuDa0ohdIKUWrNo21P7bskP5vtcM7h4rrkuW823yXXUrr7s3kt29B57a1rSrp57aXG6/cEw6a1jsdNVjQY0nS9AKIuGvxwfevf19uyvZOQNES5ZH3aZtMj3jTAzllwDu8tHo9tFE3T8D//T3+dZ599pu1wd0HcHR5G3NHoJuNzfHzMO771m7HVqqUavLfRaYwvlw+0gNamfeG9KBBwvsbTgHRDYBFBvAGvo7PoERyOBpeOd7FBwWQJgUYQD2istdE79ngJ2V4aXDRVLpg6BU7CMVoD3XrH0YhodcfBrgA6ee8+HNmz3XgE8qPzOoMxU4iPHQTB4/bWdR5z6nQEvPi10UDoOAQcuKY3SvDd+rbz6Ba391prfYFK2Wy3w+MDYxMeyZZ71D/GZUbTA14JKAUieO9QXljOF7z/fe9hOj3fGdwdHlrcNZDmnOPpp5+mKLO15a2n1S7pUpd1HG7gT6XHYngfjEl46WzPeLYHBqQzRP1z9Yxl/3hpuJuG/mt2U6QdyveP1f+e9u/axQUDddmybceVSGEkj3P9nN21bjten5uNR+s+S8+Y+c7obTVgG53HpjfaN6jtcg9iPeJCB7CNQuhTG93zvkjPbDtvnmsO9kp+7+P/Abczujs8pLirp7tYLvhj73wHTdO0HmVaJ9FAAttfPO8SF7B23G5ozMWIUrIv/cPc4QXtG+FkRJxzkVq4GBjqe239HJf94112tuR9p2H45jW3x9ngYHu3KXp+HZ3R77za3frt6Z3fQehAlLT0Rhe8lHVel/UOapNO2RY0E6Lh3bi2zQ5l83lso2o2P4edNVjHd77v21kuFni/HhjcYYeHAXc0us45/q9/8k+4+sgBYdCegkl9+EhPrntdyWv0cbDber8+WCClhEQgrL+cvlU0BMHA5fwi9AweXaCq2z7uuyVo0zc6rbFq97o8kt9eAuue8cWGtU77JpO9ZhwT39wfzuNTK6RlwiON3TOYm51fL1jn4x3peaKXXf+2drcf7+K53g2b53U2BFlf9+RNfuPX/xnW7ozuDg8f7mh0rW34Tz74QWxT0xow51srsO7V0PucotjJQjm8S9ZAgU/qBR8CV3G/1kPbcHf7BnCbIXDOBQ60J6kKZ3XBc3O9hVvQHh/ZIuW6eN41j3fL/qmzIHYsPhrAZFz7krN1OiTxutLdYyLH2jPe4fydQV0zrukaXHf8bVA9brrXmGjZ6c6fnvcW9GmKO1EMfXjvaZqG/+zP/cC6gmWHHR4S3FEyprVif3+Mp07u3SUv1UVvajOC3r3JluTFbVITbVCIzj5e6nH21gdvdZtWNhj1taH7lv0TLlInd95n83o32OSLXqH3bDtsGtZD6qg2PPi00dqAwEeK42Lb7qQu6G3ZdTDpGFtsYJ9G2hLa3HreOyEcS3P1kSsYre+6/cOOr/3ar1377pzjox/96ANqzQ6vBO5odP/BP/iHWNsgyiHo1rtNPKUnvvtx+34Qqh8Io10f/M9Ov9tfF40IFx3STc52k2Ns27OFC03H7VvyTdnTZoR+s13t+bYE4da+c+GSL24XvdS1ZRKmMSTD3dIJ6ZhrpxE2LqbHg3f3Z1uQq/PIe+2Uvs292HF07XyFAl8C4hTiHb/6q7/Me977/lfmuK9RfMd3fMdawLVpGrIs4yMf+cgDbNUOnwvuSC986EMfihxj0BG5NPBPw1BIY+nem9xNUOjzjt4TKYZ17zAhGU6RqLG9xNHse29KKYzS5HmOMRlZFv601mitIWlYe5703Ya/24JI7d/mNu14/y4G10dPNhnERBcIQWfb3tKo5JXOww3n8uGP8MC0qHhcaY3k9o5I0EoF77l/78KXOKsvdJzO+o5K3jhG2iZdx4Vr9S0pfSmF04f3HttY/sqP//d33/ghxjvf+c4Lv1djDO9617v4pm/6pgfUqh0+V9zR0/2df//b3Lj2CM71HKNkENa4wLQsLdTRzvXHxAJiIQbVNofAF4bqauP7Fu8UQItQlEMg8JRN02BdA2LxVYWPw+JtHvLm8dPny9ZvXHK4Ni+XGlwIRqr14iVO8xVBicLHQJxqr63/gvVUAhdYivXOqv/9ggzNd/use9e0Iwvvehx8GDJcPFf6P7bdpZmH8Tip/d2iiyqHdD/SVO9P/N7Ht9+0HXj3u9/N13/91291EvI851u+5VvQWvNrv/ZrD6B19w/vec97yLIgV/37f//vP+DWvDK4o9EtcnMh2LFp/DYRlnURdqXCtFW1ZkQhGJ91A6IuCZSFfbZH4ZUxlMWI8XiPshyglOL4+IjT89t4ExQGztoLx7kX9K/T94xOaH00IMnG+c77Dl99t52sG1MR8M5tTKdNNEfnMAoSpnb0PXUu0ruDwYC6rsP9UAprbTfra+OaLly773ViyYBe0kndTbrnhVZ1cum2vT54snevhZUfPrz1rW+9oy68KAre9KY3vWaN7nvf+17G4zFvetOb2vvwWjG6d6QXhoNB9yW+7dYTvMd1uUJYrSC8VVFKJcloKZR2hFlnGXiFqLhc1lnEPu+Yps2G4TVIG5ZPBkGBMowGY/YODzl45ICr167zlW/9Gr7kS97MaHyA1marsU7nuqwDSUarz+fK5oyuzhb2eNrwrVNBBEvpoooBPNaD9kKR5YhSODrqRiQ8lJbKiPdBiQ8z5lRYonSgGMrhCOs9ymiMztCmm5osKp0RJPIZfY/eObfOEsCFvA6iJNI93b7OuR7h3CHNc7mM6ujIkLDroMgvPJMd4Lu/+7spiuKu2z3yyCN827d9231o0f3F+9//fr7ma76Gt7zlLWsdzwc/+MEH2KpXDndRL4TosiTvB2Kgx/cnedFz1uJnh5Is7hOHoXH6a7BUMZNYpBH6L+mm95eG2WnUu74taKPJCsN0es7JyRFa51y/doPHn3icvcmQ3/3d3+Z8eoKtmzh9uWt3OlbKodCf/HHBSG9zMel5zht0RDiGYl2nG+6g9uE+2sauHcv1vFxiUCu2Jtx372msQytFYfJAp7gVznuMyRgVe1T1FNu4rjnR+/Q+cETO+nVjGeVvfQ+936GGXBRuzZgm+ZtSCrd5Q+g8/k3ttiBhKnZ7O1+h4NxrDE899dQdvdyEPM9bdcOv/uqvfr6bdV/w/ve/n6/6qq/CmIum6Yu/+IsfQIteedz1ySaPKHzpr0npA1NgxbeelFIpKCYgGhGD89HASI2oaL43PNCLkrPLaIye14Xn6OSYW7ducXJyxgsvfJY/+MTTfObTzzMoR7zhDW+iLIYoo1G9RDHpHOl/ay2XYc3D633f5kGv0SBrFGrnGhrCULx2FmfX8ysoQFw/8U8MYtqQ/NJojcnzNsBmdEaRj9jfu8r1Rx+jKIZhnQocsojgvOsMuay3P50oTfToL+urIZTRnTGOs+K2GdzNe9uHk06VtjO3Lw9f/uVffkE+BoFmeOSRRx5Ai155vO997+OrvuqrWg53EyLCn/2zf/Y+t+qVx+eQxFza4FL6S15VyHHbGyKveYjr7uJW3rR/lq30a0wRqQ1FMWQ2m7NarVitVtTLFefnJ/z+Jz7BJz/5SfJswBOPP4VWOUqZmHN33eDn0Yhtnn+bUb1XPjhe3Gaz23kRNo4ctNZ4gbIsyfIM631UL3T3pw3kOR/aGikGYwoODq/xxjd9OW94w5u5cf1RysEEpXpeQhzze/GgaEcWrUHtdWCtYqK3bE0tYvSFe9fOHIw/gs17uBb8ZP15vqx7+RDj9a9/Pb/zO7/DRz/6Ud761rdeWP8lX/IlfPu3f/sDaNkrh3e/+9287W1vu9TgJly9evU+tejzh7slMe9ejLVRaTKqfm1Fisq3XlMysNIAOtqgBhGzJr9aMyxhYWuwfMtDrk8sSOahqmqsdRidIwhlaajrGmctt269QJYZnnjidUxnM27d+iy19+AanOvOuaxWTEZj5vP51kBbS3akdnmi4qC9HT3uN7UfPBZBp50Bj1fhGrUTnBKyLMNVFXVdY4yhUTHDmgNFyJqmdPBaszwPp7fCcDLhicdfx97+PuPxhCIfUFUVw9EeJ+fHUC/ARadUabwEmgBUq1aQSAGo3qjDRRVKmnVnjMFaG+gC182s69+dRD9tM6FrnVkaGfWe+Q53x6c+9SkgvI/PPvvshfXGGIbD4f1u1iuGd73rXXzd131dS2e+1nFXT3eblGpz6JyqQ3gP1rmoPU2eUDxO6xOpteP2h+x3jpB3uQR8rwNwzgUPt67DUBwVtLpKs1qtODq6zXQ654nHn+Sxx57g8MoVRGu0CakOjTEYpVksFpdee99jTdNng/EJf1spBkCJZv1yAp8LwdPF+dbz9t6zXC5bgxZoGtXKuELib4t1DVkmPHrjUQbFgNOzU87Oz7HeMR6PuXJ4hSzLI5UC4MI9jjkv+gazT1P7GOwLwbXWz8Y1FoVsLI9VOnrXu+0Zbhs59D/vPN2Xj+PjY974xjdeWP6VX/mV/IW/8Bf45m/+5gfQqj88vvVbv5Vv/MZvfGgMLtwjp7v5PXk73TpHqBjRy30gEpf5NaN1z8P1u7yQ6dx1XSE93Za1Nc45rG2C191YptNzimLAZHLAaHTAo4++jkSP9JOQDwYDsiy70NH0O5g7e2f+wqdgVMNkDw1oF+RiCRpBrMOtagyCisqGdoaaCGU5wDlP0zjybMC1q4/iGs/R0RH1YkU1X7A4m7JYLDg4OGAwGLXBPS+qVSX4nkGHpEKQGAD1bXtB1jzb/l/rEW+REm5iG3++M7SfOz7+8Y/zFV/xFWvLlFLkec473/lO3v72tz+glr08fMM3fAPveMc77hg0vHbtWnBGIrTW/Pk//+fvR/M+b7ij0b23F0Tav3Z6b9RSKR2NmqgYPOuYXs+6nCgtAy4Y6u48XbuCjEkjTvC2YblcUjerjgJAyPMMbxuaqubk5ITxeI/Geuq65uDgShh2+563KbQBtbSs37C0Db12B9agH6DqAlgp2JjoAgf4aAyTwfc4RCtMnnUz8SJBKiIURQlAVVXk2YCD/SuAYrlcIpE6mE0XTKdTTk9OUEpR5GV3v3z/uSTj5xEVnk+n84oZ4SSsc3iMMReChnfqdFJnLEigLLYSDju8ElitVluXiwjvec97tnK/X0j46q/+at797ndfamPe8IY3oJTipZdeurDu1e4Vf07VgNfkW6QMW325FyCu8467FFxBFRF5Q7vmNfVVqtuRPGoRoa5Dnt/xsCTXZq3kTrVqqK3jpZdeYjGbU68qnnjsMZx1VFWDNqHkkDE5iGZ6Pse6IJPa5pH3edt2Gm/0YnH9BD++/S/Er6TlTVHB+wRwddPqc20giqMhDRUX8ixHibBaVRTFkP39A3KT4SsbZGN5AUqDVsyWC07PppyfnzMZjzGiWm4WAaVV/FtXJ/SNqtY60CcCosBiQYNX8UJgfaYgtNnP+uqHoJhIgsGLVMJl9MMO946nn356q5oh4bu+67t485vffB9b9PJwJ4furW99K08//fRr9vdxx0DaNoSJrGzYRRd92GAww8vXvVidsVr3VvvrEw/q6UrvtIG4LUZY6F50Ywy1DRH6YZkzHJYMBiOWyxWrxYLVasXt27fRRnPz0Ud54onX8dxznwrJfAiKgDzPmS9mjMd7TKdn+C3FH9sh+JrXHVsjXRKgtL7PgUpYEKb+xnwLxpi219ZaUxQFy9kcH9uktGK1WlFkBcOiBA+LxYJhMWQ4HFIMBtQ2nOP8/BytNSenDZO9g/W2927hndQiIfl790ySt9+WQ+p5/Gl/R9BaK9a57nR2zcXsZO2z31ENnxOcc2EElG+fZPIn/sSf4Gd/9md55pln7m/D7oI3v/nNfOADH7iwvK7rtdmUCcvlkrIs71fzPu+4Z6ObDKEVwPeLeyfZEVGETxj2xyq9gUZYP9Zm7oD4gTTsbQVnvtu+X1o9bJEqEQPi0VlGluUMRmNEC06ELDN4mwdPTylu3z5GG8ONGzeom4bnnvsUTVXR1J7BaMheNuD8/KylNtYMV/yslEKLWu9MeiEpiZ2Ai4G//nRc50NAS5S02rG+AfPOtZIZpTXL5ZIsKxgNJ3jnaZYVe3t77I33KMqC0XhMXg5YrSq01kynU24fHbO3f4iXDFE1OId1DTqpKDYke/3rDF5xSJITDGrgoVvOLVIp7TXFy0/pKPverpKkbfFtJ5R0vYnS0a9RT+ZzQZIv3gs+9rGP8a53vYtf+qVfYjzePqX61TKLaz6f893f/d38yq/8yoV1h4eHNE3zqqcVEu5odPsvY8qfEMxd35pKV+q7tUFJ+nXxeNsM7prhbQ0eLQ0R2NBezodoA6wH6z3GKJQSxqMhog06y8AL1XIRDYSjrmu01pydnrK3t8fVq9c4OT0l04qz82POz09QYsjzjLIYMp2exWsH70OiHqMNmcnQOkPHGTNN0wSv3Id0hYIjL4chgOUb6moVPEXnaZpodOLkiK50fRjar6qqnR1XNTUmz8izInZinjzLGZQDlBbG4wnD0YhlVUfjbEAszjfgmyA/swrnLFpp2hhn75m0QURnwfuolOh4eetsoE+SzQ2MRbC9Lnj0XgI9lOiTwFP7UBU6nqfLmEY7TKqdbbOf7dDhB3/wB5lMJve8/a//+q/zPd/zPfytv/W3GI1G7O3tfR5b9/nB2dkZP/ADP7DV4L4WcU863c740jqgSUCfNJ/9aD1ig/8rNm25VY1w0QDLmjA/LIdkKbqaYC3JAeIQ0RRZzmpVMy5K6qpCTMZ4b8J8fo7UHu+knep7+/YRjz/5JDdu3ORTz3yCyeQQvGE4HOB8GEo3jcVhqZZVlH8Jg3JEkQ8YjMcU5QjnLPPFLAynfQjQaZMxmeyT5Tm4htOTY6qqolrOEWXwTc3KNeE+Rf2tMYbJZMLZ2VlrgJumIc9KclPgmsTh5m2UGqBa1dTRA5jOzgJH7Dyz+Sxw6S5U6SCOOsJoZX0GXFI5hOcaMp+p3v2WnrKh/S1oAeUjoetbDje4tkERoZJs8DL5GKD8zui+EvjlX/5lHnvsMb7v+76PD3/4wxweHj7oJr0s/PAP/zA///M//6Cbcd9wV3phnQME8K2H4r0PEXdPKw9LovrwsiragjXeX5CGXPB8e+fsG/pWRJBkS3HrtG+WZTRNw/7+hKZpGAwGeOeorcOjAUtZ5CijyYswweD49ik3bj5GtVzyqU8+w3g05vx8imjFYDBguLeH8jC1p7gMstywv3fIZP+QycEhh4dXMCanrmu8d3z608+S5zmPPHKdshwwKEvOTm5zunfMYjHj+PSY6ekJdbMkb8IEklAWPQyZqqpq8wGfn5+TmUCXOOsQL5g8J8uyVlGglDAoB2TeM51NEXQwlt6xWiyjF90vea/wPmio+4Es530I7HkfPHWRlqGHmA9iy7NKKoj05FQMKHpCsM0733ryrRa7p93ORF8wxDt8bvjpn/5pBoMBP/ZjP8a1a9cedHPuCZ/97Gc5Ojp60M24r7hneiEgVNldl3bZNQlVXBGG3G0wLXCUm9jUbrpoYbdJKtJLn0T5zjnEd7ypMYaqqiiHgzhRQjBKkeUZy6ZitlqiaxWm3CqFtQ1NY3n0scdZLmcobRjujSmKEIRTRuFWFS+Ipq4bRqMRBweHjEZ7XLv5OIdXHmE8nlCWJYLixo0nKMqCvb39oBOua8qiYDze4+TkNuVoyPOuoao0y9USRQgcIMJwOKauK7IsZzabBuOq85BYvPEMyoKiKDBat4bXZBmiFbnW7Ok96rpmcX4K1tNUdTBvzgYJGBINbpfPN95VEg9NUmKQ+OewhYJ25lrf8K6NgFzqEONz7dVVcxdSWHbPfmd0X3n81E/9FMYYfvRHf5QbN2486ObcEc899xw/8iM/ws/93M896KbcV9yVXui8U48oh6JLGhP+D5ypYNphf/BoQgrGpHfoi++3BtLoybA6qx7Xb7yc0s1vSwoA59yaZzUYDMi0wWMpyzBFdj49p66rYChFsVwuGE8mPP66pzh66ZirV6+xWq04ODggy3OqxYp8MOL8/ITBYMhwOKbIB4zHEybjPQ6vXKEoSwbDMTcefRznLM47quWK+WyKaE0+mJKVA/ZWB9Sriun0jGFTo3Ccz87IswGZyZjPZpF/NWidoSTDNZ5MK7RWGKPIUu4DFatrGA2isXbFaFAwLUpsczt6wqF6hm3qnuF0Pa833EpFd8OTnrflY9c4dokZyqK/muQY8Zk6l0YotFx8okrWotFK2sTuu4kSnx/85E/+JMYYfuiHfoibN28+6OZcwGc+8xl+67d+i1/4hV/gZ37mZx50c+47XoZkrBO7r3uovXBJEt73g2obHO02g7uNeticHNF5UiGQp0SD7zwv53yI/HuP1jrkMlBdghZjDKPRKMqfAo/q8WRZwcDvceVQU61qrhw+QlEUOCXkhXB1MOTmzUdZVSu8BaUC9zscDhnv7TMYjiiKYQg8+Rrb1FT5iqIsGa8WHIkizwyLRclTT3lOp2fU1QrlPUe3XyTLcpbLJUYbTk6OKYohxmRUdUNmFIOyJM9jKaIipxwOKIshg8GQcjjEe2E5mzE9n7Yefr2q0Mq0ErBOxqdocx1D6OFiXuO+56lInq6PmTh9b/+k2ZX2z7VaMryut5oAACAASURBVNpCoGmSSj8/b38245audIdXED/xEz8BXCxs+YWAj3zkI/yNv/E3HnQzHhhepk53u2fivceLxBy7QnxtQxCH7kXdpntdeyGh86ToAj89UcTasNZFQ+xjhC8N10UJRVHQNDXg2ymt2hjyvMB5oalDkG+1WkSFwJI8N1RVEwzr/gRvLM5CUeYolZM6kGpVoZSmLIeMRhOMCTkWrFM0OvCteV7Q5BnT6RRjNIIwzXKuHF4FPN5ahsMh1jYsFktuvfhCrHyhaRqLEk2uNEVZkJlQB64oyvA3GCAmQ0STZYbxeMT09ATfNJgYhNMmZFSzPiZ9T+ZXFCLpvpGiohe4V9UTqLSSth4XnCqBpDJNfQ1v2/ludtK04oWWK97h84dkeHf4wsK9B9JwrR5zEyKC4PBeBx5XHD5OVRLpEmpvcoL9/dvj9I6fZlMhkJJ4h+VhWcgvm5JvWxrb4CrPaDSKCgBLlukYaAvGR5mQDMeYjExnrBYLqtWS+XzOYDDEORey9ltH0zQopRE0V6480moonbOIBE2lyTRGhykAdeMQspgUR2i0YjgcsVwuyJucoixwTYNtLDrPycwBq9WKwWBEtVqRmQznLdPzaTtLz7sQuNRKYWLBTZ2FzkNiACsrSrKybNUGdV3T2CZ0aj5JthItEDkcggQwOrSBCFI65IWII5Ww3rWdWheUSxMjaJdvBkEVcqFTDdx/6mB3fu4ODyfuKZAGIHgkJBtc53R9KgnTnz8WvwkgNr7EqvWk2nIyGy/x1jnJF3i/NHstFq+0DkQFraj3DMuS4XCIUopyNMb7UC1C62Ac6sbiddDXLmZzGteQFxlVXQHCaDSiqivqxbxNaWh0xnhyncFgQFEUIb9BUcTcEik5jG0H3FppdKbRWnFweMh8asKMt/mM1WLO6eyYsiwx0Rs9OztjPNmjyApOz08wWuOqGpMZjDFkWRGMbCuYDXRJnpd458lMhifwp4hHK4Otatoe0vfyJnQuaTCCPS+V9vl1PG9nONcNb/s5mk/Ve5bpdxEfVwisqXVvecfn7vCw4mVxurDOw6aZYsHI0pndfi0uOqlZwmYE/LIodj+Ithl4kRjISVI2EaEsy9bLTcdtGguEwFFVrfCuofEe11TMZwJKMRgOKQdDRsMRTdMwHI/whOmHx8fHTMYHoA3DyR5FnDJclCVaG3Ss82bxWGdxLuQ50FpjlMZP9tBKka+WzOdXOFeK2fk5i9WKq6PDmJayoSxKlIdimbNUGm8MOuZLMHmOzgpUnqEyg1YKrRSZyVBKsVx6hsNh4LRFyDKNm63fr00uPag+Ek3QGchNvlXFAJzEUU7H7QbiyCad9qYNjTyCdS7mn4gVhKUfHdhhh4cPd1YvREpBttjEFLxuQy8ioDoez8UXNfCCsmas4RKqQYIg7YIN3mIwiLOeUmcwHI7IssChZlmG1prcZFTVKlRn8B7b1HjXxKrDYX+T5eiqYjAc4X1I7zifzbl9dMRiteD4+ITr1495vXwxo+GQohyEKg+ZaWVwoeNJ7fQoUW3y8LwsEBV44eT8f+r3/wAxqq3ge3BwwHw+wzqLUYoyL9AimMy0eSGKsiAvSwbDEYNyFBQOSpGXoYBhXhSoOKzP8gKthabqUTQ9zrXzWtdvdOo4+x1cMLZhhJN4+bRNCJpd5OnTwdrAnI7J0tP0aXV5R7vDDq913IVeYM3gXdBqQnRs4/x63/N45GKik01czF+ggH6tsuBFJ8MN0JZr855QKVi1OWtnsxl1XbeTDIqiwChN08REGrYJNEDsDAInG4bwq+UK29hQfWE4ZLVacHZ2zmq15IUXPstqtcRZG4f7Iedua7y9i7QL0TClGxMmKBhjyLRGRHF+eoZoQ7Wc8eKLS/b39wHajiHLMrI8awNY4XzhnGHSxSgmNBfqpkGsoRwUFHmBaM2gKINMy1sk5qfwWLxPaTahJXI3YK3rpgYnHt15lNJY17S/ibaCRPsjuJwuUPEYfYWK9z5UFd0Z3h0eQtyVXmjlPxEXJV8+Rr+3vcaXv1Trnk7wsbzfLA65Tk20Q1w6TaiPErH5fN4K8YuioCzL4N1aGwtlhr8m8rTO+ZBDIebUFQlD6boOuQy89xilmE1POT8dslwuaJoqtsV1bY6ebbCxXSIY7z3WN0BonwJwnizPGY6GjIYZR0dHbZIYrRTlYMC0qdBGx04i3A9RoTiloFBKk+V5lMxBtViiyoKiHATpl3OYmOKSjXvXjjaEdkTRVyS0HHuSF0TJmLW2ra3Wzjq8RGd7YQST6OEUQEvPfSOT1A47PCy4s9FNcZgNQ5tyLjjvQlYwiRHy3pA0BVs2OdlN1UK7LCoghMAhxi26pvQ8q35+hm6mmm+9UGMMTdPgGhsNb6gmkcUkNa1HmWVM9vYwxnB6ek5dN4gIeZ4zGIw4Pj4OXq3yHB/fxjlHY2usy0KCGRVNsPftLKxk6DUgOqgYtNKI72bT5XmOrS17e3thW63xIpRFSdNUVFXVG00I3oVJIEWWtWqNLMsQQtrF2WyOT9pjCZ2OFoWVwGenuNZaNeOoONg6zPdhllq7JvWoQf6AUtIme+97sJuB0bTMpfzH7VMN599RDDs8jLgLp9txpl66BCWJJ0yaz5C0ryV46TxBANnqEXUnifFv72IUPB0kBOkuvphRH+HX0z2m3K/GGKrliizLo/cqiGic89imamerFUWOcw3n56dM9iYYozg6eok8z2mahvF4zGw2Z75YMalCPoemscGYe4+1gdqwtsHkeVv40dtwHTWQiwTDqxTiPVqrqMVVVNaGjGjasJxNEYKiIjM52oTlokJuC6UUTVNTVzWiF1gci8U8SMhMxmIxI8syhuMh6kgzGAxwvbykIgpliPkQImee+FaVZqF1nqfDh4TnPnzulA/d/e8b2vSbSLx2/4mtTwPuio/6nae7w0OKe1IvbOow07LEY4oXUtbYTk3gEdGsUQMb+yfqQqSb698/50V5kZCMcTD4wejhgu4UhNlsxrAcMJ2e41wwloiDxqJ18IT7xSDruuLWCy9S5Dmj0QDvhdVqFbndEdPpOWI0y+WK5XLJuLF464Km1YZAo7M2Jl+2eO+wFuJMkThUdygx6AyGwwFXr13jk8+cc3hln9OTc6ytuH10grWWyWRCkZWooWe5WGJt4JmNMczmM2xQzqKUpnKOl46OuLJ/wOnpKYE0COcONi0mIbKentqsNwDZ7m22/LmkjLrh2cZ5EGvPtE819HzzNU82bRNonZ2x3eHhxp2NrgjWe7QHL4K+ROIVPNRuyNilXoymeC1yflH+1Z7uDk1JnlIqnJhoiVDnzFLXlrqeI8BqMQ9BMpPhveBtSBBusiArS9xtqgZc1w3Hs1k7+aGqLaPxmKLI8ALz6awd9jdNQ71agfeYLKPI80gpNDiXcusKeEvjLdqB0QLicB4GoxFXr17n5PSE1XKJ1prj4+OgpDAZjlDbzdVV7CCkzbGrdZro0XB2dpsiLxiVOS985tNAqCqRZVm8dkXTxKfTC3aFjG8u8r/rfO9Fvl51kyV6zy15r9ueaddRbv6U1o+/LQnODjs8DLizeoF+8MpfCHq3Ufr2Re4nw4FgdHs1Ji4RxrceUfjSvrCb00+1VpFLDIZWKUfTOHTmwDc0ddOmTAwpHU2gIBQQPVitNYeHhygVSuG8dOsWRhtms1nYN9MMJxNOTm+3eQO01jhrWS5n1PWKxSIl+vZhCjDgvcV5C15QOtyPxtZowElSEQRdcDkcMh6PeeH8jOVsxt7eHp9+7jNMJhPq5ZKsCDIwkxnqqibLiAbfsaprTJGD9UxPzxgMCsqiZLlYhDSTAlVVt5yrSKxUwaZBvVxxkAzrts5xLeC24em2x/R+nT/uQUuYmtzPybDDDnfC2dnZa6ZqBNzF6Ha1wDyqo3dbiO9iLH2eTqluwgLQ5t9NBR/bPzpT0Kcw0tLNFzPwqF2TRTkQCy4EwJQH5xXeCXjBGME1FaI1ZVmGAFZjuf3SUZjqSyjhbm2NKE9VraCB+XwRZpxlBu8anK05Pz/lfHrK3jQkKK9sjTEanMXiqZqapmkQZRCBzCvEgvUNWmlAxwThgdsdDkJ5lcbWVIslN649QlVVaC0sqwXlcNAGoJomXJ/JMpytcZWnKHKMgunZKWVZslwt0EpoqvpCpdg+n9rRP73n2dIG/e3DM1FxJp9zvqs+0dsueb3JUCfHeqtBjZ23IK2Oe4cd7oZUwuq1gruoF7rKAheGnj0ZEKgwpI4SoU72EAyw6w1Rw2FbvqE7XvgU/9zF4aoKM5q0cyGIp0JwTCsTttEh1aHULmYZaxAFRhtKrcMQXQSdZXijWa5CwcrFYt7KzZIUrSyHraqgHA0REY6Pj3n8ySeoVitm52c4P0DjMVkIlNWNpbGWDIXyghePowGfhfSKSsfAYND6WldTrxYoFxQW8/k8Js4JuYG9c2HmmTaIDvdotVyGoFpVc3x0G2stTV2xWCzaahODwYDz6fmapwusGbmUXzc9o34HmZCG/955HF1Zof7zC/RON5JJ/HzXk/Z/Sr5VS4hWmJgFbYcdHja8rMoR/WXhBbLRburWOAdVQ6IEoqerYm6EO/CAbWCu85tbY5CCNG0Z8+BKQfQag/GXYKDwbdIakynyPEMRDMhisaBaVdRNRVWtWK1WgaOta6pqRWODBG6xWDAaTdBaMznYb6mHs5NTRoNhCMwpj0GHmWAmSMJwLsi0fEj2rkQQLM556nqFiNA0DcvljJPbR1SLJdOzM6pVBYCtm0hFWPSgJC8KRAnNKnjRTd1gmwZlQl6Hpg4cc1mW2BjMGwwHnLz4/Np9FaGbDSYpwXwIRCZdrnc+lGfvnjJJapb27f8eLvuddMHOzvauyf020kjusMPLRROCFa9a3H1yxGVDQAG86uk3O2fW+w0DLXLBaN/1+Fw0zkgo9e1VYphD8Kl23fTULFPtcETixASjM+bTGYvFHPFQNzWrahmM7aqiaSwpkY9tHJVd4X1MflNV7O8fYK1lOp0ym81Cpi+tKUwRyuxIHjhtYhYtolrAhXG8dUFqhoQKv6vlAm9rXrr1AtPzM4zSIUeBCGU5YDQaBXmYWlIOBigfjHXwxj3L+YIsNwyKck3ytVwu2dvf5/z8rONaew+nu9UXZXiiet825GT3iu55sfa7SMfqa4/759lhh3uFtZa/+lf/6oNuxueEe/Z0N6PNQaq7EWRJVYGJ003FtcfYnBTRHbMf2IkHpm8g1r1tC2gxEMuZN01DlgcKoCiLYBCV5sknn0Ssp24aXnjx+eDV1g11HarzLpdL6qYBhMa6mHsXRELwzTnHdDplNNlnNag5PT3l6tUrzGbniHi0JuZEyNFxSqsoE/hbJSAaLQbrG6z1NFEjPJtOOTs95vnnPs3TH/84RycvUZqSoijY29sjL+Y0Tc14PMZWlsY0SJGTS9DMLqZV9JgtUztnENUYTdNQmIxVs6RarQIfrgUn6Vl1vG0oxx4mXXjo3esu9eLmve8/tzXZ4LZthdbgb1NGbB53hx0eJtyzp9sPaq2/RL7lQlN6vzTvUy5s28F739ay9ykrWesirZ+/O28o/WJt8myDLM05S56HKL7JMw729rGpikKzas9f1zWLxYKm1dXGXLHeonXoAJyzQQMby6HP5ueYPOPKlVCI0lqPcyFgZ63DE+quiWjEebQ25CbHGBMCh06hVDhPVS2pqiUvfuZ5Pvav/xXz+RwasBK0uGenZxRlEWaf5QV5lgeqRBmyQQYuSN/qZRWUESpUyMjzUCBTFznT+SKpdcEF+++9D1Lm3hNr2d0eCRsCY2lCyeUa3hQ06ysUttEPl00XvkwyuMMODwPuObXjRU43vKxpwLiWzIS+Z9UXzq8frztmMNKdqYaUTnD9Bd3wumPC9KpeURRlMFB5mImWKAZrM0ajEavVinPvcN7jbE8O5UOSdYWgjKEcDBHRLOdzqqZiMV0wHATPMctyynJAXgzwXuGcDdF6FSZd6DgV1xiDEIyXEo3zhlrVNLbm7KUT/s1v/RbHR7d48cUXoQGlBS2GmzdvIt6zmM/ZG49BwnAqTCzQmDzD1Bm2sWQ6TFfWCMvFIpRxP9jnE596Zi1AJa0rGyY3hOBXymnRoVVVq85Yrj/3jqRNx2jLrqfnzeX6283A6A47PKy4axLzy/lX30WqYzXJGP6iSw8Yt2w1SuvoDGp65RMR6DfWbxvShimtzjusC7rUkAbR4Jqa89USrQ1pJlSe54xGY5qqCQUWncfZJpalCcm8lXiuX7vJm77ibfyzf/qPcbOaql5RLRd462gay3iyx2AwCvkNVIESTaYzRoNeHl8AF6rviobGrxDRVIuKp//9b/Px3/53nN2+hRGPGEUxGlGYkulsisk0w+GQxWLOYDQiUyH5TZEX6HKAOEeuDDZOZ26qGmsdmQiTyR71crmuNHChlFJIl7PF4EXuoVWQJeHJRpCzT9J6XLtPou994BQuGNWdod1hh3XctRrwpetapYH0hqqsvajpFd0mO0toDXmbk7Zdc8FOp5lU6/sK9arGHJq2ICW1xeHJMtp8tMPhKMifaotZGqpqTlM5vAPlPVoLXgTb1IjkfNEXfRH/4Xc+Rh2n4VZVw3AwZjTaYzQaYoxhMBi1tciUVuQxibiNEjmlYvYxo9FKOD854aP/6l9gqwWTsmRZr8iKIY89+Qb2Dg45Oz/hpRdfYCyhC5rP5xQpCbgDrwWUohiWOOcZjkbMzqeUgwGiFY1taKp1jS6qX9EjzUjznVSsnefho+46lT+66Jl6H6t1sN4hJqmCuItql0RL7eiEHXYIuCdOd6ux7EVgHB132E1+6PZPSF5n/9jd8fp8cThqCrJ1Xlf/xe00p2l6rHOOs7MZ2kFeligVKIHhcIQxeUzB6Bk2Q05PYOVD+kTnHXmWcb6qqVczTm59ihtX9nlaGWpbU9c1q1WYYlsWA4aDEVmWk5c5RZ6TaYO44D1rrdDGtO12ziEWpudTPv3JTzI7O2acK7wxFJmnyQr+yNd/I14ZTK741x/5CIvZCVpnLBaL9p4558CEXBaZ0WRZjvMekxlsYzFac3J2GjKqsT4jzPdv2WaQi76BTduvF5Vcn20Wtm/pJNfpfdd+K9Gwr42Ndh7vDjvcY8Kbbcukk4slD7XPvfZ1mxe52XazbtR6wbinDGZ3kpWFk6eaaNY2VFUduFXvKcoy5M/VQU+7p4TMaM7Pz1jMc5QtsHUFKFRmUHWNVo7jFz/BfFEzyDNWvRSGw1Ewtt5DtVoh3uJ1RsMSqS3e1Jg8IysLdJ6FYb0IjWjOjo/5g6d/j8nAoAcl1WKOKsZ8/IVztFqgzIim8bzlLV/Ov/u3/y+TyYTZdIbSmsY2WOfIKfDOUq88g8KhPGR5xsxOEYHlcsGFaWO959cnbzY/x6cWOgkVRiYhdadqd+6eo2o5YZHI7abfw4VzdyXaL0gAd9jhIcS9T45IsqMeTysinUxMOrlR8praQNolVQIEFYb3uudJtf9fNLj943fStFBufbVaUjeCs2Byg9ImnFdC4vLMGLI8x5gwO20+n9L4GjKFszUozajIGGRCoTU+c8wMlHlBUZQopRlN9lFasVrOmZ0cU2qDH88ZDAdkRlNkhmI4YLi3j9qboLMs5j4TjAi3X3qR/cmAQgpWuSYfFfz+7QWnJ59hUpZUVmOKfcbjMaPRGFBYV7dD/uVyQV1XbdVeay0TMybLDfhQ70xrHfJK9p6fUqHkkov8baI9EovuW7o2VABxKbApgQVWnjgNPOwf7LpvnwmOXhazmI+BuI3IBWO843l3eJjxMgpTdpD4zzp/F+RGmzpPuPiCrU9HTTPPoOc+pbNseEXJsDu0zsArJCbCqOoV0oDRZSydrlgul23mMGstw8GAvCwpywHaaE5eeJZmcYqtFYhmOCgxcYpqkQllUaDLkslkzJd+2RsZjvawzjI9Pebs+ecYKE12sMRPRuQi+MwgkwnGeUyeoUxINI6EROZaG/aGY8aZYrEqAM8je2Pq+RlNPUVnBcvGMigzVqsV4/GEs+lp9BTjvXbgmoYmzM+L2dI0QlBsOH9RfZC802BXU+8Zs6G1gbBI8ShZf7bpOfRyF7e8b9JUt8fdeNay3pluft5hh4cR96bT3RS2p3XQk4pxkYeQ7RrN3pdWb5tswfpZNlUM4SRKpbIyBE/WBa2uKMWyDtzr+fl5SHCTyqjHqhFZkTMcjxlP9tifTDj+7B+wmJ7gnWcwHFA1DotCqYwbjx1QTq5z9epVvuEbv4XhcMT8/IijF19g8cJnaZRiaB26OcAbgyoKlPJoo8lGJVIOMDoE+B65foOvfNvXcPLp32VgagaTEbePb/PFr3+MUWEojEFlGfOzU1y1Yqky9q8csqcPWC4XWGtDWkedhRwMWsUUlZbRYECe55zMpyiTo9RyXTubomY9uF4n1w+Eeu9RPtTw8HiUC9t6icnafAqg9oOg6499LZC6xavdebk7PMy4d3ph20qJ8iEvLQ+YBq1pWu7lB477kpQNvZNEDW7r37ZcIG3J8KTTVUpRNzV11TAcDoNkzLk28bdtGoZ7+4gIeV5gHXgvFIMhV770OtdvXOezzz3Nye0jSpMzzkrGV66yd3AFM5hw7dHXcTDZ59qjj1ItFqwWc05uvYg9P2exXGBnC576oi+DPEd7mJ3eZnF+hhkPyPYO8UqjRBgfHvIf/bF38plnnuCZf/cRiix0FiMUpQk0iBJHFvNWaK2Dh5zl2MUS0QZEMRiO0Epjck1VrYIUTGmKosRkGcT0miH4FmvO+ahakFgo0vc6Th8pIxcmUKQxRm+sEffpRh7er+fR2Ay2rXXUbr3672Wz3XbY4WHBvdELWzg50pDXw1o0jc5Ib76M6SXrJ8EOBjS94j3PbE2Ols7bpzQImca8p2ksINR1g/dhmG2txVuLZDlZljMajSjKgtFoEmavDQoODw7Z/6Iv5c1veRvz5TmL1RLxiiuPXKMcjSjKMXkxQIvBSYNgmZ6c0MznNKsly+PbVKfnXL/2KNl4zO3pKfPbL/DoU09Rn53DtRpfFoho8rzk2o3HGY/HPHrzJs/83m9TO09TVeQKlNFUjUUBe4cTDq7fpCiGzJcrJnsT6qqiqWuqmLYxJD535Canbmo8YLQmz3OqpQq165A2ruYkPbck9OueUXrGPnq0kmR+nl4gLVQCCbXTthvQrYHS3m+mb3h3NMMODyvuyegqrfDWhXpZrf8TXs42au18q0JYe5m56NGszcdv5WUbutw4/E2ie5FQOcHaFJ0Pky9S3bKyGIaEL3t7VFXFaDQKHnQ8nzGGQTmkLAcURcn+wT7Xb95kvL/PaDBA64zGNiHnrTEYVYAiTLxoLE1jKQclTzz1RVCveL5pOHrueQ5yx2J6zsAoRGluT5cU84provEq3BNRGqNDztmyKBnvHTK5eoM3zt5OvVrQVCtW9YL5qqauwJoM50OF4/PTE24f3aKOmciqumYyHlM3dbwflrqqWVUrMpOTMrz5NXc2BMICLetaNUGgalLX5qMhjkXvU2XjXkrI5DnfqwHtqyb6v4EkgdsZ3h0eRtyT0b2Mk/MoZG1V8o78NrFCt9XWAEsvUEPnL3eMg49l03XMmRCc7boOofq8LKibBussJgvVgCf7e4zH47YsezqYdRZtDOVgzGAwIS9zRAmlGgA2FpJUrOoKowzWNmilUHnB+MoV3vTVX8vrvvj1vP5L38CnPvYx8iwH5xgdPMIf+bb3Mr5xneHVR7BKo330HHWgGbCewhiMuUE9WYVZdQ6ctdTW4mzwJJfLihdf/Cyr5QJjcpx3MY1jEWaolQV1XeGcxOrBrs2pq7WmqsP9S/mME9mTgpaO+L8I4rqJKW3du56+t9Pthk7E+X49kM2Aau8pyvr6fo20l5vBbIcdXiu4o9FNHJ+nrbPY8o1tIKs/hCRtE0vTpOTXW469/UXtHat33JSNbC3pdZws4Rxt4ppVtcJkQRpWDEq89yERjFQcHx8z2dtjOBwznkzYOzjAZEJmwMbKDGK6UuFNXVHVVVRJhGvKy5LcWZrGMBqPmYwO2CvGVC8d4QRUWZIfHpLt7yN5FvhcpTBahwkkRoH2YB15OUCbHOtCFjIclIRcoV6gLC1VUzOfL7B1A1hOj09idQrB17CYL8jyDO9D2XeHY1AUVMtpO3rwkQ5opOPZBUG5ECQjcr0QDGlgFVKhUdCKtiMIPwrajJ7tSML7jc6XNR633XWtMvCO093h4cTdPd12FhK9YWZKRtMZ2n7Zlt7OPd1uZ2S36TRbI07nbSX0jy0iiNOIEuraRS/Pc3p6RpZlbbXf2WzGYDBgOp1x8MSVkLvAGIrBgGvXb1AOBmhjWCyWOOejtCwkyamrVWsczs7OKPMC7z2LxZLhoCDPc5TSZFdzlM649Xsfh+kMBgNUkaMyg4u8t2ssDYEbFULnkZmsVVtoUyB10xotnYVH0tQNhwcHLGYzzs9OGU0OMFnBYj5nOp2ijWF/b4+qbjBGh7LzZYEyId+E0RnWNXgLKEWG0HiHSs+DYDyFoOFtjWovoOmhLV7ZseyJcY/5Knx/ZMKFZ72Zw6EvZ9thh4cR96ReSFNGE/qGs5v22wuUtcsvP2Y6TnsOkf77vjEJInm5CpFQzhyfjDE0jce5moODq7EKRKikO5vNyLKMW7dusbe3R1YUKK1DhWNjcNZhvaOum5jNy7JarTDRexQJBmkZq/YWRYGXDJMpDCAG1N4eq2s3mPoXsLmmFqidIycEpZwLmiulVKguQerAFForamsxJmvvR5tukjDT7tq16yzmc7yzONtg85y9vT2a5ZLz8yWr1YqyCFUulBLGexOmZ7dpbN3K7JJRbQf0npBngiD/0tED9pu6vZ5xbANv6bl21Hq3Qf9Z9vftPedt33fY4WHCHYk15Ql835aXJXi7QJw6mmqMhcCYiodWLSu7g6OjewAAIABJREFULenJ5V6PxP3Xq83qqE313uGAVVO3XrAxBV/2ZV8GhHSIdV1zcnLCahVK2hRlyZVHrjEcjSiKEmsDb7pcrtosZGdnU5yDo6MjFosFzjmapmK1WnB2dsL8fIqvK3RUUTS4EO0vc9SwpBFFY0OtNOccdR3yIHjvsXUdaAzvcda2xsqYMHMu3ZtUbt2YkC5yPB5z8+ajjCf7jIYT9iZ7DAcDvAhaZQwGJUpCxVTfWA4PD0OeCaVwLnmvIQ9vv4NUgBFBp/WpQ3CB2lGJt5XuubdtJEgEW1nZBnfbf96bHesueLbDw447Gt2k94TtBtLHwEo7cUFc1O26WDMsBlB6L9qmt7NttlL3cq6/tEm5IEpRNQ3eCbaRWJnB8hu/8f9wfj7D2lBc8uDggKIouPLIIxwcHHJ8+zYnt29zenqKaywiIQgVEuWc4WK6xMduPs4wGyA+lD6fTqcMy5K9vYOQ4yHxqo3DedBG0wQlLEqHmV+2DjxxP1m6c44qlt1x1mLT/Yz3OXm66U9E0EYzGo+5ceMGxoQkOPPZHIByNMSpcNzRZIyIYlCM8JLhrCdTpi1dLRIiYInjBcLz2eRi+16oB/z6cxK6acEiAupiruT+b2WTVtrNTtvhYcedA2l3kPX0Z4iJqDjRIU5gEEeMjwdhfZwDAawZl01hfcK6Ye5oirYMuHg0Cmsrrl69ztHRESKe8XjIN33TN/H8p1/k5qPXODw8oGkcn3r2OT7zmc9y48ZNRuMJAsxmU+o6byP+ySCsVhVnZycs53MaG3lXramcZeUqyjitOBglR7VaMpvNWMxnrOoaeSkPkx4mE5RRrVwtZUGzNg7qYz6KZIy1MXjr2gi/S3I61R2jjMUqp9Mp5XAYSsfPQw24xjuMEhSawaCgmufMl/OoVuiUBMT0ix6itxqeY+Juu/vO2igj5HBYT1SfPm8+v37dtv72/f2MMa/6AoM77PCHwZ2TmPdlRuoyDyXMUPJr6xTJhUrSpGSi+/KhO2o813i/3j7p/6bB+4YXbz0fZWShXtpv/uZv8sQTr+NjH/s37TD9y974Rq5ff5Qnn3ySa9euh0Qu8dz9oXxRFDSNZbVYcHT0IpPJAYOyjMNxoakqFrbBmsApN1VFtVgAYK1jUc2wWlM5hxchL/K22rD3nizLMNrgBBpnMXF6bpLBKfP/s/dmv7ak513/5x2qaq21pzP0cGzHdpx4IJYc+ecQy78fP0CxoiBHSUSUSMgR4gIhbkiEQMJMN1wkv38AKUhcREFEQmK6CAgFhQhiA8kFBCcKJMGQgaS7T3efYU9rrRre4XfxvO9bVWvvM9ju7t19zn6s9tl7r7Vq1apa9dTzfp/v8/2a4hQBiT1iNFVdYaylamqWe/t8YG8PfGAYerquY9hsiH2grhu01iwWK94c+pxO82nKJ1Us4tPJiUwYITyadzvFaafUr8JgmDx392a9e85BqH7Xle6TY7PZXPUuXMdbHI+vdLNyVFKpuvB4GohQ6TnlKclJovyayaGT2J1WK/EIill6ME1ZBYYgko19u0EpmfrSWtP3A1/72tfEKWK1T1UZfv8P/oCTkzOUUqw3G45u3OLo6FDEbYwqQwYPHtzHe0dlLbdv3cFYVWx/UOnm4iMOwTVBKF6b9ZrNZoMLkXq1xLkBNwziHDzR1o0xsqgbrNbEUi2LV1xtLS6E4jOnleCrItMbqaqaG0e36Luek4f38b5nvT4nxsjh0Q2CUgx9z95+I64W1uKCVJKF7kcgRk1I3N1paCgCOE9qcO02y6ane/pZy40twUTTG/N1wr0YTdNc4C8fHR1dNxyfsXg6Pd1Ef9oNY8woYj0b46UsWx9FYbj0okvJeT69lJ83memf0ZKMYLtJ91aabYr3ve9l7t17QBMsq1VFVdVJkUvT9y0PHzp0uvhXe3uF69o0DYvDGzRLQ9f19O02NaQi0XmaZkHXtsmFYmDotmzWZ7h2Sx8iZ6eWgGHRLNHGYGxFjNLc29vboxt6TDCpMWhKknIhFCEfrVWqeOVm1jRNgkE8Vls26y3gWa32UVGx3W5p25abN2+KcM9yRVVVDMmBOKZGmsyuxLHBNjv2+RTs2vSMrIrdplg5lxMI6AJmnzev5o/tJv3rgL/wF/4Ct27duurduI63OZ7KIy2EgGF0gZ1fdKpc1LmjDZBJvJddxLvvUV4iW9t5Rr5Qx82GiZuv1lpUvZLeQoYMTk5OuHPnJTablls3b3Pz5i1WeythFISArSoWTcPe3h7W2uLSsFgs8GFg2Hb4oRf34H6gqWtcP+B6kVKsjKXrO0Ln2Jyec/+NN6CyRKVolgsePLxPVdcsV3ulWWd0hbHJUDINTuRBkzyWodJQSYhRGm3pxlA3NYc3j+iHjg995CMcP3hA10klndXUTk9PuXXrFs1ihVeaqqrou4HgPdkhLZB1kMUGaLcRdhleW850OVezV6Sb4eXfH0gqZcFfoB5ex3U8j/F1eaRdXgUJ53Q0KExVrlJE5VGYOXv+EdtX06w6Yy2A6AnkikoT/agBYa2lri0y/aaLdToIv/bmrVt853d+Gu8DBwcHDN6z2WyKQIx3Dqs1N2/cwCea13a7IabEvj5fE2JIbr8yVmusxrmB6BzrkxOO799jvd0QNxFlLPViwc3FHhFwySI9fyYfI0PXE4wclxCC0Mby59VKlvlSHstxDJGqrlFaY6sK5wbqqiLGCtf3c4YCsL/aQwVN3weRULgkJ2olvGE53GOjTSt9oVolkmCPKUGXkmyn9N5ddoJ8VwBjCpvlWnfhOp7neCxlbDZZNMF1x6Vjvuh2qEcFz1WpUbODAe5UU5mXOzp27T5Hpt9Cea5MRGltSyUu8IArle6tW7eKa+/v/M7vcHJygvMy/JB5uXVdE0Kg6zrcMNC1Lev1Oe1mw+npKev1msVyya1bt1mslgAYq4lhQOFp2zUnx/eprOHo1k3qpmG7WbPdbNi2G9puy8nJcdJDsIBKQu8qcYBdabQR5HP4KMJCWklyz7Qz7xx1VbFoFgQfGJwrWhR5e3Vdp4ad5f0f+AAq8ZpzhCQ5JuyIUFTFUFIBh0togYLohKRWlrm+k+lApcqXaIrl5vOSvw4mjkJI1wn3Yty5c0f0QSbx5S9/+RrP3QmlFB/+8Ieveje+qXgqeOFRdKHLYdkJjhsTkyFq8gU7fV5OFrLtxB/NL9/ZeK6qClUtRoIXrdaqqnAJ0z042BdcdrFgs9ngQ2Bv74C+7/mD3/99Qow0TcO2adg/2Ge12kNrXRLg2dlp8VyrqopmuWRvtQKEprZZt0CgQtG3PeebtUAAVU29DHR9x/rsmOXeir69SWUrtts18hE99WoJRmOURilwrsd7nbjMRm48mU8bIcM3zjliFIF08Xyr8aGnaRYMwyCVeErAVV1xfHzCYrGg3W7RMbE1GFXXLmglKJUq3zhjGex2NadsiFlFHGECLhXthVm1PGGsXCeTeXzf930ft2/fnv3t+7//+0uv4joktNb8+T//5/mn//Sf8rWvfe2qd+cbisdXulAqx5wI5onzIgSRMUBVtnDx4tptyMTokWVrchTLOOKkmsqc3WzDHoEQPbmQq+umMA2UUty9e5e23aRmUsS5pEZWVUQf8Km6tEan6bGe4B2KiLWWGzdusre3x6pZlGpUEvOA7wdOj094cO+eCPsYS9UsWB0dYOqK9each/fvsT07JThxEj4+Pub+g/sE7xJP15BxF+9lYCJ6Pw4sKJko82mYIt/8soNEvVhQZ++2vYM0ZSeVfAzS5FwtVyP9LITZqZje00ryRBV4YV6NSqNM6fy68fljwp3/N23M5fHiXTjpWmnsOr6RsNbyoz/6o3zyk5+86l35huLxE2kxLfoTpkdqlJUJM5USqJLkOm2ujcVq6vw/sbLRKGWThoAuDIUYYkruZUsEYnleBIytODw8Yn//gPV6zenpKXVd0zQNdVXjXI9zQ0nmWimWqxV102CMTZxRjdaGGzdvcnBwQFVVLJcrFNC1W7puS9tvUBr8MLA+P+Phw/vYSjQZ6mbBYrXP4c2bLFYrtuszTo5PODs9ZX22pm23bNbnnJ0c03ctw+BR2qJ0jbGjlm8sCTak/+Is4S4SrWjoe6ytsHXFZrtls92UIY+qFtGfrm8vQkIxD1irgsPL6ZPjGvNSI8G8MeskQ3GyVFEVo9JHxzzBXtYfuK52JT7+8Y9zdHR01bvxro2///f//oXvSl3XfN/3fd8V7dE3F49PuiDjvJMLpEgD5ipH52bapJLJEG+k8Fthjvld8k6zChgkOVpb0dQLmmbJYrlgsVwIfLBcslgusZVlsVjifaDvB5bLZZl6y8MIuZLUWomLxP4exlppnDnHInmM2cpS14KrqXRD6fuOmCbPgg+C/XYdp6fH2MqwOthn7/CA1f4ey9Ue+/uHrPb2QUG72bA+PxNrdIT+1rUt52cndN16nHhLFXokluXkeIwmuHqUxX1VVdS1DHJYW7NYLgq1DEAlzDpj3Pn1+YRnmCGfIznPFFqgrCLiCBnkRpsA67OKuVSxOYFPb7yXQES7Nj/XAZ/+9KevqWKPiS996UvPFMXwySpj6d8Lo7qzzCrODuNjXjrwCJZbKuLJtqbYYUkoQSCDkiiyEEzil8YQJ0vSSNMsqJuaoW9pWxH4zhXucrlk/2CfqlokuUdJAgcHosi13qyJMXL71i2qSswe9/f3McZgrfiLnZ1tcL0rUo4KjR8GNpsN3TCwf3TEav+Aqm5QMWKNYZsgDdd3tNs1wR0WyCLGKEk8VesyPbYAbFk9KCgVK4wQjDGGxWJB37Y0TcPe3grnBvpty2LRcOfOHdrNFuccwTMmYDVhnMgan+yXlokL8r6T86vK/5UVh8pPniZKNXndDLoYmSdaKUI6b3Gy4plNtT3H8alPfYo7d+5c+Pvf/tt/my5ZM13HsxVPGAO+GGFe5qBEN5A4e0H2GkglkB7ZDPKyixqrOWnvNtucc0TnSiIwxuCCyDnuH+wRkl7B0dE+bhDr9eXeihtHR6z29qhsTdMsODg45PDwBoMb6AexvhFFLst2vUlQgzSinHOs1+ciEWmqcoPwzrHdbDk7O8NUNYvVPkpXVHWNTrxYU1uMKH/z8PiEoe8Yhp623bJa7UnOioF2u8YaK44VjcZakxJgSoxJ9MZ7h3MCNehUFW+Pj9lsNtS1UMaccxDTsfEOnxJkZhmMzZg4OXvFmOfCeZYEnel6U/A37V/qZRYq2WS7F2GE9JwQymhcTv5cNqTxnMVHPvIRbt68eeHv//Af/sNrbYpnNJ7YybiMtZDj0boJplRACj0Csikua75dtm2llNCcohiCayUc1hBkid11LTFG9vaEsdDUDdYahqGHVEkuVws+8C0f4ODgkM3mPNmZO46ODqmqivPzc0IMLJcN3vdst2tOT44FMzWGuqkSRCKTXH3XsdluqRYLbNOglRDjjBGcxdYLlvsH3HjpRfb2V7ihT+7ErXyuJFy+aBYYrRi6Fu96nPf4CD5VoKEkXU/fb1MjUPQbDg8FwmiaBQeHhyyXe/TdwMnpGT6ANkZGLHxIgxFpm1MYgBGzR4m2bq6GVSRp8O7QBseTUxJopguq3ceUQgzYBHfP/YC8ctm5S1/HJP7iX/yLnJ2dXfVuvKvih3/4h696F96yeKK0I4zKYJc9Nv39UXjerprC9DEZd00No0lmzs0klTMAuYEn5o7OOdq2S/snHfyDw0PquoLoabsWpRTvf//7iTHQdcKdhcjLL9/h6OgG5+fneO/Fqt172rZls1kXXDj/672InA9uYNO2MlFWWdFV0Iqu7TDa0DQ1xmgWqxV7+wfcun07qYu5lECFAaGN6Fb0fY9WMKSJNZkSk0ZWTrwg3VprTRHmOTw64sUXX2K5XLLYk4agrWqaxUKgEKtk8i5tS6CRnTwXshElBSvWKSHqyXl/ZOw06C5E+SoolJGhEibfER8DPOfshe/+7u/mYx/72IW//+Iv/mLx/rsOiX/1r/7VVe/CWxaPhRdmfM2dmEEDebmYlp7ja4R/q7BEFWZV0/Q9drbMvAIaW+cyGVbTOwdRYwwJ61zSLJbUVYXWhhADt2/f5KWX38frr9/j6Gifk5Nj9veP+OAHP4T3ntdfv0sIUUTAtWIYOqw1HBwciLYC4EOgrir6vi+6upvNhlVqvBljqOuGbrvFB4fSSyoridcNA6vDQ2KMDH3HMgoG2ywburYv1kLOiRllXdfi4GAMzAYVJGmGgmdL4rJVRbNYsjl+QIgCtaDEnHKxt8/Z2RnOOXwcG5mlaTbhX0dItusQE2ZbVhlpWk7kKCd2SqWpOj1NavaHsVEW0xuktB+i0LaNnmzx+YybN2+yv79/1btxHe9wPHE4Ig8OXBaScM3k592qVri3oxftWBHPx4lHCENcgSeVsoqIE6Khahq8dwzOIzzWQNtuWK8blqt9rK24efMmH/iWD/HGm28Qg+KD3/JBDg72OTy8SV01PHjwgOPjB9L1XyzZbluqSpwWVqs9aV4oTd2I8th51+G9Z7vdcu/ePUlK1hQ5xqAiWM3gHd4PLBciNmO0oWpqlIZ2s8VoK46/XY9RlmgMbugAzWIh2gnGGPwwCC3NDTjXC1PE6MIOEUZFLxWsrYhRs1ztE8j4b8/56Zq27ajrmq0fxnM5vYHq3BOL6IkzZUJzy/nKeLBC3CLyVFveZtZykEXIoxkJIaQJOC3fGJ2mDK/jOp63+LrhhQvLyWmf5QIdLMywv2lHfhz/VZdtakYviyh0Er0OIWCTSAyQYIGeN998k4PDQ1586Q7Hx8fcvHGT7/iO72C5XIoT8P4+bdtydnZCVVXE6PHeFRWzpqmxSc/WOdG07bqOthUfsvV6zWKxYG9/n2bRYK1Nx0ZgBYDBO4ahxzmRnazriqpecXh0E43Y9PR9jzGKthU7oMrYciz6fsAHx9D3DEOLcy1dt6FLiT8Ej1LCTMjH5uDggGZRc/PmLXxUrFb7LFcrPvihDxVZyd3zmQV2okoEwOnfE5TzaDrXfGDFZA2JGC+c493vjbBRMg1Ng3l+K93Pfe5z/PE//scv/P1P/+k/zauvvnoFe/Tei8PDQ/76X//rfPGLX7zqXfm64qmkHY0xM4xpxrGcYHvzJgnEVE4pRfLtImkfWFQRVpkkBa2ZdGvS/+vkG6ZwIRKCLxW2UoGmaagqw2c+8xn+5//8n2y3HX/iT/y/DMPAf/tvv8be3h4vvPACx8cnbDZnBD/gXURpabhZW0HUOOeLPq3WlpDkIrfbLdbaknSzzGJeescYscndF7IWAmkYQ7NYHRBcS99ucX1HZbTgyLUt7IzaGvq+o64bmZSLHq0iVV2hokabKk2lZVsfWC6XVFXF/TfvpXFoODy6ydC3uLDh4YP76QZl8VnXIVetKjXLpAtISPhuhhnUBZiIAjfkhlqMkagV0Xu00rMqN8NSF8TM86CF0sTwfGsw1HU9ajWn+PznP89//I//8ZpK94i4ffs29+/fL79nCuj+/j5/82/+TQB+/dd/nV/4hV+4ql18qnhipZspR7tiJmOMxpRTJ9v5hiIhyCjtcrlXpAhhrGjVTtWVOUmaiMmczpREjLUYK4lnb2+fT/9f38Wrr97l85//PEdHB3zlK1/m9dfvcnZ2ytnZKcPQ0/dS5Q5Dx/n5Gev1GefnpwxDR91YqsoW2ccQHN45hr7l+Fg81YS/azFGIU47kjBlAEHYFJJshWObxWfquqGqa6pFjVKRtt3Qdxva7Vlannu6rsf7gb7v8G5I4vAGzQKl5L5ojKGqa0Kq+nPTbLlcslgsUBhu37rFMAzs76/oupbbt1+UG5X3xCSOUxTJEObCyDMhTZklvDfBBZfxqyFh8bvP1yOumw0256+//IZ9HRJnZ2fXCfcxcXx8fOnflVIsFgsWiwWf/exn+fznP/8O79nXF0+odCXxxcnAwjSy+Iw2THjz82pXlqOZiaBp2xatNS+++KKYLCY7knzhjvyHtFGliVHUyrSJQEBj0Kbhox/7JC+99DIvvvgCIcCv/dqv8b73vY9PfOJjvP7669y4cYMXXniB1157hZPjh1SVTQk+d+zF7XcYWpp6T7ixMRKDJwbH2dmZ+JEtHKvVfoEU5GYUExvBMAw6JVmdml4iEWlSU6yqG5z3NCvF9vyc9XrN/uEhMXqc72lqg/dDSty5yaUIMVnaeNDaopVUSMLLlQO8Wq3o+47gPa+9/jp9L7S3O+/7ACfHD+l7R20rGAa882XgJFeukWnDNJa+V4zZsl2qa59cLfLr883FakNU4DNZW4GKQUaFy3dGp/9Xk2mbp/6OXsd1PHUopfiTf/JP0nUd/+k//aer3p1L4ymSLsnO5hI34BhRE5Km0gpiZjRMnycVUcjirsCDBw8YhoGbN2/SNA1nZ2d0XZvonQq08Eu1FbqTVqJWppTlgx/+CC++9C10g6dtt/zGb/wGL7xwixdffBGtNevNmoODA5bLpShwWRkVlipVF20HpQ2r1Yqzs7MJtYuE93refPNNtNYcHOyxWh2IELnRY0d/UvnLY6ZgmdmIUimpMAX3tcQI/bYleofzPbXWeNdjrU54rynbmVaFIfgkgDNgrRWTzvTY+fk5r736Clorlsslbatouy33H7zOpz/9nbz2yiu8+od/yGpvNdvnaUOTfIyBkNqjeQrNez/Xw0gDKQoKLS2ff7l57pIEL45gPM/Qwnd/93fzPd/zPbO/fe/3fi//5b/8lyvao2cv3s3fryeyF4DR+nynOy2jnI5i9xui4LIEiotY8hUL+LK0FVUxWYKfp8rPWsudO++nqmq22zXr9Tot3zWmshweHfLyy+/D6gUPjx/w6it/xJtvvsnNo1vsH+7hnKNplty8eZvjk3s09ZKDgwPeeOMNtIqsVktkYCI1l5SwAhaLBX0/sFmfs16fAxS2RgiBxWJBCLBNBpSr1aIcl5wcs/JX1sidjjhrbZOFUCM4aYCmNrTrjeDJQVFXFSDJPFPonHOSXGOWvtQJG7elkr5//z4PHz5ku1mzv7+H90PS6zXcu/cG+IHf+K//lT4EPvxt30oIgbuv3SV6uXlIc0slfY08jJIq3JClN3eseZQipGERtBLtjR3sV07y9Hs0jm6DKjzg8JzSFy5LCNewwvMTT3SOyEIxl1W6Y+WqC/1HESYdmSRLFUYPrfnrffq7SZXlGzjnMdqwv3/Ai7dfomkWtEPPyekJv/e/f5eDgyNOTk5BKZbLFWhJeKvVir29PTabDU295PDwkM1mw/7+Pn23nbxXWuoajTaiTWttLInWJ4aB4LF1uVFYa1ksFoXPn6128r/Tjn5+fNrJt7YWB2BT0w0ti8UC53rqusJHR/Qa5RzGVDPsUynSjc1jTSWuEyEweCfSk6sVxw/vS0Mt4eonJw/l3ChYLBfcuXmDu3fv0vc9e6s9bt+8BUTatmUYXJKslCGVupax5+B8YiSEcgPJ1XFO/MI4ywZDcUSV5vIM6d80dKwgKtGeUNo87uv3TEb+vkyj67rrpPsUsbe391TPy9fBuzWekHSnvmjj36cVr1KJoZCaYGXqSeUfdPrX72w7YYj5kk2dea0CSsF2e0rXrRFvryjaCNry5ut3CVFRL5b0/UCzkCGG09NTbt++zfn5hmVSIjs9PWWxWDD0qjS6Smc9fZ6+69PnCeWzWqvLTSbfeAR6cNT1ohyD3cSaT7T3HmNN6dBnyEAZjYoBWx0QfaR3fZobkHTVdRux9lFisImvAMGOYwwE67BVhTGWqlklBwhS5Wo5Pj6m77ecnj5kGDwRhY+BN954E9Q4On337mvjF1NpotIYrVHR49ww+1y7AzIxRoYg0M9M3jFl3Aud2ZhgqNJkE/gBsqvI8xNaa77ru77rgiThF7/4RX75l3/5ivbqvROnp6dPfI73nq985Sv85//8n9+BPfrG4ql4uo/TRki/zH8fn5X+FTrVo9jwBb9M1eiY6LL5pEmuDmds1hs2W3HotbZGF76sLthsVQlGrLWm62RsN/uQDcNAn1gKmX6V9gIgLfFNGpZYJcaCKHzlm820M6+1TmO6trg3gNwIQhpjFnUtRWUbrK3Q1qIrizKGyjYYXSUXCItNXmJ919K25wxDmyQpG+q6YdEsWSxW1HVD0yxQ2rJYrahszWq5hzGW5V7N//25/4eIZnCuMC/ycQJGNobVWKtAebSeYN6XhDFGVMNE6Yaox/MtGmyX9Me0QltN1BMMOURUjDNY4nmIj370o3z/93//Ve/GMxvDMPArv/Ir7/ob2FN7pD1qJFgET8aKUDGvAGUDyZtrh26mVC6JVbIFH1+r9Kg/kOlZbhiEp5te65yj24qc4csvv8zZ2Rl937NaLYlRlUTpvC/L6PL+ijIIkRtFeZ/rumK5XBSpx5yg8n+X32Ak8o1CdHoiLngCGqUtWlusbagr0QduGknCVdWwaPalUagNpqqoq0WBLbSWhF3ZJWDKPpls8a7Enj0PWJyvz/nqr/83nHcYay7ap4PQvZQqjAKVR4wjF9S/iuhRquRNvuHEHQw3ymcukEJK0D76kd1AStAB9HNU6BojI+a7cffuXc7Pz69gj95b8YlPfOKJz/md3/kdfumXfukd2JtvLp6sp1u63RdFxi9PPHFe0D5JGCVfrWhUGu0t1VBpuIRkuSP/CkVNkure3p5Yjfd9quYim806TXcNaKXQCuJ0fBVR4Kpri/MdRlfEqIqFe37WcrksTbIyfZUfzctvY6QZlSAKociNn0vFCEnsxlY6JW3NMAw0zUqaaSoPH2gEWha9BWNy7WgIUY6DweAze0AprFZYDfv7+2Il5D3HDx6y3W546cWXeP3114hBHJAzFzoCQYlGbvkcUY65Tk3G6enK8ICHbBIyDsDEicZxiJPni9GohmKAmc9pVKCsIfpnP+saY7hz5w537tzhB37gBy48/nf+zt/hF3/xF6+vLoLmAAAgAElEQVRgz95b8d//+3+/6l14y+KxSVcSoPw8nUjKUcY+L0ms42OxcDsLRXNadRUIQ0RetJaZfJ27MkoTAlKpescQInXCf21Khi+88IKwFLThxo0jXrv7Ci/cfoG+F3zSpgSWBz26rhOLnv190ZbwgUVTzwTSjTFst9sCTWRmQZ5Gy59D68Rd1RotpR8xkmx3As5FlBpQ1sggRrXAJmdg0nLeeY9O7ITgO2FEKKFgKUySRRwnyvKx01oTgysQR1VXDMOAtRV1mnjzLqAjpUKPId0MJvoKGZ8lJ0Q1PzfyHEVAuLomjiuRGaNFCcavVJKRLEczn3dkVZRf8+5l9XzTobXmwx/+MIeHh/zZP/tnL33O1772NV577bV3eM/ee/E93/M9T6SAbbdb3nzzzXdoj765eILK2IjpZuvw3YsxL1fLdaemiEWufJh15HdjOoU2ymrP8eTgxZpcGaF6yQiuJL/tdpsaYBVd3+H9wHqTZBuNpa4qrJVBhTx1ljFYwY4pVawkVoUxkqSqqpoNRUDmLfvU+Bs/r1IarQx5xDmEfAyFilWlyboQDcZYXBppBuj7NlXUPnm2SeNJG4MKAe+zaaTCe9A6D5xk2yRF3TR0fZ+aYSrdiLTQ9jK0k0evJ+cwU7es0gIBkbnBYYSWEBKg1pqYmnRT7HeK75cR4vG0FmaDQBqzYvmZC6UUn/rUpx6ZbAF+67d+iy996Uvv+pHVd0P8m3/zbx4vMwq8+uqrfPnLX36H9uibi6fSXpAL9uLfS5Wal9RjriRPbeXRUKESTZafaroMlxdKZRzK71oblDEMzuFCxHshJnknPmX7B4cYYzg5OUk2PTXnZ2sqW+OdxzsnoHUlPmmRkHRWzGiAiUAPfd+VBJaHD7Km7kyzQElVrrXADRlakPJ2Qp1TeRIv3TSCx3stFWIwuDQ+ncdzMydXRN8pmHMkpPeUm0yMuZmXWldKqti6rtlsNsQYOT05FQaFVrzw0h3uvvqKrDjy2G46FzFObqz5TOQBByhiRTKllqCHGNHGUKx31CSB56fnfJuq56JopijPlZe++/V0X375Zd7//vd/Xa/RWl8KJUzjn//zf86//tf/+pvZtet4j8YThiNynpRL8mITbJZn03JVzRJvAg5LNRxL4lUo7WevHv/VoCxKW1BKBFsSrmytJQJt20IUpaG+79Fas7+/z/HDh+zvHTEMLlWsstzVCQKQ6z7pDyhdEqiI39QFRpEQ7moWOo9RxoSNluQpn8Oj4oj1yrCEA0z5TCF4lI+4GInGE4JLmK0VuyGVOMMKsYFPNwOtkrbujMlhSxWulLmEdqXwrqdrN1hruHnzFm/cfZ0wdOWkqnyuUqbNMEMkopUmxCBTZUoxMrEneEFewZBPd77pyF8zdkw65yF9LYQqloeL391x+/ZtPv7xj/Nt3/ZtfPSjH31Lt/3Vr371XTui+m6LH//xHy8rzWclnurTzIgIOzSyIlJTRklVWWLmCxCmEMN0y5JU5mFGSlYyM5ySnYP3EGMRFs8VqzABLFVdi+pXlMpsajeklUmVNLNxW5mQg8WiSZivDC9EItt2TWUXSdhGKFMxVbVaS6soxlBUxmLUaZQ4wxUKCFKpJn3hiKdSNWCK0lRIFb5ODbsQAyZDFUqOdD7uu8v6jFWfn5/Ttl061oIte+cTde0SvD2fx8k5CcnRoSADSqARM95JS8rNxesE1Gc3NJN8nBL9u3lE8+joiM985jO89NJL/LE/9sfe8u1/9atf5W/9rb/Fv/23//Yt3/azGD/5kz/5/CXdKY67SxvLF7zRkjylATYmXZUaK7vKZNJUc8Swu7xMfF90avUwuZDlyvXOoyv5PVe8OXEppdJwgUATVbK4yfsvS+6RY5uTZX48J845JUymwkTPVqGC/F1cbl06hJkyp0rDzftBKusoeKgPHqMU2oiimc7L+DzpZUZjTqXAKBn91aYiD5rIe3icU+Uz5+PqnOPk5JgbR4cMwzhdd36+LjegPEV2MeVN3CJ8HLm4TJtsF74Z5dSMcEK8JPHKmPHkvvGujYODA/7Mn/kzfMd3fMfbsv3f/M3f5G/8jb/Bv/t3/+5t2f51vDfiCW7AYzdfqTj5mUSkr0qzR2tN9IHImKBHrQXwycGg0M3QaUYt6TSki9UHBUZJlacNcdoEUhRjRVAYrdlutywWYrOe6V15ui1f5NrkijRhoelGksd9cxIeBWfEF622Fd4FgvEsl8s0KisfyCuP8mCK1fxUR9ajtEJHmRizuhpZGdpOxghiElLXiQOrCGkJnsVyYpBxWdG58HivChaqtFTc3nliVHRDz+///v+m71uGYSBG2Gw2SSGMGaaaul1yvlIlGnL2nDTBVPr5suq03EbV5Bcl3Ahyg7Vk23mlDe+OitcYww/90A8BIhV6mWfZWxG//du/zU/8xE/wH/7Df3hbtn8d7514AqZrRngg+FQNupRUDdpYtNJpNFWL/1XiamqtMdaQSfg+VAmXFLUsotiUh9xsyQleifpXdj3IYjuJb4ROFaH3oraVNRNGjYSsY1ulqjkJ7UwaXLkrnxkIU6ihaRoGJ/hnCAGrKxRSURtjMEoaXpmOZerJYIiSilZrlexodBrAcCyXMj7snYh+ayI+DAkeMYA8N+pxrivjwzatJJQesfX8GVTi7QYvxpZvvP6aOB+vT8VQ0w3F7DJXy5FxODs3u0qDbQrdppj+nif5ynZ08f0pGHBUSTQnJfOo9KzIfTck2x/7sR8DZP+//du//W17n1dffZW//Jf/Mvfv3+dXf/VX37b3eRbjH//jf/xUegv379/n3//7f/8O7NFbE4+HF7QWo8QYaZLweIwVSkuVaZKgeFOL/XlwQ9FZlZfLpa21iJ24YUiDDrKs72NLdBGUSU22IAnX2FJ9OicXtEGWvLmhVuxrokpVnQi4HBwc0XU9IEI9MTiWxqSmmRIWwyRpFUw10ciGYZCGktZ4F6gXkgxRitXekqF3BV+Oei7qHmJI2mqjgaQk1ZFXq7VIODZNVfYpy0IWCCStHASiiKKrG03SEbYzKh/pszx8+JA37r5Gu9nygQ98gLuv/SGKSPChVPZT1ohSKpssj0kwc7nY+XuMJdlmGGImPTlhh43pVJK8RuEnsMP0u3EZffCdireros1xfn7OF77wBdq2vZZs/Abje7/3e58Kz91ut7zyyivvwB69NfHYT9QslmUM11pL0yxKkybTnJaLFcbK8nnoOpTKRoiC5+bXa22wVipS55ws5a0ngIjLBEAZrK1K0p1rvkpCU0qjjMZoK64QvePoxmFaFU8qTu+RnrvCuYAx8ybgVDth1KwVStlqtaJt28Ttlb+bSnQctJaKeIRO4phQ1ahrMCYoyDbuWjUCE0SXYBhpsAFlGi4nspyI0ZNBgrT0F5hiFB53zjEMXSLa++Kh5mKYYaxj00tWJShp3GWebqag7SbDSMLqQyysjby9AiHE8W+pvSrHbuc7NUvW77H40pe+9NTTY845fvM3f/Nt3qNnN37+53+eF1544ap3422JxybdxWKVKhxJVHurQ5TR9P0gOKqtqCqZrhqGDj9xFciJaJq0VdIIEFeEQJ8q1BiDLEG1wdg6UaJG80qljDSUoheFqxjRwXN8/IAbN6R67bqOumnwfqBttzJooFOVHDxT94ssdKO0NL5UWgtn+tVms8Y7UI3Cx2FC3fLivpAoZ7IUlypYoIDEUggK2WhM+OuYTPu+p26EN2xtVfYnJ1BRFItpAANU1BirRv4uAa2yrbpO22x5/Y3XOT87pW23/J//83+obA0uYqtK6u5cnWZiX2JAlCnDafItjBNhLeSGZjQjVquUIvrEw9aZvTLHbjMzImPQu4M17/b4uZ/7Of7e3/t75fe7d++yXq+vboeeo/jkJz/5zLEWcjz2Uy2XS/KwgFKKw4NbRCVJtKkXaTy3pe86urbF9X1JuN772VBB3/flgnPO0fWtDDD4gCdgqopFZYu7bJleSktyIeSDDWk5bg1939O2W7quZblc0XXdZOmLiMwkSKFoFZQTGUt1nEdjgaRnq0SHd2jlcRXHVXfRcJDps0zfGqGECUVNi9tFDGCsFYWxBLlUlSH4iItuhjNPk5IxtgxfqFRJG20wWYgdcSzetlvW61OsFfra2dkZm03LaikOFmIh5GbbLnDDJRXpjFIGIwySjlJZgUg2H8eBpX4v28zbysdnl274bo5/+S//JX/lr/yVp5ITvI7r+HriiY00rTTBC1vBucBi1RTqUdttOT87ZWjbZI8ulKvMq50ul/N/4gYx4PxA5wL4AEZL0ymA9lJJxhiSfsFUONtDWnavVqtSdWw2G5bLParkHyZVYkCnxpTCoHXEWE3Eo5RGG0lgMvXlsbbGGltW4y64NCac9A5iLPujNZgyLBtnnxFGoeqmaVBKU1V1GR02JunsGktVGZzrC5QgTcFR21cp8VzLjTaUJmKISpXz4r3nj/7oD/ja//otTk8fJAt4xwsv3GZ99jDhsYknEsLspgCZjSYwSIYJ5glSqumsEibNO+RYkGAHGV8bhy6g8HoVFyGGd3P86q/+Kl/4whfo+774913HOxtf+cpX+MhHPnLVu/G2xeNFzENkSB32EAKb7RpSswcF52cndNs1wUtzKfuGwWg/ki/g3OyS57nUsJJlftYEyEnJpySPUsnZQZbjVWXRyDK4rutU6bZs1mcslnvcSEk3hEBVG2LUaakuU1beBXQSvgkeFDFNrJmUcGOBDlQAH32BIiBV30aSXkxW60mRoHzOXJnnat/asUGW4RZTWSHKaYWthepGOXapiVcwXnlfwboztU6OwRAC63VbVhTHxyf4oaPvWz7wgQ+x3pwx+A5jNM6N1X6BGdLyP+a8OaMppPM46ZLlSbOoYpnumyLFOXErFF6PThKPGvd9N1S8p6envPzyy+X3EAJ931/hHl3H4eHhE7UWcty7d4+f/dmffXt36C2OJ6qMTW3VjXFs23VphHXtluj6kkx3/4V54yQn5ayhMEqrJPzQqEnCmi+3ReYwYpcVzvmSmEIYWK/X3Lw9JntjVREfN0oXKUStTRrkGO10ckWZbRBiGiQgOJTJzTypUuu6JtPlAhGrTRK4EYaCnuCkeSgjwyy58RZCwE6S85SxMI9cPYveQggDKlRp+EKVZb4bBn77t34L73peeulF7r/5Ol235cHxCV3vCGFTqvCL75EpXZNfIAmMJ9bJDoWsnLFMQSs/z3Z9Qj17dGK96mZa13XcunVrpt52He+tyNfYeykem3Tbti14ozjr9gxeaGHr9ZowOHTiHeVKbppcpxfVMAzzx9Plm5/T+cTfRWOtkRFbNUo4hpgTbSz7lHHUvK/OSTWc8cuc7CUBz5fVOTJHN8SRPaCVhoShajsKhiulS7Up+gtg7GRkdgIv5KW8mUyaxRjF6DK5SIyVsSuVaH4u5NcmF97iQBFxLmlBOM9me85yteC117a8cfcu280xMSoePnzIzaN9Th4+RBNnDsOzCnM62FD+pOYshUIpU2MFPPmsOeL0+VGw65hm20b22SX7cIXxXrtgn/X4er8XV33j/kbisUk3c2611mw2G/b29ooW7TD0JemOUoZjdbubdNEyOeW8S+vQiI/j80kNr+12k/Rga6oq45lgtHB5jRl5rU0jWgmr1arspw8OQ03XdVKdJoaA8HHjbJmdYZMYI1VlisKW0qokzUDAKpu2PxEXLw3GEcfM2516pSnl0/HxkMTLbdXgXJhM0TUMoS0OwCN/Nw1vqPw+JHzb4J3DDT1vvPEGxw8fcL5eMyRRm5BGlg8ODjg9OcEkCp1CuLlx9r0es25JoBERGNcKJtS9qUg5TBgRatxWwYwJAtUERSYEj6sWM/u+XMd15PjlX/5lvvM7v/OpnntycsI/+Af/4G3eo7c+nph0R1GYyHq9JipwXQ8+Qgz4MB0Rlpgm3dzF98Hjoy8i2UZprJfpJY94io2DFSYlkNGxoW4W+MyMSE2cqqqSq66jH1q27bowLnLzrV40GG0YBldoYzlp5GQiyVIwWlk1G4wWOlRtFhgzbk/2XQY4gFl1ugtdymcJs5vPFIKY4t/zYYE0WJDxZBXQqsLoCq00LgZUUkZ7cO8ev/t7X8O5jhhcqtwie8s9GWCZ6Olm+CSiRtnGaZmbBymUAju36JlWIOMqYmy2kbgNcqMw5bNPj8mURnidcK/jm433Kvb+RI+03JgKIbBpt5L4gicMfcJUA8MwJIL+QN/3M6qY1rpUtCbRvjKOCRQa0m5kTNQ5wTvrqsGY0QDSWkuXtBO893TbLX4Y9zVb7Ii9tTj55iQ7tUzP+yVNIo3RaXzYZJcILmmUpemwxFKw1mCMsA4iYVbxkz5lHpvt3YCxiaY1qYjHCnxKQ5Oq2pgkS6k0QQkLI/hA27dEHB/+4IeSoE36fCj29le07aYk94LnJjrtLlZbHhonhss5nC/5VGo2jsMl0y1pPSujL0Q+P/mYX8d1fCOx2Wz46Z/+6avejW8oHlvpTpOnCGzD4HogFs+rXM1OK5cpPul3Lu1KKVwIKKMF7YuKmB0K0oUYkviLtVXCd6Vayz/rhB/v7e1BSihKwXa7pl40LFcrSXhBNBhICHJlG6qqLhXtbnNpmoxjCCirMFoq2KlLrlj0qKQDMVanamzfp+eJ/GNIVbypJFG1bYs1NSHkCTaNikJCCzEI/9eYQsKCPMaMTMQFGTler9e0Xct6vZaJwXpB13swcPf11znaX+KDK/BPpoddaHwxoYLFzCbJgw27zx0dK6afe16l50GJcdsw3oTkmL+3BiWu4+2PF198kaZprno33vZ4Inuh0LxiwGqD0rEsOaedwylXNVeySk2XseC8p5o0VFARlcS7dbrYXeKU5qVofg9JhmMzKPuCKaVoEqvAaE3ftvTLJRqFXS4haSFoxaQhBhBKkoeRWzv6wEmiyxQzrUXCMnfvVdI7zLmm0Kmi2P/IdJk02xSCjeooAuhA4h2LKLnSiqh9SorJATg19ApXN400kya/3CBTeGenp2kYYsN2e04IjuVqn6aSasCYCoJoEKNU0iiW/cv7vRtCC5ucp/S8keebjUpj+nm6LUm6+eY15Wzn7WVq3OOYDdfx/MXP/MzP8NnPfvaJz4sxcv/+/Xdgj96eeKo5O9EkULjgMZc4uE4x0pwkC3bpQ1GjMlompgyREBXRRbwaCDEWmEFrjV5UEwaAwg0DKEVlbXGZresaGEdoMxcYJRoQlR6FbHIDLEwU0GJEFNKS/i2MFXpOwColugw9yJI6jNBDHK3lheObDBuD0Lycd2ijMNoSvaf3W/RihVYU9+KyKkiwQowRTfZaE682KbUF8ohKE5XCOc+DBw84Pz9ls1nTrjdp2V7Rth3gUMqyXC7YbjfEpBlRKsw4nrvppFhRqpw8nn3eBFZQ5dzIjXYc5BhDiQW9Gt8v35TltRFFvvldx3V8fTEMAz/zMz9z1bvxDcdjMd3CZU3VavbImtLCdiuanIAzTjnl6YKsWr0S5oJQsGR2KYjdQnlt13W4pFrmvMg3RkZcVTzRGqqqoqrrgvNqBdE76rqacHGz2tlIG5PnVzO+7PTnDD2MTIeKyMj/FR1aScsZ+5Zpu4RVRw/IIEgIAecdIbr0uXpCGGbDJMLSEI3igJIkbITJoFPSJVW/MSj63nF6+hDneurKpBHgvHy3AkokxbY8fDK9OV5+whOkMAHaR/hFlXN9MZIDcpxs+3EJNc7hhuu4jifF7/7u75aff+/3fu8K9+Sbj6ewYB+XzVopotYi0ZiSrN+hiWm1c3HqPHU2UsNgXHb74GfPn8IV2cFXoYRu5jxNU5eleU6eMS3bMycWPNvNGUbP3XyzpKQxVRmvVSpeaOyMjIZcCUrpZyYd/5hGYjPXWG4aEJWsCrQWkXbBUjXaJEdlBohirjltmgmvNULSziU11STZQRI6SEMHga5r6botp2enbLZbnOvT8XMs6pqhFyZB27azm+PuzzlmegmTJtoI7SQKWP5MagojpNfJpEQ5RvIe0ogkTvnVRmQ5r0vd60jx2c9+9rEGoD/3cz/Hj/7ojxJC4F/8i3/xDu7ZWx9PhBfy8jJ7g+Vkm8d+fQjlIs1WPWqaeLVMODlyQ250Fg4TXDFfkDFG0RYQaTMAcW9QA0orujYwJDx3GIZSKeeluFKKdtvhHcSouHHjBsMgqmhSueaKd9xHredNtDHpSioKzhO1ScI5whfOOgS5ioRI9IGAwADEiAqBaET7oU5Vq9Kyn95FVKqE5eaS3jvhpjondg2kalcafCKEfn5+xnazwRrLomnw3YLg13IjspZNDKBgGKY+dWOSLW2vmAQwd5JwjnmyHv+286zJ83e/QOPUmtj2jDe267iOHD/xEz/BZz7zmcc+55/9s3/2Du3N2xtP4OmKoHdOVpcNPUi1ly7s1DjTBd9Ltt9KFfW/GKNYo++MpE6nuZSSytZrT0iuO1MR7TxlZq3FDQNVVRVBGUneUi33naHrmqJ/kFONuFfoCSNh5BNnSlkZB0b0GmIIIpgTYqr8I1GPNurRx8Q+iOhAafpZOyYXrSx57na0E4r4IEI3UnmPZpxJP34qrUMEvHOcnh5z796brM/P2W625RhmWCSvLpQSuGEYss7xpBJFGphKdi4J3swbbNMG6fRcZdhmmqxLxcskDavMTyuzaeU9ruM6niZ+5Vd+5ZmCoh5PGQsuVZ9j91mFOKl8VCE1lYs5xskVF8dEO2vcCIxg9UgRm17YGQNVg/ivmdTI2eXf5tfUdT2hrmXtXMvgOrbtOYtFU4YtYEyu+b1y9VsaaCnxkit3rUqlWZbVO6R/ee95VSlV+LzyVyqmceWa4CPGkMwg842LfETlfYoeLzDBzrfbjYj4BBEwd8N4PLI/mlDFZB9nlWXqo+V8SLLXUXGuhTFNqLsCJLvVc3n+zt/k7caG2wjXXMd1PF380i/90lXvwlsaT5B2lAvLOSecUa0KtWv+XxDql9bFAma8MyXKWYiluhXeLeKppudL2FzReudQKvF6PUVwPCfDTP8yZoo7KqwVlkHeVtd2bDZrQojs7e2JbxtJGMfM8c2pHu68iShUKykNJTFqrYh+PEa7komSrHWBHxRZOD0zJQaMqWYTf9mE0xgx1yw3ohjF6CEdy2EY6LoWowX3ripL8AMhyGfZbrcp6Y6NyQvnNv9/SuYlWT4COtjFgS9WHok+dwlqMJq1K9HQuIYYrmMSf+7P/bmnooo9K/HEiTSVOuZGJ+PJ3S54FAghl07zikglzuqksRaTdY8xoMZt7V6A+X2DD2Ln4+V9gvPEEEQDwVY0zVI+iL5I/yKCHxzn5+dst2sZRU6cY1QSOlcXE8CU3qRUEg3XhsjocpwpXbs4cPn8ilGAXAk9Le5gmrkhaLQkIx9EyyIGx8wFmXyPkkbgen3OvXtv8uDBAzbn50lnQmhqxlRJXMiQ51V24aD8vyymE1DS9Ns5B7kqlWNzCeaboJIYxp+ZLnTSL7m/FkIguzLv7td1PL/xp/7Un+LjH//4pY/9/M///DM3Mv54Pd04TmypEB/JAippIY7KYXMMlgsJKoQgS3Y9shYyTzWzJoTDKhe8jiLyErxHWcNiuUdV1QVayCPJMfrCVogxEpwnmAHvHJv1uWCcWmG1po8ypFDX9axSnVbdWV0sAymSXE3SfpkLtcOIcecpu1xRDsOAyZU68yQ/DAO2rgje4YlobREt4Oy6m5b9qRoWl1+H0RqrBW/OfN8Mj+x+UXex2WKhk28DITXKcpKcv7rgw9Pvxiyj5m9HlAYmCeKPISalsVJbA3NB9eu4jkfF//gf/+OZuzk/9ls/nZOHeVVYkg2JexsCzvuC3e0ut8fJLj0mhMwkCHOn2UxVyu+ldSQQiijO0Pes1+dsNhuc9wzDUCrHTCHTWtO2WzCKGDxu6Nhu1mzOzujbrQxcMA5XiHLaUCzdc0w/czaqLDeYONfyLF+OKDhmtkXPjT/nhKcrzx3lLmFUAYuJdTBtR+WkmG8EOcEKLziUzzGd/BpvbikJMk/08+ZYypAJaph+limMNMVxM6/64vMmhygoFAZTblnMtnXVF1Nd1/zCL/zCle7D8x5/6S/9JX74h3/4qnfjHY0nVroZL9WMF/Zc2FwnHzIZYw1AXtfmJBt9kKmznFzjmKg1Ug35CW+zJPQQiEooWFqP2gAqwtD3uKbn/OyMqqpomiZVeSJAk5tuef9UFMnDfhC3iawSpgBV18lGR7DpfKMROABiCIR8I/FBluI+zNwgpseMGIg+EI1BVxUQcG6Qql2NDhuZVWBMOkZKxoLHZJablULZ8wH63nH/zTd45ZU/5OTkhK5tZ8m2bVupWoGo50n2URjqpH1Wylx5rr7wmnxsghdjzHldnGGkkSEhIkDvzlBK8bnPfe6qd+O5jg9/+MO8733vu/Sxf/SP/tF7VknscfFUY8C5+jNaEtg00eRV5tSSPA8FVFqmobQVz7OinoUk2spYoREZjYpjMs+DCdK3ktcMw4BJtjohBIIbkhatjPv2fY8xhnbbc+PmIbYSLNXkJX5K4sPQs9mC94H9fWEuCLYrSTuXanmirrIZ8nB41wt7IgYRMS9shkllGDwKuVkExPqlrmt88MSQKt6hY7FaopTgsN4HqqopVfcYU0hGfhcjyjXWalaLhnZ9MpNKDOlGBnJzQk0ToTAi5HBcTMCip5AVziJlYCPF9GYkzC+xuR9fHwqUNN/uHFO+jut4mrh3796Vr4bejngsvDBd7oNUplN61xSDhfnFpbXwTYfg6ZNweaFWAS4GOpcm23aYA5LwxElCKmBKdZWX5ArYbrc8ePAApVTBdutazCCNzaaTwhjINjd4TxiGJPfo6fuuNK2ytY5PkEX+fAVHzT5vflwih0ugEcFAR1aDbDNpBaek5PpBlNvUyJSY4sJqJ0EqIjF6Qow8eHCfN16/y4oF7pEAABfLSURBVMOHD4oG8Qw6UNIUm9K/dmP2XuW1edhD9j8yaeYlHCl/9lGkYQ5VTPHwR73vSLF7thok13EdTxNP7GQ86sIYOZcXn1eUpVKjSilVcNecjI3WwmBgTDpT/uzYkIoMSSfXJOucEAJD0nWo69Fp11pLiI7tdk0IAWuMWJAbgzImJV+NSe+VtYLbtqVtN3SdjNO6VNECBWKok+XPtBE1vUlcgBjSeHFmdNRVLZq3UyujiQZBfn0W49k9HjHKUMfp6Rk+RFZ7eyyXTXHSKMmMxHQY228XqvHp3y5EzDoKkRhdek2uYC1S4hpg1NRNWyRLSE6hjGnynSbcx+7DdVzHMxyPhRf6fkw+hkklFSf8Vi86uzFNUpVkka5vHUkC3ONAxW6SkYwSMUolDFe4uJVOFjqATTSzIrQTIqEfePjwIV3XcXBwwIMHD7hz5w7eB4ZedGSXqxqFcHRjdETf4V1g6DYENwC+dP1BYIymadBKsVqtypK67Qfquk5Y8agIJjcBjx96gUMSe0BrK+wMxko5OyqoZPeTKWWiBaFnCWlspuXkpIQu5zqiCxw/PObs/AzXD5fitVOWQN6GvJdJf5P0LBW5mr+w3BBsoiDIAzH61OiLKKoEL+QKfdJES9+RvGnB+idcsjj+/Tqu47L46Z/+ac7Pz696N96WeGKlm7v6OSEUFa2kvQAZe02sg5CGPbPVeRbFKd300VAy05u0McVpd+QaRZx3xTIoV8sw57gSYxE+Pjw8JITASy+9xGq54uDgoFSLzrmUO8SNoa4NdZ2tc7wkTu8ozZ8YC5adm3JTKlt+zPvk+4YvVvQyykuBKvLnNslOHkhUudFfLn+mC5NfWqURXZUm0dacnDxktVywWi7LhN6jsK/dY533O2Zq1yQylEI5qyoTNUbIAFLydexaEU23k7cQYywmltPPlLd11XF0dMQrr7xy1bvxXMZf+2t/jb/7d//upY/tsoiepXhspZs1a3fdfedjuwkGmDRRIlGW/8aksftISBoJuVqFOZ44JmOpfDPbAVRiGxgRZ5ks45VSoDTHx8elKj07O8N7z97eHraqODw8YLW3j9GiaBYiKCsNOq1HxbJhGKibirqp0Uoz+AHjNFVV4ZxjsVjMKG9T006jNVGLUHpOJN45TLIcyvi3jP9WZd8lkXmqav68LHmptfCYpS6NuADHJ6ccP7zPyfExm/V5Gnu+DEYYMeEiKD95X5mQ04lbPceOp7zbC0k1RlAmNd0myT41+ohxZz92K1qVHJ31haR/VXGZNf11vP0xhc+ep3iiXY+1VsTDMwU1jlq6OWR1OS4fY4iz5lIMgYAkIs/oBlsOeGrUKBK0kA0qE91MhdQpVzI5lceBvfeEvidaS7ttaWxFvVwUr7Z60WCsplk01HXNMPQQhcGgolTnwXkqU2GU3DjcEGgahQkKFaDvWlRKmNm3rVTxMWCNxWCIupJqHhk4sJVJ1WtFxjtJn322vlDJ062p03JcbiSCGVuyP9vgA922pV23iC6mQAO5kTfn3TKl+V5gWRS8PPrJcAOIcuM4WDJ9/mSH02vkDXRa3Qi4n1gNZLqb/CkoRcQToxmNLssX5zqu4/mKJ1LGykU9oXPljvwUp0XPmQ7l9anJlJe5l97ZlBK9WT+3dpHtSSU5nbbKdC5JvAONEW7rMAwc3rwBStH1PYvBcX62QSvLchlZLhf0w1b2RY8OZN556rpKswFKqGnW4tPYqoJCT1OFFaDKaG9QoO1kmZ8wzqx+lqlfGU/NMEKGEsTUMmkDAzZVuTpR9EKqiLt+w5v3XuXNe3fpurZUlVPcFnLzL+lbqDF55pg2QDMmq5QSuUnlLz2PuxXv7vvups8Yx39DHvjYqYKfxyrnOq7jsUk3V1DOOawaO/X5sYLxpQsqc1fzxTjt6pvEYphRm8qVmRotWlHpaqbZSxw1fHPSz9brQglzJVGHELj3+huYquJbPvgt5b3y8tEHT2Vr2aZ3WG0gBBSB4D06TZz1bgAtUMd0Ce2cw1hhQRC9VOWAqeSzEaWaLzzjfNNImCow255W6XUqlONpzZx+ppTo0BJigl3EYn33OM4nANPgQi41HxcqFtx2mgMvVM6XvTRVzGEyVDGNGBVBJV3gCYQRd24W13Ed08jC+89qPLHSLS4Nmf40wTWL1U5qvgjKMGoWWGsLFDFdAudqdU4PU3jnxtHgSWTsNP+9bdsy7ut9ICBYad/37O/vs1gsePDmPTbbLbdu3abrOqqqoq5qQPRvta2yYBh91+P8AN5QxYgxmkDAMGruFhqcA2NlSe21MB90mpiLIaISBCGvE+cKo8VCPUxWCtMIIaDCRfeK2c9RxNz39g5YLfc40w+k3ziR2mQ8C4jnzm61mhPrhBNM1kOOxFTV71bF85OBcK6z4kaBGiZPEDpLuhEJDCFiN1Kx757/67iOafzsz/4sx8fHV70bb1s8Udpx2kTTWi7+gvcRJ3ieQJVKKUic2DgZHJCl+nxZmilX3nsqbfBx1KQtlZtKTbUY0BGMGuldMUaMliV8cB5bG9Zn5yilWB7ss1zusVwu2d/fZ7VaASJWUzUkviwQA4FApS1Vaor5EGlqgQUyHlsbNUnAfsbeIAjEkI/ZWCFTFNiCD9hFjQKMrtBmdCCOMaLSjUulRlqM0kDTSqb5usFxenZGPwyFTiZV7xjzBCkVr1jsRGLIlfBIQcvPUgWnZTdPT7aWRXdy3k2NOBKrYQo15WeHSYMvitFn9B5HhiUe8+W7jut4RuOpxoBzxEnVAjKhlnm0avKcDEnkJLtbLeUqJ1fR1hiIYKtKSP15+Z2w41lTJ7ntzmhj6bmDG6hsxXa7Zbm/lwYnLF3XYRItzdg8iGHEgiiI71re5yZG6qpCRUVVVzMsWpk0zlwSDpAbUpMEPcO+iYnXqsp+5sRUKHN6rKgLLKGnGUnR9x3n63OOHz4UoR83p1zNj3EesMjDHHN2Q9rrscqdkGZH7dtd1oJQvabnIjt05BXO+Nxc4Y7fhxjBI5OJwYufnPc773Edz0289NJLfOu3futV78aVxBONKedDAJPGSfqbCz7RjhRRjdKOojk7qoWFGAs0UbiquxhvCAVeyO8xhSVkFHjElGOMLJdLcZFQDkLEh6GIi6/Xa5qm4caNw/L6uqqIhJwiISiikc9a14tidZNx4vy6qCTpWJMw4WLnniABpQmzJCJ8Y2N0EW83WQlMx/KzncARMxyX8dhkXnDfinD5smk41YY+XLL8vxBjAoRJxQrlp5gw2RHOGMkoM2y3NMfEmWNc8YwpekzK4++7K6MYAspHuSFdx3MZP/iDP8iP//iPX/VuXEk8NuluNltWq+XkootF6FolUqp08JOEoQJSxZZJ8SqzBCbVXybh65Jwk+hMqu6CD0lHl1IhZqzYJlGa3FhrmkZszr28f27eCU57irWGo6MDDg4OUnKzaE0RuvEDbLctdd1QN4ty01BKJxUxuXHYuhIxd2OIQQRwMm5rTJPobE4qx2z/Yyx6MhABYK28HpjdWMpxyrZABWaIeO84Pzvl3r03OD05Tk4Y0lCbcnRH6CCvSHIVO6k8yzjYxWp2WsWSt2JUonmNkIRKVDXUxGi0LHXGZB1KlRuLxkaGZmKInJ6ePe7rdx3X8UzGYyfSNtt2VnUKSpi61aUiS3P+Ol/IkyoxbWe3CsqCNdGLBGJwTiriCATx69IoTE7ocT4+HGOk73u895ydneGcR2mD81481Yxh6Dq0gu1mzenJCZv1Gu9DqrQrERS3lqgUtlpSNytsVZfPpfWYlPKSP0RJHBlysDYPMBiUNpiqwiatByEPmFLxTZtnGZkYfdmsOFGUW4ws95XQQhj6nuMH93lw/w0ePLjHdnOOH/oR1silpijdSFJPLg4on9gJeYAlZDzgQpKdRhmEAYKKBKUIUf6LQRed3myWqQLgk95v0sbIfOm+7+m6jq5t6dqOdrtlu91y7/7DJ3w935lYrVZ88YtfvOrduI7nJB5b6X7iE5/g+OH9GQyg8wRaFtWOI+0nZgghYZUzLHZn29poSRAxouKoQ1AadSX5afROgijygsxdguu6TsLenrZtaVZL+r5ns9nIkEfdsFqt0FphtTgc26pmuVwVDd6p7U+mek1vFjHKEjm7EefufYxRWAxqxL6VTv+qqXhNRCtbBjyMthhTlepYBh+QpJm2NfQD5+s15+szhqGn7zqcT7KKEYSNoMp+ZI84VCSGVAFrJnVt0lxQj6KGqfK5BLYVjFhgBXksJM1gEAaFT9+FXOFmMaHccJz/6wgBPvaxjz3h6/nOxMHBAT/5kz/JP/kn/+Sqd+W5iE984hN84QtfuOrduLJ4bKX7Uz/1U5OONLJE9UG69bnBNcECZUk9Gj/u0p8uqGaREoFOiVrrYgU+1YjN2wDKkMaUfpYfz9SurHmw2Wxo25au6+iHgbZtOTs7Z7vd4lMTqGkalqsVzWKBtdUs4e5OcuXHxjHoSUNJZUgkf06ISboxHTxJ6KYWgRtlC6ygdYWxNcbWxY9u3EZMFWNH33fi/Ot7QpCEWirOOCbQqGJqSCqKD1oGaRlpXjO8dQYFQAwQfPo3MEucLgZc8AxOdDn6oaftO9quY9u2bFMlu9lsyjnIf5P/Wrbblp/6qf/v6/7Cvl1x69Yt/upf/atXvRvPRXzqU5/iR37kR656N64sHpt0f/AHfzAti9PwgICzZEp8yAKCE1wx460zfYb8eJy47CbO7zRZxulrifgoFkDTBJwr6CwZmTUQYhRhcmE1aCpriD6gEXPLPIHmkj25VhatLJWtWTSiq1BVCXKIY4Ms/v/tXb+PFMkV/qqqu+cH7GhA5jKfCFaWcXqO8UnESMiBJQuBEIE5+A9syTmBDRLBEV1+PieIk3WyLFvmctsJGTLODXhv92Znuqu6qhy8elXVs2tAtpixlvpWo92dnemp7en56tWr730PQJ31UJNSom5qOO+gjY6RPudikxKBjHWUyqVaaQKK50xKyCqoKaSCkqEnmxRBLQCY3qBtl9kEkKJUWtdz4iekehxNhunetLlFv62XDIfncITLBAxSqNjsPbDWhglAw2iNrqNJretaIteWiLVtO3SdRtfp+LPWBlqb2GLo8uXL/9VF+y4wn89x8+bNbQ+j4D3AG70X9vf3sbNzOmyEJWmRgoT14QPtcw1rIltWGkTzllypEKItfh2pkl3igJSySJPLd3OZGKcExuMxeeOuONLt0VuLSin0xmCxWGAynqKZN5iMp2Bvg7oehVy0oLxsBSjBUi5FVWq5bjiQkQxqjtj7TUgqVJD0PBqfiORNhSTp/nh8RT4KPGFJWUUrTCbW1arF3jd7ODjYh9YmRJ8uMLmLeWMb+tNBiLiZxT3rWB9N55bbBKeJEggm9fy+BIa2LtlwpkpBm0W+lCdOKwwX73fOD66H/D01xkRvj4ICxpMnT7C/v7/tYbxTvPaKr6oKn3/+BT65dQtWcENF9tKlUti0xB6mAmI6Id1J6YRjKpFihLtWpZa/Hj/GCvIm4L9z1RsTAkfAbCnZtit07YQ8bPECAG2IfefcuUDyDt4Po1RuApmPQwgiy1zelX8/bsyk5RXB0UxCKqqCI/9c8twVQsWWRGkFkHdkoBSFyie0KPViRW3KHQOhT10o713XT+fnnI9BkS/J+lhZweePU0h5eoE1zaSsYCJmZUIiY0AMCDeWAEuJ33zx20K4BUfw/Plz6vN3gvHa9IKUEj/6+GO8evUqtM9JS83oFpVpS/mDzCXAMiPQo85kPvr0sj1i3v4nJ728cotzuvkGmvceXdfFzRuucgOC3OrwMHb61VrDGBMfz80rmQB47Ozvq9YkXzzJ5KS7TsC8AaeUQqWaQTScyoDJgYz8F2iikEqFnHhO+LTkH08mmM/nmEwmqWW8zzpFZFEu22mmUt2hdGvwuw9dlm0P0/fobY9Oa3RGQxuNru1C+mB4o24bbUwhdF0Xzy0T8nqEy9+N6XHp0qUj5dDbxu7uLh4+fLjtYRSccLyxG3DTNPjz11/jJz++AtPrWFbgLIn8hRdh5zpJqYAQYWWkW68VV6i1dIIQAlUo5/XBBNxZCwhaylPhQzJSz4sKckWDkFRA0TQNutAUcjIZU4cFa9E0VZSDsXetdwJNU0PrLh63GY2wWq3QsH5WBWKULpBnDSnTJMKkqmRIpQRFgayo6y8pE1iVEfxbBbt6VVCSxHfOh47AUsJ5Sw0tvYubUdY69H3IcQtuvBzSA1zk4BG9MqKaLOpsqdtHlADGScxSKsHSGLwLhNznES5FtOzy5lwWa3OqCYnwhafrwwtAOgHnLerxGH/80x8wmUz+pwv3XWA6neL69evw3uPOnTvbHk7BCcVrQw3vyW5wd3cXo+k05Cypt1ldVZkAN0VTXIGWR1X8Ic1JkpF7GNhQYOHD80WlovyMSTrPG0f3sDWbybquo/TLGIPDxSG6toUxBsvVEtbZ2BGDIuMKJIaTaJpRbOc+nU4hGzIdz6Vk1lr4oH4g+VeTRbFV3BSrqxpSKMBLKFkBglIMHrSBxnlkLoagdENwB/OAsx59kL/1QfPati1MIDznj6YM+Iu312wQnHgPOEdpARNuvLHFN8OrgJYi2a7tSJWgOcKliJbzsdYmWRinFeIqhUldJLkdpMS5Dz7AhQvfP5Ji2jQePHhw7P3T6RQ3btzA/fv3NzyigsePH78XXTzeqhvwbDbDl1/+DtVoBB9yfUyiFL36wdL6dSmHHEyc3KHChQ9qb/ugi0gdiDmdwOMChh0sVivyyR2Px/SPBZIej8eDtjsvX77E3t4eEfByia4zsJY6/CpVAZ4Ivq5ZPqZQheU8bYZR8YFHGJOikoacbKWoIQRH9iJE4RJK1nHTkf4HCSmGBRZsKkTnR6DXPdq2xXK5xOFyCWN0POf5uWBQcQJFtjbkabnTBU806zci8y7c2pgq4O9aa+jexAj7uEac61pfNvphaZ3uDaSq8dVXv8eZM2e33q1hb2/vPxLrZDLB7du3cffu3Q2P6uTj4sWL+Oyzz47922KxSCZSJxhvJF2ANqu+++GH+PkvfgnnHIzWVP0V87CIRQlp6Tnc8V6XHfFjBtravoc2JpYae+9RBVJumjqSIUvF+IM+nU4xm82ws7NDFo5NA2ctuq7Dt98uYsqgbVucPXsW8/k8i8xdjKKVqgcbZ4l4ZSJKIQaWlXQXRa1cXUuROaUTeutilEmvmXqWkYqBjyupKadIuW2tNRYL0hV7AKpS1CsNwxLb9XytCxVhFNXaQXUY51zzvCwTbU6y/Hh+X1NEezRXm7brAnwohomVw7Ri+tW9X+P8+fPhHG/fS/fg4AD37t079m+j0Qg7OzsbHtHJR13XmM1m2x7GVvHW28eTyQRXr17FdDrFJ7d+hjOzGazro+ieiyJMcBc7KhFLm19sQM4376mSbVw3EKEyrO/p2Dzz9dnPknuKSVqWd6s2Vp1NRmNSMIDImjfsqqqCsxbLxRJ7/9rHctxiNBqhbir0zqBWFeqaWhMBZIVYx2aZgPMpsleVgjUuFigIyY5hIWUgUscL56lcVwgB4SxUTV11lWogISGRiFdK8txFKNU1WuPli3/iH8//jlcvXkB3XdrIHNaXJSIMqQTvkrlMnubJJ7xk25kmhXzD0zry2YX3oaSNfMmcEKgg0IdcPryHCsPhkTnvgWAQ1HUGnz78FFeuXImrmrfoiboRaK23PYSC9wxvlV4AiDRPnz6Nq1d/ir/89W/43oUf4JuDQ9jexZJgH56jwu5/bsrNOdH0oUOUdklJuVRVqUgedVWhFjKQYPKezY8Zo2WVcr1d12G1WkEqiaZpooQs32yTUmE6PYXRaBKaJJKS4NSpGZpmjLoeQ6kqmo4rpVBX1HGYx8uRqvNDVUY0rMmUD/zasqqDi1qoeqtqSO5EAawdh/9nVm4QgUbSDcoD51MFnrMefe9ilGoztUZMEwTPCo5g2cGM87SDohYkXTJtlvXwgdUtPFUsO8ojOEEbeXwNWA9o7fHRRz/E06dPce3aNczn82OvrYKCR48e4dmzZ9sexkYg1nNxBQUFBQXvDv8fa7yCgoKC9wSFdAsKCgo2iEK6BQUFBRtEId2CgoKCDaKQbkFBQcEGUUi3oKCgYIP4N6SqeECFO87qAAAAAElFTkSuQmCC\n",
"image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<!-- Created with matplotlib (https://matplotlib.org/) -->\n<svg height=\"181.699943pt\" version=\"1.1\" viewBox=\"0 0 349.2 181.699943\" width=\"349.2pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n <defs>\n <style type=\"text/css\">\n*{stroke-linecap:butt;stroke-linejoin:round;}\n </style>\n </defs>\n <g id=\"figure_1\">\n <g id=\"patch_1\">\n <path d=\"M 0 181.699943 \nL 349.2 181.699943 \nL 349.2 0 \nL 0 0 \nz\n\" style=\"fill:none;\"/>\n </g>\n <g id=\"axes_1\">\n <g clip-path=\"url(#p58ad9a7e6d)\">\n <image height=\"153\" id=\"image6a21407320\" transform=\"scale(1 -1)translate(0 -153)\" width=\"153\" x=\"7.2\" xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAJkAAACZCAYAAAA8XJi6AAAABHNCSVQICAgIfAhkiAAAIABJREFUeJx0vUuvZUl2HvatiNiP87qPzKysKnWzKTatNgk0YNMUQMCgKcMaGIIhzSRrbI0kAx7YHhmGQcCA/R/kIUH4B/SYgDywKUpNWObAzZdksLurWZVZlY9773nsvSNiebDWioh9MusUbuW955z9ilixHt/61gpiZgYAZi4/OUf8+Z//Jf7ZP/2n+MM//EPsNiOICJkAIgIAeO/RhYCcM3LOYGaEEMoPAKSUkHNGSgnee/RdD0dAYkbOGY4IOSbAETKzXCNn2Mt+DyHAM5Dsb+fhnEPXd2BkvQ4wDAN2ux3unz3HJ59+jtu7W3z66ae4u3+G3X6DcRxx2N8ALPcLMMgRuhDKcy1xwvl8xjAMiDGCiNB1PbwL6LoORA7eB3gv90BEYCwAE7puhPcdyANdGGQsfA/nHch5gKhcx16Pj0f827/8C/w//+b/xr/7t3+Kr776Am/fvcWyzAAnvU+UMQYTUuIyrgAQY9R5k/dSkuNs/OVYIKX84RgzA5C/iQjIDAJkrgH5XyIwMQgM1uOI5D5SBn77t/9j/PN//r/h+9//1dW42LM6Eyw78OnpCb//+/87fvM/+g38+U/+X9zd7OCDAzk5iFQgU4yY57k8kA1E1PftxcxwziHnjHmekGIqN7DEiIUzlhjL8e0gmYA555CTDJZzDsMwYLPZIKeMeZ7LIMuDJx3IhNPpiGk6w3sCMyPGBcfjA+b5gmW5IKUogq7XXeIkg6L3K5PEcORXwsEqpDaRJowpJeS4AFkWas4ZKS7IMYFzM5HlPPbMWe/bgZxbTZQjB0fynvcezhNCcOi6Dn3fw4eAvu/L3/bjvZfFqf+GIIvkY0Jg8woQiAKIHEAEDwJngB0BIDgGPKoMeAL6nvDHf/xj/PCHP8Tv/d7v4d27d6u5Z2aEdlWdz2f8/u//Pv6n//F/wH4cscwzQienjTkj51Q0mE0GESGqkBARnHMAgMvlUs7bdV1ZaZdlRp5ZB5DkYWwAASSIQCzLIoLjPMAZw2YsA+gcsMwzGAnzPIPIYbsVbeu8x3a/xf2zW2w2W4TQYZkjdts9iIG4JIAZzgM+EGKaRRupkNok5Jj0GQnkGAAhpQznMxw6HcCElCMADyIPgMDOI2VCII+cM8gTCAyS2ULKGYEIRA4RjK7v8eKTl/iV7/8qjqcHvH3/dbkHzgRU3QEQIRGDiZFZhCR4D2YPzhmcM5wjhODR9x1ijEi6OOuiyfCeihakxGB9ftPuDICYEXWhOyIwgASAVL0RsyxQBmKc4R3hv/mv/xmmecI/+a/+CYZhUA3pEGxwY4z42U9/iv/1f/mf4ZyYBNabYpaTm1bxvq5s01QAyuCU1aFaQh5MjiHnEADEZSnmI6r2ciBAtYOZX7vO6XTC5XLBMAzYbkfM8wznPYZhwDAMmKYFfd9jHEe8efMGm+0W2+1OBcfpoBKADO97EQgGYlxWK9s5BzAjxojQdyqAAJBlhbP8LVpMhC8EB6L6zERcNCqRg3MZzA7EGQQHBsuYOoe+77Hf77HZbMQyxSQTCJLJx/plmteRg3cO6uuAsxPt4v1qzHPm8neMcSVw9v3EVRCJHYCMnOnqyld3YuNS3iYsy4L//r/97/B3/7O/ix/84AcyXoAIGQA8PDzg7//9/wJxmkRlq8pc0iKmIHPRRiZQJmSmCWylmE9mwmYmsOs60V7MCD4Uk+hVMB1IhE/PK5rElWtsNhuklHC5XGSw9XqXywVdN5Tvv3jxAvf39+i6DtvtFsPQwXuC84RljgB1SEmeQ+7Xg1OG7zoZfPXVCE5NNcN5MYGAU3OSQSB4HwCwPh8h5QVBfVVZcBmZEyiLIIIYmRzI2/gwQh8wjiO22y122y3OxyfVQqLJ7PmLoDknyyWzahkCqztjc2Q+nAmZLFoTQC4Cx8yIWX23mPQYrBSEzaN6aVXsyMRLfulCh5gW/L2/95/jxz/+Y9ze3om5JhIJ/Mu//EtMpxPIEThlRLD4EU6lleRkZlLswvZgzrniy9QBbgbFAobGFEEdfyZZ1akZHDsmpVS0ZyvQ5sfFGGVy9js450WwNlt4/b3rOjjnxP/yHgzxDZkZwzDgfD5jP44g1QBiNlTzOg+AkTkiR11YGSCKADkQJGDpOg/vgJQjiAJyWsSM5SxCkhOYgpg0ynAuiMZwEB/Le4zjiKB+1TiOiMsMTgs4E6qbrpPNMt2qx+BIJpxhLotei8X0VYFzSJmRU4YPHpwZmTNC1ADNJzCLEFZf06lwMZir+bZzEzOIJSjMOQOc8frVK/zkJ3+K3/qt3xJNRkSY5xn/6e/8DvbbDbwnIGfAOThPSOW06kPpTZMJURM4LDGWVWYrKsZYNJsJR1HNRCLUzMiqvjMYXgW5FSoAxQRxFsG7XC4ggpjO8wW393cg7zHPETnLypABznCe9H25J2Jg5kn+zRmd90CKxcQwZYAWMFczKveXxdVQ8+gdIScCsxwbfAcwwKwLiXsw5FlSVpeCHMAZyKRmz4PIYRxHbDYbeO8QgkNcnPiBDmCWcfAgMEn0BzVHxFXHlKjZm5kHOHhkZuRMcBnqkmRkNZPe+TK+rSk1vzhnNNpRfooGs/lhRnYA4LEsC/7O3/kdvH37DofDASHnjP/jX/wLPH/+HGmZVr4FiICcdHVXrWQ3VLSUPpxpuHbVdV1XBNP8hzZAsIFpNSBR9ZFMGC2qzDkjJy6wSMoZ3gfsdzt0XYfQRFvDIDDCsiwABdFmTv2z3JxDhTcE09Aowm1a1e7RtHXRKgAyMTyrJues2k6c8BgXuMBwzACCmHgQXLPwAAYRcDmf8e7dO5zPZ8zzLNchCRzkeq4iATrBOqgrc3oNk5if6MiJwGZG8BrZ5gwODPCHQmYLvQYPjJyT/mR1OdpouSqFrgv4gz/4A/yDf/APEGKM+Mf/+B/hcNgXB7869TaY8litSSy4WuujacTh9OZsQOxBnRNn1R6i/bz1wTy5IgAm0JvNpgjmPEWklCQwcB7juMEwjtjtdvjkxSd4+fIl7u7vixNM1JWHl4FM8FSDFLu2QAq8Eu6cM7quK460DWQ7RkRUtAJhAReNr4EAEzITPNWwPnOGgy0mJzhWWWRU/BybQvvb8EQHgFiQqxa7sn9bP85MLAjIJKYNENelA8SEqi9tiz3nVHy4lLKa0YyUHFKq77c+n/3Y+PyX/+gf4vHpiOC9x+3tLXKOooaZQXrXibOulmom68qgIhT2kolxQHFqZXjaVdHekA1gbnA6MSupnLed7MvlLOau+GkBLnTwoUPoOuz3e3R9h3mZcb6cELoDco5YFqDrNoD6mSkmMHHx1RhcgOXiP6oJBwg5iVbKjhB80PtKGmXaOHj1HwmEoMIq36Pk4dUtIOeQOcFlAjsufu5mM+L+7h43N7d4enyH6VKDHvG1xDn25CUwaQSLyJnXpOPGEDFEq2jALILlWEBdhmmVqq2L9soJXmGRVoullJFyQk4ZLTxy/QMAfS+4XPjRj36kbzo4D6QY4VnsPkGdW/U0i3CoRruOegCRR7uIDRBUuEwzFcHU6Ig8IbbnaGAVM7kVx3JYFkaMGUQJ4zAgI8N5CRwYQOg6mWCOyAwsMeMyEZwjLEtETgm+70UASHTEMs8IfV8EO86Lmt+u4kcNCEtlPMQxds7re4Y0VWiHc0aODATxcC3Q4SyYF5FEZuO4reaQxISJD+p0xcpsEAgQpAX2rupi+RqrdBGqniPSRa/3XnQjChTSZgJ8CMUsgvFRQZIIOBXFYShCxeSAH/3oR/B/8id/8rvT5byyp845gBp/C9W0tdmBVtCuTaP5AeVRuPwPOgfV/DIjN6q2YECN/0Y6SNVPEiT7cHODrutwc3OD7XaLw80N7u/vMY4DgvcgJ4FNToK620C0EbLdhw8VVhF4wsN7EyKNspr7MOECKograyeJgDiolrFjJPqCI4lO9YfBWOYZX3/9NV599dc4HY+Ies8g0VrVVzaEXv+jOqbOXftl9fvreTGfG5DHsu+56tbAAh4Byr3zCAqaWwbhY7/Xfx1CCPiXf/RHCH/2Z3+Gl588XwlJttDYiT/BWQdOb5QB0QaNo28mpn3llEFcQVtZwlUgqVlB16bYVkcrzMwSScqEStjvFNDcbrc4HA7Y7XYFNGYIThSXGZfTCeM4luAhaiRs17AI2DSuKwCuwikQvy6nrOPDBZUnX4MXG7fEEYhitkSjMzwEU3Ps1EeqC7brO+x3O+x3B3Rdj34YkJZFoBodW1nMBhslEHn52wmMYD51WSYNrIHm3/Yl46vngwdYrU2WeyfnkMFgx3AKQmfVjy3w2+Kkoskkwv+Lv/gLhO1mXGNTEFBUJUPVv7wnOI9Ifpkgu1msH8iiSedd0QwOkOiLSCEG1sFzIKqwhqHufd+DmXE4HHA6HQv+JIEDoRsGZAY22x1ubm+x3e3gvVMhEj9KXA5GXM6YKcNrHs6ASu9tEWj2gpwAnLnNPABECSBCilF9H4KDA5NMtj1zNZEAfGuCogK81bSBAXay+Lq+x92z53j2/CXev3+LlBbM04yYosAhBQvlAp9Uy+BV49H6faom8dsEDFAo1DAwE3yX4Vj8NTIr5mQmHct5Ja3UEiuyQiUJOXfgzHjx/B5hu92swvL2xjjL0sgsoCKcK6ujrHgiRclRLiiTJv9aNESaSuHMYEfCTCgwQdVigrCL1jRfbJomxChAIeeq3fqhx2a7x83NLTabTTlHzhEp5+KvIEd0ISDFBfN0wTBsyrVFG1NZlXAKz9j954ycHZgngAT9lzFwyJTBiHAOxafs+x4xJnjXlXEomjlnsEf17XIGuwwiLzDM4QYvXrzEN998hePxAY/v38sTrOAJVCFSYS+rhCrozWi/96FglQgUAKfGrTHzS+KSlKwCNWcrXyV4VBfGewlKUhYgmhKje/4MwTQSUMHOVkhSSgKYqvAVTWWRYvOeo5qhN6e9TKblNIsZy2ut11yv86FENUSE8/ksT5VFCwTfFZhkt9thHEcsizjqKSXMiznAOkA5Iae5mNwQegRep6wK0AvCUpLmxshIClVknXSLqLkAm1CBTjnDqw+Zc0bo3ErQXLPyzdclQvFt+nFEyozzNCHlJHhrXs/Jhy+upkTvw3xpXd6Kq314iAlc8asL/FF91m8T1Bb6EetgEa4XReEkK+G+7QTmbHtfhbDFY8yPKZQUtz5VYTOoCYnGfVqWwtoAPoRFWozHhNWA35wzOo32NpsNQKR0o1gS5SEEFWhR2zFKNDlNc/W9iDAvC5gqqp1SAscE1myA+RVl2iiAyMPSLC3u5tWE2ABdL5prf8VMM+dWaBh9P2C/2+Pu/h7b7VYS7x+ZG72KarVcHXlUraJ3vfrXAi4TQiq/lcf8IIAr1KDr6zdphho8ETon3D/nHJx36LogCfJiKjRctQNFnRK882oy9T4VQ7ILF4Q4hHLLhqyXFeyc5ODU0bT35boK4jrhTtnkFCpRmisoCsb+sBcHfl5wPh9xPo94enoCABwOe+ScscwLHLEEcgAcROCXnBFyxtBpXtTrmueMOTMCBXhICN9GjeRlEDmz+mu5DCyrv+ScQ5wXEfLAyOxAQTSu8wCTmtBiJllBVVlEQxdwczig77oCQzjykspaCRo3v9vilgDAnP8qVQp8qMbEh3qlkTEVOQtnIXQmOaeQFldSD0kL5hIKiNsETwjkinIIQGVWBk1bmCksgsBcMu6F5qOmxlYoAFDOAhvoU7YIudyf0Hfg1uxMqOC2qapxHEvKJ/MCRxL99X2PZVmQAXz3l76L7W6Hvu8xDAOICPMywxFEMFNEcB7IQo0Jviv3l1JG72tEKYi3JNNd8ICTxWURtGggeY6krgCrd78sC7Jvvqv/835NEqhRcvVbW63NBPjgcDw+4nQ+Fq3XLlQ5TiWBDT9rhbAK2MqXYwKYyuet5ms1YZ2Q5jvUwr+tjBX7LK6JZurtOS2DEOzmvfdAqimidtLJzuca+EFvoE1FfCxVtMJkIBDAkuKVWq+TYEK8LAsulwu894hLwjhWKs/zl5/AeY95WbBpzBIAeOcxL2ekFOGpwfHIwTWQSx86dM7XqFInsMAfihsVn22pxMwKVxhSnhFjBZzbScscRT/wWFI3Zg1WPxKuiaY3AuSV6WrH00BgkcIP538tMFfOfiNo1+7JB4fqe+b8X39GJNCGeaXtMTauwSaOgMKsaDGfVbSYuEAYdgLBlLyatVQnu9VicgKFLdSsxCponDJy45PZZIcQCi15ThFdEOrO5XIRnljfI3QB+8MW+8NWtdxUgo+UU2GODEMPy0AwMxInUBTNaGNn92wOMKBUGBfgGIhpKdE3q6kQn65OmPHmZPwqNSolRhcki9F3vboD6qYQaRbFY+i3+OTF38AnL77E+ekBcV7AuaZ9WqGTC1+9BzTjTh985gMJHtbM7fV32pfJRtL7XH+/vuXIQRA0Kr5mmUsb2JhSMZfXN2wn5LoExBZnXkVL8hChJMHXjqJgLiJUqZyb9eYF1vAg1lXTRKiu7+GIMG5G+L4rE9n3veTQYsZ0mRDnWfJqMYIoin/ne4S+QwaQmODJlYgveQY7oB/GmnBXYclJ7supwCQkpLxAsj1ONWyGDwExLgq3CByRUoIQHvXZjaER5Rpi7yzpHJWWI77qsBkx7kYB2ljZE+5b5qUZ3g8EEOpjkZNxX8EZRhn/NhVo0aplMFrTSwUYbrWtMNS9HOcdjGhARAhW9OFcLRYxE1rVuWJOJGYkmY8TPDzUbyOC7zpAheP64Y2BUXjntgLUue+HQc1pTY4DRoRj3N3fo+/7Alm8fPlSuGqKpR2fnoT/7wFwFEH2EoHFSOi6QaqN4DBPEX3vMPgRIYgzbmbffAl7z7RusmRxYrjgyoKySTVz07KCa8bCF1Np40uEoqVdZngn2jE44O72Bnf3z/H+3TdwLuF8OpUoeK196u9mps0KWaKckWR9N6aMRTy+RcCg6imWIKNCNvXztem2c9b78t6e00moYQxSO6hNZBeaNSrY6i2nmWqmwCtKb3+bT2eRZ05S9CGDxLAwKPhqFpm58M9Mk0kelTBNUkn08PAA5xxevXqF0/mEx8fHlXAQALCwBeY5YZ4t1eM1DydRo+XpTCicc1iWZeW02mdCRgwAPJwPxQ8j/cysgUXUtshYiZNGB7Jn+iDoySylaHqOzWaH29t7nM4XnM7nWlTzLZrneqxDEX6JCtfycYXzlQxChSJEoXgQBQDuo9dtQV+iSmhtn8nOFSx1A5QAtCRHS/Rj8ENzcos+PVCot2Zy7Hvt6hLnm5FyLnWXRIQlKzPTAVFNSFl1juD6Drf7Pe7u7tB1HTabDeZ5lrKwPoCYkaJo46enR3BO8MTwzqPf7tD3Uj4XQleSt1ZK1vUjYmIMvVREjX2HnBNc18N4UoBhXYDvenmOrMn2HOUeEbRsLQjAmsXPhKfCtEhpAdCtimMMpjBBIQJc8KAwgILD3f0dkCccnx4wz40mK35hDZpaQahCrGjYletTrksZUoMkaSmwCVa9LxG2gpCWqLposua85uYUQdRfi+P/sVcRDiVAtr6BPRgTVpScktgGIWaL+GraqgUq5fxipkOnEe2Syrk61TZW3zmOo2g3CoXEuMwzaBFsSnJwWUq5OldMlEEifT8WzeOVMdv3fVkIswKxobdJybAxawtnbBEx1zxrzhnzMotP2kaYTRK85ci1/Drnqt/rnMfNzQHeEU7HI87nCSnV2tUygWhA1mau2ijxW7WfsWOITARUgABmdXUowwDAKjfropbqDnzoM7b3FVrJM4FwqNqnlHa5dRhcByug04ruxDVV5BgIWpSaIQ7+0pgKmQiBNbxCCUbBCV78I3iHzWaD25sbEYJ5xmazwTRN2PKAFAVIJgjAGe2hg4frOjgX4JxH3w/FPBYh05+a3nLwvtK+r/3TVTTMAkzaeOac1R/0xSFnZoS+k6iL6+JaO+7VkxG/xoFI8sHPnj3Hy08/gwPjzTevSvFLIydQMGuV2lvJ0rUGU0ExAiSp9gKtPxehEY9OJMytzvWt2NrVde334qWGIGG6DVqrcUgwyCJwzjmNJmqkkWMC+yZoAABHWJRX3/pv9WahbFyhInddJ8wGtedOTZzlJU3rbLaDUEkWE1gHT4S+60Deo+t6bDc77HZ7DINosK7r64qzcL5UU6vAqzklZJCv97zWXlZwTCWT0Gl9pnfCXwMBoRuQEzRT4Ao7xLRrO6lVeOXvYRiwGXeIMeN0mUTw3Lp6i5P4hEytdjWhNU22Npf1OwZPAUBeaUTzK733SkbwV5GrCZwdw8XnLE90pdWcre6WT3UtqSlpYahGEQ5rPyArBcjOZWAmoJpRI9frouCcpepJtKRQfIvbQEDX9wihx/5wKMlvcawXnDTiOp2OhbfPKih91yt3bMQwbtCZloFgga2wk2kCu08SwfDewanpuzY7YgYdyAfxV3MC4BBCB8Fora2AlPxLMENF4LTSpEyWaDHJtIAYfR/w/JOX+M53fgm3t7eFN2eTN46jAONurUm+HZJAyVMSrn20NT/fBE3ID1Ug68sYwkl/lJ61huVWr2BIe9JcZGvPq2kQjeq8IvJYh8oldE65lLgVHyJrGZlzyE3hLjOv/BxmQtAolYhAwWO320ui2HuwM56Y8KsMchjHDZAYFAJCN2Cz3WG736MfNwhNpVTbDOZjfoP9G2NC1wvijmbhtK0YdNaEceBr8tjK/yxCJkIDzKJQYYh84/TrtQEx0SyTXFIyyjK152iFrQpFOcOV1llrnIparDXNt2khO74k2qmei1VpEonQGbGq/Z7JQTAVDAa8rWp8+CqOJtXV0GoyU/vgmoohImROYjaoIuJ2cYMQRGgVPjFB8KIhlmWGpQCtkYhzXTE9RMJNk4KSgO1uj34YETptRtKYyZbxYf+KaZe8pLKrwCzXNoLhNT5VJsK3KLzCLx+ZMEC5cRDAWWCUdelfTU4LxOB9gA8BKWdE1fgmeO0YtsHAh84366xp7tGV3z7y+jBIIPOTqH0WKsJVjnMEcC2NtJdZC1e0EEsNHtxagMQ0VKYBmsmyi9hKKhcg6aoTU1LU+mMOKMp1RUOSIN8WxqsZSnHBNAmfzDSZXV9oO4DvgvaT2ElS22mbhaZvxfX1bSGIeyCNUzgnEGoLpsy1dM9+Wqo4WPAg86ey4WGo1zOsL2l3IO+CNEtxYSVkOn1aGyla/MWLT/Ds2TNs98I6kcBE4BDRkKlkFa61kf0HCJjtwPBAYbm2gmSQxMcEzcxYKSRXig+3X2BVTKqFGXklPyW6bM0I5YpjGRZiAF7OuQBv8n0Rde+9ZSHqZLCE5ERr6KJw1UJAUL+mpUWXlBQzkqtRYdBCD6kDTBgG8VWGccB2u8M4bopmtOtYCN5Gy/ZZMT+Z4b2luDyQZdQyV46+PXtLtgTLMS54jfIU8sgZcB4hdGXcylhq8YhFd/V982tEu0pxshAYY0xSZdWQEHa7DWKcYXanNelF2OTpi0QQVR8UH3zTAoFrc9t8S8fww+NNXyrwTWvcLngX4ByrGVInNNZEd1nFrR1fXZzKzXu9UVYBu27Kcm2mgg/oukGqelB9DhvMw+FQwNPW+XYuIGcRni4M2Ix7eN+h6waE4BFCBYFNoIC6cJjFtEmyWlJhTIxM1echEuGjRlPIfVfcqEAcaFNwddHmzOh6mRTvG8YDoLBB9aOMnm7aRJD/rVZuyTh5cjhfhDfXadkfc0sBaubIgmg25aOmXxdvK0TFmuVatng9b+34F7NejoVaLNFiaMZex6LTiTDbzUiUV5PUYj8EiDNPVdOxmVGuk1HyelcPX6JKIvjg4TvBhZBZtVotUdtst1iWpTjwlhi3ye66Af2wUc6+aRwTRK9atNJb2meyMF20lSTEfR9kcTgCHEBtHwwogBklhUIWiV4NaOYoGQySQAgsTFxZzEHXp2jIGuGiOtdmGULAzc0dXrz4BI+P75HTjOPDUibashDQhR0Lx60FTGVGMymLyIT5Soiuo+dWAA2Qrn+jWLdGGmE20zBWNGSLUCdfzENhqLZZgManMtttDrG8XFFuNqF2bQfpSrN6eJ0kS36DCD44OCdcq2Hoiwa7JvsZJXq332Icd9jvD8W8mKl1hRRZ77elGZnglESyCrw53BZVrgZcJ420eMYWg0Ef1kdDil20i5ALYDjNe1K9H98hCfSKYB4HZ2SWCihycj/7/QGb7RYxRVymCXO8IGtbp0U77nCWcrp5vqy0DsoVFYss9ZtVk1UZuQ4YrrG3MhKNoLVvWzCxVkY236HFRExw6Ip810Zn1xBHkcEGd5LKzSxsFRaud9ty03tfqrOdZgvIAT54zTG6lUDYKrXBkc89NttDiTjtJUJpKZvaLbEV1FajMqO4CYD0YRNKtrJIc/VlWgG1CanZA8lbOgc41wHk4PX5KuxhAYN2/Cn4ogOz5E9hpg5OcL5hg5vDDabTE5bpjLic4X3QCnqJVqWQZirz8q2BlmljkQuYuhAsULoIrJABZYu0FsLOZHhD0aao1s857emmrbfCitGqDYK9qsPr/hVr+g8XQcyo7At7AG9FqDkjJQVxLROg55aIKYgQ2Dw0CPg8z9IWSrP5xV9yPcgHzPOCoe8KKwKotG4iIRR6XxF2G7gqjMZ7A8gRUloQgpa0eY/OBy170yip5ZvlrAxW7cnRCN4wDHDBwbluBUBn7fGR84LgAsDSU4J8EAOahfuVIXhj3wfc3NwjhB7zkopgAdAeHnLOlGJxyE0Urs14ealGZ+s7BxTtX/AvfOj0ryNPM1VXste+mpxtyW9Y7jE4r30asjAJmpteRQwa6dnKtldMCR0BUfEUdOKPUM5SgQwUjr8Uh4ipDZ1OICplyJoOExEG7VMh6aUB3TCUwtwKCGdYHaIIk5igGOOqpN6unXNG14tJY4ZGR7rokhAVyResB6WdZ5b+r6QNWCywIB8kfeVxU0SAAAAgAElEQVQlgux7YXNI9yFGigk5RYCCCBJLWiqlKM+SJUKTajRpZfXs2TPs9zfYbnfSmfsxIqUFm+0epxPAfMH5fAFlBgoiYEU70ABgbQZl/iud2twlA1aJDH+rQRDQpo/kxDW/e+WXFj6bNKwJLbc9G15CFu7nYt5aU2OrpLR2aoLa4KVwwySfAaFXp4QMIfoFLdAwjRCCFmGoiTNYwXAxsPDRfAjohwH9OKLvOvRDb0OnAmU+mWonApgTKFTBK/WVgjWA2WnXHmFzJmOOeAdmi6DMB2FF7F0xwdYnAiCxAKTWAFDhFuCYIIXJGlrJ9SQUh0AZDvBN2kcX3jAMONzc4HA4YIkXUMyY5iPiMmFaJtzutzidTghkRSUS3cN/zL9q3gNDu+qs3q8KQ90Fg+Gppfh86DK1EW6rkJgBVzpPO+n4xwC60MPDSejb+CA1oUyrNM0QahUQEWEx05kyXJIrFT+oQasBRoyzVPvoyokxFTOZc8bxeGxQbmCzkV4X2bA0R5hjlIGAQ4wL5lnap7eAq73a3miWAkuJpd9HAVolUOBkxTR19TJntKwFOYZLKsgGdxxHWGJbzLIKOIQCTgzkmMr1gFQiOSvqdeSECTxI77UYI6b5IkDxEvHZp5/i5uZGLEWJglvYqUW9muDLBM6t368vIzbac1/7dx/ayetz2Lgxs6SVWnLhtttIZj9luK4H0rLSWtcnLf6Zc0iwgUQ5JqeMxvyvXjFGDQCEFx85FuGw1qCDMi+89xg2G/gulIkzJqskkC26q8W0FfOqlCUocdK7ToQIQN/JphNgCwgYRAnkCch1wMqCW8EPNtjVJ+tChxRlIiwh7r2Hg91XbZDHnCCQJMFDesp6DkhRMiFjP4IQ8Fc/+6lE/46kMgqM49MJn336Qu6Jcw2QGnzs+kWAMjfWc7l+FhYLQJLua5UCAF1UH5lQGwlXmdEA4Nq26d577Kwt5tCLyiVhJJgW+9iPCal3Hh15OH3AnDMihAlrNyrXE4fZePfWA2yeLiVtZGtoWRYcj0eBNLoRm3G3EvIYI+bLpBhaKHSgFmiUgTHmgGorVsA5M+Z0KQyPZZlF63DCnGZkKO0pA1k13vViu6Yot8GFvW9RfNWqBq1oloClOV7Ki1RIkQMyo+t6PHvxAt//lb+FcdyBXNAF73A8X4Rp3AQ7BNtgQjZ3cFwxOACAd9IpGwDFDMq8us/2uaqWrXiZEDVbAqfAKG3A0JZZqtWrxQ3b7RYANOrTwlo4OKoA5rWUrl5ZuioXyIIAj8pHYgDeB93EQTIMcm2Z4JSl0UlSMxVCKNz+0+mE7f5Og4UAIJe2BLZil2UB0do/SCkVmCNl2Sug5eQDWCH2IjAijJaqccGjruJcFheg/P9QeWnMkk3IzAiharFpPorpKL1ixR8sfi1HOHYgHa+olU2h67U96TO8efMKp4cHxOUM52UxPz4+SgRvwlCg/pXIfLAAPKgENSsw1pHUG5Cl44C271k7BhKkYaUyK55Zd54J4ziu0i19P8BrysW5gOlyBsd5BWFY+qGtzTRTstqEINdWBlA/bhgGpcJUG55zRlwWZNUmhnNdsyfGcdRJovIdZmn3VAtRPiz1KvsLdHUFZmS4rPUJ0e5DBDIpxsMkGiVFE6y1Zqx7SGV4X8fhcrmgHwcscZIoXIOYTgtm6mQI+8N7D2L9UbMcgvLfgsd2s8f5dMG42eDlZ5/hm9eEx8f3uL+/w8O7N9LLbJ4+6DQONFqsrR0xYcwojOfy/UZTOzIsTAS3eqb6/fJlWmnLa5wutCsSAJwP2Iw7pJSw3QJPj+8xnY/IKa6iMxtoO6nhUtXRls0VpCUoF6wpJYb3WUiBDTfKJivnjGVaSqsEKR7usNvtVqaHcwVp2THGvgcjS2Bb2h7VxSM+lzShs6gXLoCRtQCiLhwBRxN6Ly2QMiVpLqx+VNtMpqVyt4vNFodBKa2w25jJ942NIbhaPbZCBKHr8Gu//uv4V//6LV69+iukRZrHPLu7xXR6wGYz4lHfa5+ZFEbSwHgdBNgcsmQeiOrnZBCF2HFJu7E6+xZum3nEt2nPKnCBHKHzlr902G52GDd9YxaAJ0dYLhfVbgnmS7RMjVbzyPcAcoxpyUIq1BL8EskayswVqmAGliUiLVH78UvkuT/cYrs7YKfm3NpGtry2lDJCkPSUDiOct4S2OLIxRd0JRTWwk7YG0hrK10FLSQqNUwK7APisQroWHhMu08rmYnRdhxzFR0SWjcNaQRTQOKl/Kh28ycn2M0iW0ZAUX+c8druxKIG7u1s8Pci1Ht69A1JE5we0O8C1RFJmlqZ1Sgtv9nsQ08hcqowKlZvEl+OcC4hehI9VaInhraUVJJfdClqrzQJzQoYVnoofsSy6Y0c/Yhw28I4wjyMulxNisytb65fVlUnaCC5imk8Az0gEJEh/L+9qIartRtY68eIWifk7nU6lM7ZsXzMU8p5MqgYkRBrtEFJUv4ehCeqEvh+gjb4Q0wznJckeXMC8XEoaCyQ5VRiUAOshbYvomqgJTNMF3gektKDvO4TQS2TmCIyo2r1mF0wDViFQ2jlkYy2ZxARiWYSk4/Xd7/4y/ta/9zX+dI6YzicQebz+6itsNzKGornjChivWhwgXtMVi5/MBsvIrnCZ7D0ICYBblVUamYJASHpGzYl8K/gfzuezSr+asyhFFPO8YBxH7T8RELoOA8ZyIkPRW2qOMSRsNzlQ1vZMTtp1s3Sfdj7BkTAScqMZsvYIiznJNjgxoR8H6dOv3LFe97Q8n8+a+5Q7Ej79OrqRJJWa86baygTl6ekJ/WCZhmaQyAGaMqpmuwY8a9gnA5RFE+UIRFk8/RCwLAkhdCUrUFNCdavFlCKIpRCGc0Z2snMJUVQqum69OG6w291oasnhcNji+PgeS5wUZ6xtutrImvS80AVzXT8LmI8lm4sV38sE1OhORYnUZtUmfeZzm9C2CgcAwuVyKkJiu3ewgrCX8xNCCNiMW0321obBVtJmKQgzlzlHhKBRGTn0XYclLmB28oQ5IUVp71RbK4k5y2kRDMbMoPO4u3sm2tRL0CD05Q7jKBiTI/H5pDq8ppfKas2VnSr37AA4bLcb3QjMwZO1UReN4IJHipX9Kg17SYthnDQCdur4luu5Mm6iyefC9QdM8xmo3RWBEDRC+4q43Ai39PgyQen7EZ++/BT7ww0up/f43vd+CT/76b9DzFGCJkB3PSl5CVCuPXizhoptUx0TNmHJAJ4ZFBnW1I2Za+ag+M4G8daggCFjQFeCVjTZdJFwGMyIYEzTRVUlCY3ZB1ymC4Z+wNAPyHGpoB8q+9Q5QuYoiPsyqXZKuqlolDIyJ2Bp3f4vrHwHyblFabgXZQIPhxvBxqhqJfH9rKSNwDlKV0KysjHbsCo3k1lbLxAcTqcTnJMMQxdqMDEMI5Y5fuDflIwFZzj4VRTNbLWXA0CKfWXVjU3Ng/lstiBli8QFxNrWqsyfOOIlgmXSApUBn3/+Ob7+6heYpsmc6hWyWmNKFRZeN0zJ1xFA0UcqHA2A3YpU8cXKe/VKlqUs52o0KTMjoKHvTMsMkDT2JSJk55F8B7csSDEK1147DdqE+SCDQ0RIOaLuvcMAazoIDKG3aC9VzrB9jlrzlWwFUhvFeTBVUzyOoxIOnSbXqURRBhhaAGKC1S6IlBKGYcASVSPr1i5934MBTJe5wgBY06OJSDtBWsoFxZckqpXodr5rLV+yI7421pPKJQdnW/MAyEmi8gKLaIbh/v4eLz/7HON2gy+++EK1kBTSiIBQ8ZFYgdiVIJFkMFrnqY3ufaPlWp+toP0F/jBdpoAyCIVCbMJKNZ0XrF3jOk2UAfWXwOYMJiSnBa+oobrPhj1B+z20N56QGIBGZlT4VA6cE1KMq8Z0tiJtqzzvBa/b7gTlt0Ajxhned5jnBd4DwQuTwfj45iuaYLW+lHMO0zTBaMuuk023GNA2CKl0+yayaLVJKZVd3arPY5kMu7++HyVIcITgrCyP1Xdbm3Q5vrYKsOS2nbdqbad7mTu8/PRz/PUXfwXZfxPwoattvWxhwKJBU0HVi4JmAdqmdq3T3qbMiOp8Fm1HpJ0b6/ull4UcuDLHjo1guCryFCKF7Hu4CEYWI+I8IcZFdvTVIlvZz3vGskwlNWOrt7RmN1dRL+ydVM8wJ3BOdWAUf6FmNFLOZTNV4Yd5jdgUpNXJysmKV7JCEjXSbYMT+1sAVOHhCyvX43w+q28hzq7swlE3tW81r3O+0H8cuaK9xcxFgTz02UsnIWe9Z2XhWbAl8IJsvmqdh6wARVI2spUiEWPoevzNv/mrWhUvfLXtdlu0kKhC8UWrECheRkItN+JDGWUd9zL/aMxkI5jt70LlVpmSyAFtzq21UAFYh7Orahz7jIHEtbe9jk5ZIUSGYuPDF6lWKCZGhJhJd0lrrm9gn++8Mm1lRY7jiGmaCsV6nmdsxhFSl7ku/Rfz3WJ4rgCzovqrX1QAS5CaWRMmE3ihhJv/VD8XDpz3TX8ymIYj5MRYENEFhxCqqS2usy1kzpLFSYuUAhY2Ra3ot/MzCz54e3uHn/3sZ5paE790v9+VDS5Mi/EHk9GUB5LBDlV+rlt12icWzJDha/SxL3Ld1BVrrQjUli4rAWnD0HIbTd9Rwa4YBvTajZOepzqOTgXw+rKyuZP0YZBaz5oPVCDUubKxvWlH2TQiYplnxLgICZBqBsLMkF2t9uyqC+dymYqGk4BAMhzLEgtVxzAxhiHvGUQBts9QSwywJV7frxkA0t6vy7IUYoBouooxRo6A43KsjX8brLS+3H6/xzgOZdyd1gM4cmVj+laL2PEFoVcznHKSvRmIyp7nzQElfSTAq/2vasdW2DJaIVVvrRGoUH+vN9hqtnYFrvCR9jjdHuWDtUOMUoxJ9QZUpwMcdfUFeKdVQMSIKaJ30s8CJI3vjC3y9PSE0HW4TBfxg6KE/hbh2f6b4FSyCpamGYYetTW6V0G3AhJLC2n1FEMCHA8Q+cbngQYlQSEd4bI5J71zQ9chOGHIsmZGxJci2bRCAUzOQrHOLEl7uyfRklyAVUb6cI2C4UOPYdzifDri7ds3YqLVFNpIswUA1jpC0cCiwTVsNJuRqSl709RAmbYipfK/GmHSSsEQc/G9s45VCQmupb88ThMMyCQ2UYaqfettbyAemn9FXqtTaslW+ztn8e88EYLiUM5LEYZttpVSwu3tLYgI0zRjf9hhibOW1AU4bz33A4LvwMoBM9qQ0Wn6fijPYzBMzhkGSK+eialoHKsSFxehrcZicNO23LQYKYU9eK/PImVxhh1KZlqO8Yr7eW9FJ5bH1HSV+soxRszzXAKOm9sbJSEwvvzyy6JhWE2hmEDSzEMbEaI2DgbUdzNZUStFVMilZn6LBmsUmZlZhiEObKZMtKFG/C7rAF4zKtqX/E2NILY6i8pF7fiPvVq/ixqhK5MFaejiQOAkNYt1o3WUpnc5Zwz9AO877LbSkGXcjGVLaNso3mg6tRVmjU4tV2rpsWVZZAvp5j7b7+bGoWXO2r7AgE47RnKAJedLrAUedcH1/QgBbT1CGOBDj9AN8G6Ac31x+E2zWhBQIIXMmKdJun6HDjkzXr58qQFQNYVyjkYiIIwK23ALzT21eU6DP3LOqz1N2zm0+4AplSZQK+aVa9gDwPZWqvSTjwmY3YDZ2mttJSGr4GD1b15/B6IdvJMw3ZxZkdkM5z26LiAuCeBYose4LOBhwNdff43vfOc7uFwumKYFn3/2HczzInABSbWTRZWAcOL6YUTX98gsTNh1Ul8E6ebmBk9PT8XnucbWzNSymmIp/tViYSfEQu8ZzndgOPTdAIKki5ZlAYPRdwO6vlPKD+D8IEIprSEBCCXboZJHq+kkkAvI2fzRRavfF8xzxHYrMAsvlRlry5hLYEZF6yBX/+t6rhlqUhlIxCpA60o0ZKUo6fMXlWOajVHwOtJgxNnFWom29z5mPlWK1p+35uNbvm8ApjmrAAozALBGI4QQdCsZPd/lcsHxeCytN6Py+bdb2YnksD9ontR46TZgKLv5Bj/A6h8rE0K+dT6fcblcStPkNulv38uNybS0VHUDCEwOcE7SRfBq2gSLm6YTYlxgEAFRRgiA8wxh587IeQawwBHKtj81RcaImRGz5FoFevG4e3aPzWaLV69fiSA5J1CGZkE8ETo46XZpvhih1DWguEGNL55Z9kVqp0w/K64S6nsy31xKIk2bAqrdliTM2/WKzR8VLBKdX4TRHNqVaeS2dOwqslGPwGr7Sq2nqtxVcUqnvokeG0LAsJFGKl999RUOBynoPZ3OIBIhZGYE1YTXjX+NuWpcMbvneV5wPl904lLx4Voz+m2mvy1IFsDVwyGDc0TOETFOmJcLpumsEfGCZZlwmZ4AygJELwvm5VLglJxZkt3xDCCVe0pJUnPESavUGX3fYb/b4z/8D34DwQekmFamrQRrGoFmQV81KOK6+0nzKjw31VjJsE6qi83wCfPzbH4dCJ6k3b4pjQRGdkB2qF19PjagK5PXaqnV/Zkl9gDSt+AoVfJlmzqpj7SIiiBgZj8MipZ7Kdl3hBhnZO1Wk3PG4XDA09MJIUx4/vwZHh4eMI4jzqcn2CaqhqdlizhLrCH3Zo30cmYpJ9Oqq8vlolVGlQhgfmAbKFgNp3dOMU81USSCEuMCzhGcGJwSljihyx7e9Zh0DwE4Qt+NcBCWcEoLUpoETOYMRwuksYzc/+V0whJnnM5HpBRxPs74V3/0f4GgjNuYChaac4YDVZIkObBmKqwdVpnXq1cp/iASv4tR5tRaK3xwVNacrnNgMmGtghjE91nn1q6FTrAnjVhY9/5hi2YFjuAPL90EDEDtNQrZPY09NpsD7m5vMQwjLsuM9w/vkXLGJ59+hvfvHwAi9H0nHDTncHNzg2laFONKmCahN1cHPup4qFb1knjuhx5x0d4RKjjTtKy0twi8kAitiupau9v3AGUSN+bB0lYxJ8QpYro8wrsgKaJiKgnDsFUWBiDaXarGyRPAVKPjnHFJp6Lp3j+8xTevv8ai5IObm3t0nUecGJ4cnr98hq9ev8ayLNhtd3h+/wyAaPplMQ2tOx53EkzkmAqV6VoTdpYlIZS9Tm3Sc/21QGit9QDpM0MgoVISF2Ozx3gjYEbfgTU2YzXWBrKyheQf98eMi28T8+zZc3Rdj/P5iOPxiL/+6hcAOfgu4Ob2Bp9++jmCk16vRB1ev36N7ei1P6yQ9V6+fIl377/GwwPwySef4NWrV3Ba/SwTJyYzJcG1s5rCzThgmuQ+eq1If/fuXUHXY4x4enrCdjuu2nDKudYN8YBaLOxcQAiidUGyQx05h8vlgv3trQQ8FErjY/NJC5uV9Z4hzV66Qbt/L4SHhwe8ffsW59OxuBaAw+3tvfzOwOV8wU+Pv8Av//IvIeeML//6S3zxxRfFr5NOiFLyRyQt63W160a4lYdmwhKjtEEw2k8bQUIF6xoybTFUKYlRzX93s/1dGUiDN9aiYpGlhdak1dKwCEYPJt8KFBXMZ1kW3N7e4nA4YJ5nvH//Fo+P7zBNZ91BLcMHiWKmacKbr1/jm69f49nz5/js8+/i/vlL3N/f4XyZsN/LHuPLMgutRid6t9thmSccj6eGBSLVT1JjKdHaze0t+i6g70eM44C+78uOJrvdBtvtviD3IdQ+ILZji5lJg0uqj2d+3oQUF5xPZ5yfztjfHjBspPs2uaBwR4UOCkN4BQPVegPnPDabLW72NzI233xTtE6MC96+eY3Hp0f8+g9/CALw87/6KeKylM5MNn/y7xp4MqAUgO6TtabSk0aJDhD6NlWqVdF4RQquXKtWLojg7253vyurqva7al/i9KOGpGyRZ/slwDY8ELUrJ3/+/Dm89zgej0KlVoapIerEHrIbvPllGSDpqvz+3Xv84ou/xs3NLW5v7/D97/8Knp6OePPmG+z3e9ze3uLNmzdwzuFwOODx8QHHp0cAFfdaloiYhNj44sULbDeb0kzFKqeISO/N0HFB3NuyN6kNrUGFJdmtuFiizSymaZ5wOT6hGwcMmw3GYaM9MkYAwDBUvr6ZqJp0D/ChQz+MpY0Xs/bQyBnbzRan0xmXyxkpLXj7zddw5PGnf/ZnWOYJgQhxiSpQNTXYQhBsbo9GhYCg/+VedJ6dr5uieucKRFGAW2ZYjUURgkbI2pe/u93/rqyedfXRStAg2hbFwa0SLF8QA+207nAcBTh9eHhYtZgUkLLRs5pechCcDETIWbRmhvDT3r17gzdvXmO33+P166/xt//2b+L9+/f4+c9/gc8//xt48+YbTNOE73znu+iHAfM8oQsBUSMuRx7b7R53d8/lvDmV5sAh9LIBa4wYhk1heVg0ahvQy9bT1iutmlvr1gh4xOWCGCdMpzOICIfbW4AY47BD1w0AdI8AqjQh6WYk/pnTnYFJiZQWceeU8PjwWFrc397c4Hh8wu6ww7/+l/8nhk2P4/ERvMwy/w30wSzQhoHEAMpWkqapDI5iZu2zC+3ZYQTPCnkY+i/B6VUPOxNd4prHVIXkb9VctgUO1xgZgVbN8T6QVhU676XlwOVyFgpNA2xW5xk1RVFiYvnXey9bBqaInBk5OaVuM77++mv8+7/2a/jxj3+MEHr89m//J9hstrhcztjvDzgcDpqXlCXX9z36YcQ4bjEMG3RhEH9H6zKdCwidFaQEbLdbPD4+qhkL+uNLFkHoQVLXYH31rQRwnhekZcJymZAzsNlssd3foBtGOHLowohhFKyu7wfllAlOxiQNZUiZIilleCddlRat1np4/yBFzpTx9u07AIzj6YSf//T/w9t33wC5MmRSkw4yANY0EKEgTTVKbP1vM5VqvURIFRVQbr+Bx60gty8u5xKBJUfwdzdiLk3STSjag0v+Ch/aXtJeWnKO1ERj1Vdp4QOwJVBbTZjVxNae+CBfsZqc4X2P169f4wc/+AG+971fxqtXXyHGiB/+8IcAgMfHB9zf3yFn0g2+ZIOGvh9U4HoMg2xlaL0tQif5Sds7oBS0KHXIgoO+78E5SstR9c1MCFOKOD09ImuE67seruuw3W4Rl4i+7zAOW5CTfmyAdOsmLUiOyyxlgNY3TAFfy6kejwJZKNcBr776Ep0nMDu8e/s1Hsy/bQK3MuEqYLaHaPs+NRJR57LpIWfaFiTt7plLXrOVj7U2QwkUHItFYlKczAa43MS1c9b8+WEmwAnD0TVOY6MNeaW1Kqq2GhDVbjlG9OOAlBLmWfaChKZ7xrHHi08+wePDA4L3+M53v4dXr1/hJz/5CQ6HPe7v7/H49ASpiL/Fu3dvEEJfiIkAME1zacFgq3EYBgTl3u92O3z11VcAS/9aoxrlnLDMM4a+R6c9yELwWOYF87xgmU+4nM7YHu4Q1JSmVN2GJUWMWjXf9z2YHChYxEagUCnUrG3hjcNPRHh8fIR3DsfTAzwxTqcnzAvhZz/9aWlQcz13IiBcdpHhRqCcqrZrZdJOuL0t0Xku1gyt49/ITdFsmcEkgBexlOiHFgv62AXl/aS+mLYSV0VFJmSwOvGmSqi95aK91vvyWOK6njBimYGuG9EFj2WRSGoct9jtdgCkUPabb77B669f4/nze9DtAT/7+c9we7vH+/fvsN/f4pe/9yu4vb3Fw8MDcmZsNmPpvTHPF+x2O0yXC+YpYxxHbMaxMDFevHiBd9+8LR3AnXNwTEDM6Dah0L5TSjg+PuFyesTx/XskIgw5wvktuqEvnLigHblzivBezKdXDI5DJSfID5cKrr7vcTmfkeMCoozz6YTpeML5fBZe2c0O4zjgzdv3FUpgmYGiW4zsQVQ3sWV12htFUQpcCpuiTJGaylqVtI5R168SPGRB+rNQcOEPu/F3vy1H2fpRBAh8QfWzahahuNm1/2VmtNZB2rnqNdrftUdaFt56TAtSygofiAN7OZ/w+PgOD49vcZkuWJYF3/veL4m/0w04Xy44nU64ubnD3d0dmKW/xzhucDgcAAjKv91uxeHX9u2mgb33iIsQI512CUIWIuRm3OrnImTLdMbTu7c4PR2xORzQDSO22x2cC9jttzAAehgGhK5H1w8qfIqGs6XYjDJuboRM6nQ+43R8BIgQ5wVvv3mL4+lJ22wlvP36FY7HR0B3lGHmZlOPOk9mKhmAbwDkNqMBQBWFzXULrTSTVGxpmTU50jUWSv1izqIBPzCX13nHD/OQKMLVKiytM/rosbXesUGOG4HMmsMEaW8G4kJKDJ1TQU04Hh8RZ+my44LHOIxgZvziF7/A559/B8OwVSIi4auvvsTd3R0Oh5uSjzSn3ftQNNf5fC69a5kZXeiwHUe8fnzEsESkGMHBYdwMSDlhmhRwvUw4PT3izTffaKRlNO1Om/QBISgDl4FRgwdyTgiQKrgSUFiqS1nnIByfnvDmm6+Rk9RVztOEuMyYLhfkvofXtltOAc6cU3VFqve9mjNSv0o6kiu151vy1UV+PoKdrgROXR1OXHw9+7ZX31JKGr4FumgF68P3U0F4bYOAlS9/da5vy41a5AJyYAj6zSQ0mGVZMAySSzwenzBNE6Z5klrJrgcUUT6fLvji51/g8fEB2+0e4yjFwO/fP0gvjf0ejhzO5wne99hsdri5vUPX94gpYZ6WgoY7J/uhbzcb6f8xTcrrkwwCmBHnC85Pj3j36jWOTycE3b9J+GKyglOccJkuSJnRDaNwx7yHJ9UmQInWBRLZlGBlWRY8PDzgdHzCNF3w+PCA8/mIfuhwe3OAd8oMQU2d2XxbbziTASMxgrnuyKxJchOKVrNRnRxYE+MSbV5/JrgXYJT3bP4fo5X40Dri9nJF33C1Z8oIbWLOqpWMZt3CHrRuwtICj62WI9LmvoqT2Xcla5UxzxP6occyZ7x//4RxHOBzxvl4KoPQdUm1gPhgz5+9EBD4dMTbt2+x2Wyw2W0xXRjYRAMAACAASURBVC4oG7eGgMPhgMtlQlwV8wZsthvE+YA337zG5fSEzThimaUvRF5mnC8nPLx9g3dv3oDh0fUDuq6XHeugfiY5jJstNts9hn5Qs6T4l42Rc3CsewfA/LIFaVmwGUfE7Rbv378vhAKQUpFYC7tZuV0aHBUyQLP8GY2P1rzaiHL9gQpR6Vq8Ljppj9XZhyhmLgwduzJJT1LdLEK/fG0eqWBYH8aGBOl10VKJr7VVG0xYaC6pCl/OVo5TX8HaJdm1pumC8+UJtzfPMQxeOyGmwngNIWAcc9lmhpnx+PiIYeiVuDjgdD5jS2K6LpezbqMTVNBuC8Mh5yTtEzrBzY4PHZ7ev8fQD+jCVFb9Mk94enjA+XTC7fPP4EIPBimo24sPuNlisztgs9kJ6OoJHLk8t9RpxtU4pZRwuVyQkyT/j8cT5nlG13d4fDzhyy+/xKBa03mUBoHWcrNObitEXAMDVN+zgGfG5bcx/wiyYNmD1haywhnm4ki9wjo1VdpfiEpdS3WVRhR6bRHs2nAfNTAkLW9fMxY+uFtYiVUd2MwMjgui9aJXsxUzMC8iTM4TLpczDocbEHV4eHgsLAnZFifCeunnzFiWGXFZcLhx8JsNuhBwOYtwySCydgZCCQxiipLO0Q6PNGTc3Nzh8eEBp8cnBN/p5BIulxNOxyeAgXG7xW5/UC0mK3oYR+wPtwjdRuGRiJRzSUY776/GqI22xUIsy4J5nqSvyDLjcr6As+R3N5sNOGfM86y+5GW1yLMKV3HdCYahoiqJxkyCm62eGmFqcTRu5KMIIn8ok00QZf8Gs9lESpVlgB2tJF5sNRVzVhx/mOBLSPOtDmR5iWSnXBO4bR7UFpNUMdamcd57pLjg4eE9AJaO0OOI4/EowGgnLZus96xp5fPppJx/MWeyJU3Gu7cPSCljt9tjWWYMvbA+mDM4ZilS6Trs9gfc3z/H5XxGill3skt4eP8el9MJN7fPcHt3i8PNDTYHiSqHccTh9g59LzlKzhGcZ+TM6Met7lQs+F8hHTT7VHnvcdKO313f43x6QpoXbDcbTJctlkUYvMu8aL3DiCd6Wk2+gbBOhaf4XwUjs+9qNOmoNrMoNcIqld+OWKBVba1glU/1mWRrH3PAnROMY/VFO9laWrFahR8PGGr0KL5OXR65nIP4Y6V0DsxRHFmy0jVhORARNhs5/2effYZpWtCFXqnbWvy7LBIxdh3gCDFlDIMrxSVm2t69eyvbTjOw324hIYwFNBk3fY+UIv76Zz8FUsKyJCyXC9K8YLfd4/75C2wON3ChwzBssNvtBMLwQTRJToXLXFoneF/6skI3tyAXVj1WrUp+vlwwT7IRxPH4iGm66BaFsrNeSgmn8wmWQpLuPS2o3sAKjda69pGLWInGWWk8RnVlrrVWUQp2jvK9KnhiLhthMA21NnltUFpvuNwaMYQRK0vm2qmvEi59tYr6BqTB2gcQCZV/QeLLSKX4IhgTEZ6ennBW87fbHXC4uVGz1+Ozzz/HZZImfpvNBptxRPBek9wD5mWBCx6sfo8g6+LU932HtETs9htpn5AZ/dhjv93h8viAaZkxn89gIuwOdxj3B/TjCOc9NpsdtrsbDP0oUSgyEjJ8FzTzoJVHQuQv40zEpfLbWnfJziUJ8zSLWZym4oeacMQ54u7uFm/evpLNOJpqMzQTfQ0rtfNSP7gSngKGXn2X1n53y/m341pGsb3CCphrfK+aZFVHEusGHVbQK8G9L8LxMZ+sOvK1Jx998J32Exsg2SHEOhXKKg+lt6xRffphwG/8xm9KS08VqNNJnGbvHPrdTgHRDqQFuDGKtksp4fh0xOl0xN3NrWi2mOGD7Iw7jjvc3j3H49u3eP/mLZgZh2fPsdlusRm3GIcNbm5vG0eeYWVyFsVaQxZQU7CDjMiyQSxUe/kQMC8LLtMFzjt0Qfb1bNtNzfMs/Lkl4hdffLESMEAS0wlqlRiyHQ7MfFbXZJ2K0oQ2Sb2SkT9bX5FJG01cuUQlnWRRKK+VSxGyazt6XYMpDrIMhpU7WSQi3fnwwWpoz/eBr9b4eoABkKbUuFRXmyEVOo3DMI5gRmFFbDYbjOOIt2/e4E/+5N/g/v4Znj1/gc1mi77rpJo7yF7eMWc8vnsHZsb9/T3GzYicMtIiG4TFeSnpk5xZ0ki+wxICdre3uHv+AvPxCegCdocbbA57RN2CL3SdBkTaToAIXd9LwxbloVnKhyATb8lmWKUVhHXBWgaYYsS8SG40KQO3FY6n0xHsMvreYZ4SUmwCMX1lw69MWaj/nJsArQhDgSFaa6IChuqft/ewNo3Q7bWpLKamdZRNtNOaOl4JRjV1GkGi+lAfN68fvlYarvy/lUqbIIs8paOOlK9JC9AUxVxst9uyEG5vb/Hll69wOOzw5u03SAkYxy1ubm4xaDt2R4RpmmRPJ23vdLlccHNzh6H3mBSpzxvRymJWR0H6M2PoByycsb3Z4/nLl5gzY9jvEboRz+6fY9xu4IMkxUMIGDf9mvVK1YTIaqojaC3MM0To5un/r+zNenVL0juvX8Qa32lPZ8p0le02nS64cKsNtGU3FgIkREstcYXgAyDRXHKHEHf+BH0D4o4b+gsg07gFTXULWYBayCqpJzurypWZdTJPnmkP77imiODiiYgVa+13n2pWaufZ+33XECviiWd+/k/H9u6B5nDkl998BRjKqhSC82k/19fX9H1P2xzJnKXte0LTL2lvLRwrEPE5IJXUzTDqalMCmmoyaqLfPVpbPBaaziYMKxz5/OT5YYxJ0kQSjpf+HWX2YwPhLBdjSpDOBYMgiMxgfYXdZpJOHJIbb4zlzZu3rJbrWBjS911Miy7LmsvLC19tnVFWC5xzPqt0wPQtzcGQ5YrSux9QAsacKYk3aiRp8LjTLFcbTvUOrGNzsaG+WLParFksF6xWS+lc54H6qqJEZWNhTnizUuuY9pw2KnNGwGSkQUbGYAeWqwXGdBwOe3b7PcuyIluvORwOrNYLTqdj7EkV1k6YgOTyj41l07lnoqifk2LnfpeVdskyj7lk0YCDWNmUujEAcu0jQs66R7V4gM/eVF6pZyTnWeOe+Y5IBzPfNfiJP8/5ZKdrrSl0Rms6yWD1le7WSieSH/7wN3j37h0oR98bfvSvfcHLl5/zwx/+kBcvXuJ8ZdJgpCQs89DnNzfPGAbD/e17Pt6+ZbO5gqTuUynBEBsRtMVvtlytOC2XdMagc2nVkxeFbxNURk5WFAUOqTay1pJriYTkfoPkWkdL0jqpa0QJV+v7jvuHW47HA0VVUZiCTBfs9we2D/e4TLFer+n6XohyGHwlVAx+ECu17BhLTu33YC3OIQjmR6o2zZd7GsOWI4SuQnFySmh59IeoKeeZcqFQnW2T8yKOMqHaPu2fGAb5iMBScpqJZsAnERKhDJTKefn8BR8/fvRVQTl/8Ad/wHffvuN3f/evc319xTBYvvnla06nryTPfjBsLi6FGyV1k8ZblHW9oFosePbspcQuffWOzjPysqQuSwrfzMKUBRlw2m3JMs2qrtlcXXNxc8NqvY5iOGTQWmulX4DOyHUGHh0xJELawUSrDW9sGWPpO9HD+rbjdNhz+/GjiErTczjsWGUllOIGMdbSNEfKouRopKKe1FtqR9AV0aG9mLaPiWO+HoH4JlzwzPrNC1VSuglHSE/P5yc/JgJkgMHiiGatjoO3aoQnEkIZ+w89RWDTz9Jyee3hRxUGiTN++PDOE3fJfn/kH/7DH7NcLrm9+0CWKYqi5NVnn/Pq1WcY4zjsd6wvLlmt1pRlQdM0UR9zzvkM2SVtfcJpJ8Ho45FSZ1S6BONwOijEmtK3AmyXK4p6wc3z5yw3F+RFQV0vYkldSMdGCzcUi3z0fYleliDm+Dl2dkTuaU4NXduSaU1zPCIOZMNAT60KMRqM4XRqafuOXOcYF/qCTudXQeyxeW5t5wSSisG56JzTxzxZ8ZGO598HQKdUe84SVKSWoIJAXMqnrITU6zOyPbVQ0pebvuT0ZUPSnrOW0ne8zXJHlkm/oT/8w7/JZrMi8+Gd+/v7WC52f3/H9c0NVzc3XF5eij/MjRCeFxcXaK05nU589/23HPsTTom1ul6vOTYN2+29RBl8TySVa7QCMxhy72Kxvp4zizUCIziK1prSuy10lpElKkLqQ0o3oRkMh/2et2/fMgw9i8WC5Uq6rzSHI9pHSA67Pc5ZTu0B5STc1tthann62G/0dSV4I/M1CFZlkIVxbUB6TgW/mH2sr8+Nw3RN5+udi06gAjlNBqCUio7CAHHpxJ8xmrUxAHEeyecp1jxR9JNBhoJXrXIUUOUF7WAZBsswtHz55ZcAscgjtk7Mc9qm4fbje+E6bcNiWUvPJSftdLqu4+JiTdu2PHv2bNQZ8pIsK1guxcemihLjIFeKHMXgwDUd9thQ+nTt3BNWURSx74D21qwjVJiPLpgsWZBAbMHBut/v+f77N+x3DxyOO5rTSWARnMNYwewoi4KLiwtUnnF3dyfwDVYiBsZ65Tuuj8ywVWA8xwx95Z1z+BqPGBbM3GOGYHGTZl4qaFVKPTp3Qi9naCCPH7jHOllKjeEn7lil4iDP0dGc9U5dGFMCnLhRvP6nfAMr0anwhF6w2+1YrVYR4Uc62hpevHhBVQoBBFA9MwzoRY32pW1lWVJVBXVd03uId0HgGWICZN/3FLnGdA6nMzCWw3bL9v1bOBzIVmsKB4VP2VFuzAsLPin8wtqAXasVQ99H5Tms3dAPHI9H3r9/x/v37zgeDz4w3rHf79FWesIHp7H058zZb3eiR6bZF951YZy0pkaBco5cJdIoegOShfDjDR878HBRTzOIc6JxLmZT+gl965hzshHekrgzzpbD4fOGmA7qnI434XDxtcZ7OxeUT+dLxRR5UVHa0vvFLri7u4sYrCsPvb5er2iO0mOgLEva04n3797yKvuc5aJmdbEi1BcEcZyXBUM/YJRglEkrRMtiUeOsoR96MqVo7h94+9Of0334SK2A0wnbdth+IC8LcTzm4nS1IAgODt8QQ/kExgFjXKSu0AvUDIa7+3t5p+bEYXfPw919zOI1w8But6UoCxSOsq5ojG++oRSD6Udry1p6z2msCuVuoD1wxcRfpqa/Kq1wxiVR6LG+EiX1Acq5uEmeUn/Cv2lIyTlHHqLykZjCQIPXn2AUT8lCB9tZ7nTWJfG0qPSXJUaRPM9jrwZvsXMonyUb8+U9t3LW0p4alstltOyur68pfBnbfrdjsVhwsbmiH6Cuc99lxaMPaU1V1oQmB8MgsO1d00hP877l+PCR73/xNd/85Cd88dmvoxcVtmno7u4oS+k5pWrjLUopGraDla54ynn4+VZqSK24Q3pjpAMwjqbpuL/9yMPdB+7ubrm7vaNrW9FFfU+jxXKBRkWYBG0MJx+bdV6UKKVQTlE4FcFdrPJoh747n/J6VQRLUS627fZTP6o8fqMov7ZPKjyJrhbpIqnfDTQQnbEST5vKWLGAxM80fuW8pfdJ+nnC4pCAcDooYeJjLFPr0C88yHkxhbvOt4EeBjItNY3VqqI9nsA4qucBhl2ILNPCCZrTnrzM0CzIsoLGDDhr0XlOrivQCmOFIxhrMV3L/v6B1z/7c777+c/45l/+OVdlQffsFawWHLb3/Pk/+wmf/9Zv8a///u+zXn/h1QUl/Q36AYWjGzoO2w90hwN9e2LoWtr+xLHt6TsweYF1mcRY25Zh6NBKewirls16zWl/jKB2ocNJ6Gh89HMUw2BBSQ/uKBi5GW4Ui8F9khhcIfXcr7y4Vmbr+ZRPMyxkKAJOXR+B2CKRSW3dWLFCGGSkKJU4a30fxJmiN1cK3YTTBVYZ7i6/T4PlAelwfGHnBJs/t+IiCJ2Gy7Kk6zrZ9Ywm86k5CkexFodB4WiPDR+GlmOz49Q2KKe5efaCerWiqteU1YJM5VjlaE4Nr7/+BW++/gqz2/JsWVHrgsV6Q1Ev6bqGm3XNzbKkcAZlvb7jdaauazgdHth/+I6vfvrPefv65wxdR6nFD9cNhu19z15VXL38jKpacmxa0bH8a5dFQds0NK0Uuzy7vqEoC6qy4nZ7R0jsVEpJFoyfVitMC+20XypR3qUWUs4LQMIZTPLawpqFloapz+wphT6sYkqIqRsr/J10JJheEW/qd8aoFap48lxEprI6wBr5p8ddNz7usSHg6dFbsp5YrSHLxXrsu4GiyH0mhnjwVZbhkGzYh4eeqpGyuPVyydCvcMby7Vdf8v3rn3F/+5E6L8mLmvXNcy6ubsgXG158/htcbS558fnnODLWV1fkyyXqeKS8vuFqfclivSErS26uLtlcrqmvLikuNlAW0tXXCYG9f/stb776kq/+2T+hKhSHu/cYNLbQoDKyIsPSsb3b0fSW569eMRjHbrtDKQQxsqromobr62vJji0KirxAQXQeC7wm3sEqs6gT5V5Eo5suGaPHQgjAT7pYVnJlUiAUCOdTDvWUbp4KUSXW5RPyz4k+NIrIVPzJS+mzrBRv4Xj0+MCqHw1i5HwqIcYoLhHlucgrwYhA0o5TjIosz2makw8LwWJZo5SjbY58ePNL7r7/Baf9vSzIckG3u+f97TuGAXRVU29e8vz5c/7w3/sPuLx6QbVYcvXiJae2ZXl5yavnr6g3S8o8p6wqLl7esLy8pt5comKPT8f+7o7/+x/9mPtv/4JV3tPj2D7cs3U5L68vWOYa6zS950RSFOIwfUcWikS04nQ80DYNRR/CVFLy1rYNQ98LIUS/WyAiyYgJmrPoack6IPqZTop3A9/IRCny9/CEOGvIla7RxGsAk9K7sLYp55u0vXnCAIkefO/pmBGhGndL8pDRZeEf6AloSswjoU10AiS8lOdhJhy51gxeHJSFpDZvNpuIsBMcocvFAuUcx/2e+7tb7t/+kuH0gOl7UBn7U0OeaapyiR0M7z9+oFcfePtmTVbk/Lv//t+iWix59vIV275noTOWVy+oNitKpSiLnGq1obq4olhtkmb3ho/v3vJPf/JnPKsadKE5tSfA8ZffvGO1XJJZi3KKbHGJbnbUVU17bNnuH6ICZYaeoe0EKVtrbGYpioyu7yT12hqs95HNJcPcMy+czbua/H94MZhiwdqg6CeGX7DKIgdL3R6MTEL4znlPQrh//ujbuOwpWw0P0zEhb/pC8t0505ZI0fHps6fM2bDzInPMRyJ0XysqyZQ1+OokAcALfipJ8jOcjh2H3Z43b14zHB/A9FjTg87YnlqeX12Q5QNt39O0LftW0K9/9uVf8G//3t9kWT9jc3mNdlBnOav1FYvlgiLPqIqccrmgvLgkq2uUIjYsNdZgzMB2t6dVlrY5Ua4qPm73FMsL8rqmMxl1dcnpzS2XVxX7/c6rBNLwwjqL04hhUo6Ae23TgBO9VKvpRo6LHtZJxw9jvv+o47sggoQBMNKPcqPREKIxcU1mrqrJ+s4419ytMRWXgYDTgcNYocSY7hEGoVRy/ZnDYWM9XuBqU3Y6Eu04ZpU8Q9oWt23L1WpDUeQcT0dBmvaWIkqyHJxzDH3H6XBgt9tyOh0ZmhbTd4BFF3Boe1a9Iy8Np94wDNB0bayLPOweuLm6pqqXVK8WVGXO5fqCuhKIzyLPycuCoq7QRRGbIjgcg3PcPH/Jm5+9IaOnOx3RTcvQD1xefY7OV2AVXSOo24fDnsP+QFmXaJ+eXtcLqrLEOseiqqXiXXt91DeISGHgx4UNKrOQTnAJCYcb1zVwtmjgxTK2QGzeGIg463IVmpgMPZV8KjUyx3VP1jQK3pRuxxMt3jyJBubIrbxnPgEcPhdITTx8BJ/LSMf2EdU/PuQNRqTGnLIsKDPpgdQ2jVRma03Xtmwfttze3nI8nei6jlPT0nQDx7anawesBWM11y//Kn/1R7/DyVf9hPEfDwf6XtrLlFVFsVigqoJ8WVOuFyI2lwtUnkVQXwm8Gy6ur/mtL36b3Wng9qFh1zje3+7ZLNcYu6AbNDrP+Bf/4p9LZfhux2B6SSfKcuqyRuc5WVmyXC0pFzXlopZMhqwg05mU3qm5g4EJR5p6B8YlkJ9QKebAWd9dzsYTx/WwnoEkBPMkIxmjOqlXIfw+0ckeXaxAjfTtjYCUy4SM/PPFI/N7OxUyD8K0pOVwakLA49SJ/y4tpLi4uIBemmEFDNmmOXI8njjs9zzcP3BqTnTdUeAtLTgn7atzrSmqFVcvfoOvv/qXDL4Bl1QBCdpi057IckXet1i3EqyyqqIMfjUkqE0AkvESaL1Z84Pf/E1WF9e8+e6AHQaavqOocv7ff/J/cXF1zXZ3z4d3b3nx4jnG9CyXK6q6pioFsM9lCucMmfK6sBKITmsMPY6ri0vpw0Q7yQmLxOXG+SRwncDFZht6HvKLn3mwlehUDT/ApHhEefGaOGXPMZpHOf7pMQYX3LjcbmZlBMvlE97Z1B0SPGaBiFLxKIPUj69VjqIqYk7YsqjReeEhB3Lvn+o4Hg/s9jt2+x1t12LtIP4/J4HiwTiyQnBZnev4xS9+gbMpFlnO8bTncNhirbSRdk6gp6y1WGPpkHQeB2CtVE8jISJjHZurK/7N3/t3ePcnf8zDfsdgLE174KuvfkqVS4/Oq6tLtEggiVhUNeViSb1Y4pwR67LtMNaPvR9oTidQisVqSV5WcNqPk2SdePbjShHHKPaEiwq9bxcRvaHzdUuV/LjGEIt/z+ndKTz/OTdHhI6aU3QgiuQPEY2JwyWYsFEvmw+WVEENiY+ph8w9Oi+8WIh/iYsiA1VEULk8LyiritVy5Yt2HW0res7Dwz1tJ5gXkufu6xKCra8y2q7lm69/QXs80htDXS/ZXF5y/eyGV69eje4TpSgrzWK5IityjDUS4/SIPDjJzbdGiLkfGozrqcslD9t7tvs9u8MRO0jYqm97bm5uKOsKnedcXV2xXK1QOqOsakDRtCdOxwPHw5Guk35Kdhjo2obe9KAVD4c9p/1WjATnXUhq9IuJ33yiXHsCJH6vZmsUFzVeor0hQTQS1BMO2XPrnh4T6zKlxDGsNPKywL4nro6E4pk9IFB3QDYcX2M62OnA5GmRVTsFThpjBeJru45FvYhJgsPQczgc6DoJzWjlkQt9y2iHKMYWx9D3NO39BIF6sV5Q+CxaEb0niiJnuZTuKCISBvrW0Fmpz6SqIv6G9UaItZY8K7h4fsVf/xt/gzfff49SOafjMaYjkUke3mK5JMsLcMSiE+csQ9czdD12MFFfE2IvaQ8HHu7u2axW3GstWLH4KQpiUYW1cnFzpzOrAWeJmCOTyAwuKEbEcFW6nv7fR7UDTK3dR66UlBjm0fN04RUjmJrczDfkenTu9MG5N83l0qAfTF9+dCw6nJM6S2nLF86T9jjGGJq24XQ88u7dO7KiIFMZRV7FFyyKQpL+PAZGABpGZbFeU6kxmbEoClbLjU8ZGhiGjiyTFoNaCSS8wvrGKwJUZ8xAN3T0wwDGd4/z6UhlKbj9Lz//NX733/o9lssl5CNi9sXlBZuLCxGTZYFzoogPduB4OtCcBBo0NCoz1sYClaIoMG3HerkYlQ7fv8jqqSRK8igeOUqzbIwznuM8aQzyXOb0nKDOZeekBDy6MM4Qixq19DhglcQvo+3qH2rPKPwBEnPU40ZuNuec4fcMcE7gBFACjtIPrZTzH0xMFvzyyy9R2lEvqli+b62hyHP6oZfmWn0vldiZJsvEaSsWoUzgarWiqguqsuDyUpTq1WrDZnPBcn3Bol5RVQt0kWOHHtf3GOvIbQ7OYLzrQQizBJWByuj6nl/74Q/44kc/4rPdZ+RagJaVUtS1QBo458jKjLzIGdqO7nhkaAXmvTc9RZFTlxVt30XO2w49l3lNWVUYO/hGrYmbIir5BkXA3ZiQ0CMjMV2zKWd7QmcjKPvT6+bFKZPY5VPBT6/d+oGHfth4fSwMZprZel4EPn2kFo7zD7BSCeHrFJVPw5FMU+erk6QgpCYvffWPESW6riq6tqMfOvI8k8yMvIgoOYOxZLmmLHNWq3XkMPv9js9/7Qes12vBtFivqJfLCE6sM4E2CKVhEu5CeiIhWR86K+J7V/UClRU8f/EqjknmRfSdYRioFzVVXcu4OmnqKmVyVuo5M82paWjbVoD8tJaEy75js7mgaY7e+iNo+H4+z0kWn+WctBdMCeT/13oF8lJT+o26qjfwHjtjOUMkDpQSgLbw/Xzgkwck7ou5eZyGmeS64JtKXlRrSfsNRO/1vNDjUfn43tBL5Y9kLuRgQZfyLBGVS5yzNL4V4PF45Oj1oiDe63rpRYeAm2w2G3GPXF2yubhitZKayrquKYoctKZXCrTg2o7IkFb6JumMoqjiGOt6xdXNMz6+f8sGadwqRcaZj1BI5XjXtrSNYJ9prSXbohbxfzqdUL4KPaAbZVnG6dSwWl3y7t2bZO2IndqCeiMAeTZahUoprEl0pYTzpFXlT3kcxs+DZTqSQPzOeR9bNsLW5yQcZE6xstBIOpmz0jwUJv4ylMch9bLbJlwxWJ9hYeUaF38CgcV7Od9VNtEDskwqyPO8RFmHGwRV0RjDYlFTVxXKCcla71Iwg2C9VtWCqlzgHBG//9QELC9fwFuMXeaur6+Fs1UVq81FrCYvcmlaX+TSogetsUrgMTWC25rlBQEfTSlBD8p0QVEtIlz6crn0fcSFk+VFgfE6nrNORH5dg5Kw2cXVJaYf2O+k3eJ2uyXPc/YPD1z7lkIxhjmz8FPfpdc6HonJqHdlCu3xXefEFfSzVDcLKk9IOp2cHwjOSphNa00eAGoVko+Uzah3EmaC2HRTCGn0M9sZ55rL5jjAcJ+gt87keJZpD1ziLS7rdQqd49Tgi2F9gwrlGAbn/VkCP3A6nVgul9w8f+Yry1uKU0me5RwOB8q8IisylptNxD8zFnRWsNlcsllfslpvyFVGmRViyuuMDCjz53l7wQAAIABJREFUQtB+nOB8WW1RWUbhe4fj58ci3PfofVl5VlAuar59/YbNZoN1jmW1oD2esE5qLkPfdGsceVWiy1Kq4ruO9cUldjDU1YLtfkdeFrHxxHyO07l3sR3J4yNVzK0JEuX8eROnb8IQzqpZSvhoyhnzwGVCC+GQPzYONHU1jNQclfiZ8zQd/NTJmrhE1LgF0oE653zrGnF5hE5zee4tmKwkd6GeUDGiCSnv1ReCK8uS0+kUA+fPX7ygPTWoTMeMja43bC4v0VrzzetfioGSZdS19EJaLBYUZSkdSLSEsIwZpH+UcmIVKk2eFRD7hWsvtyzN8ch+v4/Pe//+jViaKIq6Fl2ulWKSFLu/qhcs12ssju2wZX15QaYc94c70GJhCiRpEUW9c85XlT1WfZ7St6OijuMRBJRSE+711D2fcsIaZyff57gRFVkx6kfzgSnGbIPg01HRIJi+4LmXii9HqslNj+CykEEHApRcKqUyikIQpofBsqxFDFk7oJRD5QoGwZYIxbyC2y/+MV3kXK+W0i3DKUrvcmjbFuVguV6R5wncQFVRFYXgS1iLynMPGep7USpJQsxVJn2RUJKRiuF0OPDhwzse7m65vrmk8QmItx/vpa6Smq4bMB53Lcu075xiY/isWtQ8f/4CZy0fPn7k1ec/oDmeyPMtt7cf6bqOYRir+IMMC3pRIPhzHC79W5DLg/ya6tOTyM4jo+4x25tbmBHVZ/JARawhHIlLRasuqO7yIDxBTCuJ5w8NfPhcOUI6qJGww3fhRTxKo8DGAErMf2NZrzeAELu1BquUoP8MI1ic1pq8KLh5tqFtpJtJWQrM+nq95uc//5k4MwdDXVcitvJM+gkpyDOpotJZRuaEyzonuK9RT8GJnuYMtu85Hk98eP+eTDnubu99H6aSly9fev3Pcv9wJx3lnCMri8Rnt6Jc1BR1BUh2yW9dXrHb7rh8vuJweJCZtM6rrh4O1LfGdgGWKi6Fm+hT6dwrJRvDBoh8v7zKK1xzhhOiC7Em/ZxK5C3MlBLy1H8fW/oyFvY6Z6V6RQVROorO0dIYfS9nOVjQ1ZRGYRgLgRUuKn1T5+94r8cW7DAMLBYeCHggpr9Y6wTa0/uP2rZjtVqz2VyKTja03Nw8RyklZXDDwGq1xNiOspQGErkvb9NKyuek9jPzSX4+8yRTqDyj8NCczovIIO6lS55Yg0PbMpiBqq6iCNoftr5MTlKVClt4rIyCopTwWblYUJU1CrEyF4sVg7Uc90cPU3DyhC4IR87ZEAAIU+43+MhRUm6kUWCsz4iN7k7PE9RkvsNmVSpZ/4Ro09y/tBW2iuLSk905mY0SBD6HpPpor4elOltKHOdcIZPzwps8ITSjv8yOkYTUAnVO+TaF4gWv6xqXZThjqarST/RA73H5Q3PV3XbrcSuqGOsUF8GBsqxEr7GK6+sb4Xw+rSZ0oRE4euXLy4iuDxWrxIXItV8YB2JFOst2u+X6+tqLMem00pxO4MNFfouitNRjtn1PYSwlkqCIEzVltVrG/qHKeagp5yMReFcFespd3WPHa0qFYRXiPo9LMg0pzUXndF29RGJa7ebCekPM4HlSFwsma/g5M9on3mJOuMFtkTGl56nzMFig6fVKST78crmMVlXwfwWOkfrbAjaFlNf1HhLUxy8TAlVKMVjLan3J5vLKg6eU/p00QTyPKeSjdR2yhjMtYTPj+5y3fUffdRwPR+7u7iOcZ5aJx785nQTJZzBSmheIwoY+7tJnqe86mqalHwYRn0VO25xEF9aawVfAj2G68f3nBDb3f0UHbqYEdUiNgDCfwkVJ12jyuwdPDJAWkeNpD5sVrUcXFjwRVXGUMtUxOS15gFJTCv7VhDYPrsakk1FP0KNV69wIZeScpFsHQsmyjDIv6LqWLCsk7WfQOCuhlqIssdYx9ANDPrBab1j4Rl6996NdXIjP7dWrz6gqARkehkHgCooc64SDZglHj1htns05JyLc9J1HCDrgzEBZL7i5usRaK4DDnnv1fU/fCWhf7v10fS/PVM0p4vxLM9icIstoTtLyxxnDqW1Y5LlYurR+A+iR65OypunaZJmOXCp4MZUWnXYknKlUCrcI6zy3Ji3SMzTUe4y+Mq/4Kzw8AL4IdEYSQQ1TSMcvZTX4pqg6dV+4x6QWZPnsw/MkmRC1vIRwEeFGMtjj8YDWGZeXl2PQOMvFI48QoxRbSDfcoiglqK+170ISxOSJi6tL8iIXzAmdcXlxjXHSEXcYBvqupapr6sWSIpdAtgvjRGFdQEk3dE3LyfdB+v77N+y29yyWNb0v22vblvv7e5/BmzFYS9O19G1LboI3Pydv2iiOcaGOoaRrWprmRNe2klqEo+9ajBmt+nNKfdCV030exLNW05rKiGjrRq4NoiYEl0RM0grcypfjOZy05skyjDUxTTs8dlZIMqXUeDOXdtcJIjWR9yqw4PN+mblpO6U5L4aYsXNSbjbqZql7QmuNygVSyjkwRvxkg8nJs4xlvaCuKgY7UFYFh+MJa6SnZd91DMNAURR89tln4qsyA8fdFlNVNFpTVRVFWXrdR4g+05nMhzc2zGDY77Yc9zuatuX+/pb2JIS6qGvatvGcVwyLvhVcscEa+mEQ562xDF2HKXJsprF5gaksxlppj22ddAg+Hr2eJvFbrRXDzMKb6ExKalgneo6b/APMQoLB6PMKqQ26KL4uIL2XaP++nXUoNgnPG5lJPlnURHk7G7sKBBb/8j6ZWCowdUmcUxTHjo4TSpvoFlFsh+u1Bie9ubVSNE1D5qHU21NDUYRuuZIZUngHap7nLFaSbaq0ou8Ni8UiwpQvlkuf0p2R5wX73Y7Oh4CsNTx7/oLr568kZ0yNprkDqUxqLUPXcn93R9OcOB6lOMQO0kJHedSgtm3JskyI8XDAOsNgZMdL+EnT9y1dN7aQxjnv7pAGEf3QowgNvBTGGrKsGOdMpYW8yttZdmJvRcnkVaOg+Y8MIRGRCk8oclJqTYb11cEsJYhH76hKfWskAXJhmdLRQk9JYLyxynz5uwKnUWrw33oUwVmAbO5jic/RiiwdnH/haeUynigF91WmMyPPcuEuHhag8EB3YiGK78gOYBgoi4Le9AI1VVRYqwhhkrZtWdYluZPelA7D7e1HmVCluLhY45xYiWVf43VpjJXuwYICJG0Jj8eDIFS3HW3Txo1iuo7jYYsxA6dTw8PDPYfDDq0zBp+5myl88yuH8cAtxhhMP9B1LfUik9BT29A3TSzoLYqCQXmDRo2iLDzb+6N8LBUBx2PUvUJE0PrMjMC9wvIF6ZP5TSXK/ZQeLGMFWvJF4jtQI5HNyAnOEJko/BJOEU4TKlymTevPhZPmv0flcHJ+2CVyRtgF2jsZwzlSOV5EKy/PC5QX17I4HZ1pWS6X5IVMUVUt6LoG5wx9P7BaLSiKjLY5YbtBHKk2KNagdc7zFzdYaySdRiuqaikcxPWYQZqc9l1H355om5amldbNpu942G/puxbtHB9v31EUJU3TYEwffWh5XtD1Q/Aa4nyYqG0bdJ6j0BRFjnOGrrPs9wdpcpbnDMZQLxaYZojoR37WHhFL1EXma+OIoMjJxMd5njvk01qOqE6hJtelngEcETXokU6WFiKknweymHwe6TE8+HHxZ6rfhahC9P6nFuzsBcc0msSBqMUyK+vKZ2Es/LkiFoZh4Hg4kOca56SUTKHo+5ZTs+X27o7nz19we/eRqytpqtq1Ldv9lt3unsViyXK5pioXHI8n6vpItX2g71oWyw5rxL1gnaVrWo6HPcfT0ddP7mjbA19//XP2+61EHbDsDlvKYkGRFxwPB8DRtkcGU5CpgsFI92DnJH2pbzuUOuGsIysynJbwldKK9WbNYb/HOUdRlZitB/KzAiwzEpishYs72YtOiGtkgkhUkrsf0rGt8TpyoCq/9lrpGEmIDe9tYATTDI40RAlJtVIUZ1ZjNeiUSPzgXOKCcASFPdQhz8zelESTHRSsjriJVBoHnVwUrd5QpZTirIYWMFYPOGdojgfJO9M6wkc5Z6nrBUPf8O03X6OznNevv6KqJPFP5xrbdrx9+8ZzuBXmqsOuLtjvdxSlpPjUdc397R3ff/+Gqq64uBC3hOl77u8+cDwcuL+/ZX/c8v79G7ruJK1rEIeqXThaEJdFnmPsIBhmmXDiYRCv/TBY+sFQOCdgxdbiBoPKJH37/tTSto3g2TqJ6RoTeaFHDNfR7xj2sMCfK088YT29cTi3Sj1CQFDbwrqFXqLES0b6GIZh6kWwI0dzzo0dScZD/EEq4TbS4Iq4S7R3so1iTuR+GrkPx5yInwRnSYjTWhtjg4ELBujN1WoTuVjww/SdQGWuqhrtu4OE3P48z/jlN1/z7u171qs1h9MRlUnLHJRFO9jfP2CBrjtKxGAYMEo4Tp6XPtPB8u23v6QsS549e0ldS3Ow7f0tD/d3nE4H7h7uOOz29EPDMPTRUYxzHE8Hqqqi78Xo2O124KBciPO378VAKMvSB80zhr7HlUJ4+8Oe4/EoE5Up8lISMNEZDJKxq9Bjy6C5GhIozo1xSqle8MZYYmGm6zH6yZJqdESPdHZMuz7XYjz8/mSOvw3mrIeknLs1VLRm0nJ4dXagE0JidFeMg5kSZXp2+Lvve1bLDafTifXFhqZpKPOCssjoMYClaVv0oDHWsLnYcH1zydt33/H9999y/eyG3XbHZrPGOoNSjv12i8XQ9SIKu1646ul0ZLd/4OOH91hrOJ4O4CRjt+97vnv9NZvNpaA62oGH+zu6rqNrjlJNPvR0doiTH5TtsixjaKiqKu+E7ajyGrxDV5zAhedUjlNzoh0G30JRCppRmmpRe67uCcovuohJO5lPHVwSPgN1DAPJEbwcc33aU2dUoazPssVzRq3URLJEqzRIIGfI0NMucXLypM5lItfD5wK2m4MyOBfUu9F8TY9JBCHcM7g+xOyJ7FdcB36CnFQVye/yAm3fcXWxIDQc1Sj22x3WDgymR1pWi2iRziM9b99+z3q9Ybu7Y7/fcTzsyPKMuipomoN/d7EujYNTc2AYOpr26EFQPMYrTmCwnEVhGdqj73oyeMeoQdkEuz4Uz/gkg2EYIoGlDRm6vkGhqCspLGm7jrquxc0ClFWBVRJGW9RLmuaE1orVciUWvg5QEsR1JOFgo6Iv5rFkbPh5tWJtat+2O9KVcyLyAtPzPkrtjYigOkke2tPxHgdYNROXMYnN+/e1ClEAX4AQsIW8LFcJ1aYWzORBiesiOurGLO5EJ9CR0MYvfXhNKd9yz7E/HLnYaGkkUZaUdUHXWDBSwaS05uLyksViwffvBLZ86ASKabO5wjnY7ba0zX4U/fFQo2thGHBNurPDFnMe+txKMa0b431hUsOGxHlHpd/AwzAImMrge2nmgtajnODXSi6cJCQWuRS3OAVlLbUGImIztMpB+bRxI6qNsUOsaEqXYBRjEue0niOLH1JgHlLEzNATJDDD0F4w81VPwpntyCH9mwf1yvqXd85RaFEVzhb3aseMOkcLLgg85wzYDKVDhcr0OBtc9btMnMl+wR6dH1VGFEEHlDGYvqd3jtPBSeFtUdB4fH4BJcl58fwZL1684OPte95+/x1de8IaEU99d+J4OrBeX7Dfd964GBXY8GhjTLTGIGXOKu4x00+La5RSArfOqDiP3oFRd1Fa0/dNvCZwrcNxx3KxIq8qDs0J5xQXaoNTiv5hB8ButxMu/UyyOpTrcVZqHrRKE+cf85a5L8skhAKp159oMYbrQhc7zSgenQvtFGVe7JxiPOFansAni/mW6UyRQBQQiGIUhWPFt4urEl4s6CbRYAhca7zbuWHgCLDuEmBeLBR5pkT/6Toe7rYoJQXEVVVxc3PDer3hcNzz+vU3viOckUZXStH51J++v0MpF+GmwljHkIqbTJoeMZPiqK1yk/kI40Ux1kF47hWgnowxwrm887PrOyqko/Cp63BtxmUlqd+mN5Jl4hzWuyMWiwVa51xdXvCwvZsSzkyvTY2t+VoIs5FVTgt8nRsXPFwf3slZN0mrJuF+lsdAxlGHd4+/e7zQqTHAzM8VFYHRCkxR18JOUXh2m9zp3G5Lj7SeQHBiM/bHhs6nuGgVsCpyikzz/PlzFqslRVXy+rvv0JnUVpqhQynHMHTgDOvNkizkh50R7fEzR0QfjM5NraZczf8TutYG8Yl1qLAgRS4N5xHYzOA5D07Uru+wTvo9te2Rh4d7uqFHlZIa1HatwBEYy6pecHmxZrPZsNvvGUKjW2/yWWP9z8iV03dyzid3hjI5CxmZdPe2/kVgkvOvGN1OwdqPnNk7ZPXsObJ+IwPS52KUjw8Xf6Ll4n0x1uNNSNB41FvCAOOOcONncRCPnWPTBbcW54w008pyqYHMq4ly23U9KhNguqurK/b7LXkmBRf397e+PmBsNaMcUXeZlIAlOqXym2QcO37nhk3nCO6coEcqpSS5EfFFheul4FXjjOBcODumC4naIWAxINZn15+4f7gFrCRlOlHOV+sF6/Way6srrLW0XTPOl0rXZZQyzuIJTsUf5dHsJOFZxXLAOOfnLMzkkHUTdd8GI+5XHDpeOLtRuNn4XQC8m+JlRBC85B7nOMTZ41cQeHh2UZSJjqQEn8v7wayz6Dxjvd7Qtid2u3sOh3vevPkGZoq5UpLmEyp8Ju/uHj/3iVE9+i3kujkHBjA6EZmAweEyja4KBqSrnvHNHAJnCFZjnmu6/sT7D2/QueLZs2cUi4pyuWBxsWaxWHB/f8/pdPB6pPMWb+A00znXITnPje8l8+E3gE7dUlMQlrnP89w6nuNe8/N0+uHcnTElUplIpZikdkCqqofPpkrxtDD0Uzslfdnxvjqk3RSF37E+CdBjiD17dsN6veT1t7/ku+9ec3d7K7j6g+8WMgwM1sS6gLOTlIzJJsUx+J/5xClvWYnPbXJHjP87Q4FOuKhX9FOuGQqScU70NZUJfGfvePP2Daf2xOXFJRebDZnS7Pd7bu9u6fvOtzSUZRQ9yUsclY4mmGp4w8SXF44yBZ1nolfpqd4dUsnjXJxZw/lannPm5pOTgt7uhzfZs1HGzxfBL5ArgMFjLWTxBYPITAeUfiafh6kIdsN4HghUkzHS/AEUTTOQ59Lq78WLV7x48ZJvv/2Gd+/eYMxocaXvVpdVTNd+lEgZxuTHpXRi7s84dHBNjAGRaeImDpSR6wYc+CYXDgnuD2YAX9GjtEKFOlLjQFm6U0NZV6jMcTzs+OoXP2Vzfc1nr37AcrEmz3LJ7GjbOI5QPR/aM4eEgaByhPx750SnDrHFsKGDby8YAUqpMe/fGwMpQ5nYg7O1RIWsmnHNn0S/9pd783i6NSTa7+LCqDDvEwTa4KSdWjvnFjgxSJMjBGwH2vbIarVhGHy1UpaxXqz4/Nc+58Xz5xyOW15/+zXGdhLATvCzwq4L3Xmf2nmPiOmTIjMd5uw8NQLxBnz8kHbdNI3sbC0GwEioI5fQmcRks6JAKuRb7u/e83B3z3p9wc3Nc5qTOKDHAXt9y6+t85p66qOM6tvoZo8iNFqCSmGHscAl3n68eHKPcQrG9XWJGyS8UwwrqUCBo96OKLVjUnZw4IVqbe0VQOVz3zOVCbdxEhZB20RZHh86Z7XnxehoLGgUz66usUp7/LCSly9ecX1zye7hjp/+9M9p2iN2MNgZF5tYOVo/Qo5Ox4Wa+oHi50yJMA2djdkKYS5kGQYcuVMUWcaAnbRmthA97LJBBTlJa/E3SYmbIysrHBpjeqzpMNtOIKb6o+dSwfzVPp1aQnzBIInz7adfnO0eXCWxPqPbxhMYQGhqrj0IzrljHkIEJj7WQEp5aAjlPPmlojLlMKluL/9qr6N5i8ULfUn/CBM6srewiIqpbyYw4GARzRmDtVLZ07cDl8+fsVzWlHlNvVjw7etvef36LzkethJMDpZgckRfzoy7hX8nnPUJO2QSmmFqEEm4iWT3y+dGQeYcWZ4lld7MGumGe4zzgJKGrcoKBKizjsVqzWAFdfFweEBlo9XrjWyi+1dJXcTEpaASdICEe8ZNrBRE+M5kUyk+SWBnK5tcQmhKKCk/nk5c+HL+cL9MjUSR3FW+tkwJIjQYcBZrBGbKKcETk401uuqiqqnSyVWM8UsIYhLCLrNgBw6nPepO050EN/bu7iMPu1uG/pSgND6ekLkYTI/UIAmOZOHACRmMdEXgEtGHl/aV8Up1ODNTYkG2fSfEPL6dF2nEs128N15fk3sEmKfmGDI4BixGqr5DKMuOye9WhVqI8d2VUpErhdlWWlwqcV7sjPMpFR3o8aJw7kRXOz+v8XPg1HbkbTdMb5hO+hPHxHUBkaOl3FBeViZOB9EQCPWMCEr/nj/fDgNNe6BpD2gPu2TsAMpghl7EN0nG5r+qPjUbw1wXUeF/bnp+fI5XiAOXSLmzvHe6UCp+HgnKz4UKYmN86iP+cTqdgFm1djSQxhT2MH+Td1PEtJw0EP4py/Dc4VIjINHnHh3RioLddo/6a3/td9z7d9+B7/wBoAKYmkoGHDhMsksmd0smSOkkVMGUnY4GqVT/BLl8ziR2Tgp1M6XJihxZJBtZdRC9+B7cqSiYT/ZcwZ1P9OS5KQOPYxxFUzw/6KR+XlJis4zvpJWPa0Yuo2JBTchscCGd3Y5pU+cWMxBZ+lkMwk+IIaxZoh5YEZc6jHc215NVDGOdcK4wx4nide5QgFEobVldvUT/0R/9kedAkm8UQiBeY/cPCH+HO0giY6pgBt+W0inRnXkRJ5wver7PHClxWGsZrPFB8BB77JOe3MGnF1b/X21Hzr3c8Wd+TpTxyeufG6/yxBmI2evkyvnFjVPqU5Ydo6XnnOf63veGwC/JfRN/5ExEhR8TfG3p3MkficEhYHeBRub3COeE93j0roE7RDr49KGUIssz/uv/5r9FdV3j/spv/joO6V4bM3oYHxQGdt7jL19GjqBtmG0/y1NO4bUNJIKQiV0wY+/nxJ3oCY9FqR5pC+vxW89NQuQqcz3s3POS1nqPRDqjjylwssn3UYebX5feXlYy3DvF309vpxMuGfTBxyhIT6sGKUGNitvj8+JcBO75ieNXbeD0yIqcP/1//inaGMvDw97HHT91ozF294j6k3PkJ2P0k83wZtXkH3+PqXh79ORgYifoNOO5oZJdPVrY9Pq5u+RTz5tfMx/nJz+b6XXhcEh1kFNMuM5kDPML45g//T5PjS2It5GLMoYxSRQdP+ZfRWDhub9K5xWJYPjw8Tb0Zs/5n/7e35PGBcE3FexsNR381Nj0u9CzdNBeVCrEBB1BctMGEzKA0aM052JPWYlaa5RT6CRxRJ6qQz+J8cMnJki+dhOdazIxyXNVYvafu96F7zzUeNRZA+efZZomdx45RphjRtGqJs9/rEvG+ROFLt7/3GHdY+swUlvgBwoPjXp+3ubqxKeel85dnuf8d//D/yjrNgyD2x/2vHr1kpurDcqNuPtxcsOYfF5TKmak6AQgA9Wf3f1Tl0XYP9mox50Z5Pw+WmuUR42xyscCtaZwAhI87x4bxzcnEjW+07miliA2Ytbn7JTUskw3iAo6mX9Fh4tl/UqldYvyb7rxglsDiI5apZQEvpWUqU3m3DlC1nuKpv6rvAL+JL87x039FP5++H4yN7PP5vOrMylmef3dW/7iL79juVyglVIs6gX/6Mf/mDzPI4TQ9GHTB04WUzLw4jnzh0s671ykjO87f4Hz8zLdRcGPEx2QZyYjXZQ50Qbl+9wx4WBndm/qFpiOL5mCZLHTBXTpZel4kudrYERzmeJUPFIZmKsuqdR5LFLDs0KbnLnxM5mfT6gKT/0uFxvINP/z3//fqBcLlPKyR2vNF198Qdv00/OTyfG3TNj2mHgorzvG0gIRyAAE/nJCaH5FRjcIj154TjRxssPETgc6cTjOOVpYnOm4plGAcJz77Nx9g0UZudjkmeO7PtWHaLqQ09oGZZ2EOhJ15ewmPKOrpeMLP3P3jMuUB/R7vPmAR5LsKc527rldZ7jfNvz2j/6NKCm0UgKAdn19zT/+P/+UrKwIzQykBxGoYAkqifAbM8SNqHyWpVYFipwAPRR3nxKnqbAHhUOjEWytwDVmI47IfCNxmshhFeJzyoA8gl2Jr11bb20mOz/VZwDxdH9CpXCM6M2oMen83CKLWZOEspwDZ3HKRE6DIjZOGPUxvKhTk7CWiGoHGnSehLvU+H3kkOPHca4DZPx0OmdqByp4qyLHns9Reo+nJIwi2QxOnPNWOerlgv/l7/8D1uvNONZQffLHf/zH/PjH/we///t/CDpjcAMOSV1On+McSWzMxgcmj37yJUczOdzrU9YdXlwBZL47riccK2nGzkbjPyrdYRhzEZmKr5C6HbqzzYcfcdfcWJ3zqQlP58Z7wcaXiCJrfDERm6HHUwhKu3HsWsX3BaK3fjo/0zjvfGxPGVBBDeCJ78M5n7JeRY+TphlKCTh/lgvMwt/5L/8rfv3X/0rcPEoplPWwe6EIdbfb87/+yT/g7/wX/zmff3ZD15xQaDJdxBePCruC0ecV9pYTX9n49sF9Sywp9Tj4RMV/KpY9aLsPgQQEv0EakeqxDF9uH6p0xiiD1Uwcneeq2s8eilGEzDrUTquyZ0cCY+nSsT1aQ+UzZp1wMT+HgUPLTDzmRIGxphJC+05vKrG2J/rfI72aR5+nx68yGs4TuWyGvKx4/+EDf/fv/vf8h3/rb1N5qNSU6wIC85jnBVdXF/xn/+l/wp/92U/44rd/h/cftuz30s9bdJuJ+hrmLnn440HFFw07yCu1cunTfhe5JnDHQCwu4S7nOao8ZpzMlP3PRcHcQIg3SM+ZjelJc16F/6mz8Fti1QqBqSc5yePFn4835RIp4YXKqLR24VPK/fz+c3fFOZ0unGeMYX888v5uyw9+4wv+5H//U/6jv/0fe2UfGL3MW1veAAAAB0lEQVRw/H+b2Lc3Rg46LgAAAABJRU5ErkJggg==\" y=\"-21.499943\"/>\n </g>\n <g id=\"text_1\">\n <!-- Train Image -->\n <defs>\n <path d=\"M -0.296875 72.90625 \nL 61.375 72.90625 \nL 61.375 64.59375 \nL 35.5 64.59375 \nL 35.5 0 \nL 25.59375 0 \nL 25.59375 64.59375 \nL -0.296875 64.59375 \nz\n\" id=\"DejaVuSans-84\"/>\n <path d=\"M 41.109375 46.296875 \nQ 39.59375 47.171875 37.8125 47.578125 \nQ 36.03125 48 33.890625 48 \nQ 26.265625 48 22.1875 43.046875 \nQ 18.109375 38.09375 18.109375 28.8125 \nL 18.109375 0 \nL 9.078125 0 \nL 9.078125 54.6875 \nL 18.109375 54.6875 \nL 18.109375 46.1875 \nQ 20.953125 51.171875 25.484375 53.578125 \nQ 30.03125 56 36.53125 56 \nQ 37.453125 56 38.578125 55.875 \nQ 39.703125 55.765625 41.0625 55.515625 \nz\n\" id=\"DejaVuSans-114\"/>\n <path d=\"M 34.28125 27.484375 \nQ 23.390625 27.484375 19.1875 25 \nQ 14.984375 22.515625 14.984375 16.5 \nQ 14.984375 11.71875 18.140625 8.90625 \nQ 21.296875 6.109375 26.703125 6.109375 \nQ 34.1875 6.109375 38.703125 11.40625 \nQ 43.21875 16.703125 43.21875 25.484375 \nL 43.21875 27.484375 \nz\nM 52.203125 31.203125 \nL 52.203125 0 \nL 43.21875 0 \nL 43.21875 8.296875 \nQ 40.140625 3.328125 35.546875 0.953125 \nQ 30.953125 -1.421875 24.3125 -1.421875 \nQ 15.921875 -1.421875 10.953125 3.296875 \nQ 6 8.015625 6 15.921875 \nQ 6 25.140625 12.171875 29.828125 \nQ 18.359375 34.515625 30.609375 34.515625 \nL 43.21875 34.515625 \nL 43.21875 35.40625 \nQ 43.21875 41.609375 39.140625 45 \nQ 35.0625 48.390625 27.6875 48.390625 \nQ 23 48.390625 18.546875 47.265625 \nQ 14.109375 46.140625 10.015625 43.890625 \nL 10.015625 52.203125 \nQ 14.9375 54.109375 19.578125 55.046875 \nQ 24.21875 56 28.609375 56 \nQ 40.484375 56 46.34375 49.84375 \nQ 52.203125 43.703125 52.203125 31.203125 \nz\n\" id=\"DejaVuSans-97\"/>\n <path d=\"M 9.421875 54.6875 \nL 18.40625 54.6875 \nL 18.40625 0 \nL 9.421875 0 \nz\nM 9.421875 75.984375 \nL 18.40625 75.984375 \nL 18.40625 64.59375 \nL 9.421875 64.59375 \nz\n\" id=\"DejaVuSans-105\"/>\n <path d=\"M 54.890625 33.015625 \nL 54.890625 0 \nL 45.90625 0 \nL 45.90625 32.71875 \nQ 45.90625 40.484375 42.875 44.328125 \nQ 39.84375 48.1875 33.796875 48.1875 \nQ 26.515625 48.1875 22.3125 43.546875 \nQ 18.109375 38.921875 18.109375 30.90625 \nL 18.109375 0 \nL 9.078125 0 \nL 9.078125 54.6875 \nL 18.109375 54.6875 \nL 18.109375 46.1875 \nQ 21.34375 51.125 25.703125 53.5625 \nQ 30.078125 56 35.796875 56 \nQ 45.21875 56 50.046875 50.171875 \nQ 54.890625 44.34375 54.890625 33.015625 \nz\n\" id=\"DejaVuSans-110\"/>\n <path id=\"DejaVuSans-32\"/>\n <path d=\"M 9.8125 72.90625 \nL 19.671875 72.90625 \nL 19.671875 0 \nL 9.8125 0 \nz\n\" id=\"DejaVuSans-73\"/>\n <path d=\"M 52 44.1875 \nQ 55.375 50.25 60.0625 53.125 \nQ 64.75 56 71.09375 56 \nQ 79.640625 56 84.28125 50.015625 \nQ 88.921875 44.046875 88.921875 33.015625 \nL 88.921875 0 \nL 79.890625 0 \nL 79.890625 32.71875 \nQ 79.890625 40.578125 77.09375 44.375 \nQ 74.3125 48.1875 68.609375 48.1875 \nQ 61.625 48.1875 57.5625 43.546875 \nQ 53.515625 38.921875 53.515625 30.90625 \nL 53.515625 0 \nL 44.484375 0 \nL 44.484375 32.71875 \nQ 44.484375 40.625 41.703125 44.40625 \nQ 38.921875 48.1875 33.109375 48.1875 \nQ 26.21875 48.1875 22.15625 43.53125 \nQ 18.109375 38.875 18.109375 30.90625 \nL 18.109375 0 \nL 9.078125 0 \nL 9.078125 54.6875 \nL 18.109375 54.6875 \nL 18.109375 46.1875 \nQ 21.1875 51.21875 25.484375 53.609375 \nQ 29.78125 56 35.6875 56 \nQ 41.65625 56 45.828125 52.96875 \nQ 50 49.953125 52 44.1875 \nz\n\" id=\"DejaVuSans-109\"/>\n <path d=\"M 45.40625 27.984375 \nQ 45.40625 37.75 41.375 43.109375 \nQ 37.359375 48.484375 30.078125 48.484375 \nQ 22.859375 48.484375 18.828125 43.109375 \nQ 14.796875 37.75 14.796875 27.984375 \nQ 14.796875 18.265625 18.828125 12.890625 \nQ 22.859375 7.515625 30.078125 7.515625 \nQ 37.359375 7.515625 41.375 12.890625 \nQ 45.40625 18.265625 45.40625 27.984375 \nz\nM 54.390625 6.78125 \nQ 54.390625 -7.171875 48.1875 -13.984375 \nQ 42 -20.796875 29.203125 -20.796875 \nQ 24.46875 -20.796875 20.265625 -20.09375 \nQ 16.0625 -19.390625 12.109375 -17.921875 \nL 12.109375 -9.1875 \nQ 16.0625 -11.328125 19.921875 -12.34375 \nQ 23.78125 -13.375 27.78125 -13.375 \nQ 36.625 -13.375 41.015625 -8.765625 \nQ 45.40625 -4.15625 45.40625 5.171875 \nL 45.40625 9.625 \nQ 42.625 4.78125 38.28125 2.390625 \nQ 33.9375 0 27.875 0 \nQ 17.828125 0 11.671875 7.65625 \nQ 5.515625 15.328125 5.515625 27.984375 \nQ 5.515625 40.671875 11.671875 48.328125 \nQ 17.828125 56 27.875 56 \nQ 33.9375 56 38.28125 53.609375 \nQ 42.625 51.21875 45.40625 46.390625 \nL 45.40625 54.6875 \nL 54.390625 54.6875 \nz\n\" id=\"DejaVuSans-103\"/>\n <path d=\"M 56.203125 29.59375 \nL 56.203125 25.203125 \nL 14.890625 25.203125 \nQ 15.484375 15.921875 20.484375 11.0625 \nQ 25.484375 6.203125 34.421875 6.203125 \nQ 39.59375 6.203125 44.453125 7.46875 \nQ 49.3125 8.734375 54.109375 11.28125 \nL 54.109375 2.78125 \nQ 49.265625 0.734375 44.1875 -0.34375 \nQ 39.109375 -1.421875 33.890625 -1.421875 \nQ 20.796875 -1.421875 13.15625 6.1875 \nQ 5.515625 13.8125 5.515625 26.8125 \nQ 5.515625 40.234375 12.765625 48.109375 \nQ 20.015625 56 32.328125 56 \nQ 43.359375 56 49.78125 48.890625 \nQ 56.203125 41.796875 56.203125 29.59375 \nz\nM 47.21875 32.234375 \nQ 47.125 39.59375 43.09375 43.984375 \nQ 39.0625 48.390625 32.421875 48.390625 \nQ 24.90625 48.390625 20.390625 44.140625 \nQ 15.875 39.890625 15.1875 32.171875 \nz\n\" id=\"DejaVuSans-101\"/>\n </defs>\n <g transform=\"translate(48.199347 16.318125)scale(0.12 -0.12)\">\n <use xlink:href=\"#DejaVuSans-84\"/>\n <use x=\"46.333984\" xlink:href=\"#DejaVuSans-114\"/>\n <use x=\"87.447266\" xlink:href=\"#DejaVuSans-97\"/>\n <use x=\"148.726562\" xlink:href=\"#DejaVuSans-105\"/>\n <use x=\"176.509766\" xlink:href=\"#DejaVuSans-110\"/>\n <use x=\"239.888672\" xlink:href=\"#DejaVuSans-32\"/>\n <use x=\"271.675781\" xlink:href=\"#DejaVuSans-73\"/>\n <use x=\"301.167969\" xlink:href=\"#DejaVuSans-109\"/>\n <use x=\"398.580078\" xlink:href=\"#DejaVuSans-97\"/>\n <use x=\"459.859375\" xlink:href=\"#DejaVuSans-103\"/>\n <use x=\"523.335938\" xlink:href=\"#DejaVuSans-101\"/>\n </g>\n </g>\n </g>\n <g id=\"axes_2\">\n <g clip-path=\"url(#pf02e2d733d)\">\n <image height=\"153\" id=\"imageb081ed1ee7\" transform=\"scale(1 -1)translate(0 -153)\" width=\"153\" x=\"189.818182\" xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAJkAAACZCAYAAAA8XJi6AAAABHNCSVQICAgIfAhkiAAADEVJREFUeJzt3V9MW2UfB/Bv2xUKsm44Fkt0Gpdtzo3ojM4MnasJ4mYUsmxqvMBsmYlx80+miboLY7hxGnahhkVNVDSMhKibYEC2MmAONkWQbR3IYIVRkilsDAoUO1tOe96LveN9m7bQQp/zPE/5fRKS9ZyTnm/gu/Ocnp4/usLCQhUAkpOTsW/fPoTz2Wef4dVXXw07j8xOTk4O6urqQqZXVlbCbrdzSMSOnncAEmzr1q1YuXIl7xhxNVWypKQknjlIAtMDgNlsxltvvRV2Aa/XC7fbrWmo+WBychLj4+O8Y2hCn56ejjfffDPszOvXr+Pzzz+PuK9GZq+xsREvvfRS2HlpaWkwGAwaJ2JH/8Ybb4Sd4fF48O2330YsIGEnPz8ft99+O+8YcRN2x9/j8aC0tBR79uzROg9JQGFL1tPTg927d2udhSQoOoQhqOXLl8NkMvGOERdUMkFZrVYsWrSId4y4CCnZ6OgoSkpKeGQhCSqkZCMjI/j00095ZJl32tvbceTIEd4xmAsqmdvtxnvvvccry7zT3d2No0eP8o7BXFDJPB4PysvLeWUhCYp2/AlzVDLOqqqqcPDgQd4xmKKScXb16lU4nU7eMZiikhHmqGQC27lzJxYvXsw7xpxRyQRmMpmg0+l4x5gzKhlhjkpGmAsqmd/v55VjXgsEAlBVlXcMZqZKNjY2llBnY8rk448/xgcffBB23oIFCzROE380XApuz549SEtL4x1jTqhkhDkqGWGOSiaBjIwMqY+X6QFAVVU0NzfzzjKv9ff3Y2BgIOy8HTt2SH2Fvx4AfD4ftmzZwjvLvPbVV1+hoqKCdwwmaLiUxJo1a6QdMqlkAmlsbMTFixfDzsvPz4deL+efS87UCeq7775DS0sL7xhxRyWTSE5ODu8Is0Ilk0h2draU+2VUMsEUFxfjzJkz0y7z3HPPYfv27RolmjsqmWBaWlrw999/R5xfUFCANWvWICsrCy+88IKGyWaPSiaZ5cuXT/377rvv5pgkelQyiRmNRuzatYt3jBlRyQS0a9euqA5l6HQ6LFmyRINEc0MlE9DQ0BC8Xi/vGHFDJZNcamqq8LddpZIlANHP0KCSCcpqteL8+fNRLbto0SKh7/E7VbJEum98Ioj16iWRvwnQAzce3jUyMgKTyTT1I/ommATT6XTCbiimtmRmsxnXr1+f+jl58iQWL16M1NRUnvnmtfHxcQQCgaiWzcjIwM6dO9kGmqWI+2QbNmyAy+XCoUOHYDabtcxE/uuxxx5DX18f7xhzNuOVo9u2bYPH40FhYeHUtMHBQfzzzz8sc5EEEtXlyQUFBSgoKJh6/c477+D48eNRrUBRFHR0dMwuHUFnZyfuuusuqa8k16mMb8IwMTGBp556Cv/++y/++OMPlqtKWAMDA7BYLDMud/nyZXz99dcaJIqN3uFwwOFwoLe3l8kK0tLS0NTUhJ9++glPP/00NmzYwGQ9iayurg6Kosy4XEpKipD3M9HdfAa5wWBAfn4+gBthWT2iuKurC7t378Yvv/zC5P0T1ejoaFSPwent7UVZWZkGiaI39enS7/ejoqICFRUVqKqqwoULF5iscPXq1SguLsYTTzzB5P2JeMIewnC73bDZbGhsbERXV1fcV5qVlYUDBw5g8+bNcX9vIh7D448/XhhuhtfrhdPpxODgIEZHR6GqKm699da4rdhisSArKwtOp5PZ/mAimZiYwJNPPjnjtZculyvq7zy1MuPn4uHhYfz222+4dOkSOjs7Y3pzvV6PZ555JuL8devW4dFHH4XNZovpfeejgwcP4sCBA1Ieyog68ZUrV3DlypWY3lyn00FRFGzdujXiMs8++yxaWlpQXV0d03sTeUQcLuNlaGgI/f396Ovrw+rVq0PmL126FOvXr4fD4UBPTw/LKNL79ddf8eKLL057xkVKSgoMBgP6+/s1TDY95tveQCCAvr4+GAwGXLt2DRaLJWQIXblyJTIzM1lHkd6JEydmPAUoJSUFS5cu1ShRdDQ7adHv9+Ovv/7CuXPnwg6N+/fvR25urlZxpLV27VreEWKm+Zmxfr8fbrc7ZLrFYpH+Brxa6O7unnGZe+65R6j7ZnA5/bqnpwc1NTU8Vj0vGI1GZGdnw2q18o4CgFPJAoEA2traUFtbGzS9vLxcmF+MyKI5v89gMGDTpk145JFHNEg0PW4XkgQCgZAnoCQnJ0t7ozctRXsun16vF+L3yTVBuE9KIvxSSHxx/Yu2trbixIkTQdPq6urw0EMPcUqUeER4ZpN831GQqKiqilOnTuH06dO8o4hZsoULF0Kv10d9pc58E+lpvqqqTt1Dw263o6GhQctYEXEvmc/nw+TkJIxG49S0hoYGWK1WnDp1iooWxvDwcNBrVVUxMTGBgYEBlJeXc0oVGfeSNTc3w2w2Izs7O2j6yZMnsWzZMly+fJlTMnmMj4/jk08+4R0jIvooR5gTomQulwsTExO8YxBGhChZa2srHA5HyPTc3NygfTUC5OXl8Y4QMyFKFklJSQkWLlzIO4ZQZHzIlzAl6+vrg8vlCpn+8ssvS3nKMfkfYUrW3t6OwcHBkOkffvghkpOTOSQi8SJMyQDg3LlzGBkZ4R1DWEVFRVJ+tytU4osXL2JsbIx3DGG9/vrrIef3+3y+kFOmREM7OxJTFAWHDx8O+8lcJEJtyQCgtrY25GuTmpoaYW9VyUsgEEBZWZnwBQMELNng4GDIgxI2bdok9I13eVBVVajL3qYjXMlI4qGSEeaELFlpaSkdykggQpbM6/WGnEc2NjZG+2WSErJk4dDzBOQlTcmIvKhkhDkqGWGOSkaYo5IR5qhkhDkqGWGOSkaYo5IR5qhkhDkqGWGOSkaYo5IR5qhkhDkqGWGOSkaYo5IR5qhkhDkqGWGOSkaYo5JJymAw4O233+YdIypUMonJcnNAKhlhjkomkcnJSd4RZoVKJhGz2Rzy+EYZUMkIc1QywhyVjDBHJSPMUckIc1QywhyVjDBHJSPMUckIc1QywpwcX+PP0d69e/Hggw/yjhGipaUFxcXFvGMwl/Al27t3L959911YLBbeUULk5OQgNzcXFRUV+Oabb3jHYSahS/baa69h3759uO2223hHCSszMxN5eXl44IEHoCgKDh06xDsSEwm7T/bKK6/g/fffF7Zg/++OO+5AUVERnn/+ed5RmEjILdmOHTuwf/9+pKen844SNYvFgiVLlvCOwURClWzLli0oKSnBLbfcArPZzDtOzD766CNcvXoVR44c4R0lroQdLr/88ku43e6ol9+4cSN++OEHZGZmSlkw4MZJiaWlpdi8eTPvKHEl7JbM5/NBVdWoll23bh3q6+uRlJQUcZmysjI4nc44pYuPe++9F9u3bw+alpqaiqqqKvj9fqxfvx4dHR1T81wul5QPlxW2ZLHQ6/XTFuz7779Hb2+vhomi09HRgaSkJOTl5QVNNxqNMBqNIQ+1N5lMWsaLG2GHy2itWLECbW1tEedXVlbiwoULGiaKzXRba7vdjhUrVkj/dDzpS5acnBx2uqqqOHbsGOx2u8aJYnP27FnYbLaIZXM4HAgEAsjIyAiZJ8tFJVIPl6tWrQraZwFuPABeURQ0NTXh999/55QsNs3NzUhKSoLVag0ZIm8aGhoKeu33+1FUVKRFvDmTtmTp6eno7u4Omd7e3o7KykoOieamsbERRqMR2dnZUu7cT0eq4fLOO+8EAOh0OixbtixkvqIo8Hg8WseKm/r6erS2tkJRFN5R4kqqLZnT6cTatWthMpnC7uz39vaitraWQ7L4sdlsWLBgAe6//34YjcaIy127dk3DVHMjVckA4M8//ww73ev1Ynh4WOM0bPz888/Q6XS47777whZNVVV88cUXHJLNjtDDZX9/f8gD78Px+Xxoa2vD8ePHNUiljerqapw/fz7s0Hnp0iUOiWZP6JL9+OOP8Hq9My43PDycUAW7qbq6GmfOnEFnZ2fQf7aysjKOqWIn/HBpt9vx8MMPR/xo7/V60dXVpXEq7Rw9ehTAjS//p9tHE5nwJbPZbJicnMTGjRtDjnz7fD40NTXh9OnTnNJp59ixY7wjzJrQw+VNDQ0NIUfEFUVBfX39vCiY7ITfkt1UU1MT9DoQCODs2bOc0pBYSFOy6b4EJ2KTYrgkcqOSEeaoZIQ5KhlhjkpGmKOSEeaoZIQ5Khlh7j+IobnQcdL/mQAAAABJRU5ErkJggg==\" y=\"-21.499943\"/>\n </g>\n <g id=\"text_2\">\n <!-- Label -->\n <defs>\n <path d=\"M 9.8125 72.90625 \nL 19.671875 72.90625 \nL 19.671875 8.296875 \nL 55.171875 8.296875 \nL 55.171875 0 \nL 9.8125 0 \nz\n\" id=\"DejaVuSans-76\"/>\n <path d=\"M 48.6875 27.296875 \nQ 48.6875 37.203125 44.609375 42.84375 \nQ 40.53125 48.484375 33.40625 48.484375 \nQ 26.265625 48.484375 22.1875 42.84375 \nQ 18.109375 37.203125 18.109375 27.296875 \nQ 18.109375 17.390625 22.1875 11.75 \nQ 26.265625 6.109375 33.40625 6.109375 \nQ 40.53125 6.109375 44.609375 11.75 \nQ 48.6875 17.390625 48.6875 27.296875 \nz\nM 18.109375 46.390625 \nQ 20.953125 51.265625 25.265625 53.625 \nQ 29.59375 56 35.59375 56 \nQ 45.5625 56 51.78125 48.09375 \nQ 58.015625 40.1875 58.015625 27.296875 \nQ 58.015625 14.40625 51.78125 6.484375 \nQ 45.5625 -1.421875 35.59375 -1.421875 \nQ 29.59375 -1.421875 25.265625 0.953125 \nQ 20.953125 3.328125 18.109375 8.203125 \nL 18.109375 0 \nL 9.078125 0 \nL 9.078125 75.984375 \nL 18.109375 75.984375 \nz\n\" id=\"DejaVuSans-98\"/>\n <path d=\"M 9.421875 75.984375 \nL 18.40625 75.984375 \nL 18.40625 0 \nL 9.421875 0 \nz\n\" id=\"DejaVuSans-108\"/>\n </defs>\n <g transform=\"translate(249.721278 16.318125)scale(0.12 -0.12)\">\n <use xlink:href=\"#DejaVuSans-76\"/>\n <use x=\"55.712891\" xlink:href=\"#DejaVuSans-97\"/>\n <use x=\"116.992188\" xlink:href=\"#DejaVuSans-98\"/>\n <use x=\"180.46875\" xlink:href=\"#DejaVuSans-101\"/>\n <use x=\"241.992188\" xlink:href=\"#DejaVuSans-108\"/>\n </g>\n </g>\n </g>\n </g>\n <defs>\n <clipPath id=\"p58ad9a7e6d\">\n <rect height=\"152.181818\" width=\"152.181818\" x=\"7.2\" y=\"22.318125\"/>\n </clipPath>\n <clipPath id=\"pf02e2d733d\">\n <rect height=\"152.181818\" width=\"152.181818\" x=\"189.818182\" y=\"22.318125\"/>\n </clipPath>\n </defs>\n</svg>\n", "text/plain": [
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAC2CAYAAAB6fF5CAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOy9e9BtyVne93u7e1327bucObe5acRF6MJFAgwi3CxLBklIKiiqcLliBSjKuOxUkn+SGEyRyKbiFGW7SjjGKZKq2KGgHBcxRQVfgGAnNgYHy8SSHYONGMFIMxpp5sx3vtu+rrW6O39091pr729/55xBo3M0c/ZT9Z2z97r2Wmuvt99+3qffV7z37LDDDjvscH+gHnQDdthhhx0eJuyM7g477LDDfcTO6O6www473EfsjO4OO+yww33EzujusMMOO9xH7IzuDjvssMN9xM7o9iAivyQi3/eg27HDDl8oEJF/KiJ/+n7v+1rGq97oisi09+dEZNH7/qdezrG89+/13v/0H7Idz4jIH//D7LvDDvcDu9/oFwbMg27A5wrv/Th9FpFngD/tvf/Hm9uJiPHeN/ezbTvssMMOm3jVe7qXQUTeISLPicgPichngb8tIoci8g9E5JaIHMfPT/T2aYdDIvL9IvLrIvLX4rZ/ICLvvcdzf7+I/IaIfFhETkTk90XkG+PyZ0XkxT6NISLvE5GPishZXP8XN473vSLySRE5EpH/pu+xiIgSkR8WkU/E9T8nIldeiXu4w2sfd3snIr5ERD4Sf5//R//3JSLfICL/Iv7O/42IvOP+XsGrD69ZoxtxE7gCPAX8GcL1/u34/XXAAvjJO+z/duB3gavAXwH+FxGRezz324F/CzwC/B3g7wJfB3wp8EHgJ0Ukeekz4HuBA+B9wJ8Tke8CEJG3AP8j8KeAR4F94PHeef5z4LuAPwo8BhwDf/Me27jDDvfyTnwv8AOE318D/A8AIvI48A+B/47wnv1XwM+LyLX70vJXK7z3r5k/4Bngj8fP7wAqoLzD9m8Djnvf/ymBngD4fuDp3roh4IGb93Du7wd+r7fuK+O+N3rLjoC3XXKsnwA+HD//t8D/ttGOqneufw+8q7f+UaAGzIN+Hru/L6y//m/0Dttseyd+vPf9LfH3p4EfAn5mY/9fAb6vt++fftDX/YX296rndO+CW977ZfoiIkPgw8B7gMO4eCIi2ntvt+z/2fTBez+PTu54y3bb8ELv8yIeY3PZOLbr7cCPA18B5EAB/O9xu8eAZzfacdQ7zlPAL4iI6y2zwA3g0/fY1h0eUtzjO/Fsb5dPAhlh9PcU8D0i8oHe+gz4vz+/rX5147VOL2ymUPsvgTcCb/fe7wHfGpffK2Xw+cLfAX4ReNJ7vw/8FF2bPgP0eecBgbJIeBZ4r/f+oPdXeu93BneHe8G9vBNP9j6/jjCSeonw2/uZjd/eyHv/4/ej4a9WvNaN7iYmBA/zJAYDPvSA25MwAW5775ci8vXAf9xb9/eAD8RAXA78RdZfiJ8C/rKIPAUgItdE5DvvU7t3ePUhE5Ey/RG827u9Ex8UkbdEr/jHgL8XveCfJfw23y0iOh7zHVsCcTv08LAZ3Z8ABoRe+jeBX36wzWnxnwI/JiLnBA7359IK7/1vE4Jlf5fg9U6BF4FV3OSvE7zk/zPu/5uEIN4OO2zDPyIY2fR3wN3fiZ8B/lcC3VYC/wWA9/5Z4DuBHwFuETzf/5qHz668LEgkvHd4lSAqHk6AN3jv/+BBt2eHHXZ4edj1SK8CiMgHRGQoIiPgrwH/HyESvcMOO7zKsDO6rw58J/B8/HsD8Cf9boiyww6vSuzohR122GGH+4idp7vDDjvscB+xM7o77LDDDvcRd5yRlnhD5xzOWc7Pp/yjX/pl/swP/gCP3rxCtVwgKLTKsNbivUdEEIEwecsBChFFkJZ6RPUmTokggCB4FIIH8XgfDiBCWEY3y0GHoyBKAAVe8DQoD0opRHzv8Dp+cqSUCU6F88XrQ6n1fufS1AoS2ysCzvd3QJS0134BLt0TwffbdoHVEUSpcHVeUPEeeu9Jd8nj1vcQwcdF6fzee5SO7en1qUqpNE2z/T9tn9Bf3sel17axX79dIoK1FpMX3HrpJT784b/JH3/3d1CUZbu+vfAHgL/0l/7Sjlfb4fOKD33oQ1t/23c0us6FN/oXf/EXef75T/PP//mv8S9+459xcDBgtZyBV4gIzrnW2HrvCO+gRGMbPgf41tClY4fVwSBbAeWjMWD9bQxbdJsHYxONuO8MiPdu86VeNxgia/PU+gZl2z6bRkmQjXU+Xut227G2bX8T1Z2zPYWA93H44TxOYqfg14+X7ncfqfNIzwHAu+7amqZpt7sbj7/NiPbXbR5j814pCc9XoXBVxfWre/zlv/yjHB0d8Sf/1PdSlOVdDfkOO7xWcUd6QSmFUooPfOADvPOd7+Jf/svfAGcxYhByvO8ZDII9c61nFw7dM1lr3+CiQcP71i7d2bOi9YTBorVuDa0ohdIKUWrNo21P7bskP5vtcM7h4rrkuW823yXXUrr7s3kt29B57a1rSrp57aXG6/cEw6a1jsdNVjQY0nS9AKIuGvxwfevf19uyvZOQNES5ZH3aZtMj3jTAzllwDu8tHo9tFE3T8D//T3+dZ599pu1wd0HcHR5G3NHoJuNzfHzMO771m7HVqqUavLfRaYwvlw+0gNamfeG9KBBwvsbTgHRDYBFBvAGvo7PoERyOBpeOd7FBwWQJgUYQD2istdE79ngJ2V4aXDRVLpg6BU7CMVoD3XrH0YhodcfBrgA6ee8+HNmz3XgE8qPzOoMxU4iPHQTB4/bWdR5z6nQEvPi10UDoOAQcuKY3SvDd+rbz6Ba391prfYFK2Wy3w+MDYxMeyZZ71D/GZUbTA14JKAUieO9QXljOF7z/fe9hOj3fGdwdHlrcNZDmnOPpp5+mKLO15a2n1S7pUpd1HG7gT6XHYngfjEl46WzPeLYHBqQzRP1z9Yxl/3hpuJuG/mt2U6QdyveP1f+e9u/axQUDddmybceVSGEkj3P9nN21bjten5uNR+s+S8+Y+c7obTVgG53HpjfaN6jtcg9iPeJCB7CNQuhTG93zvkjPbDtvnmsO9kp+7+P/Abczujs8pLirp7tYLvhj73wHTdO0HmVaJ9FAAttfPO8SF7B23G5ozMWIUrIv/cPc4QXtG+FkRJxzkVq4GBjqe239HJf94112tuR9p2H45jW3x9ngYHu3KXp+HZ3R77za3frt6Z3fQehAlLT0Rhe8lHVel/UOapNO2RY0E6Lh3bi2zQ5l83lso2o2P4edNVjHd77v21kuFni/HhjcYYeHAXc0us45/q9/8k+4+sgBYdCegkl9+EhPrntdyWv0cbDber8+WCClhEQgrL+cvlU0BMHA5fwi9AweXaCq2z7uuyVo0zc6rbFq97o8kt9eAuue8cWGtU77JpO9ZhwT39wfzuNTK6RlwiON3TOYm51fL1jn4x3peaKXXf+2drcf7+K53g2b53U2BFlf9+RNfuPX/xnW7ozuDg8f7mh0rW34Tz74QWxT0xow51srsO7V0PucotjJQjm8S9ZAgU/qBR8CV3G/1kPbcHf7BnCbIXDOBQ60J6kKZ3XBc3O9hVvQHh/ZIuW6eN41j3fL/qmzIHYsPhrAZFz7krN1OiTxutLdYyLH2jPe4fydQV0zrukaXHf8bVA9brrXmGjZ6c6fnvcW9GmKO1EMfXjvaZqG/+zP/cC6gmWHHR4S3FEyprVif3+Mp07u3SUv1UVvajOC3r3JluTFbVITbVCIzj5e6nH21gdvdZtWNhj1taH7lv0TLlInd95n83o32OSLXqH3bDtsGtZD6qg2PPi00dqAwEeK42Lb7qQu6G3ZdTDpGFtsYJ9G2hLa3HreOyEcS3P1kSsYre+6/cOOr/3ar1377pzjox/96ANqzQ6vBO5odP/BP/iHWNsgyiHo1rtNPKUnvvtx+34Qqh8Io10f/M9Ov9tfF40IFx3STc52k2Ns27OFC03H7VvyTdnTZoR+s13t+bYE4da+c+GSL24XvdS1ZRKmMSTD3dIJ6ZhrpxE2LqbHg3f3Z1uQq/PIe+2Uvs292HF07XyFAl8C4hTiHb/6q7/Me977/lfmuK9RfMd3fMdawLVpGrIs4yMf+cgDbNUOnwvuSC986EMfihxj0BG5NPBPw1BIY+nem9xNUOjzjt4TKYZ17zAhGU6RqLG9xNHse29KKYzS5HmOMRlZFv601mitIWlYe5703Ya/24JI7d/mNu14/y4G10dPNhnERBcIQWfb3tKo5JXOww3n8uGP8MC0qHhcaY3k9o5I0EoF77l/78KXOKsvdJzO+o5K3jhG2iZdx4Vr9S0pfSmF04f3HttY/sqP//d33/ghxjvf+c4Lv1djDO9617v4pm/6pgfUqh0+V9zR0/2df//b3Lj2CM71HKNkENa4wLQsLdTRzvXHxAJiIQbVNofAF4bqauP7Fu8UQItQlEMg8JRN02BdA2LxVYWPw+JtHvLm8dPny9ZvXHK4Ni+XGlwIRqr14iVO8xVBicLHQJxqr63/gvVUAhdYivXOqv/9ggzNd/use9e0Iwvvehx8GDJcPFf6P7bdpZmH8Tip/d2iiyqHdD/SVO9P/N7Ht9+0HXj3u9/N13/91291EvI851u+5VvQWvNrv/ZrD6B19w/vec97yLIgV/37f//vP+DWvDK4o9EtcnMh2LFp/DYRlnURdqXCtFW1ZkQhGJ91A6IuCZSFfbZH4ZUxlMWI8XiPshyglOL4+IjT89t4ExQGztoLx7kX9K/T94xOaH00IMnG+c77Dl99t52sG1MR8M5tTKdNNEfnMAoSpnb0PXUu0ruDwYC6rsP9UAprbTfra+OaLly773ViyYBe0kndTbrnhVZ1cum2vT54snevhZUfPrz1rW+9oy68KAre9KY3vWaN7nvf+17G4zFvetOb2vvwWjG6d6QXhoNB9yW+7dYTvMd1uUJYrSC8VVFKJcloKZR2hFlnGXiFqLhc1lnEPu+Yps2G4TVIG5ZPBkGBMowGY/YODzl45ICr167zlW/9Gr7kS97MaHyA1marsU7nuqwDSUarz+fK5oyuzhb2eNrwrVNBBEvpoooBPNaD9kKR5YhSODrqRiQ8lJbKiPdBiQ8z5lRYonSgGMrhCOs9ymiMztCmm5osKp0RJPIZfY/eObfOEsCFvA6iJNI93b7OuR7h3CHNc7mM6ujIkLDroMgvPJMd4Lu/+7spiuKu2z3yyCN827d9231o0f3F+9//fr7ma76Gt7zlLWsdzwc/+MEH2KpXDndRL4TosiTvB2Kgx/cnedFz1uJnh5Is7hOHoXH6a7BUMZNYpBH6L+mm95eG2WnUu74taKPJCsN0es7JyRFa51y/doPHn3icvcmQ3/3d3+Z8eoKtmzh9uWt3OlbKodCf/HHBSG9zMel5zht0RDiGYl2nG+6g9uE+2sauHcv1vFxiUCu2Jtx372msQytFYfJAp7gVznuMyRgVe1T1FNu4rjnR+/Q+cETO+nVjGeVvfQ+936GGXBRuzZgm+ZtSCrd5Q+g8/k3ttiBhKnZ7O1+h4NxrDE899dQdvdyEPM9bdcOv/uqvfr6bdV/w/ve/n6/6qq/CmIum6Yu/+IsfQIteedz1ySaPKHzpr0npA1NgxbeelFIpKCYgGhGD89HASI2oaL43PNCLkrPLaIye14Xn6OSYW7ducXJyxgsvfJY/+MTTfObTzzMoR7zhDW+iLIYoo1G9RDHpHOl/ay2XYc3D633f5kGv0SBrFGrnGhrCULx2FmfX8ysoQFw/8U8MYtqQ/NJojcnzNsBmdEaRj9jfu8r1Rx+jKIZhnQocsojgvOsMuay3P50oTfToL+urIZTRnTGOs+K2GdzNe9uHk06VtjO3Lw9f/uVffkE+BoFmeOSRRx5Ai155vO997+OrvuqrWg53EyLCn/2zf/Y+t+qVx+eQxFza4FL6S15VyHHbGyKveYjr7uJW3rR/lq30a0wRqQ1FMWQ2m7NarVitVtTLFefnJ/z+Jz7BJz/5SfJswBOPP4VWOUqZmHN33eDn0Yhtnn+bUb1XPjhe3Gaz23kRNo4ctNZ4gbIsyfIM631UL3T3pw3kOR/aGikGYwoODq/xxjd9OW94w5u5cf1RysEEpXpeQhzze/GgaEcWrUHtdWCtYqK3bE0tYvSFe9fOHIw/gs17uBb8ZP15vqx7+RDj9a9/Pb/zO7/DRz/6Ud761rdeWP8lX/IlfPu3f/sDaNkrh3e/+9287W1vu9TgJly9evU+tejzh7slMe9ejLVRaTKqfm1Fisq3XlMysNIAOtqgBhGzJr9aMyxhYWuwfMtDrk8sSOahqmqsdRidIwhlaajrGmctt269QJYZnnjidUxnM27d+iy19+AanOvOuaxWTEZj5vP51kBbS3akdnmi4qC9HT3uN7UfPBZBp50Bj1fhGrUTnBKyLMNVFXVdY4yhUTHDmgNFyJqmdPBaszwPp7fCcDLhicdfx97+PuPxhCIfUFUVw9EeJ+fHUC/ARadUabwEmgBUq1aQSAGo3qjDRRVKmnVnjMFaG+gC182s69+dRD9tM6FrnVkaGfWe+Q53x6c+9SkgvI/PPvvshfXGGIbD4f1u1iuGd73rXXzd131dS2e+1nFXT3eblGpz6JyqQ3gP1rmoPU2eUDxO6xOpteP2h+x3jpB3uQR8rwNwzgUPt67DUBwVtLpKs1qtODq6zXQ654nHn+Sxx57g8MoVRGu0CakOjTEYpVksFpdee99jTdNng/EJf1spBkCJZv1yAp8LwdPF+dbz9t6zXC5bgxZoGtXKuELib4t1DVkmPHrjUQbFgNOzU87Oz7HeMR6PuXJ4hSzLI5UC4MI9jjkv+gazT1P7GOwLwbXWz8Y1FoVsLI9VOnrXu+0Zbhs59D/vPN2Xj+PjY974xjdeWP6VX/mV/IW/8Bf45m/+5gfQqj88vvVbv5Vv/MZvfGgMLtwjp7v5PXk73TpHqBjRy30gEpf5NaN1z8P1u7yQ6dx1XSE93Za1Nc45rG2C191YptNzimLAZHLAaHTAo4++jkSP9JOQDwYDsiy70NH0O5g7e2f+wqdgVMNkDw1oF+RiCRpBrMOtagyCisqGdoaaCGU5wDlP0zjybMC1q4/iGs/R0RH1YkU1X7A4m7JYLDg4OGAwGLXBPS+qVSX4nkGHpEKQGAD1bXtB1jzb/l/rEW+REm5iG3++M7SfOz7+8Y/zFV/xFWvLlFLkec473/lO3v72tz+glr08fMM3fAPveMc77hg0vHbtWnBGIrTW/Pk//+fvR/M+b7ij0b23F0Tav3Z6b9RSKR2NmqgYPOuYXs+6nCgtAy4Y6u48XbuCjEkjTvC2YblcUjerjgJAyPMMbxuaqubk5ITxeI/Geuq65uDgShh2+563KbQBtbSs37C0Db12B9agH6DqAlgp2JjoAgf4aAyTwfc4RCtMnnUz8SJBKiIURQlAVVXk2YCD/SuAYrlcIpE6mE0XTKdTTk9OUEpR5GV3v3z/uSTj5xEVnk+n84oZ4SSsc3iMMReChnfqdFJnLEigLLYSDju8ElitVluXiwjvec97tnK/X0j46q/+at797ndfamPe8IY3oJTipZdeurDu1e4Vf07VgNfkW6QMW325FyCu8467FFxBFRF5Q7vmNfVVqtuRPGoRoa5Dnt/xsCTXZq3kTrVqqK3jpZdeYjGbU68qnnjsMZx1VFWDNqHkkDE5iGZ6Pse6IJPa5pH3edt2Gm/0YnH9BD++/S/Er6TlTVHB+wRwddPqc20giqMhDRUX8ixHibBaVRTFkP39A3KT4SsbZGN5AUqDVsyWC07PppyfnzMZjzGiWm4WAaVV/FtXJ/SNqtY60CcCosBiQYNX8UJgfaYgtNnP+uqHoJhIgsGLVMJl9MMO946nn356q5oh4bu+67t485vffB9b9PJwJ4furW99K08//fRr9vdxx0DaNoSJrGzYRRd92GAww8vXvVidsVr3VvvrEw/q6UrvtIG4LUZY6F50Ywy1DRH6YZkzHJYMBiOWyxWrxYLVasXt27fRRnPz0Ud54onX8dxznwrJfAiKgDzPmS9mjMd7TKdn+C3FH9sh+JrXHVsjXRKgtL7PgUpYEKb+xnwLxpi219ZaUxQFy9kcH9uktGK1WlFkBcOiBA+LxYJhMWQ4HFIMBtQ2nOP8/BytNSenDZO9g/W2927hndQiIfl790ySt9+WQ+p5/Gl/R9BaK9a57nR2zcXsZO2z31ENnxOcc2EElG+fZPIn/sSf4Gd/9md55pln7m/D7oI3v/nNfOADH7iwvK7rtdmUCcvlkrIs71fzPu+4Z6ObDKEVwPeLeyfZEVGETxj2xyq9gUZYP9Zm7oD4gTTsbQVnvtu+X1o9bJEqEQPi0VlGluUMRmNEC06ELDN4mwdPTylu3z5GG8ONGzeom4bnnvsUTVXR1J7BaMheNuD8/KylNtYMV/yslEKLWu9MeiEpiZ2Ai4G//nRc50NAS5S02rG+AfPOtZIZpTXL5ZIsKxgNJ3jnaZYVe3t77I33KMqC0XhMXg5YrSq01kynU24fHbO3f4iXDFE1OId1DTqpKDYke/3rDF5xSJITDGrgoVvOLVIp7TXFy0/pKPverpKkbfFtJ5R0vYnS0a9RT+ZzQZIv3gs+9rGP8a53vYtf+qVfYjzePqX61TKLaz6f893f/d38yq/8yoV1h4eHNE3zqqcVEu5odPsvY8qfEMxd35pKV+q7tUFJ+nXxeNsM7prhbQ0eLQ0R2NBezodoA6wH6z3GKJQSxqMhog06y8AL1XIRDYSjrmu01pydnrK3t8fVq9c4OT0l04qz82POz09QYsjzjLIYMp2exWsH70OiHqMNmcnQOkPHGTNN0wSv3Id0hYIjL4chgOUb6moVPEXnaZpodOLkiK50fRjar6qqnR1XNTUmz8izInZinjzLGZQDlBbG4wnD0YhlVUfjbEAszjfgmyA/swrnLFpp2hhn75m0QURnwfuolOh4eetsoE+SzQ2MRbC9Lnj0XgI9lOiTwFP7UBU6nqfLmEY7TKqdbbOf7dDhB3/wB5lMJve8/a//+q/zPd/zPfytv/W3GI1G7O3tfR5b9/nB2dkZP/ADP7DV4L4WcU863c740jqgSUCfNJ/9aD1ig/8rNm25VY1w0QDLmjA/LIdkKbqaYC3JAeIQ0RRZzmpVMy5K6qpCTMZ4b8J8fo7UHu+knep7+/YRjz/5JDdu3ORTz3yCyeQQvGE4HOB8GEo3jcVhqZZVlH8Jg3JEkQ8YjMcU5QjnLPPFLAynfQjQaZMxmeyT5Tm4htOTY6qqolrOEWXwTc3KNeE+Rf2tMYbJZMLZ2VlrgJumIc9KclPgmsTh5m2UGqBa1dTRA5jOzgJH7Dyz+Sxw6S5U6SCOOsJoZX0GXFI5hOcaMp+p3v2WnrKh/S1oAeUjoetbDje4tkERoZJs8DL5GKD8zui+EvjlX/5lHnvsMb7v+76PD3/4wxweHj7oJr0s/PAP/zA///M//6Cbcd9wV3phnQME8K2H4r0PEXdPKw9LovrwsiragjXeX5CGXPB8e+fsG/pWRJBkS3HrtG+WZTRNw/7+hKZpGAwGeOeorcOjAUtZ5CijyYswweD49ik3bj5GtVzyqU8+w3g05vx8imjFYDBguLeH8jC1p7gMstywv3fIZP+QycEhh4dXMCanrmu8d3z608+S5zmPPHKdshwwKEvOTm5zunfMYjHj+PSY6ekJdbMkb8IEklAWPQyZqqpq8wGfn5+TmUCXOOsQL5g8J8uyVlGglDAoB2TeM51NEXQwlt6xWiyjF90vea/wPmio+4Es530I7HkfPHWRlqGHmA9iy7NKKoj05FQMKHpCsM0733ryrRa7p93ORF8wxDt8bvjpn/5pBoMBP/ZjP8a1a9cedHPuCZ/97Gc5Ojp60M24r7hneiEgVNldl3bZNQlVXBGG3G0wLXCUm9jUbrpoYbdJKtJLn0T5zjnEd7ypMYaqqiiHgzhRQjBKkeUZy6ZitlqiaxWm3CqFtQ1NY3n0scdZLmcobRjujSmKEIRTRuFWFS+Ipq4bRqMRBweHjEZ7XLv5OIdXHmE8nlCWJYLixo0nKMqCvb39oBOua8qiYDze4+TkNuVoyPOuoao0y9USRQgcIMJwOKauK7IsZzabBuOq85BYvPEMyoKiKDBat4bXZBmiFbnW7Ok96rpmcX4K1tNUdTBvzgYJGBINbpfPN95VEg9NUmKQ+OewhYJ25lrf8K6NgFzqEONz7dVVcxdSWHbPfmd0X3n81E/9FMYYfvRHf5QbN2486ObcEc899xw/8iM/ws/93M896KbcV9yVXui8U48oh6JLGhP+D5ypYNphf/BoQgrGpHfoi++3BtLoybA6qx7Xb7yc0s1vSwoA59yaZzUYDMi0wWMpyzBFdj49p66rYChFsVwuGE8mPP66pzh66ZirV6+xWq04ODggy3OqxYp8MOL8/ITBYMhwOKbIB4zHEybjPQ6vXKEoSwbDMTcefRznLM47quWK+WyKaE0+mJKVA/ZWB9Sriun0jGFTo3Ccz87IswGZyZjPZpF/NWidoSTDNZ5MK7RWGKPIUu4DFatrGA2isXbFaFAwLUpsczt6wqF6hm3qnuF0Pa833EpFd8OTnrflY9c4dokZyqK/muQY8Zk6l0YotFx8okrWotFK2sTuu4kSnx/85E/+JMYYfuiHfoibN28+6OZcwGc+8xl+67d+i1/4hV/gZ37mZx50c+47XoZkrBO7r3uovXBJEt73g2obHO02g7uNeticHNF5UiGQp0SD7zwv53yI/HuP1jrkMlBdghZjDKPRKMqfAo/q8WRZwcDvceVQU61qrhw+QlEUOCXkhXB1MOTmzUdZVSu8BaUC9zscDhnv7TMYjiiKYQg8+Rrb1FT5iqIsGa8WHIkizwyLRclTT3lOp2fU1QrlPUe3XyTLcpbLJUYbTk6OKYohxmRUdUNmFIOyJM9jKaIipxwOKIshg8GQcjjEe2E5mzE9n7Yefr2q0Mq0ErBOxqdocx1D6OFiXuO+56lInq6PmTh9b/+k2ZX2z7VaMryut5oAACAASURBVNpCoGmSSj8/b38245audIdXED/xEz8BXCxs+YWAj3zkI/yNv/E3HnQzHhhepk53u2fivceLxBy7QnxtQxCH7kXdpntdeyGh86ToAj89UcTasNZFQ+xjhC8N10UJRVHQNDXg2ymt2hjyvMB5oalDkG+1WkSFwJI8N1RVEwzr/gRvLM5CUeYolZM6kGpVoZSmLIeMRhOMCTkWrFM0OvCteV7Q5BnT6RRjNIIwzXKuHF4FPN5ahsMh1jYsFktuvfhCrHyhaRqLEk2uNEVZkJlQB64oyvA3GCAmQ0STZYbxeMT09ATfNJgYhNMmZFSzPiZ9T+ZXFCLpvpGiohe4V9UTqLSSth4XnCqBpDJNfQ1v2/ludtK04oWWK97h84dkeHf4wsK9B9JwrR5zEyKC4PBeBx5XHD5OVRLpEmpvcoL9/dvj9I6fZlMhkJJ4h+VhWcgvm5JvWxrb4CrPaDSKCgBLlukYaAvGR5mQDMeYjExnrBYLqtWS+XzOYDDEORey9ltH0zQopRE0V6480moonbOIBE2lyTRGhykAdeMQspgUR2i0YjgcsVwuyJucoixwTYNtLDrPycwBq9WKwWBEtVqRmQznLdPzaTtLz7sQuNRKYWLBTZ2FzkNiACsrSrKybNUGdV3T2CZ0aj5JthItEDkcggQwOrSBCFI65IWII5Ww3rWdWheUSxMjaJdvBkEVcqFTDdx/6mB3fu4ODyfuKZAGIHgkJBtc53R9KgnTnz8WvwkgNr7EqvWk2nIyGy/x1jnJF3i/NHstFq+0DkQFraj3DMuS4XCIUopyNMb7UC1C62Ac6sbiddDXLmZzGteQFxlVXQHCaDSiqivqxbxNaWh0xnhyncFgQFEUIb9BUcTcEik5jG0H3FppdKbRWnFweMh8asKMt/mM1WLO6eyYsiwx0Rs9OztjPNmjyApOz08wWuOqGpMZjDFkWRGMbCuYDXRJnpd458lMhifwp4hHK4Otatoe0vfyJnQuaTCCPS+V9vl1PG9nONcNb/s5mk/Ve5bpdxEfVwisqXVvecfn7vCw4mVxurDOw6aZYsHI0pndfi0uOqlZwmYE/LIodj+Ithl4kRjISVI2EaEsy9bLTcdtGguEwFFVrfCuofEe11TMZwJKMRgOKQdDRsMRTdMwHI/whOmHx8fHTMYHoA3DyR5FnDJclCVaG3Ss82bxWGdxLuQ50FpjlMZP9tBKka+WzOdXOFeK2fk5i9WKq6PDmJayoSxKlIdimbNUGm8MOuZLMHmOzgpUnqEyg1YKrRSZyVBKsVx6hsNh4LRFyDKNm63fr00uPag+Ek3QGchNvlXFAJzEUU7H7QbiyCad9qYNjTyCdS7mn4gVhKUfHdhhh4cPd1YvREpBttjEFLxuQy8ioDoez8UXNfCCsmas4RKqQYIg7YIN3mIwiLOeUmcwHI7IssChZlmG1prcZFTVKlRn8B7b1HjXxKrDYX+T5eiqYjAc4X1I7zifzbl9dMRiteD4+ITr1495vXwxo+GQohyEKg+ZaWVwoeNJ7fQoUW3y8LwsEBV44eT8f+r3/wAxqq3ge3BwwHw+wzqLUYoyL9AimMy0eSGKsiAvSwbDEYNyFBQOSpGXoYBhXhSoOKzP8gKthabqUTQ9zrXzWtdvdOo4+x1cMLZhhJN4+bRNCJpd5OnTwdrAnI7J0tP0aXV5R7vDDq913IVeYM3gXdBqQnRs4/x63/N45GKik01czF+ggH6tsuBFJ8MN0JZr855QKVi1OWtnsxl1XbeTDIqiwChN08REGrYJNEDsDAInG4bwq+UK29hQfWE4ZLVacHZ2zmq15IUXPstqtcRZG4f7Iedua7y9i7QL0TClGxMmKBhjyLRGRHF+eoZoQ7Wc8eKLS/b39wHajiHLMrI8awNY4XzhnGHSxSgmNBfqpkGsoRwUFHmBaM2gKINMy1sk5qfwWLxPaTahJXI3YK3rpgYnHt15lNJY17S/ibaCRPsjuJwuUPEYfYWK9z5UFd0Z3h0eQtyVXmjlPxEXJV8+Rr+3vcaXv1Trnk7wsbzfLA65Tk20Q1w6TaiPErH5fN4K8YuioCzL4N1aGwtlhr8m8rTO+ZBDIebUFQlD6boOuQy89xilmE1POT8dslwuaJoqtsV1bY6ebbCxXSIY7z3WN0BonwJwnizPGY6GjIYZR0dHbZIYrRTlYMC0qdBGx04i3A9RoTiloFBKk+V5lMxBtViiyoKiHATpl3OYmOKSjXvXjjaEdkTRVyS0HHuSF0TJmLW2ra3Wzjq8RGd7YQST6OEUQEvPfSOT1A47PCy4s9FNcZgNQ5tyLjjvQlYwiRHy3pA0BVs2OdlN1UK7LCoghMAhxi26pvQ8q35+hm6mmm+9UGMMTdPgGhsNb6gmkcUkNa1HmWVM9vYwxnB6ek5dN4gIeZ4zGIw4Pj4OXq3yHB/fxjlHY2usy0KCGRVNsPftLKxk6DUgOqgYtNKI72bT5XmOrS17e3thW63xIpRFSdNUVFXVG00I3oVJIEWWtWqNLMsQQtrF2WyOT9pjCZ2OFoWVwGenuNZaNeOoONg6zPdhllq7JvWoQf6AUtIme+97sJuB0bTMpfzH7VMN599RDDs8jLgLp9txpl66BCWJJ0yaz5C0ryV46TxBANnqEXUnifFv72IUPB0kBOkuvphRH+HX0z2m3K/GGKrliizLo/cqiGic89imamerFUWOcw3n56dM9iYYozg6eok8z2mahvF4zGw2Z75YMalCPoemscGYe4+1gdqwtsHkeVv40dtwHTWQiwTDqxTiPVqrqMVVVNaGjGjasJxNEYKiIjM52oTlokJuC6UUTVNTVzWiF1gci8U8SMhMxmIxI8syhuMh6kgzGAxwvbykIgpliPkQImee+FaVZqF1nqfDh4TnPnzulA/d/e8b2vSbSLx2/4mtTwPuio/6nae7w0OKe1IvbOow07LEY4oXUtbYTk3gEdGsUQMb+yfqQqSb698/50V5kZCMcTD4wejhgu4UhNlsxrAcMJ2e41wwloiDxqJ18IT7xSDruuLWCy9S5Dmj0QDvhdVqFbndEdPpOWI0y+WK5XLJuLF464Km1YZAo7M2Jl+2eO+wFuJMkThUdygx6AyGwwFXr13jk8+cc3hln9OTc6ytuH10grWWyWRCkZWooWe5WGJt4JmNMczmM2xQzqKUpnKOl46OuLJ/wOnpKYE0COcONi0mIbKentqsNwDZ7m22/LmkjLrh2cZ5EGvPtE819HzzNU82bRNonZ2x3eHhxp2NrgjWe7QHL4K+ROIVPNRuyNilXoymeC1yflH+1Z7uDk1JnlIqnJhoiVDnzFLXlrqeI8BqMQ9BMpPhveBtSBBusiArS9xtqgZc1w3Hs1k7+aGqLaPxmKLI8ALz6awd9jdNQ71agfeYLKPI80gpNDiXcusKeEvjLdqB0QLicB4GoxFXr17n5PSE1XKJ1prj4+OgpDAZjlDbzdVV7CCkzbGrdZro0XB2dpsiLxiVOS985tNAqCqRZVm8dkXTxKfTC3aFjG8u8r/rfO9Fvl51kyV6zy15r9ueaddRbv6U1o+/LQnODjs8DLizeoF+8MpfCHq3Ufr2Re4nw4FgdHs1Ji4RxrceUfjSvrCb00+1VpFLDIZWKUfTOHTmwDc0ddOmTAwpHU2gIBQQPVitNYeHhygVSuG8dOsWRhtms1nYN9MMJxNOTm+3eQO01jhrWS5n1PWKxSIl+vZhCjDgvcV5C15QOtyPxtZowElSEQRdcDkcMh6PeeH8jOVsxt7eHp9+7jNMJhPq5ZKsCDIwkxnqqibLiAbfsaprTJGD9UxPzxgMCsqiZLlYhDSTAlVVt5yrSKxUwaZBvVxxkAzrts5xLeC24em2x/R+nT/uQUuYmtzPybDDDnfC2dnZa6ZqBNzF6Ha1wDyqo3dbiO9iLH2eTqluwgLQ5t9NBR/bPzpT0Kcw0tLNFzPwqF2TRTkQCy4EwJQH5xXeCXjBGME1FaI1ZVmGAFZjuf3SUZjqSyjhbm2NKE9VraCB+XwRZpxlBu8anK05Pz/lfHrK3jQkKK9sjTEanMXiqZqapmkQZRCBzCvEgvUNWmlAxwThgdsdDkJ5lcbWVIslN649QlVVaC0sqwXlcNAGoJomXJ/JMpytcZWnKHKMgunZKWVZslwt0EpoqvpCpdg+n9rRP73n2dIG/e3DM1FxJp9zvqs+0dsueb3JUCfHeqtBjZ23IK2Oe4cd7oZUwuq1gruoF7rKAheGnj0ZEKgwpI4SoU72EAyw6w1Rw2FbvqE7XvgU/9zF4aoKM5q0cyGIp0JwTCsTttEh1aHULmYZaxAFRhtKrcMQXQSdZXijWa5CwcrFYt7KzZIUrSyHraqgHA0REY6Pj3n8ySeoVitm52c4P0DjMVkIlNWNpbGWDIXyghePowGfhfSKSsfAYND6WldTrxYoFxQW8/k8Js4JuYG9c2HmmTaIDvdotVyGoFpVc3x0G2stTV2xWCzaahODwYDz6fmapwusGbmUXzc9o34HmZCG/955HF1Zof7zC/RON5JJ/HzXk/Z/Sr5VS4hWmJgFbYcdHja8rMoR/WXhBbLRburWOAdVQ6IEoqerYm6EO/CAbWCu85tbY5CCNG0Z8+BKQfQag/GXYKDwbdIakynyPEMRDMhisaBaVdRNRVWtWK1WgaOta6pqRWODBG6xWDAaTdBaMznYb6mHs5NTRoNhCMwpj0GHmWAmSMJwLsi0fEj2rkQQLM556nqFiNA0DcvljJPbR1SLJdOzM6pVBYCtm0hFWPSgJC8KRAnNKnjRTd1gmwZlQl6Hpg4cc1mW2BjMGwwHnLz4/Np9FaGbDSYpwXwIRCZdrnc+lGfvnjJJapb27f8eLvuddMHOzvauyf020kjusMPLRROCFa9a3H1yxGVDQAG86uk3O2fW+w0DLXLBaN/1+Fw0zkgo9e1VYphD8Kl23fTULFPtcETixASjM+bTGYvFHPFQNzWrahmM7aqiaSwpkY9tHJVd4X1MflNV7O8fYK1lOp0ym81Cpi+tKUwRyuxIHjhtYhYtolrAhXG8dUFqhoQKv6vlAm9rXrr1AtPzM4zSIUeBCGU5YDQaBXmYWlIOBigfjHXwxj3L+YIsNwyKck3ytVwu2dvf5/z8rONaew+nu9UXZXiiet825GT3iu55sfa7SMfqa4/759lhh3uFtZa/+lf/6oNuxueEe/Z0N6PNQaq7EWRJVYGJ003FtcfYnBTRHbMf2IkHpm8g1r1tC2gxEMuZN01DlgcKoCiLYBCV5sknn0Ssp24aXnjx+eDV1g11HarzLpdL6qYBhMa6mHsXRELwzTnHdDplNNlnNag5PT3l6tUrzGbniHi0JuZEyNFxSqsoE/hbJSAaLQbrG6z1NFEjPJtOOTs95vnnPs3TH/84RycvUZqSoijY29sjL+Y0Tc14PMZWlsY0SJGTS9DMLqZV9JgtUztnENUYTdNQmIxVs6RarQIfrgUn6Vl1vG0oxx4mXXjo3esu9eLmve8/tzXZ4LZthdbgb1NGbB53hx0eJtyzp9sPaq2/RL7lQlN6vzTvUy5s28F739ay9ykrWesirZ+/O28o/WJt8myDLM05S56HKL7JMw729rGpikKzas9f1zWLxYKm1dXGXLHeonXoAJyzQQMby6HP5ueYPOPKlVCI0lqPcyFgZ63DE+quiWjEebQ25CbHGBMCh06hVDhPVS2pqiUvfuZ5Pvav/xXz+RwasBK0uGenZxRlEWaf5QV5lgeqRBmyQQYuSN/qZRWUESpUyMjzUCBTFznT+SKpdcEF+++9D1Lm3hNr2d0eCRsCY2lCyeUa3hQ06ysUttEPl00XvkwyuMMODwPuObXjRU43vKxpwLiWzIS+Z9UXzq8frztmMNKdqYaUTnD9Bd3wumPC9KpeURRlMFB5mImWKAZrM0ajEavVinPvcN7jbE8O5UOSdYWgjKEcDBHRLOdzqqZiMV0wHATPMctyynJAXgzwXuGcDdF6FSZd6DgV1xiDEIyXEo3zhlrVNLbm7KUT/s1v/RbHR7d48cUXoQGlBS2GmzdvIt6zmM/ZG49BwnAqTCzQmDzD1Bm2sWQ6TFfWCMvFIpRxP9jnE596Zi1AJa0rGyY3hOBXymnRoVVVq85Yrj/3jqRNx2jLrqfnzeX6283A6A47PKy4axLzy/lX30WqYzXJGP6iSw8Yt2w1SuvoDGp65RMR6DfWbxvShimtzjusC7rUkAbR4Jqa89USrQ1pJlSe54xGY5qqCQUWncfZJpalCcm8lXiuX7vJm77ibfyzf/qPcbOaql5RLRd462gay3iyx2AwCvkNVIESTaYzRoNeHl8AF6rviobGrxDRVIuKp//9b/Px3/53nN2+hRGPGEUxGlGYkulsisk0w+GQxWLOYDQiUyH5TZEX6HKAOEeuDDZOZ26qGmsdmQiTyR71crmuNHChlFJIl7PF4EXuoVWQJeHJRpCzT9J6XLtPou994BQuGNWdod1hh3XctRrwpetapYH0hqqsvajpFd0mO0toDXmbk7Zdc8FOp5lU6/sK9arGHJq2ICW1xeHJMtp8tMPhKMifaotZGqpqTlM5vAPlPVoLXgTb1IjkfNEXfRH/4Xc+Rh2n4VZVw3AwZjTaYzQaYoxhMBi1tciUVuQxibiNEjmlYvYxo9FKOD854aP/6l9gqwWTsmRZr8iKIY89+Qb2Dg45Oz/hpRdfYCyhC5rP5xQpCbgDrwWUohiWOOcZjkbMzqeUgwGiFY1taKp1jS6qX9EjzUjznVSsnefho+46lT+66Jl6H6t1sN4hJqmCuItql0RL7eiEHXYIuCdOd6ux7EVgHB132E1+6PZPSF5n/9jd8fp8cThqCrJ1Xlf/xe00p2l6rHOOs7MZ2kFeligVKIHhcIQxeUzB6Bk2Q05PYOVD+kTnHXmWcb6qqVczTm59ihtX9nlaGWpbU9c1q1WYYlsWA4aDEVmWk5c5RZ6TaYO44D1rrdDGtO12ziEWpudTPv3JTzI7O2acK7wxFJmnyQr+yNd/I14ZTK741x/5CIvZCVpnLBaL9p4558CEXBaZ0WRZjvMekxlsYzFac3J2GjKqsT4jzPdv2WaQi76BTduvF5Vcn20Wtm/pJNfpfdd+K9Gwr42Ndh7vDjvcY8Kbbcukk4slD7XPvfZ1mxe52XazbtR6wbinDGZ3kpWFk6eaaNY2VFUduFXvKcoy5M/VQU+7p4TMaM7Pz1jMc5QtsHUFKFRmUHWNVo7jFz/BfFEzyDNWvRSGw1Ewtt5DtVoh3uJ1RsMSqS3e1Jg8IysLdJ6FYb0IjWjOjo/5g6d/j8nAoAcl1WKOKsZ8/IVztFqgzIim8bzlLV/Ov/u3/y+TyYTZdIbSmsY2WOfIKfDOUq88g8KhPGR5xsxOEYHlcsGFaWO959cnbzY/x6cWOgkVRiYhdadqd+6eo2o5YZHI7abfw4VzdyXaL0gAd9jhIcS9T45IsqMeTysinUxMOrlR8praQNolVQIEFYb3uudJtf9fNLj943fStFBufbVaUjeCs2Byg9ImnFdC4vLMGLI8x5gwO20+n9L4GjKFszUozajIGGRCoTU+c8wMlHlBUZQopRlN9lFasVrOmZ0cU2qDH88ZDAdkRlNkhmI4YLi3j9qboLMs5j4TjAi3X3qR/cmAQgpWuSYfFfz+7QWnJ59hUpZUVmOKfcbjMaPRGFBYV7dD/uVyQV1XbdVeay0TMybLDfhQ70xrHfJK9p6fUqHkkov8baI9EovuW7o2VABxKbApgQVWnjgNPOwf7LpvnwmOXhazmI+BuI3IBWO843l3eJjxMgpTdpD4zzp/F+RGmzpPuPiCrU9HTTPPoOc+pbNseEXJsDu0zsArJCbCqOoV0oDRZSydrlgul23mMGstw8GAvCwpywHaaE5eeJZmcYqtFYhmOCgxcYpqkQllUaDLkslkzJd+2RsZjvawzjI9Pebs+ecYKE12sMRPRuQi+MwgkwnGeUyeoUxINI6EROZaG/aGY8aZYrEqAM8je2Pq+RlNPUVnBcvGMigzVqsV4/GEs+lp9BTjvXbgmoYmzM+L2dI0QlBsOH9RfZC802BXU+8Zs6G1gbBI8ShZf7bpOfRyF7e8b9JUt8fdeNay3pluft5hh4cR96bT3RS2p3XQk4pxkYeQ7RrN3pdWb5tswfpZNlUM4SRKpbIyBE/WBa2uKMWyDtzr+fl5SHCTyqjHqhFZkTMcjxlP9tifTDj+7B+wmJ7gnWcwHFA1DotCqYwbjx1QTq5z9epVvuEbv4XhcMT8/IijF19g8cJnaZRiaB26OcAbgyoKlPJoo8lGJVIOMDoE+B65foOvfNvXcPLp32VgagaTEbePb/PFr3+MUWEojEFlGfOzU1y1Yqky9q8csqcPWC4XWGtDWkedhRwMWsUUlZbRYECe55zMpyiTo9RyXTubomY9uF4n1w+Eeu9RPtTw8HiUC9t6icnafAqg9oOg6499LZC6xavdebk7PMy4d3ph20qJ8iEvLQ+YBq1pWu7lB477kpQNvZNEDW7r37ZcIG3J8KTTVUpRNzV11TAcDoNkzLk28bdtGoZ7+4gIeV5gHXgvFIMhV770OtdvXOezzz3Nye0jSpMzzkrGV66yd3AFM5hw7dHXcTDZ59qjj1ItFqwWc05uvYg9P2exXGBnC576oi+DPEd7mJ3eZnF+hhkPyPYO8UqjRBgfHvIf/bF38plnnuCZf/cRiix0FiMUpQk0iBJHFvNWaK2Dh5zl2MUS0QZEMRiO0Epjck1VrYIUTGmKosRkGcT0miH4FmvO+ahakFgo0vc6Th8pIxcmUKQxRm+sEffpRh7er+fR2Ay2rXXUbr3672Wz3XbY4WHBvdELWzg50pDXw1o0jc5Ib76M6SXrJ8EOBjS94j3PbE2Ols7bpzQImca8p2ksINR1g/dhmG2txVuLZDlZljMajSjKgtFoEmavDQoODw7Z/6Iv5c1veRvz5TmL1RLxiiuPXKMcjSjKMXkxQIvBSYNgmZ6c0MznNKsly+PbVKfnXL/2KNl4zO3pKfPbL/DoU09Rn53DtRpfFoho8rzk2o3HGY/HPHrzJs/83m9TO09TVeQKlNFUjUUBe4cTDq7fpCiGzJcrJnsT6qqiqWuqmLYxJD535Canbmo8YLQmz3OqpQq165A2ruYkPbck9OueUXrGPnq0kmR+nl4gLVQCCbXTthvQrYHS3m+mb3h3NMMODyvuyegqrfDWhXpZrf8TXs42au18q0JYe5m56NGszcdv5WUbutw4/E2ie5FQOcHaFJ0Pky9S3bKyGIaEL3t7VFXFaDQKHnQ8nzGGQTmkLAcURcn+wT7Xb95kvL/PaDBA64zGNiHnrTEYVYAiTLxoLE1jKQclTzz1RVCveL5pOHrueQ5yx2J6zsAoRGluT5cU84provEq3BNRGqNDztmyKBnvHTK5eoM3zt5OvVrQVCtW9YL5qqauwJoM50OF4/PTE24f3aKOmciqumYyHlM3dbwflrqqWVUrMpOTMrz5NXc2BMICLetaNUGgalLX5qMhjkXvU2XjXkrI5DnfqwHtqyb6v4EkgdsZ3h0eRtyT0b2Mk/MoZG1V8o78NrFCt9XWAEsvUEPnL3eMg49l03XMmRCc7boOofq8LKibBussJgvVgCf7e4zH47YsezqYdRZtDOVgzGAwIS9zRAmlGgA2FpJUrOoKowzWNmilUHnB+MoV3vTVX8vrvvj1vP5L38CnPvYx8iwH5xgdPMIf+bb3Mr5xneHVR7BKo330HHWgGbCewhiMuUE9WYVZdQ6ctdTW4mzwJJfLihdf/Cyr5QJjcpx3MY1jEWaolQV1XeGcxOrBrs2pq7WmqsP9S/mME9mTgpaO+L8I4rqJKW3du56+t9Pthk7E+X49kM2Aau8pyvr6fo20l5vBbIcdXiu4o9FNHJ+nrbPY8o1tIKs/hCRtE0vTpOTXW469/UXtHat33JSNbC3pdZws4Rxt4ppVtcJkQRpWDEq89yERjFQcHx8z2dtjOBwznkzYOzjAZEJmwMbKDGK6UuFNXVHVVVRJhGvKy5LcWZrGMBqPmYwO2CvGVC8d4QRUWZIfHpLt7yN5FvhcpTBahwkkRoH2YB15OUCbHOtCFjIclIRcoV6gLC1VUzOfL7B1A1hOj09idQrB17CYL8jyDO9D2XeHY1AUVMtpO3rwkQ5opOPZBUG5ECQjcr0QDGlgFVKhUdCKtiMIPwrajJ7tSML7jc6XNR633XWtMvCO093h4cTdPd12FhK9YWZKRtMZ2n7Zlt7OPd1uZ2S36TRbI07nbSX0jy0iiNOIEuraRS/Pc3p6RpZlbbXf2WzGYDBgOp1x8MSVkLvAGIrBgGvXb1AOBmhjWCyWOOejtCwkyamrVWsczs7OKPMC7z2LxZLhoCDPc5TSZFdzlM649Xsfh+kMBgNUkaMyg4u8t2ssDYEbFULnkZmsVVtoUyB10xotnYVH0tQNhwcHLGYzzs9OGU0OMFnBYj5nOp2ijWF/b4+qbjBGh7LzZYEyId+E0RnWNXgLKEWG0HiHSs+DYDyFoOFtjWovoOmhLV7ZseyJcY/5Knx/ZMKFZ72Zw6EvZ9thh4cR96ReSFNGE/qGs5v22wuUtcsvP2Y6TnsOkf77vjEJInm5CpFQzhyfjDE0jce5moODq7EKRKikO5vNyLKMW7dusbe3R1YUKK1DhWNjcNZhvaOum5jNy7JarTDRexQJBmkZq/YWRYGXDJMpDCAG1N4eq2s3mPoXsLmmFqidIycEpZwLmiulVKguQerAFForamsxJmvvR5tukjDT7tq16yzmc7yzONtg85y9vT2a5ZLz8yWr1YqyCFUulBLGexOmZ7dpbN3K7JJRbQf0npBngiD/0tED9pu6vZ5xbANv6bl21Hq3Qf9Z9vftPedt33fY4WHCHYk15Ql835aXJXi7QJw6mmqMhcCYiodWLSu7g6OjewAAIABJREFULenJ5V6PxP3Xq83qqE313uGAVVO3XrAxBV/2ZV8GhHSIdV1zcnLCahVK2hRlyZVHrjEcjSiKEmsDb7pcrtosZGdnU5yDo6MjFosFzjmapmK1WnB2dsL8fIqvK3RUUTS4EO0vc9SwpBFFY0OtNOccdR3yIHjvsXUdaAzvcda2xsqYMHMu3ZtUbt2YkC5yPB5z8+ajjCf7jIYT9iZ7DAcDvAhaZQwGJUpCxVTfWA4PD0OeCaVwLnmvIQ9vv4NUgBFBp/WpQ3CB2lGJt5XuubdtJEgEW1nZBnfbf96bHesueLbDw447Gt2k94TtBtLHwEo7cUFc1O26WDMsBlB6L9qmt7NttlL3cq6/tEm5IEpRNQ3eCbaRWJnB8hu/8f9wfj7D2lBc8uDggKIouPLIIxwcHHJ8+zYnt29zenqKaywiIQgVEuWc4WK6xMduPs4wGyA+lD6fTqcMy5K9vYOQ4yHxqo3DedBG0wQlLEqHmV+2DjxxP1m6c44qlt1x1mLT/Yz3OXm66U9E0EYzGo+5ceMGxoQkOPPZHIByNMSpcNzRZIyIYlCM8JLhrCdTpi1dLRIiYInjBcLz2eRi+16oB/z6cxK6acEiAupiruT+b2WTVtrNTtvhYcedA2l3kPX0Z4iJqDjRIU5gEEeMjwdhfZwDAawZl01hfcK6Ye5oirYMuHg0Cmsrrl69ztHRESKe8XjIN33TN/H8p1/k5qPXODw8oGkcn3r2OT7zmc9y48ZNRuMJAsxmU+o6byP+ySCsVhVnZycs53MaG3lXramcZeUqyjitOBglR7VaMpvNWMxnrOoaeSkPkx4mE5RRrVwtZUGzNg7qYz6KZIy1MXjr2gi/S3I61R2jjMUqp9Mp5XAYSsfPQw24xjuMEhSawaCgmufMl/OoVuiUBMT0ix6itxqeY+Juu/vO2igj5HBYT1SfPm8+v37dtv72/f2MMa/6AoM77PCHwZ2TmPdlRuoyDyXMUPJr6xTJhUrSpGSi+/KhO2o813i/3j7p/6bB+4YXbz0fZWShXtpv/uZv8sQTr+NjH/s37TD9y974Rq5ff5Qnn3ySa9euh0Qu8dz9oXxRFDSNZbVYcHT0IpPJAYOyjMNxoakqFrbBmsApN1VFtVgAYK1jUc2wWlM5hxchL/K22rD3nizLMNrgBBpnMXF6bpLBKfP/s/dmv7ak513/5x2qaq21pzP0cGzHdpx4IJYc+ecQy78fP0CxoiBHSUSUSMgR4gIhbkiEQMJMN1wkv38AKUhcREFEQmK6CAgFhQhiA8kFBCcKJMGQgaS7T3efYU9rrRre4XfxvO9bVWvvM9ju7t19zn6s9tl7r7Vq1apa9dTzfp/v8/2a4hQBiT1iNFVdYaylamqWe/t8YG8PfGAYerquY9hsiH2grhu01iwWK94c+pxO82nKJ1Us4tPJiUwYITyadzvFaafUr8JgmDx392a9e85BqH7Xle6TY7PZXPUuXMdbHI+vdLNyVFKpuvB4GohQ6TnlKclJovyayaGT2J1WK/EIill6ME1ZBYYgko19u0EpmfrSWtP3A1/72tfEKWK1T1UZfv8P/oCTkzOUUqw3G45u3OLo6FDEbYwqQwYPHtzHe0dlLbdv3cFYVWx/UOnm4iMOwTVBKF6b9ZrNZoMLkXq1xLkBNwziHDzR1o0xsqgbrNbEUi2LV1xtLS6E4jOnleCrItMbqaqaG0e36Luek4f38b5nvT4nxsjh0Q2CUgx9z95+I64W1uKCVJKF7kcgRk1I3N1paCgCOE9qcO02y6ane/pZy40twUTTG/N1wr0YTdNc4C8fHR1dNxyfsXg6Pd1Ef9oNY8woYj0b46UsWx9FYbj0okvJeT69lJ83memf0ZKMYLtJ91aabYr3ve9l7t17QBMsq1VFVdVJkUvT9y0PHzp0uvhXe3uF69o0DYvDGzRLQ9f19O02NaQi0XmaZkHXtsmFYmDotmzWZ7h2Sx8iZ6eWgGHRLNHGYGxFjNLc29vboxt6TDCpMWhKknIhFCEfrVWqeOVm1jRNgkE8Vls26y3gWa32UVGx3W5p25abN2+KcM9yRVVVDMmBOKZGmsyuxLHBNjv2+RTs2vSMrIrdplg5lxMI6AJmnzev5o/tJv3rgL/wF/4Ct27duurduI63OZ7KIy2EgGF0gZ1fdKpc1LmjDZBJvJddxLvvUV4iW9t5Rr5Qx82GiZuv1lpUvZLeQoYMTk5OuHPnJTablls3b3Pz5i1WeythFISArSoWTcPe3h7W2uLSsFgs8GFg2Hb4oRf34H6gqWtcP+B6kVKsjKXrO0Ln2Jyec/+NN6CyRKVolgsePLxPVdcsV3ulWWd0hbHJUDINTuRBkzyWodJQSYhRGm3pxlA3NYc3j+iHjg995CMcP3hA10klndXUTk9PuXXrFs1ihVeaqqrou4HgPdkhLZB1kMUGaLcRdhleW850OVezV6Sb4eXfH0gqZcFfoB5ex3U8j/F1eaRdXgUJ53Q0KExVrlJE5VGYOXv+EdtX06w6Yy2A6AnkikoT/agBYa2lri0y/aaLdToIv/bmrVt853d+Gu8DBwcHDN6z2WyKQIx3Dqs1N2/cwCea13a7IabEvj5fE2JIbr8yVmusxrmB6BzrkxOO799jvd0QNxFlLPViwc3FHhFwySI9fyYfI0PXE4wclxCC0Mby59VKlvlSHstxDJGqrlFaY6sK5wbqqiLGCtf3c4YCsL/aQwVN3weRULgkJ2olvGE53GOjTSt9oVolkmCPKUGXkmyn9N5ddoJ8VwBjCpvlWnfhOp7neCxlbDZZNMF1x6Vjvuh2qEcFz1WpUbODAe5UU5mXOzp27T5Hpt9Cea5MRGltSyUu8IArle6tW7eKa+/v/M7vcHJygvMy/JB5uXVdE0Kg6zrcMNC1Lev1Oe1mw+npKev1msVyya1bt1mslgAYq4lhQOFp2zUnx/eprOHo1k3qpmG7WbPdbNi2G9puy8nJcdJDsIBKQu8qcYBdabQR5HP4KMJCWklyz7Qz7xx1VbFoFgQfGJwrWhR5e3Vdp4ad5f0f+AAq8ZpzhCQ5JuyIUFTFUFIBh0togYLohKRWlrm+k+lApcqXaIrl5vOSvw4mjkJI1wn3Yty5c0f0QSbx5S9/+RrP3QmlFB/+8Ieveje+qXgqeOFRdKHLYdkJjhsTkyFq8gU7fV5OFrLtxB/NL9/ZeK6qClUtRoIXrdaqqnAJ0z042BdcdrFgs9ngQ2Bv74C+7/mD3/99Qow0TcO2adg/2Ge12kNrXRLg2dlp8VyrqopmuWRvtQKEprZZt0CgQtG3PeebtUAAVU29DHR9x/rsmOXeir69SWUrtts18hE99WoJRmOURilwrsd7nbjMRm48mU8bIcM3zjliFIF08Xyr8aGnaRYMwyCVeErAVV1xfHzCYrGg3W7RMbE1GFXXLmglKJUq3zhjGex2NadsiFlFHGECLhXthVm1PGGsXCeTeXzf930ft2/fnv3t+7//+0uv4joktNb8+T//5/mn//Sf8rWvfe2qd+cbisdXulAqx5wI5onzIgSRMUBVtnDx4tptyMTokWVrchTLOOKkmsqc3WzDHoEQPbmQq+umMA2UUty9e5e23aRmUsS5pEZWVUQf8Km6tEan6bGe4B2KiLWWGzdusre3x6pZlGpUEvOA7wdOj094cO+eCPsYS9UsWB0dYOqK9each/fvsT07JThxEj4+Pub+g/sE7xJP15BxF+9lYCJ6Pw4sKJko82mYIt/8soNEvVhQZ++2vYM0ZSeVfAzS5FwtVyP9LITZqZje00ryRBV4YV6NSqNM6fy68fljwp3/N23M5fHiXTjpWmnsOr6RsNbyoz/6o3zyk5+86l35huLxE2kxLfoTpkdqlJUJM5USqJLkOm2ujcVq6vw/sbLRKGWThoAuDIUYYkruZUsEYnleBIytODw8Yn//gPV6zenpKXVd0zQNdVXjXI9zQ0nmWimWqxV102CMTZxRjdaGGzdvcnBwQFVVLJcrFNC1W7puS9tvUBr8MLA+P+Phw/vYSjQZ6mbBYrXP4c2bLFYrtuszTo5PODs9ZX22pm23bNbnnJ0c03ctw+BR2qJ0jbGjlm8sCTak/+Is4S4SrWjoe6ytsHXFZrtls92UIY+qFtGfrm8vQkIxD1irgsPL6ZPjGvNSI8G8MeskQ3GyVFEVo9JHxzzBXtYfuK52JT7+8Y9zdHR01bvxro2///f//oXvSl3XfN/3fd8V7dE3F49PuiDjvJMLpEgD5ipH52bapJLJEG+k8Fthjvld8k6zChgkOVpb0dQLmmbJYrlgsVwIfLBcslgusZVlsVjifaDvB5bLZZl6y8MIuZLUWomLxP4exlppnDnHInmM2cpS14KrqXRD6fuOmCbPgg+C/XYdp6fH2MqwOthn7/CA1f4ey9Ue+/uHrPb2QUG72bA+PxNrdIT+1rUt52cndN16nHhLFXokluXkeIwmuHqUxX1VVdS1DHJYW7NYLgq1DEAlzDpj3Pn1+YRnmCGfIznPFFqgrCLiCBnkRpsA67OKuVSxOYFPb7yXQES7Nj/XAZ/+9KevqWKPiS996UvPFMXwySpj6d8Lo7qzzCrODuNjXjrwCJZbKuLJtqbYYUkoQSCDkiiyEEzil8YQJ0vSSNMsqJuaoW9pWxH4zhXucrlk/2CfqlokuUdJAgcHosi13qyJMXL71i2qSswe9/f3McZgrfiLnZ1tcL0rUo4KjR8GNpsN3TCwf3TEav+Aqm5QMWKNYZsgDdd3tNs1wR0WyCLGKEk8VesyPbYAbFk9KCgVK4wQjDGGxWJB37Y0TcPe3grnBvpty2LRcOfOHdrNFuccwTMmYDVhnMgan+yXlokL8r6T86vK/5UVh8pPniZKNXndDLoYmSdaKUI6b3Gy4plNtT3H8alPfYo7d+5c+Pvf/tt/my5ZM13HsxVPGAO+GGFe5qBEN5A4e0H2GkglkB7ZDPKyixqrOWnvNtucc0TnSiIwxuCCyDnuH+wRkl7B0dE+bhDr9eXeihtHR6z29qhsTdMsODg45PDwBoMb6AexvhFFLst2vUlQgzSinHOs1+ciEWmqcoPwzrHdbDk7O8NUNYvVPkpXVHWNTrxYU1uMKH/z8PiEoe8Yhp623bJa7UnOioF2u8YaK44VjcZakxJgSoxJ9MZ7h3MCNehUFW+Pj9lsNtS1UMaccxDTsfEOnxJkZhmMzZg4OXvFmOfCeZYEnel6U/A37V/qZRYq2WS7F2GE9JwQymhcTv5cNqTxnMVHPvIRbt68eeHv//Af/sNrbYpnNJ7YybiMtZDj0boJplRACj0Csikua75dtm2llNCcohiCayUc1hBkid11LTFG9vaEsdDUDdYahqGHVEkuVws+8C0f4ODgkM3mPNmZO46ODqmqivPzc0IMLJcN3vdst2tOT44FMzWGuqkSRCKTXH3XsdluqRYLbNOglRDjjBGcxdYLlvsH3HjpRfb2V7ihT+7ErXyuJFy+aBYYrRi6Fu96nPf4CD5VoKEkXU/fb1MjUPQbDg8FwmiaBQeHhyyXe/TdwMnpGT6ANkZGLHxIgxFpm1MYgBGzR4m2bq6GVSRp8O7QBseTUxJopguq3ceUQgzYBHfP/YC8ctm5S1/HJP7iX/yLnJ2dXfVuvKvih3/4h696F96yeKK0I4zKYJc9Nv39UXjerprC9DEZd00No0lmzs0klTMAuYEn5o7OOdq2S/snHfyDw0PquoLoabsWpRTvf//7iTHQdcKdhcjLL9/h6OgG5+fneO/Fqt172rZls1kXXDj/672InA9uYNO2MlFWWdFV0Iqu7TDa0DQ1xmgWqxV7+wfcun07qYu5lECFAaGN6Fb0fY9WMKSJNZkSk0ZWTrwg3VprTRHmOTw64sUXX2K5XLLYk4agrWqaxUKgEKtk8i5tS6CRnTwXshElBSvWKSHqyXl/ZOw06C5E+SoolJGhEibfER8DPOfshe/+7u/mYx/72IW//+Iv/mLx/rsOiX/1r/7VVe/CWxaPhRdmfM2dmEEDebmYlp7ja4R/q7BEFWZV0/Q9drbMvAIaW+cyGVbTOwdRYwwJ61zSLJbUVYXWhhADt2/f5KWX38frr9/j6Gifk5Nj9veP+OAHP4T3ntdfv0sIUUTAtWIYOqw1HBwciLYC4EOgrir6vi+6upvNhlVqvBljqOuGbrvFB4fSSyoridcNA6vDQ2KMDH3HMgoG2ywburYv1kLOiRllXdfi4GAMzAYVJGmGgmdL4rJVRbNYsjl+QIgCtaDEnHKxt8/Z2RnOOXwcG5mlaTbhX0dItusQE2ZbVhlpWk7kKCd2SqWpOj1NavaHsVEW0xuktB+i0LaNnmzx+YybN2+yv79/1btxHe9wPHE4Ig8OXBaScM3k592qVri3oxftWBHPx4lHCENcgSeVsoqIE6Khahq8dwzOIzzWQNtuWK8blqt9rK24efMmH/iWD/HGm28Qg+KD3/JBDg72OTy8SV01PHjwgOPjB9L1XyzZbluqSpwWVqs9aV4oTd2I8th51+G9Z7vdcu/ePUlK1hQ5xqAiWM3gHd4PLBciNmO0oWpqlIZ2s8VoK46/XY9RlmgMbugAzWIh2gnGGPwwCC3NDTjXC1PE6MIOEUZFLxWsrYhRs1ztE8j4b8/56Zq27ajrmq0fxnM5vYHq3BOL6IkzZUJzy/nKeLBC3CLyVFveZtZykEXIoxkJIaQJOC3fGJ2mDK/jOp63+LrhhQvLyWmf5QIdLMywv2lHfhz/VZdtakYviyh0Er0OIWCTSAyQYIGeN998k4PDQ1586Q7Hx8fcvHGT7/iO72C5XIoT8P4+bdtydnZCVVXE6PHeFRWzpqmxSc/WOdG07bqOthUfsvV6zWKxYG9/n2bRYK1Nx0ZgBYDBO4ahxzmRnazriqpecXh0E43Y9PR9jzGKthU7oMrYciz6fsAHx9D3DEOLcy1dt6FLiT8Ej1LCTMjH5uDggGZRc/PmLXxUrFb7LFcrPvihDxVZyd3zmQV2okoEwOnfE5TzaDrXfGDFZA2JGC+c493vjbBRMg1Ng3l+K93Pfe5z/PE//scv/P1P/+k/zauvvnoFe/Tei8PDQ/76X//rfPGLX7zqXfm64qmkHY0xM4xpxrGcYHvzJgnEVE4pRfLtImkfWFQRVpkkBa2ZdGvS/+vkG6ZwIRKCLxW2UoGmaagqw2c+8xn+5//8n2y3HX/iT/y/DMPAf/tvv8be3h4vvPACx8cnbDZnBD/gXURpabhZW0HUOOeLPq3WlpDkIrfbLdbaknSzzGJeescYscndF7IWAmkYQ7NYHRBcS99ucX1HZbTgyLUt7IzaGvq+o64bmZSLHq0iVV2hokabKk2lZVsfWC6XVFXF/TfvpXFoODy6ydC3uLDh4YP76QZl8VnXIVetKjXLpAtISPhuhhnUBZiIAjfkhlqMkagV0Xu00rMqN8NSF8TM86CF0sTwfGsw1HU9ajWn+PznP89//I//8ZpK94i4ffs29+/fL79nCuj+/j5/82/+TQB+/dd/nV/4hV+4ql18qnhipZspR7tiJmOMxpRTJ9v5hiIhyCjtcrlXpAhhrGjVTtWVOUmaiMmczpREjLUYK4lnb2+fT/9f38Wrr97l85//PEdHB3zlK1/m9dfvcnZ2ytnZKcPQ0/dS5Q5Dx/n5Gev1GefnpwxDR91YqsoW2ccQHN45hr7l+Fg81YS/azFGIU47kjBlAEHYFJJshWObxWfquqGqa6pFjVKRtt3Qdxva7Vlannu6rsf7gb7v8G5I4vAGzQKl5L5ojKGqa0Kq+nPTbLlcslgsUBhu37rFMAzs76/oupbbt1+UG5X3xCSOUxTJEObCyDMhTZklvDfBBZfxqyFh8bvP1yOumw0256+//IZ9HRJnZ2fXCfcxcXx8fOnflVIsFgsWiwWf/exn+fznP/8O79nXF0+odCXxxcnAwjSy+Iw2THjz82pXlqOZiaBp2xatNS+++KKYLCY7knzhjvyHtFGliVHUyrSJQEBj0Kbhox/7JC+99DIvvvgCIcCv/dqv8b73vY9PfOJjvP7669y4cYMXXniB1157hZPjh1SVTQk+d+zF7XcYWpp6T7ixMRKDJwbH2dmZ+JEtHKvVfoEU5GYUExvBMAw6JVmdml4iEWlSU6yqG5z3NCvF9vyc9XrN/uEhMXqc72lqg/dDSty5yaUIMVnaeNDaopVUSMLLlQO8Wq3o+47gPa+9/jp9L7S3O+/7ACfHD+l7R20rGAa882XgJFeukWnDNJa+V4zZsl2qa59cLfLr883FakNU4DNZW4GKQUaFy3dGp/9Xk2mbp/6OXsd1PHUopfiTf/JP0nUd/+k//aer3p1L4ymSLsnO5hI34BhRE5Km0gpiZjRMnycVUcjirsCDBw8YhoGbN2/SNA1nZ2d0XZvonQq08Eu1FbqTVqJWppTlgx/+CC++9C10g6dtt/zGb/wGL7xwixdffBGtNevNmoODA5bLpShwWRkVlipVF20HpQ2r1Yqzs7MJtYuE93refPNNtNYcHOyxWh2IELnRY0d/UvnLY6ZgmdmIUimpMAX3tcQI/bYleofzPbXWeNdjrU54rynbmVaFIfgkgDNgrRWTzvTY+fk5r736Clorlsslbatouy33H7zOpz/9nbz2yiu8+od/yGpvNdvnaUOTfIyBkNqjeQrNez/Xw0gDKQoKLS2ff7l57pIEL45gPM/Qwnd/93fzPd/zPbO/fe/3fi//5b/8lyvao2cv3s3fryeyF4DR+nynOy2jnI5i9xui4LIEiotY8hUL+LK0FVUxWYKfp8rPWsudO++nqmq22zXr9Tot3zWmshweHfLyy+/D6gUPjx/w6it/xJtvvsnNo1vsH+7hnKNplty8eZvjk3s09ZKDgwPeeOMNtIqsVktkYCI1l5SwAhaLBX0/sFmfs16fAxS2RgiBxWJBCLBNBpSr1aIcl5wcs/JX1sidjjhrbZOFUCM4aYCmNrTrjeDJQVFXFSDJPFPonHOSXGOWvtQJG7elkr5//z4PHz5ku1mzv7+H90PS6zXcu/cG+IHf+K//lT4EPvxt30oIgbuv3SV6uXlIc0slfY08jJIq3JClN3eseZQipGERtBLtjR3sV07y9Hs0jm6DKjzg8JzSFy5LCNewwvMTT3SOyEIxl1W6Y+WqC/1HESYdmSRLFUYPrfnrffq7SZXlGzjnMdqwv3/Ai7dfomkWtEPPyekJv/e/f5eDgyNOTk5BKZbLFWhJeKvVir29PTabDU295PDwkM1mw/7+Pn23nbxXWuoajTaiTWttLInWJ4aB4LF1uVFYa1ksFoXPn6128r/Tjn5+fNrJt7YWB2BT0w0ti8UC53rqusJHR/Qa5RzGVDPsUynSjc1jTSWuEyEweCfSk6sVxw/vS0Mt4eonJw/l3ChYLBfcuXmDu3fv0vc9e6s9bt+8BUTatmUYXJKslCGVupax5+B8YiSEcgPJ1XFO/MI4ywZDcUSV5vIM6d80dKwgKtGeUNo87uv3TEb+vkyj67rrpPsUsbe391TPy9fBuzWekHSnvmjj36cVr1KJoZCaYGXqSeUfdPrX72w7YYj5kk2dea0CSsF2e0rXrRFvryjaCNry5ut3CVFRL5b0/UCzkCGG09NTbt++zfn5hmVSIjs9PWWxWDD0qjS6Smc9fZ6+69PnCeWzWqvLTSbfeAR6cNT1ohyD3cSaT7T3HmNN6dBnyEAZjYoBWx0QfaR3fZobkHTVdRux9lFisImvAMGOYwwE67BVhTGWqlklBwhS5Wo5Pj6m77ecnj5kGDwRhY+BN954E9Q4On337mvjF1NpotIYrVHR49ww+1y7AzIxRoYg0M9M3jFl3Aud2ZhgqNJkE/gBsqvI8xNaa77ru77rgiThF7/4RX75l3/5ivbqvROnp6dPfI73nq985Sv85//8n9+BPfrG4ql4uo/TRki/zH8fn5X+FTrVo9jwBb9M1eiY6LL5pEmuDmds1hs2W3HotbZGF76sLthsVQlGrLWm62RsN/uQDcNAn1gKmX6V9gIgLfFNGpZYJcaCKHzlm820M6+1TmO6trg3gNwIQhpjFnUtRWUbrK3Q1qIrizKGyjYYXSUXCItNXmJ919K25wxDmyQpG+q6YdEsWSxW1HVD0yxQ2rJYrahszWq5hzGW5V7N//25/4eIZnCuMC/ycQJGNobVWKtAebSeYN6XhDFGVMNE6Yaox/MtGmyX9Me0QltN1BMMOURUjDNY4nmIj370o3z/93//Ve/GMxvDMPArv/Ir7/ob2FN7pD1qJFgET8aKUDGvAGUDyZtrh26mVC6JVbIFH1+r9Kg/kOlZbhiEp5te65yj24qc4csvv8zZ2Rl937NaLYlRlUTpvC/L6PL+ijIIkRtFeZ/rumK5XBSpx5yg8n+X32Ak8o1CdHoiLngCGqUtWlusbagr0QduGknCVdWwaPalUagNpqqoq0WBLbSWhF3ZJWDKPpls8a7Enj0PWJyvz/nqr/83nHcYay7ap4PQvZQqjAKVR4wjF9S/iuhRquRNvuHEHQw3ymcukEJK0D76kd1AStAB9HNU6BojI+a7cffuXc7Pz69gj95b8YlPfOKJz/md3/kdfumXfukd2JtvLp6sp1u63RdFxi9PPHFe0D5JGCVfrWhUGu0t1VBpuIRkuSP/CkVNkure3p5Yjfd9quYim806TXcNaKXQCuJ0fBVR4Kpri/MdRlfEqIqFe37WcrksTbIyfZUfzctvY6QZlSAKociNn0vFCEnsxlY6JW3NMAw0zUqaaSoPH2gEWha9BWNy7WgIUY6DweAze0AprFZYDfv7+2Il5D3HDx6y3W546cWXeP3114hBHJAzFzoCQYlGbvkcUY65Tk3G6enK8ICHbBIyDsDEicZxiJPni9GohmKAmc9pVKCsIfpnP+saY7hz5w537tzhB37gBy48/nf+zt/hF3/xF6+vLoLmAAAgAElEQVRgz95b8d//+3+/6l14y+KxSVcSoPw8nUjKUcY+L0ms42OxcDsLRXNadRUIQ0RetJaZfJ27MkoTAlKpescQInXCf21Khi+88IKwFLThxo0jXrv7Ci/cfoG+F3zSpgSWBz26rhOLnv190ZbwgUVTzwTSjTFst9sCTWRmQZ5Gy59D68Rd1RotpR8xkmx3As5FlBpQ1sggRrXAJmdg0nLeeY9O7ITgO2FEKKFgKUySRRwnyvKx01oTgysQR1VXDMOAtRV1mnjzLqAjpUKPId0MJvoKGZ8lJ0Q1PzfyHEVAuLomjiuRGaNFCcavVJKRLEczn3dkVZRf8+5l9XzTobXmwx/+MIeHh/zZP/tnL33O1772NV577bV3eM/ee/E93/M9T6SAbbdb3nzzzXdoj765eILK2IjpZuvw3YsxL1fLdaemiEWufJh15HdjOoU2ymrP8eTgxZpcGaF6yQiuJL/tdpsaYBVd3+H9wHqTZBuNpa4qrJVBhTx1ljFYwY4pVawkVoUxkqSqqpoNRUDmLfvU+Bs/r1IarQx5xDmEfAyFilWlyboQDcZYXBppBuj7NlXUPnm2SeNJG4MKAe+zaaTCe9A6D5xk2yRF3TR0fZ+aYSrdiLTQ9jK0k0evJ+cwU7es0gIBkbnBYYSWEBKg1pqYmnRT7HeK75cR4vG0FmaDQBqzYvmZC6UUn/rUpx6ZbAF+67d+iy996Uvv+pHVd0P8m3/zbx4vMwq8+uqrfPnLX36H9uibi6fSXpAL9uLfS5Wal9RjriRPbeXRUKESTZafaroMlxdKZRzK71oblDEMzuFCxHshJnknPmX7B4cYYzg5OUk2PTXnZ2sqW+OdxzsnoHUlPmmRkHRWzGiAiUAPfd+VBJaHD7Km7kyzQElVrrXADRlakPJ2Qp1TeRIv3TSCx3stFWIwuDQ+ncdzMydXRN8pmHMkpPeUm0yMuZmXWldKqti6rtlsNsQYOT05FQaFVrzw0h3uvvqKrDjy2G46FzFObqz5TOQBByhiRTKllqCHGNHGUKx31CSB56fnfJuq56JopijPlZe++/V0X375Zd7//vd/Xa/RWl8KJUzjn//zf86//tf/+pvZtet4j8YThiNynpRL8mITbJZn03JVzRJvAg5LNRxL4lUo7WevHv/VoCxKW1BKBFsSrmytJQJt20IUpaG+79Fas7+/z/HDh+zvHTEMLlWsstzVCQKQ6z7pDyhdEqiI39QFRpEQ7moWOo9RxoSNluQpn8Oj4oj1yrCEA0z5TCF4lI+4GInGE4JLmK0VuyGVOMMKsYFPNwOtkrbujMlhSxWulLmEdqXwrqdrN1hruHnzFm/cfZ0wdOWkqnyuUqbNMEMkopUmxCBTZUoxMrEneEFewZBPd77pyF8zdkw65yF9LYQqloeL391x+/ZtPv7xj/Nt3/ZtfPSjH31Lt/3Vr371XTui+m6LH//xHy8rzWclnurTzIgIOzSyIlJTRklVWWLmCxCmEMN0y5JU5mFGSlYyM5ySnYP3EGMRFs8VqzABLFVdi+pXlMpsajeklUmVNLNxW5mQg8WiSZivDC9EItt2TWUXSdhGKFMxVbVaS6soxlBUxmLUaZQ4wxUKCFKpJn3hiKdSNWCK0lRIFb5ODbsQAyZDFUqOdD7uu8v6jFWfn5/Ttl061oIte+cTde0SvD2fx8k5CcnRoSADSqARM95JS8rNxesE1Gc3NJN8nBL9u3lE8+joiM985jO89NJL/LE/9sfe8u1/9atf5W/9rb/Fv/23//Yt3/azGD/5kz/5/CXdKY67SxvLF7zRkjylATYmXZUaK7vKZNJUc8Swu7xMfF90avUwuZDlyvXOoyv5PVe8OXEppdJwgUATVbK4yfsvS+6RY5uTZX48J845JUymwkTPVqGC/F1cbl06hJkyp0rDzftBKusoeKgPHqMU2oiimc7L+DzpZUZjTqXAKBn91aYiD5rIe3icU+Uz5+PqnOPk5JgbR4cMwzhdd36+LjegPEV2MeVN3CJ8HLm4TJtsF74Z5dSMcEK8JPHKmPHkvvGujYODA/7Mn/kzfMd3fMfbsv3f/M3f5G/8jb/Bv/t3/+5t2f51vDfiCW7AYzdfqTj5mUSkr0qzR2tN9IHImKBHrQXwycGg0M3QaUYt6TSki9UHBUZJlacNcdoEUhRjRVAYrdlutywWYrOe6V15ui1f5NrkijRhoelGksd9cxIeBWfEF622Fd4FgvEsl8s0KisfyCuP8mCK1fxUR9ajtEJHmRizuhpZGdpOxghiElLXiQOrCGkJnsVyYpBxWdG58HivChaqtFTc3nliVHRDz+///v+m71uGYSBG2Gw2SSGMGaaaul1yvlIlGnL2nDTBVPr5suq03EbV5Bcl3Ahyg7Vk23mlDe+OitcYww/90A8BIhV6mWfZWxG//du/zU/8xE/wH/7Df3hbtn8d7514AqZrRngg+FQNupRUDdpYtNJpNFWL/1XiamqtMdaQSfg+VAmXFLUsotiUh9xsyQleifpXdj3IYjuJb4ROFaH3oraVNRNGjYSsY1ulqjkJ7UwaXLkrnxkIU6ihaRoGJ/hnCAGrKxRSURtjMEoaXpmOZerJYIiSilZrlexodBrAcCyXMj7snYh+ayI+DAkeMYA8N+pxrivjwzatJJQesfX8GVTi7QYvxpZvvP6aOB+vT8VQ0w3F7DJXy5FxODs3u0qDbQrdppj+nif5ynZ08f0pGHBUSTQnJfOo9KzIfTck2x/7sR8DZP+//du//W17n1dffZW//Jf/Mvfv3+dXf/VX37b3eRbjH//jf/xUegv379/n3//7f/8O7NFbE4+HF7QWo8QYaZLweIwVSkuVaZKgeFOL/XlwQ9FZlZfLpa21iJ24YUiDDrKs72NLdBGUSU22IAnX2FJ9OicXtEGWvLmhVuxrokpVnQi4HBwc0XU9IEI9MTiWxqSmmRIWwyRpFUw10ciGYZCGktZ4F6gXkgxRitXekqF3BV+Oei7qHmJI2mqjgaQk1ZFXq7VIODZNVfYpy0IWCCStHASiiKKrG03SEbYzKh/pszx8+JA37r5Gu9nygQ98gLuv/SGKSPChVPZT1ohSKpssj0kwc7nY+XuMJdlmGGImPTlhh43pVJK8RuEnsMP0u3EZffCdireros1xfn7OF77wBdq2vZZs/Abje7/3e58Kz91ut7zyyivvwB69NfHYT9QslmUM11pL0yxKkybTnJaLFcbK8nnoOpTKRoiC5+bXa22wVipS55ws5a0ngIjLBEAZrK1K0p1rvkpCU0qjjMZoK64QvePoxmFaFU8qTu+RnrvCuYAx8ybgVDth1KwVStlqtaJt28Ttlb+bSnQctJaKeIRO4phQ1ahrMCYoyDbuWjUCE0SXYBhpsAFlGi4nspyI0ZNBgrT0F5hiFB53zjEMXSLa++Kh5mKYYaxj00tWJShp3GWebqag7SbDSMLqQyysjby9AiHE8W+pvSrHbuc7NUvW77H40pe+9NTTY845fvM3f/Nt3qNnN37+53+eF1544ap3422JxybdxWKVKhxJVHurQ5TR9P0gOKqtqCqZrhqGDj9xFciJaJq0VdIIEFeEQJ8q1BiDLEG1wdg6UaJG80qljDSUoheFqxjRwXN8/IAbN6R67bqOumnwfqBttzJooFOVHDxT94ssdKO0NL5UWgtn+tVms8Y7UI3Cx2FC3fLivpAoZ7IUlypYoIDEUggK2WhM+OuYTPu+p26EN2xtVfYnJ1BRFItpAANU1BirRv4uAa2yrbpO22x5/Y3XOT87pW23/J//83+obA0uYqtK6u5cnWZiX2JAlCnDafItjBNhLeSGZjQjVquUIvrEw9aZvTLHbjMzImPQu4M17/b4uZ/7Of7e3/t75fe7d++yXq+vboeeo/jkJz/5zLEWcjz2Uy2XS/KwgFKKw4NbRCVJtKkXaTy3pe86urbF9X1JuN772VBB3/flgnPO0fWtDDD4gCdgqopFZYu7bJleSktyIeSDDWk5bg1939O2W7quZblc0XXdZOmLiMwkSKFoFZQTGUt1nEdjgaRnq0SHd2jlcRXHVXfRcJDps0zfGqGECUVNi9tFDGCsFYWxBLlUlSH4iItuhjNPk5IxtgxfqFRJG20wWYgdcSzetlvW61OsFfra2dkZm03LaikOFmIh5GbbLnDDJRXpjFIGIwySjlJZgUg2H8eBpX4v28zbysdnl274bo5/+S//JX/lr/yVp5ITvI7r+HriiY00rTTBC1vBucBi1RTqUdttOT87ZWjbZI8ulKvMq50ul/N/4gYx4PxA5wL4AEZL0ymA9lJJxhiSfsFUONtDWnavVqtSdWw2G5bLParkHyZVYkCnxpTCoHXEWE3Eo5RGG0lgMvXlsbbGGltW4y64NCac9A5iLPujNZgyLBtnnxFGoeqmaVBKU1V1GR02JunsGktVGZzrC5QgTcFR21cp8VzLjTaUJmKISpXz4r3nj/7oD/ja//otTk8fJAt4xwsv3GZ99jDhsYknEsLspgCZjSYwSIYJ5glSqumsEibNO+RYkGAHGV8bhy6g8HoVFyGGd3P86q/+Kl/4whfo+774913HOxtf+cpX+MhHPnLVu/G2xeNFzENkSB32EAKb7RpSswcF52cndNs1wUtzKfuGwWg/ki/g3OyS57nUsJJlftYEyEnJpySPUsnZQZbjVWXRyDK4rutU6bZs1mcslnvcSEk3hEBVG2LUaakuU1beBXQSvgkeFDFNrJmUcGOBDlQAH32BIiBV30aSXkxW60mRoHzOXJnnat/asUGW4RZTWSHKaYWthepGOXapiVcwXnlfwboztU6OwRAC63VbVhTHxyf4oaPvWz7wgQ+x3pwx+A5jNM6N1X6BGdLyP+a8OaMppPM46ZLlSbOoYpnumyLFOXErFF6PThKPGvd9N1S8p6envPzyy+X3EAJ931/hHl3H4eHhE7UWcty7d4+f/dmffXt36C2OJ6qMTW3VjXFs23VphHXtluj6kkx3/4V54yQn5ayhMEqrJPzQqEnCmi+3ReYwYpcVzvmSmEIYWK/X3Lw9JntjVREfN0oXKUStTRrkGO10ckWZbRBiGiQgOJTJzTypUuu6JtPlAhGrTRK4EYaCnuCkeSgjwyy58RZCwE6S85SxMI9cPYveQggDKlRp+EKVZb4bBn77t34L73peeulF7r/5Ol235cHxCV3vCGFTqvCL75EpXZNfIAmMJ9bJDoWsnLFMQSs/z3Z9Qj17dGK96mZa13XcunVrpt52He+tyNfYeykem3Tbti14ozjr9gxeaGHr9ZowOHTiHeVKbppcpxfVMAzzx9Plm5/T+cTfRWOtkRFbNUo4hpgTbSz7lHHUvK/OSTWc8cuc7CUBz5fVOTJHN8SRPaCVhoShajsKhiulS7Up+gtg7GRkdgIv5KW8mUyaxRjF6DK5SIyVsSuVaH4u5NcmF97iQBFxLmlBOM9me85yteC117a8cfcu280xMSoePnzIzaN9Th4+RBNnDsOzCnM62FD+pOYshUIpU2MFPPmsOeL0+VGw65hm20b22SX7cIXxXrtgn/X4er8XV33j/kbisUk3c2611mw2G/b29ooW7TD0JemOUoZjdbubdNEyOeW8S+vQiI/j80kNr+12k/Rga6oq45lgtHB5jRl5rU0jWgmr1arspw8OQ03XdVKdJoaA8HHjbJmdYZMYI1VlisKW0qokzUDAKpu2PxEXLw3GEcfM2516pSnl0/HxkMTLbdXgXJhM0TUMoS0OwCN/Nw1vqPw+JHzb4J3DDT1vvPEGxw8fcL5eMyRRm5BGlg8ODjg9OcEkCp1CuLlx9r0es25JoBERGNcKJtS9qUg5TBgRatxWwYwJAtUERSYEj6sWM/u+XMd15PjlX/5lvvM7v/OpnntycsI/+Af/4G3eo7c+nph0R1GYyHq9JipwXQ8+Qgz4MB0Rlpgm3dzF98Hjoy8i2UZprJfpJY94io2DFSYlkNGxoW4W+MyMSE2cqqqSq66jH1q27bowLnLzrV40GG0YBldoYzlp5GQiyVIwWlk1G4wWOlRtFhgzbk/2XQY4gFl1ugtdymcJs5vPFIKY4t/zYYE0WJDxZBXQqsLoCq00LgZUUkZ7cO8ev/t7X8O5jhhcqtwie8s9GWCZ6Olm+CSiRtnGaZmbBymUAju36JlWIOMqYmy2kbgNcqMw5bNPj8mURnidcK/jm433Kvb+RI+03JgKIbBpt5L4gicMfcJUA8MwJIL+QN/3M6qY1rpUtCbRvjKOCRQa0m5kTNQ5wTvrqsGY0QDSWkuXtBO893TbLX4Y9zVb7Ii9tTj55iQ7tUzP+yVNIo3RaXzYZJcILmmUpemwxFKw1mCMsA4iYVbxkz5lHpvt3YCxiaY1qYjHCnxKQ5Oq2pgkS6k0QQkLI/hA27dEHB/+4IeSoE36fCj29le07aYk94LnJjrtLlZbHhonhss5nC/5VGo2jsMl0y1pPSujL0Q+P/mYX8d1fCOx2Wz46Z/+6avejW8oHlvpTpOnCGzD4HogFs+rXM1OK5cpPul3Lu1KKVwIKKMF7YuKmB0K0oUYkviLtVXCd6Vayz/rhB/v7e1BSihKwXa7pl40LFcrSXhBNBhICHJlG6qqLhXtbnNpmoxjCCirMFoq2KlLrlj0qKQDMVanamzfp+eJ/GNIVbypJFG1bYs1NSHkCTaNikJCCzEI/9eYQsKCPMaMTMQFGTler9e0Xct6vZaJwXpB13swcPf11znaX+KDK/BPpoddaHwxoYLFzCbJgw27zx0dK6afe16l50GJcdsw3oTkmL+3BiWu4+2PF198kaZprno33vZ4Inuh0LxiwGqD0rEsOaedwylXNVeySk2XseC8p5o0VFARlcS7dbrYXeKU5qVofg9JhmMzKPuCKaVoEqvAaE3ftvTLJRqFXS4haSFoxaQhBhBKkoeRWzv6wEmiyxQzrUXCMnfvVdI7zLmm0Kmi2P/IdJk02xSCjeooAuhA4h2LKLnSiqh9SorJATg19ApXN400kya/3CBTeGenp2kYYsN2e04IjuVqn6aSasCYCoJoEKNU0iiW/cv7vRtCC5ucp/S8keebjUpj+nm6LUm6+eY15Wzn7WVq3OOYDdfx/MXP/MzP8NnPfvaJz4sxcv/+/Xdgj96eeKo5O9EkULjgMZc4uE4x0pwkC3bpQ1GjMlompgyREBXRRbwaCDEWmEFrjV5UEwaAwg0DKEVlbXGZresaGEdoMxcYJRoQlR6FbHIDLEwU0GJEFNKS/i2MFXpOwColugw9yJI6jNBDHK3lheObDBuD0Lycd2ijMNoSvaf3W/RihVYU9+KyKkiwQowRTfZaE682KbUF8ohKE5XCOc+DBw84Pz9ls1nTrjdp2V7Rth3gUMqyXC7YbjfEpBlRKsw4nrvppFhRqpw8nn3eBFZQ5dzIjXYc5BhDiQW9Gt8v35TltRFFvvldx3V8fTEMAz/zMz9z1bvxDcdjMd3CZU3VavbImtLCdiuanIAzTjnl6YKsWr0S5oJQsGR2KYjdQnlt13W4pFrmvMg3RkZcVTzRGqqqoqrrgvNqBdE76rqacHGz2tlIG5PnVzO+7PTnDD2MTIeKyMj/FR1aScsZ+5Zpu4RVRw/IIEgIAecdIbr0uXpCGGbDJMLSEI3igJIkbITJoFPSJVW/MSj63nF6+hDneurKpBHgvHy3AkokxbY8fDK9OV5+whOkMAHaR/hFlXN9MZIDcpxs+3EJNc7hhuu4jifF7/7u75aff+/3fu8K9+Sbj6ewYB+XzVopotYi0ZiSrN+hiWm1c3HqPHU2UsNgXHb74GfPn8IV2cFXoYRu5jxNU5eleU6eMS3bMycWPNvNGUbP3XyzpKQxVRmvVSpeaOyMjIZcCUrpZyYd/5hGYjPXWG4aEJWsCrQWkXbBUjXaJEdlBohirjltmgmvNULSziU11STZQRI6SEMHga5r6botp2enbLZbnOvT8XMs6pqhFyZB27azm+PuzzlmegmTJtoI7SQKWP5MagojpNfJpEQ5RvIe0ogkTvnVRmQ5r0vd60jx2c9+9rEGoD/3cz/Hj/7ojxJC4F/8i3/xDu7ZWx9PhBfy8jJ7g+Vkm8d+fQjlIs1WPWqaeLVMODlyQ250Fg4TXDFfkDFG0RYQaTMAcW9QA0orujYwJDx3GIZSKeeluFKKdtvhHcSouHHjBsMgqmhSueaKd9xHredNtDHpSioKzhO1ScI5whfOOgS5ioRI9IGAwADEiAqBaET7oU5Vq9Kyn95FVKqE5eaS3jvhpjondg2kalcafCKEfn5+xnazwRrLomnw3YLg13IjspZNDKBgGKY+dWOSLW2vmAQwd5JwjnmyHv+286zJ83e/QOPUmtj2jDe267iOHD/xEz/BZz7zmcc+55/9s3/2Du3N2xtP4OmKoHdOVpcNPUi1ly7s1DjTBd9Ltt9KFfW/GKNYo++MpE6nuZSSytZrT0iuO1MR7TxlZq3FDQNVVRVBGUneUi33naHrmqJ/kFONuFfoCSNh5BNnSlkZB0b0GmIIIpgTYqr8I1GPNurRx8Q+iOhAafpZOyYXrSx57na0E4r4IEI3UnmPZpxJP34qrUMEvHOcnh5z796brM/P2W625RhmWCSvLpQSuGEYss7xpBJFGphKdi4J3swbbNMG6fRcZdhmmqxLxcskDavMTyuzaeU9ruM6niZ+5Vd+5ZmCoh5PGQsuVZ9j91mFOKl8VCE1lYs5xskVF8dEO2vcCIxg9UgRm17YGQNVg/ivmdTI2eXf5tfUdT2hrmXtXMvgOrbtOYtFU4YtYEyu+b1y9VsaaCnxkit3rUqlWZbVO6R/ee95VSlV+LzyVyqmceWa4CPGkMwg842LfETlfYoeLzDBzrfbjYj4BBEwd8N4PLI/mlDFZB9nlWXqo+V8SLLXUXGuhTFNqLsCJLvVc3n+zt/k7caG2wjXXMd1PF380i/90lXvwlsaT5B2lAvLOSecUa0KtWv+XxDql9bFAma8MyXKWYiluhXeLeKppudL2FzReudQKvF6PUVwPCfDTP8yZoo7KqwVlkHeVtd2bDZrQojs7e2JbxtJGMfM8c2pHu68iShUKykNJTFqrYh+PEa7komSrHWBHxRZOD0zJQaMqWYTf9mE0xgx1yw3ohjF6CEdy2EY6LoWowX3ripL8AMhyGfZbrcp6Y6NyQvnNv9/SuYlWT4COtjFgS9WHok+dwlqMJq1K9HQuIYYrmMSf+7P/bmnooo9K/HEiTSVOuZGJ+PJ3S54FAghl07zikglzuqksRaTdY8xoMZt7V6A+X2DD2Ln4+V9gvPEEEQDwVY0zVI+iL5I/yKCHxzn5+dst2sZRU6cY1QSOlcXE8CU3qRUEg3XhsjocpwpXbs4cPn8ilGAXAk9Le5gmrkhaLQkIx9EyyIGx8wFmXyPkkbgen3OvXtv8uDBAzbn50lnQmhqxlRJXMiQ51V24aD8vyymE1DS9Ns5B7kqlWNzCeaboJIYxp+ZLnTSL7m/FkIguzLv7td1PL/xp/7Un+LjH//4pY/9/M///DM3Mv54Pd04TmypEB/JAippIY7KYXMMlgsJKoQgS3Y9shYyTzWzJoTDKhe8jiLyErxHWcNiuUdV1QVayCPJMfrCVogxEpwnmAHvHJv1uWCcWmG1po8ypFDX9axSnVbdWV0sAymSXE3SfpkLtcOIcecpu1xRDsOAyZU68yQ/DAO2rgje4YlobREt4Oy6m5b9qRoWl1+H0RqrBW/OfN8Mj+x+UXex2WKhk28DITXKcpKcv7rgw9Pvxiyj5m9HlAYmCeKPISalsVJbA3NB9eu4jkfF//gf/+OZuzk/9ls/nZOHeVVYkg2JexsCzvuC3e0ut8fJLj0mhMwkCHOn2UxVyu+ldSQQiijO0Pes1+dsNhuc9wzDUCrHTCHTWtO2WzCKGDxu6Nhu1mzOzujbrQxcMA5XiHLaUCzdc0w/czaqLDeYONfyLF+OKDhmtkXPjT/nhKcrzx3lLmFUAYuJdTBtR+WkmG8EOcEKLziUzzGd/BpvbikJMk/08+ZYypAJaph+limMNMVxM6/64vMmhygoFAZTblnMtnXVF1Nd1/zCL/zCle7D8x5/6S/9JX74h3/4qnfjHY0nVroZL9WMF/Zc2FwnHzIZYw1AXtfmJBt9kKmznFzjmKg1Ug35CW+zJPQQiEooWFqP2gAqwtD3uKbn/OyMqqpomiZVeSJAk5tuef9UFMnDfhC3iawSpgBV18lGR7DpfKMROABiCIR8I/FBluI+zNwgpseMGIg+EI1BVxUQcG6Qql2NDhuZVWBMOkZKxoLHZJablULZ8wH63nH/zTd45ZU/5OTkhK5tZ8m2bVupWoGo50n2URjqpH1Wylx5rr7wmnxsghdjzHldnGGkkSEhIkDvzlBK8bnPfe6qd+O5jg9/+MO8733vu/Sxf/SP/tF7VknscfFUY8C5+jNaEtg00eRV5tSSPA8FVFqmobQVz7OinoUk2spYoREZjYpjMs+DCdK3ktcMw4BJtjohBIIbkhatjPv2fY8xhnbbc+PmIbYSLNXkJX5K4sPQs9mC94H9fWEuCLYrSTuXanmirrIZ8nB41wt7IgYRMS9shkllGDwKuVkExPqlrmt88MSQKt6hY7FaopTgsN4HqqopVfcYU0hGfhcjyjXWalaLhnZ9MpNKDOlGBnJzQk0ToTAi5HBcTMCip5AVziJlYCPF9GYkzC+xuR9fHwqUNN/uHFO+jut4mrh3796Vr4bejngsvDBd7oNUplN61xSDhfnFpbXwTYfg6ZNweaFWAS4GOpcm23aYA5LwxElCKmBKdZWX5ArYbrc8ePAApVTBdutazCCNzaaTwhjINjd4TxiGJPfo6fuuNK2ytY5PkEX+fAVHzT5vflwih0ugEcFAR1aDbDNpBaek5PpBlNvUyJSY4sJqJ0EqIjF6Qow8eHCfN16/y4oF7pEAABfLSURBVMOHD4oG8Qw6UNIUm9K/dmP2XuW1edhD9j8yaeYlHCl/9lGkYQ5VTPHwR73vSLF7thok13EdTxNP7GQ86sIYOZcXn1eUpVKjSilVcNecjI3WwmBgTDpT/uzYkIoMSSfXJOucEAJD0nWo69Fp11pLiI7tdk0IAWuMWJAbgzImJV+NSe+VtYLbtqVtN3SdjNO6VNECBWKok+XPtBE1vUlcgBjSeHFmdNRVLZq3UyujiQZBfn0W49k9HjHKUMfp6Rk+RFZ7eyyXTXHSKMmMxHQY228XqvHp3y5EzDoKkRhdek2uYC1S4hpg1NRNWyRLSE6hjGnynSbcx+7DdVzHMxyPhRf6fkw+hkklFSf8Vi86uzFNUpVkka5vHUkC3ONAxW6SkYwSMUolDFe4uJVOFjqATTSzIrQTIqEfePjwIV3XcXBwwIMHD7hz5w7eB4ZedGSXqxqFcHRjdETf4V1g6DYENwC+dP1BYIymadBKsVqtypK67Qfquk5Y8agIJjcBjx96gUMSe0BrK+wMxko5OyqoZPeTKWWiBaFnCWlspuXkpIQu5zqiCxw/PObs/AzXD5fitVOWQN6GvJdJf5P0LBW5mr+w3BBsoiDIAzH61OiLKKoEL+QKfdJES9+RvGnB+idcsjj+/Tqu47L46Z/+ac7Pz696N96WeGKlm7v6OSEUFa2kvQAZe02sg5CGPbPVeRbFKd300VAy05u0McVpd+QaRZx3xTIoV8sw57gSYxE+Pjw8JITASy+9xGq54uDgoFSLzrmUO8SNoa4NdZ2tc7wkTu8ozZ8YC5adm3JTKlt+zPvk+4YvVvQyykuBKvLnNslOHkhUudFfLn+mC5NfWqURXZUm0dacnDxktVywWi7LhN6jsK/dY533O2Zq1yQylEI5qyoTNUbIAFLydexaEU23k7cQYywmltPPlLd11XF0dMQrr7xy1bvxXMZf+2t/jb/7d//upY/tsoiepXhspZs1a3fdfedjuwkGmDRRIlGW/8aksftISBoJuVqFOZ44JmOpfDPbAVRiGxgRZ5ks45VSoDTHx8elKj07O8N7z97eHraqODw8YLW3j9GiaBYiKCsNOq1HxbJhGKibirqp0Uoz+AHjNFVV4ZxjsVjMKG9T006jNVGLUHpOJN45TLIcyvi3jP9WZd8lkXmqav68LHmptfCYpS6NuADHJ6ccP7zPyfExm/V5Gnu+DEYYMeEiKD95X5mQ04lbPceOp7zbC0k1RlAmNd0myT41+ohxZz92K1qVHJ31haR/VXGZNf11vP0xhc+ep3iiXY+1VsTDMwU1jlq6OWR1OS4fY4iz5lIMgYAkIs/oBlsOeGrUKBK0kA0qE91MhdQpVzI5lceBvfeEvidaS7ttaWxFvVwUr7Z60WCsplk01HXNMPQQhcGgolTnwXkqU2GU3DjcEGgahQkKFaDvWlRKmNm3rVTxMWCNxWCIupJqHhk4sJVJ1WtFxjtJn322vlDJ062p03JcbiSCGVuyP9vgA922pV23iC6mQAO5kTfn3TKl+V5gWRS8PPrJcAOIcuM4WDJ9/mSH02vkDXRa3Qi4n1gNZLqb/CkoRcQToxmNLssX5zqu4/mKJ1LGykU9oXPljvwUp0XPmQ7l9anJlJe5l97ZlBK9WT+3dpHtSSU5nbbKdC5JvAONEW7rMAwc3rwBStH1PYvBcX62QSvLchlZLhf0w1b2RY8OZN556rpKswFKqGnW4tPYqoJCT1OFFaDKaG9QoO1kmZ8wzqx+lqlfGU/NMEKGEsTUMmkDAzZVuTpR9EKqiLt+w5v3XuXNe3fpurZUlVPcFnLzL+lbqDF55pg2QDMmq5QSuUnlLz2PuxXv7vvups8Yx39DHvjYqYKfxyrnOq7jsUk3V1DOOawaO/X5sYLxpQsqc1fzxTjt6pvEYphRm8qVmRotWlHpaqbZSxw1fHPSz9brQglzJVGHELj3+huYquJbPvgt5b3y8tEHT2Vr2aZ3WG0gBBSB4D06TZz1bgAtUMd0Ce2cw1hhQRC9VOWAqeSzEaWaLzzjfNNImCow255W6XUqlONpzZx+ppTo0BJigl3EYn33OM4nANPgQi41HxcqFtx2mgMvVM6XvTRVzGEyVDGNGBVBJV3gCYQRd24W13Ed08jC+89qPLHSLS4Nmf40wTWL1U5qvgjKMGoWWGsLFDFdAudqdU4PU3jnxtHgSWTsNP+9bdsy7ut9ICBYad/37O/vs1gsePDmPTbbLbdu3abrOqqqoq5qQPRvta2yYBh91+P8AN5QxYgxmkDAMGruFhqcA2NlSe21MB90mpiLIaISBCGvE+cKo8VCPUxWCtMIIaDCRfeK2c9RxNz39g5YLfc40w+k3ziR2mQ8C4jnzm61mhPrhBNM1kOOxFTV71bF85OBcK6z4kaBGiZPEDpLuhEJDCFiN1Kx757/67iOafzsz/4sx8fHV70bb1s8Udpx2kTTWi7+gvcRJ3ieQJVKKUic2DgZHJCl+nxZmilX3nsqbfBx1KQtlZtKTbUY0BGMGuldMUaMliV8cB5bG9Zn5yilWB7ss1zusVwu2d/fZ7VaASJWUzUkviwQA4FApS1Vaor5EGlqgQUyHlsbNUnAfsbeIAjEkI/ZWCFTFNiCD9hFjQKMrtBmdCCOMaLSjUulRlqM0kDTSqb5usFxenZGPwyFTiZV7xjzBCkVr1jsRGLIlfBIQcvPUgWnZTdPT7aWRXdy3k2NOBKrYQo15WeHSYMvitFn9B5HhiUe8+W7jut4RuOpxoBzxEnVAjKhlnm0avKcDEnkJLtbLeUqJ1fR1hiIYKtKSP15+Z2w41lTJ7ntzmhj6bmDG6hsxXa7Zbm/lwYnLF3XYRItzdg8iGHEgiiI71re5yZG6qpCRUVVVzMsWpk0zlwSDpAbUpMEPcO+iYnXqsp+5sRUKHN6rKgLLKGnGUnR9x3n63OOHz4UoR83p1zNj3EesMjDHHN2Q9rrscqdkGZH7dtd1oJQvabnIjt05BXO+Nxc4Y7fhxjBI5OJwYufnPc773Edz0289NJLfOu3futV78aVxBONKedDAJPGSfqbCz7RjhRRjdKOojk7qoWFGAs0UbiquxhvCAVeyO8xhSVkFHjElGOMLJdLcZFQDkLEh6GIi6/Xa5qm4caNw/L6uqqIhJwiISiikc9a14tidZNx4vy6qCTpWJMw4WLnniABpQmzJCJ8Y2N0EW83WQlMx/KzncARMxyX8dhkXnDfinD5smk41YY+XLL8vxBjAoRJxQrlp5gw2RHOGMkoM2y3NMfEmWNc8YwpekzK4++7K6MYAspHuSFdx3MZP/iDP8iP//iPX/VuXEk8NuluNltWq+XkootF6FolUqp08JOEoQJSxZZJ8SqzBCbVXybh65Jwk+hMqu6CD0lHl1IhZqzYJlGa3FhrmkZszr28f27eCU57irWGo6MDDg4OUnKzaE0RuvEDbLctdd1QN4ty01BKJxUxuXHYuhIxd2OIQQRwMm5rTJPobE4qx2z/Yyx6MhABYK28HpjdWMpxyrZABWaIeO84Pzvl3r03OD05Tk4Y0lCbcnRH6CCvSHIVO6k8yzjYxWp2WsWSt2JUonmNkIRKVDXUxGi0LHXGZB1KlRuLxkaGZmKInJ6ePe7rdx3X8UzGYyfSNtt2VnUKSpi61aUiS3P+Ol/IkyoxbWe3CsqCNdGLBGJwTiriCATx69IoTE7ocT4+HGOk73u895ydneGcR2mD81481Yxh6Dq0gu1mzenJCZv1Gu9DqrQrERS3lqgUtlpSNytsVZfPpfWYlPKSP0RJHBlysDYPMBiUNpiqwiatByEPmFLxTZtnGZkYfdmsOFGUW4ws95XQQhj6nuMH93lw/w0ePLjHdnOOH/oR1silpijdSFJPLg4on9gJeYAlZDzgQpKdRhmEAYKKBKUIUf6LQRed3myWqQLgk95v0sbIfOm+7+m6jq5t6dqOdrtlu91y7/7DJ3w935lYrVZ88YtfvOrduI7nJB5b6X7iE5/g+OH9GQyg8wRaFtWOI+0nZgghYZUzLHZn29poSRAxouKoQ1AadSX5afROgijygsxdguu6TsLenrZtaVZL+r5ns9nIkEfdsFqt0FphtTgc26pmuVwVDd6p7U+mek1vFjHKEjm7EefufYxRWAxqxL6VTv+qqXhNRCtbBjyMthhTlepYBh+QpJm2NfQD5+s15+szhqGn7zqcT7KKEYSNoMp+ZI84VCSGVAFrJnVt0lxQj6KGqfK5BLYVjFhgBXksJM1gEAaFT9+FXOFmMaHccJz/6wgBPvaxjz3h6/nOxMHBAT/5kz/JP/kn/+Sqd+W5iE984hN84QtfuOrduLJ4bKX7Uz/1U5OONLJE9UG69bnBNcECZUk9Gj/u0p8uqGaREoFOiVrrYgU+1YjN2wDKkMaUfpYfz9SurHmw2Wxo25au6+iHgbZtOTs7Z7vd4lMTqGkalqsVzWKBtdUs4e5OcuXHxjHoSUNJZUgkf06ISboxHTxJ6KYWgRtlC6ygdYWxNcbWxY9u3EZMFWNH33fi/Ot7QpCEWirOOCbQqGJqSCqKD1oGaRlpXjO8dQYFQAwQfPo3MEucLgZc8AxOdDn6oaftO9quY9u2bFMlu9lsyjnIf5P/Wrbblp/6qf/v6/7Cvl1x69Yt/upf/atXvRvPRXzqU5/iR37kR656N64sHpt0f/AHfzAti9PwgICzZEp8yAKCE1wx460zfYb8eJy47CbO7zRZxulrifgoFkDTBJwr6CwZmTUQYhRhcmE1aCpriD6gEXPLPIHmkj25VhatLJWtWTSiq1BVCXKIY4Ms/v/tXb+PFMkV/qqqu+cH7GhA5jKfCFaWcXqO8UnESMiBJQuBEIE5+A9syTmBDRLBEV1+PieIk3WyLFvmctsJGTLODXhv92Znuqu6qhy8elXVs2tAtpixlvpWo92dnemp7en56tWr730PQJ31UJNSom5qOO+gjY6RPudikxKBjHWUyqVaaQKK50xKyCqoKaSCkqEnmxRBLQCY3qBtl9kEkKJUWtdz4iekehxNhunetLlFv62XDIfncITLBAxSqNjsPbDWhglAw2iNrqNJretaIteWiLVtO3SdRtfp+LPWBlqb2GLo8uXL/9VF+y4wn89x8+bNbQ+j4D3AG70X9vf3sbNzOmyEJWmRgoT14QPtcw1rIltWGkTzllypEKItfh2pkl3igJSySJPLd3OZGKcExuMxeeOuONLt0VuLSin0xmCxWGAynqKZN5iMp2Bvg7oehVy0oLxsBSjBUi5FVWq5bjiQkQxqjtj7TUgqVJD0PBqfiORNhSTp/nh8RT4KPGFJWUUrTCbW1arF3jd7ODjYh9YmRJ8uMLmLeWMb+tNBiLiZxT3rWB9N55bbBKeJEggm9fy+BIa2LtlwpkpBm0W+lCdOKwwX73fOD66H/D01xkRvj4ICxpMnT7C/v7/tYbxTvPaKr6oKn3/+BT65dQtWcENF9tKlUti0xB6mAmI6Id1J6YRjKpFihLtWpZa/Hj/GCvIm4L9z1RsTAkfAbCnZtit07YQ8bPECAG2IfefcuUDyDt4Po1RuApmPQwgiy1zelX8/bsyk5RXB0UxCKqqCI/9c8twVQsWWRGkFkHdkoBSFyie0KPViRW3KHQOhT10o713XT+fnnI9BkS/J+lhZweePU0h5eoE1zaSsYCJmZUIiY0AMCDeWAEuJ33zx20K4BUfw/Plz6vN3gvHa9IKUEj/6+GO8evUqtM9JS83oFpVpS/mDzCXAMiPQo85kPvr0sj1i3v4nJ728cotzuvkGmvceXdfFzRuucgOC3OrwMHb61VrDGBMfz80rmQB47Ozvq9YkXzzJ5KS7TsC8AaeUQqWaQTScyoDJgYz8F2iikEqFnHhO+LTkH08mmM/nmEwmqWW8zzpFZFEu22mmUt2hdGvwuw9dlm0P0/fobY9Oa3RGQxuNru1C+mB4o24bbUwhdF0Xzy0T8nqEy9+N6XHp0qUj5dDbxu7uLh4+fLjtYRSccLyxG3DTNPjz11/jJz++AtPrWFbgLIn8hRdh5zpJqYAQYWWkW68VV6i1dIIQAlUo5/XBBNxZCwhaylPhQzJSz4sKckWDkFRA0TQNutAUcjIZU4cFa9E0VZSDsXetdwJNU0PrLh63GY2wWq3QsH5WBWKULpBnDSnTJMKkqmRIpQRFgayo6y8pE1iVEfxbBbt6VVCSxHfOh47AUsJ5Sw0tvYubUdY69H3IcQtuvBzSA1zk4BG9MqKaLOpsqdtHlADGScxSKsHSGLwLhNznES5FtOzy5lwWa3OqCYnwhafrwwtAOgHnLerxGH/80x8wmUz+pwv3XWA6neL69evw3uPOnTvbHk7BCcVrQw3vyW5wd3cXo+k05Cypt1ldVZkAN0VTXIGWR1X8Ic1JkpF7GNhQYOHD80WlovyMSTrPG0f3sDWbybquo/TLGIPDxSG6toUxBsvVEtbZ2BGDIuMKJIaTaJpRbOc+nU4hGzIdz6Vk1lr4oH4g+VeTRbFV3BSrqxpSKMBLKFkBglIMHrSBxnlkLoagdENwB/OAsx59kL/1QfPati1MIDznj6YM+Iu312wQnHgPOEdpARNuvLHFN8OrgJYi2a7tSJWgOcKliJbzsdYmWRinFeIqhUldJLkdpMS5Dz7AhQvfP5Ji2jQePHhw7P3T6RQ3btzA/fv3NzyigsePH78XXTzeqhvwbDbDl1/+DtVoBB9yfUyiFL36wdL6dSmHHEyc3KHChQ9qb/ugi0gdiDmdwOMChh0sVivyyR2Px/SPBZIej8eDtjsvX77E3t4eEfByia4zsJY6/CpVAZ4Ivq5ZPqZQheU8bYZR8YFHGJOikoacbKWoIQRH9iJE4RJK1nHTkf4HCSmGBRZsKkTnR6DXPdq2xXK5xOFyCWN0POf5uWBQcQJFtjbkabnTBU806zci8y7c2pgq4O9aa+jexAj7uEac61pfNvphaZ3uDaSq8dVXv8eZM2e33q1hb2/vPxLrZDLB7du3cffu3Q2P6uTj4sWL+Oyzz47922KxSCZSJxhvJF2ANqu+++GH+PkvfgnnHIzWVP0V87CIRQlp6Tnc8V6XHfFjBtravoc2JpYae+9RBVJumjqSIUvF+IM+nU4xm82ws7NDFo5NA2ctuq7Dt98uYsqgbVucPXsW8/k8i8xdjKKVqgcbZ4l4ZSJKIQaWlXQXRa1cXUuROaUTeutilEmvmXqWkYqBjyupKadIuW2tNRYL0hV7AKpS1CsNwxLb9XytCxVhFNXaQXUY51zzvCwTbU6y/Hh+X1NEezRXm7brAnwohomVw7Ri+tW9X+P8+fPhHG/fS/fg4AD37t079m+j0Qg7OzsbHtHJR13XmM1m2x7GVvHW28eTyQRXr17FdDrFJ7d+hjOzGazro+ieiyJMcBc7KhFLm19sQM4376mSbVw3EKEyrO/p2Dzz9dnPknuKSVqWd6s2Vp1NRmNSMIDImjfsqqqCsxbLxRJ7/9rHctxiNBqhbir0zqBWFeqaWhMBZIVYx2aZgPMpsleVgjUuFigIyY5hIWUgUscL56lcVwgB4SxUTV11lWogISGRiFdK8txFKNU1WuPli3/iH8//jlcvXkB3XdrIHNaXJSIMqQTvkrlMnubJJ7xk25kmhXzD0zry2YX3oaSNfMmcEKgg0IdcPryHCsPhkTnvgWAQ1HUGnz78FFeuXImrmrfoiboRaK23PYSC9wxvlV4AiDRPnz6Nq1d/ir/89W/43oUf4JuDQ9jexZJgH56jwu5/bsrNOdH0oUOUdklJuVRVqUgedVWhFjKQYPKezY8Zo2WVcr1d12G1WkEqiaZpooQs32yTUmE6PYXRaBKaJJKS4NSpGZpmjLoeQ6kqmo4rpVBX1HGYx8uRqvNDVUY0rMmUD/zasqqDi1qoeqtqSO5EAawdh/9nVm4QgUbSDcoD51MFnrMefe9ilGoztUZMEwTPCo5g2cGM87SDohYkXTJtlvXwgdUtPFUsO8ojOEEbeXwNWA9o7fHRRz/E06dPce3aNczn82OvrYKCR48e4dmzZ9sexkYg1nNxBQUFBQXvDv8fa7yCgoKC9wSFdAsKCgo2iEK6BQUFBRtEId2CgoKCDaKQbkFBQcEGUUi3oKCgYIP4N6SqeECFO87qAAAAAElFTkSuQmCC\n" "<Figure size 432x288 with 2 Axes>"
]
}, },
"metadata": { "metadata": {
"needs_background": "light" "needs_background": "light"
} },
"output_type": "display_data"
} }
], ],
"source": [ "source": [
...@@ -446,7 +464,7 @@ ...@@ -446,7 +464,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 6, "execution_count": 25,
"metadata": { "metadata": {
"colab": {}, "colab": {},
"colab_type": "code", "colab_type": "code",
...@@ -509,7 +527,7 @@ ...@@ -509,7 +527,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 7, "execution_count": 26,
"metadata": { "metadata": {
"colab": {}, "colab": {},
"colab_type": "code", "colab_type": "code",
...@@ -569,7 +587,7 @@ ...@@ -569,7 +587,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 8, "execution_count": 27,
"metadata": { "metadata": {
"colab": {}, "colab": {},
"colab_type": "code", "colab_type": "code",
...@@ -591,7 +609,7 @@ ...@@ -591,7 +609,7 @@
" kernel_size=3, \n", " kernel_size=3, \n",
" padding='same')\n", " padding='same')\n",
" self.bn = paddle.nn.BatchNorm2d(out_channels)\n", " self.bn = paddle.nn.BatchNorm2d(out_channels)\n",
" self.upsample = paddle.nn.UpSample(scale_factor=2.0)\n", " self.upsample = paddle.nn.Upsample(scale_factor=2.0)\n",
" self.residual_conv = paddle.nn.Conv2d(in_channels, \n", " self.residual_conv = paddle.nn.Conv2d(in_channels, \n",
" out_channels, \n", " out_channels, \n",
" kernel_size=1, \n", " kernel_size=1, \n",
...@@ -630,7 +648,7 @@ ...@@ -630,7 +648,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 9, "execution_count": 30,
"metadata": { "metadata": {
"colab": {}, "colab": {},
"colab_type": "code", "colab_type": "code",
...@@ -638,9 +656,9 @@ ...@@ -638,9 +656,9 @@
}, },
"outputs": [], "outputs": [],
"source": [ "source": [
"class PetModel(paddle.nn.Layer):\n", "class PetNet(paddle.nn.Layer):\n",
" def __init__(self, num_classes):\n", " def __init__(self, num_classes):\n",
" super(PetModel, self).__init__()\n", " super(PetNet, self).__init__()\n",
"\n", "\n",
" self.conv_1 = paddle.nn.Conv2d(3, 32, \n", " self.conv_1 = paddle.nn.Conv2d(3, 32, \n",
" kernel_size=3,\n", " kernel_size=3,\n",
...@@ -706,7 +724,7 @@ ...@@ -706,7 +724,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 11, "execution_count": 31,
"metadata": { "metadata": {
"colab": { "colab": {
"base_uri": "https://localhost:8080/", "base_uri": "https://localhost:8080/",
...@@ -719,17 +737,56 @@ ...@@ -719,17 +737,56 @@
}, },
"outputs": [ "outputs": [
{ {
"output_type": "stream",
"name": "stdout", "name": "stdout",
"text": "--------------------------------------------------------------------------------\n Layer (type) Input Shape Output Shape Param #\n================================================================================\n Conv2d-22 [-1, 3, 160, 160] [-1, 32, 80, 80] 896\n BatchNorm2d-9 [-1, 32, 80, 80] [-1, 32, 80, 80] 64\n ReLU-9 [-1, 32, 80, 80] [-1, 32, 80, 80] 0\n ReLU-12 [-1, 256, 20, 20] [-1, 256, 20, 20] 0\n Conv2d-33 [-1, 128, 20, 20] [-1, 128, 20, 20] 1,152\n Conv2d-34 [-1, 128, 20, 20] [-1, 256, 20, 20] 33,024\nSeparableConv2d-11 [-1, 128, 20, 20] [-1, 256, 20, 20] 0\n BatchNorm2d-12 [-1, 256, 20, 20] [-1, 256, 20, 20] 512\n Conv2d-35 [-1, 256, 20, 20] [-1, 256, 20, 20] 2,304\n Conv2d-36 [-1, 256, 20, 20] [-1, 256, 20, 20] 65,792\nSeparableConv2d-12 [-1, 256, 20, 20] [-1, 256, 20, 20] 0\n MaxPool2d-6 [-1, 256, 20, 20] [-1, 256, 10, 10] 0\n Conv2d-37 [-1, 128, 20, 20] [-1, 256, 10, 10] 33,024\n Encoder-6 [-1, 128, 20, 20] [-1, 256, 10, 10] 0\n ReLU-16 [-1, 32, 80, 80] [-1, 32, 80, 80] 0\nConvTranspose2d-15 [-1, 64, 80, 80] [-1, 32, 80, 80] 18,464\n BatchNorm2d-16 [-1, 32, 80, 80] [-1, 32, 80, 80] 64\nConvTranspose2d-16 [-1, 32, 80, 80] [-1, 32, 80, 80] 9,248\n UpSample-8 [-1, 64, 80, 80] [-1, 64, 160, 160] 0\n Conv2d-41 [-1, 64, 160, 160] [-1, 32, 160, 160] 2,080\n Decoder-8 [-1, 64, 80, 80] [-1, 32, 160, 160] 0\n Conv2d-42 [-1, 32, 160, 160] [-1, 4, 160, 160] 1,156\n================================================================================\nTotal params: 167,780\nTrainable params: 167,780\nNon-trainable params: 0\n--------------------------------------------------------------------------------\nInput size (MB): 0.29\nForward/backward pass size (MB): 43.16\nParams size (MB): 0.64\nEstimated Total Size (MB): 44.10\n--------------------------------------------------------------------------------\n\n" "output_type": "stream",
"text": [
"--------------------------------------------------------------------------------\n",
" Layer (type) Input Shape Output Shape Param #\n",
"================================================================================\n",
" Conv2d-38 [-1, 3, 160, 160] [-1, 32, 80, 80] 896\n",
" BatchNorm2d-14 [-1, 32, 80, 80] [-1, 32, 80, 80] 128\n",
" ReLU-14 [-1, 32, 80, 80] [-1, 32, 80, 80] 0\n",
" ReLU-17 [-1, 256, 20, 20] [-1, 256, 20, 20] 0\n",
" Conv2d-49 [-1, 128, 20, 20] [-1, 128, 20, 20] 1,152\n",
" Conv2d-50 [-1, 128, 20, 20] [-1, 256, 20, 20] 33,024\n",
"SeparableConv2d-17 [-1, 128, 20, 20] [-1, 256, 20, 20] 0\n",
" BatchNorm2d-17 [-1, 256, 20, 20] [-1, 256, 20, 20] 1,024\n",
" Conv2d-51 [-1, 256, 20, 20] [-1, 256, 20, 20] 2,304\n",
" Conv2d-52 [-1, 256, 20, 20] [-1, 256, 20, 20] 65,792\n",
"SeparableConv2d-18 [-1, 256, 20, 20] [-1, 256, 20, 20] 0\n",
" MaxPool2d-9 [-1, 256, 20, 20] [-1, 256, 10, 10] 0\n",
" Conv2d-53 [-1, 128, 20, 20] [-1, 256, 10, 10] 33,024\n",
" Encoder-9 [-1, 128, 20, 20] [-1, 256, 10, 10] 0\n",
" ReLU-21 [-1, 32, 80, 80] [-1, 32, 80, 80] 0\n",
"ConvTranspose2d-17 [-1, 64, 80, 80] [-1, 32, 80, 80] 18,464\n",
" BatchNorm2d-21 [-1, 32, 80, 80] [-1, 32, 80, 80] 128\n",
"ConvTranspose2d-18 [-1, 32, 80, 80] [-1, 32, 80, 80] 9,248\n",
" Upsample-8 [-1, 64, 80, 80] [-1, 64, 160, 160] 0\n",
" Conv2d-57 [-1, 64, 160, 160] [-1, 32, 160, 160] 2,080\n",
" Decoder-9 [-1, 64, 80, 80] [-1, 32, 160, 160] 0\n",
" Conv2d-58 [-1, 32, 160, 160] [-1, 4, 160, 160] 1,156\n",
"================================================================================\n",
"Total params: 168,420\n",
"Trainable params: 167,140\n",
"Non-trainable params: 1,280\n",
"--------------------------------------------------------------------------------\n",
"Input size (MB): 0.29\n",
"Forward/backward pass size (MB): 43.16\n",
"Params size (MB): 0.64\n",
"Estimated Total Size (MB): 44.10\n",
"--------------------------------------------------------------------------------\n",
"\n"
]
}, },
{ {
"output_type": "execute_result",
"data": { "data": {
"text/plain": "{'total_params': 167780, 'trainable_params': 167780}" "text/plain": [
"{'total_params': 168420, 'trainable_params': 167140}"
]
}, },
"execution_count": 31,
"metadata": {}, "metadata": {},
"execution_count": 11 "output_type": "execute_result"
} }
], ],
"source": [ "source": [
...@@ -737,7 +794,7 @@ ...@@ -737,7 +794,7 @@
"\n", "\n",
"paddle.disable_static()\n", "paddle.disable_static()\n",
"num_classes = 4\n", "num_classes = 4\n",
"model = paddle.Model(PetModel(num_classes))\n", "model = paddle.Model(PetNet(num_classes))\n",
"model.summary((3, 160, 160))" "model.summary((3, 160, 160))"
] ]
}, },
...@@ -765,7 +822,7 @@ ...@@ -765,7 +822,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 13, "execution_count": 18,
"metadata": { "metadata": {
"colab": {}, "colab": {},
"colab_type": "code", "colab_type": "code",
...@@ -793,7 +850,7 @@ ...@@ -793,7 +850,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 13, "execution_count": 19,
"metadata": { "metadata": {
"colab": {}, "colab": {},
"colab_type": "code", "colab_type": "code",
...@@ -846,15 +903,12 @@ ...@@ -846,15 +903,12 @@
" epsilon=1e-07, \n", " epsilon=1e-07, \n",
" centered=False,\n", " centered=False,\n",
" parameters=model.parameters())\n", " parameters=model.parameters())\n",
"model = paddle.Model(PetModel(num_classes, model_tools))\n", "model = paddle.Model(PetModel(num_classes))\n",
"model.prepare(optim, \n", "model.prepare(optim, SoftmaxWithCrossEntropy())\n",
" SoftmaxWithCrossEntropy())\n",
"\n",
"model.fit(train_dataset, \n", "model.fit(train_dataset, \n",
" val_dataset, \n", " val_dataset, \n",
" epochs=EPOCHS, \n", " epochs=EPOCHS, \n",
" batch_size=BATCH_SIZE\n", " batch_size=BATCH_SIZE)"
")"
] ]
}, },
{ {
...@@ -887,7 +941,8 @@ ...@@ -887,7 +941,8 @@
"metadata": { "metadata": {
"colab": {}, "colab": {},
"colab_type": "code", "colab_type": "code",
"id": "Ur088_vjeSdR" "id": "Ur088_vjeSdR",
"tags": []
}, },
"outputs": [], "outputs": [],
"source": [ "source": [
...@@ -912,10 +967,12 @@ ...@@ -912,10 +967,12 @@
"metadata": { "metadata": {
"colab": {}, "colab": {},
"colab_type": "code", "colab_type": "code",
"id": "1mfaFkO5S1PU" "id": "1mfaFkO5S1PU",
"tags": []
}, },
"outputs": [], "outputs": [],
"source": [ "source": [
"print(len(predict_results))\n",
"plt.figure(figsize=(10, 10))\n", "plt.figure(figsize=(10, 10))\n",
"\n", "\n",
"i = 0\n", "i = 0\n",
...@@ -934,8 +991,9 @@ ...@@ -934,8 +991,9 @@
" plt.title('Label')\n", " plt.title('Label')\n",
" plt.axis(\"off\")\n", " plt.axis(\"off\")\n",
" \n", " \n",
" \n", " # 模型只有一个输出,所以我们通过predict_results[0]来取出1000个预测的结果\n",
" data = val_preds[0][mask_idx][0].transpose((1, 2, 0))\n", " # 映射原始图片的index来取出预测结果,提取mask进行展示\n",
" data = predict_results[0][mask_idx][0].transpose((1, 2, 0))\n",
" mask = np.argmax(data, axis=-1)\n", " mask = np.argmax(data, axis=-1)\n",
" mask = np.expand_dims(mask, axis=-1)\n", " mask = np.expand_dims(mask, axis=-1)\n",
"\n", "\n",
...@@ -961,7 +1019,7 @@ ...@@ -961,7 +1019,7 @@
"kernelspec": { "kernelspec": {
"display_name": "Python 3.7.4 64-bit", "display_name": "Python 3.7.4 64-bit",
"language": "python", "language": "python",
"name": "python_defaultSpec_1599452401282" "name": "python37464bitc4da1ac836094043840bff631bedbf7f"
}, },
"language_info": { "language_info": {
"codemirror_mode": { "codemirror_mode": {
...@@ -973,9 +1031,9 @@ ...@@ -973,9 +1031,9 @@
"name": "python", "name": "python",
"nbconvert_exporter": "python", "nbconvert_exporter": "python",
"pygments_lexer": "ipython3", "pygments_lexer": "ipython3",
"version": "3.7.4-final" "version": "3.7.4"
} }
}, },
"nbformat": 4, "nbformat": 4,
"nbformat_minor": 1 "nbformat_minor": 4
} }
\ No newline at end of file
基于U型语义分割模型实现的宠物图像分割 基于U-Net卷积神经网络实现宠物图像分割
===================================== =====================================
本示例教程当前是基于2.0-beta版本Paddle做的案例实现,未来会随着2.0的系列版本发布进行升级。 本示例教程当前是基于2.0-beta版本Paddle做的案例实现,未来会随着2.0的系列版本发布进行升级。
...@@ -33,7 +33,7 @@ ...@@ -33,7 +33,7 @@
.. parsed-literal:: .. parsed-literal::
'0.0.0' '2.0.0-beta0'
...@@ -65,6 +65,17 @@ Pet数据集,官网:https://www.robots.ox.ac.uk/~vgg/data/pets 。 ...@@ -65,6 +65,17 @@ Pet数据集,官网:https://www.robots.ox.ac.uk/~vgg/data/pets 。
!tar -xf images.tar.gz !tar -xf images.tar.gz
!tar -xf annotations.tar.gz !tar -xf annotations.tar.gz
.. parsed-literal::
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 755M 100 755M 0 0 1707k 0 0:07:32 0:07:32 --:--:-- 2865k0 0:12:48 524k 0 0:13:34 0:02:41 0:10:53 668k 0 0:12:45 0:03:06 0:09:39 1702k 0 1221k 0 0:10:33 0:03:25 0:07:08 3108k37 282M 0 0 1243k 0 0:10:21 0:03:52 0:06:29 719k0:05:53 566k0 1237k 0 0:10:25 0:04:43 0:05:42 1593k 0 0:09:46 0:05:28 0:04:18 2952k 1467k 0 0:08:47 0:06:43 0:02:04 1711k
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 18.2M 100 18.2M 0 0 1602k 0 0:00:11 0:00:11 --:--:-- 3226k
3.2 数据集概览 3.2 数据集概览
~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~
...@@ -89,10 +100,10 @@ Pet数据集,官网:https://www.robots.ox.ac.uk/~vgg/data/pets 。 ...@@ -89,10 +100,10 @@ Pet数据集,官网:https://www.robots.ox.ac.uk/~vgg/data/pets 。
├── test.txt ├── test.txt
├── trainval.txt ├── trainval.txt
├── trimaps ├── trimaps
   ├── Abyssinian_1.png ├── Abyssinian_1.png
   ├── Abyssinian_10.png ├── Abyssinian_10.png
   ├── ...... ├── ......
    └── yorkshire_terrier_99.png └── yorkshire_terrier_99.png
└── xmls └── xmls
├── Abyssinian_1.xml ├── Abyssinian_1.xml
├── Abyssinian_10.xml ├── Abyssinian_10.xml
...@@ -315,7 +326,7 @@ DataLoader(多进程数据集加载)。 ...@@ -315,7 +326,7 @@ DataLoader(多进程数据集加载)。
.. image:: pets_image_segmentation_U_Net_like_files/pets_image_segmentation_U_Net_like_12_0.svg .. image:: https://github.com/PaddlePaddle/FluidDoc/tree/develop/doc/paddle/tutorial/cv_case/image_segmentation/pets_image_segmentation_U_Net_like_files/pets_image_segmentation_U_Net_like_001.png?raw=true
4.模型组网 4.模型组网
...@@ -436,7 +447,7 @@ Layer类,整个过程是把\ ``filter_size * filter_size * num_filters``\ 的C ...@@ -436,7 +447,7 @@ Layer类,整个过程是把\ ``filter_size * filter_size * num_filters``\ 的C
kernel_size=3, kernel_size=3,
padding='same') padding='same')
self.bn = paddle.nn.BatchNorm2d(out_channels) self.bn = paddle.nn.BatchNorm2d(out_channels)
self.upsample = paddle.nn.UpSample(scale_factor=2.0) self.upsample = paddle.nn.Upsample(scale_factor=2.0)
self.residual_conv = paddle.nn.Conv2d(in_channels, self.residual_conv = paddle.nn.Conv2d(in_channels,
out_channels, out_channels,
kernel_size=1, kernel_size=1,
...@@ -467,9 +478,9 @@ Layer类,整个过程是把\ ``filter_size * filter_size * num_filters``\ 的C ...@@ -467,9 +478,9 @@ Layer类,整个过程是把\ ``filter_size * filter_size * num_filters``\ 的C
.. code:: ipython3 .. code:: ipython3
class PetModel(paddle.nn.Layer): class PetNet(paddle.nn.Layer):
def __init__(self, num_classes): def __init__(self, num_classes):
super(PetModel, self).__init__() super(PetNet, self).__init__()
self.conv_1 = paddle.nn.Conv2d(3, 32, self.conv_1 = paddle.nn.Conv2d(3, 32,
kernel_size=3, kernel_size=3,
...@@ -531,7 +542,7 @@ Layer类,整个过程是把\ ``filter_size * filter_size * num_filters``\ 的C ...@@ -531,7 +542,7 @@ Layer类,整个过程是把\ ``filter_size * filter_size * num_filters``\ 的C
paddle.disable_static() paddle.disable_static()
num_classes = 4 num_classes = 4
model = paddle.Model(PetModel(num_classes)) model = paddle.Model(PetNet(num_classes))
model.summary((3, 160, 160)) model.summary((3, 160, 160))
...@@ -540,32 +551,32 @@ Layer类,整个过程是把\ ``filter_size * filter_size * num_filters``\ 的C ...@@ -540,32 +551,32 @@ Layer类,整个过程是把\ ``filter_size * filter_size * num_filters``\ 的C
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
Layer (type) Input Shape Output Shape Param # Layer (type) Input Shape Output Shape Param #
================================================================================ ================================================================================
Conv2d-22 [-1, 3, 160, 160] [-1, 32, 80, 80] 896 Conv2d-38 [-1, 3, 160, 160] [-1, 32, 80, 80] 896
BatchNorm2d-9 [-1, 32, 80, 80] [-1, 32, 80, 80] 64 BatchNorm2d-14 [-1, 32, 80, 80] [-1, 32, 80, 80] 128
ReLU-9 [-1, 32, 80, 80] [-1, 32, 80, 80] 0 ReLU-14 [-1, 32, 80, 80] [-1, 32, 80, 80] 0
ReLU-12 [-1, 256, 20, 20] [-1, 256, 20, 20] 0 ReLU-17 [-1, 256, 20, 20] [-1, 256, 20, 20] 0
Conv2d-33 [-1, 128, 20, 20] [-1, 128, 20, 20] 1,152 Conv2d-49 [-1, 128, 20, 20] [-1, 128, 20, 20] 1,152
Conv2d-34 [-1, 128, 20, 20] [-1, 256, 20, 20] 33,024 Conv2d-50 [-1, 128, 20, 20] [-1, 256, 20, 20] 33,024
SeparableConv2d-11 [-1, 128, 20, 20] [-1, 256, 20, 20] 0 SeparableConv2d-17 [-1, 128, 20, 20] [-1, 256, 20, 20] 0
BatchNorm2d-12 [-1, 256, 20, 20] [-1, 256, 20, 20] 512 BatchNorm2d-17 [-1, 256, 20, 20] [-1, 256, 20, 20] 1,024
Conv2d-35 [-1, 256, 20, 20] [-1, 256, 20, 20] 2,304 Conv2d-51 [-1, 256, 20, 20] [-1, 256, 20, 20] 2,304
Conv2d-36 [-1, 256, 20, 20] [-1, 256, 20, 20] 65,792 Conv2d-52 [-1, 256, 20, 20] [-1, 256, 20, 20] 65,792
SeparableConv2d-12 [-1, 256, 20, 20] [-1, 256, 20, 20] 0 SeparableConv2d-18 [-1, 256, 20, 20] [-1, 256, 20, 20] 0
MaxPool2d-6 [-1, 256, 20, 20] [-1, 256, 10, 10] 0 MaxPool2d-9 [-1, 256, 20, 20] [-1, 256, 10, 10] 0
Conv2d-37 [-1, 128, 20, 20] [-1, 256, 10, 10] 33,024 Conv2d-53 [-1, 128, 20, 20] [-1, 256, 10, 10] 33,024
Encoder-6 [-1, 128, 20, 20] [-1, 256, 10, 10] 0 Encoder-9 [-1, 128, 20, 20] [-1, 256, 10, 10] 0
ReLU-16 [-1, 32, 80, 80] [-1, 32, 80, 80] 0 ReLU-21 [-1, 32, 80, 80] [-1, 32, 80, 80] 0
ConvTranspose2d-15 [-1, 64, 80, 80] [-1, 32, 80, 80] 18,464 ConvTranspose2d-17 [-1, 64, 80, 80] [-1, 32, 80, 80] 18,464
BatchNorm2d-16 [-1, 32, 80, 80] [-1, 32, 80, 80] 64 BatchNorm2d-21 [-1, 32, 80, 80] [-1, 32, 80, 80] 128
ConvTranspose2d-16 [-1, 32, 80, 80] [-1, 32, 80, 80] 9,248 ConvTranspose2d-18 [-1, 32, 80, 80] [-1, 32, 80, 80] 9,248
UpSample-8 [-1, 64, 80, 80] [-1, 64, 160, 160] 0 Upsample-8 [-1, 64, 80, 80] [-1, 64, 160, 160] 0
Conv2d-41 [-1, 64, 160, 160] [-1, 32, 160, 160] 2,080 Conv2d-57 [-1, 64, 160, 160] [-1, 32, 160, 160] 2,080
Decoder-8 [-1, 64, 80, 80] [-1, 32, 160, 160] 0 Decoder-9 [-1, 64, 80, 80] [-1, 32, 160, 160] 0
Conv2d-42 [-1, 32, 160, 160] [-1, 4, 160, 160] 1,156 Conv2d-58 [-1, 32, 160, 160] [-1, 4, 160, 160] 1,156
================================================================================ ================================================================================
Total params: 167,780 Total params: 168,420
Trainable params: 167,780 Trainable params: 167,140
Non-trainable params: 0 Non-trainable params: 1,280
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
Input size (MB): 0.29 Input size (MB): 0.29
Forward/backward pass size (MB): 43.16 Forward/backward pass size (MB): 43.16
...@@ -579,7 +590,7 @@ Layer类,整个过程是把\ ``filter_size * filter_size * num_filters``\ 的C ...@@ -579,7 +590,7 @@ Layer类,整个过程是把\ ``filter_size * filter_size * num_filters``\ 的C
.. parsed-literal:: .. parsed-literal::
{'total_params': 167780, 'trainable_params': 167780} {'total_params': 168420, 'trainable_params': 167140}
...@@ -629,15 +640,12 @@ Layer类,整个过程是把\ ``filter_size * filter_size * num_filters``\ 的C ...@@ -629,15 +640,12 @@ Layer类,整个过程是把\ ``filter_size * filter_size * num_filters``\ 的C
epsilon=1e-07, epsilon=1e-07,
centered=False, centered=False,
parameters=model.parameters()) parameters=model.parameters())
model = paddle.Model(PetModel(num_classes, model_tools)) model = paddle.Model(PetModel(num_classes))
model.prepare(optim, model.prepare(optim, SoftmaxWithCrossEntropy())
SoftmaxWithCrossEntropy())
model.fit(train_dataset, model.fit(train_dataset,
val_dataset, val_dataset,
epochs=EPOCHS, epochs=EPOCHS,
batch_size=BATCH_SIZE batch_size=BATCH_SIZE)
)
6.模型预测 6.模型预测
---------- ----------
...@@ -660,6 +668,7 @@ Layer类,整个过程是把\ ``filter_size * filter_size * num_filters``\ 的C ...@@ -660,6 +668,7 @@ Layer类,整个过程是把\ ``filter_size * filter_size * num_filters``\ 的C
.. code:: ipython3 .. code:: ipython3
print(len(predict_results))
plt.figure(figsize=(10, 10)) plt.figure(figsize=(10, 10))
i = 0 i = 0
...@@ -678,8 +687,9 @@ Layer类,整个过程是把\ ``filter_size * filter_size * num_filters``\ 的C ...@@ -678,8 +687,9 @@ Layer类,整个过程是把\ ``filter_size * filter_size * num_filters``\ 的C
plt.title('Label') plt.title('Label')
plt.axis("off") plt.axis("off")
# 模型只有一个输出,所以我们通过predict_results[0]来取出1000个预测的结果
data = val_preds[0][mask_idx][0].transpose((1, 2, 0)) # 映射原始图片的index来取出预测结果,提取mask进行展示
data = predict_results[0][mask_idx][0].transpose((1, 2, 0))
mask = np.argmax(data, axis=-1) mask = np.argmax(data, axis=-1)
mask = np.expand_dims(mask, axis=-1) mask = np.expand_dims(mask, axis=-1)
......
<?xml version="1.0" encoding="utf-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Created with matplotlib (https://matplotlib.org/) -->
<svg height="181.699943pt" version="1.1" viewBox="0 0 349.2 181.699943" width="349.2pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<defs>
<style type="text/css">
*{stroke-linecap:butt;stroke-linejoin:round;}
</style>
</defs>
<g id="figure_1">
<g id="patch_1">
<path d="M 0 181.699943
L 349.2 181.699943
L 349.2 0
L 0 0
z
" style="fill:none;"/>
</g>
<g id="axes_1">
<g clip-path="url(#p58ad9a7e6d)">
<image height="153" id="image6a21407320" transform="scale(1 -1)translate(0 -153)" width="153" x="7.2" xlink:href="data:image/png;base64,
iVBORw0KGgoAAAANSUhEUgAAAJkAAACZCAYAAAA8XJi6AAAABHNCSVQICAgIfAhkiAAAIABJREFUeJx0vUuvZUl2HvatiNiP87qPzKysKnWzKTatNgk0YNMUQMCgKcMaGIIhzSRrbI0kAx7YHhmGQcCA/R/kIUH4B/SYgDywKUpNWObAzZdksLurWZVZlY9773nsvSNiebDWioh9MusUbuW955z9ilixHt/61gpiZgYAZi4/OUf8+Z//Jf7ZP/2n+MM//EPsNiOICJkAIgIAeO/RhYCcM3LOYGaEEMoPAKSUkHNGSgnee/RdD0dAYkbOGY4IOSbAETKzXCNn2Mt+DyHAM5Dsb+fhnEPXd2BkvQ4wDAN2ux3unz3HJ59+jtu7W3z66ae4u3+G3X6DcRxx2N8ALPcLMMgRuhDKcy1xwvl8xjAMiDGCiNB1PbwL6LoORA7eB3gv90BEYCwAE7puhPcdyANdGGQsfA/nHch5gKhcx16Pj0f827/8C/w//+b/xr/7t3+Kr776Am/fvcWyzAAnvU+UMQYTUuIyrgAQY9R5k/dSkuNs/OVYIKX84RgzA5C/iQjIDAJkrgH5XyIwMQgM1uOI5D5SBn77t/9j/PN//r/h+9//1dW42LM6Eyw78OnpCb//+/87fvM/+g38+U/+X9zd7OCDAzk5iFQgU4yY57k8kA1E1PftxcxwziHnjHmekGIqN7DEiIUzlhjL8e0gmYA555CTDJZzDsMwYLPZIKeMeZ7LIMuDJx3IhNPpiGk6w3sCMyPGBcfjA+b5gmW5IKUogq7XXeIkg6L3K5PEcORXwsEqpDaRJowpJeS4AFkWas4ZKS7IMYFzM5HlPPbMWe/bgZxbTZQjB0fynvcezhNCcOi6Dn3fw4eAvu/L3/bjvZfFqf+GIIvkY0Jg8woQiAKIHEAEDwJngB0BIDgGPKoMeAL6nvDHf/xj/PCHP8Tv/d7v4d27d6u5Z2aEdlWdz2f8/u//Pv6n//F/wH4cscwzQienjTkj51Q0mE0GESGqkBARnHMAgMvlUs7bdV1ZaZdlRp5ZB5DkYWwAASSIQCzLIoLjPMAZw2YsA+gcsMwzGAnzPIPIYbsVbeu8x3a/xf2zW2w2W4TQYZkjdts9iIG4JIAZzgM+EGKaRRupkNok5Jj0GQnkGAAhpQznMxw6HcCElCMADyIPgMDOI2VCII+cM8gTCAyS2ULKGYEIRA4RjK7v8eKTl/iV7/8qjqcHvH3/dbkHzgRU3QEQIRGDiZFZhCR4D2YPzhmcM5wjhODR9x1ijEi6OOuiyfCeihakxGB9ftPuDICYEXWhOyIwgASAVL0RsyxQBmKc4R3hv/mv/xmmecI/+a/+CYZhUA3pEGxwY4z42U9/iv/1f/mf4ZyYBNabYpaTm1bxvq5s01QAyuCU1aFaQh5MjiHnEADEZSnmI6r2ciBAtYOZX7vO6XTC5XLBMAzYbkfM8wznPYZhwDAMmKYFfd9jHEe8efMGm+0W2+1OBcfpoBKADO97EQgGYlxWK9s5BzAjxojQdyqAAJBlhbP8LVpMhC8EB6L6zERcNCqRg3MZzA7EGQQHBsuYOoe+77Hf77HZbMQyxSQTCJLJx/plmteRg3cO6uuAsxPt4v1qzHPm8neMcSVw9v3EVRCJHYCMnOnqyld3YuNS3iYsy4L//r/97/B3/7O/ix/84AcyXoAIGQA8PDzg7//9/wJxmkRlq8pc0iKmIHPRRiZQJmSmCWylmE9mwmYmsOs60V7MCD4Uk+hVMB1IhE/PK5rElWtsNhuklHC5XGSw9XqXywVdN5Tvv3jxAvf39+i6DtvtFsPQwXuC84RljgB1SEmeQ+7Xg1OG7zoZfPXVCE5NNcN5MYGAU3OSQSB4HwCwPh8h5QVBfVVZcBmZEyiLIIIYmRzI2/gwQh8wjiO22y122y3OxyfVQqLJ7PmLoDknyyWzahkCqztjc2Q+nAmZLFoTQC4Cx8yIWX23mPQYrBSEzaN6aVXsyMRLfulCh5gW/L2/95/jxz/+Y9ze3om5JhIJ/Mu//EtMpxPIEThlRLD4EU6lleRkZlLswvZgzrniy9QBbgbFAobGFEEdfyZZ1akZHDsmpVS0ZyvQ5sfFGGVy9js450WwNlt4/b3rOjjnxP/yHgzxDZkZwzDgfD5jP44g1QBiNlTzOg+AkTkiR11YGSCKADkQJGDpOg/vgJQjiAJyWsSM5SxCkhOYgpg0ynAuiMZwEB/Le4zjiKB+1TiOiMsMTgs4E6qbrpPNMt2qx+BIJpxhLotei8X0VYFzSJmRU4YPHpwZmTNC1ADNJzCLEFZf06lwMZir+bZzEzOIJSjMOQOc8frVK/zkJ3+K3/qt3xJNRkSY5xn/6e/8DvbbDbwnIGfAOThPSOW06kPpTZMJURM4LDGWVWYrKsZYNJsJR1HNRCLUzMiqvjMYXgW5FSoAxQRxFsG7XC4ggpjO8wW393cg7zHPETnLypABznCe9H25J2Jg5kn+zRmd90CKxcQwZYAWMFczKveXxdVQ8+gdIScCsxwbfAcwwKwLiXsw5FlSVpeCHMAZyKRmz4PIYRxHbDYbeO8QgkNcnPiBDmCWcfAgMEn0BzVHxFXHlKjZm5kHOHhkZuRMcBnqkmRkNZPe+TK+rSk1vzhnNNpRfooGs/lhRnYA4LEsC/7O3/kdvH37DofDASHnjP/jX/wLPH/+HGmZVr4FiICcdHVXrWQ3VLSUPpxpuHbVdV1XBNP8hzZAsIFpNSBR9ZFMGC2qzDkjJy6wSMoZ3gfsdzt0XYfQRFvDIDDCsiwABdFmTv2z3JxDhTcE09Aowm1a1e7RtHXRKgAyMTyrJues2k6c8BgXuMBwzACCmHgQXLPwAAYRcDmf8e7dO5zPZ8zzLNchCRzkeq4iATrBOqgrc3oNk5if6MiJwGZG8BrZ5gwODPCHQmYLvQYPjJyT/mR1OdpouSqFrgv4gz/4A/yDf/APEGKM+Mf/+B/hcNgXB7869TaY8litSSy4WuujacTh9OZsQOxBnRNn1R6i/bz1wTy5IgAm0JvNpgjmPEWklCQwcB7juMEwjtjtdvjkxSd4+fIl7u7vixNM1JWHl4FM8FSDFLu2QAq8Eu6cM7quK460DWQ7RkRUtAJhAReNr4EAEzITPNWwPnOGgy0mJzhWWWRU/BybQvvb8EQHgFiQqxa7sn9bP85MLAjIJKYNENelA8SEqi9tiz3nVHy4lLKa0YyUHFKq77c+n/3Y+PyX/+gf4vHpiOC9x+3tLXKOooaZQXrXibOulmom68qgIhT2kolxQHFqZXjaVdHekA1gbnA6MSupnLed7MvlLOau+GkBLnTwoUPoOuz3e3R9h3mZcb6cELoDco5YFqDrNoD6mSkmMHHx1RhcgOXiP6oJBwg5iVbKjhB80PtKGmXaOHj1HwmEoMIq36Pk4dUtIOeQOcFlAjsufu5mM+L+7h43N7d4enyH6VKDHvG1xDn25CUwaQSLyJnXpOPGEDFEq2jALILlWEBdhmmVqq2L9soJXmGRVoullJFyQk4ZLTxy/QMAfS+4XPjRj36kbzo4D6QY4VnsPkGdW/U0i3CoRruOegCRR7uIDRBUuEwzFcHU6Ig8IbbnaGAVM7kVx3JYFkaMGUQJ4zAgI8N5CRwYQOg6mWCOyAwsMeMyEZwjLEtETgm+70UASHTEMs8IfV8EO86Lmt+u4kcNCEtlPMQxds7re4Y0VWiHc0aODATxcC3Q4SyYF5FEZuO4reaQxISJD+p0xcpsEAgQpAX2rupi+RqrdBGqniPSRa/3XnQjChTSZgJ8CMUsgvFRQZIIOBXFYShCxeSAH/3oR/B/8id/8rvT5byyp845gBp/C9W0tdmBVtCuTaP5AeVRuPwPOgfV/DIjN6q2YECN/0Y6SNVPEiT7cHODrutwc3OD7XaLw80N7u/vMY4DgvcgJ4FNToK620C0EbLdhw8VVhF4wsN7EyKNspr7MOECKograyeJgDiolrFjJPqCI4lO9YfBWOYZX3/9NV599dc4HY+Ies8g0VrVVzaEXv+jOqbOXftl9fvreTGfG5DHsu+56tbAAh4Byr3zCAqaWwbhY7/Xfx1CCPiXf/RHCH/2Z3+Gl588XwlJttDYiT/BWQdOb5QB0QaNo28mpn3llEFcQVtZwlUgqVlB16bYVkcrzMwSScqEStjvFNDcbrc4HA7Y7XYFNGYIThSXGZfTCeM4luAhaiRs17AI2DSuKwCuwikQvy6nrOPDBZUnX4MXG7fEEYhitkSjMzwEU3Ps1EeqC7brO+x3O+x3B3Rdj34YkJZFoBodW1nMBhslEHn52wmMYD51WSYNrIHm3/Yl46vngwdYrU2WeyfnkMFgx3AKQmfVjy3w2+Kkoskkwv+Lv/gLhO1mXGNTEFBUJUPVv7wnOI9Ifpkgu1msH8iiSedd0QwOkOiLSCEG1sFzIKqwhqHufd+DmXE4HHA6HQv+JIEDoRsGZAY22x1ubm+x3e3gvVMhEj9KXA5GXM6YKcNrHs6ASu9tEWj2gpwAnLnNPABECSBCilF9H4KDA5NMtj1zNZEAfGuCogK81bSBAXay+Lq+x92z53j2/CXev3+LlBbM04yYosAhBQvlAp9Uy+BV49H6faom8dsEDFAo1DAwE3yX4Vj8NTIr5mQmHct5Ja3UEiuyQiUJOXfgzHjx/B5hu92swvL2xjjL0sgsoCKcK6ujrHgiRclRLiiTJv9aNESaSuHMYEfCTCgwQdVigrCL1jRfbJomxChAIeeq3fqhx2a7x83NLTabTTlHzhEp5+KvIEd0ISDFBfN0wTBsyrVFG1NZlXAKz9j954ycHZgngAT9lzFwyJTBiHAOxafs+x4xJnjXlXEomjlnsEf17XIGuwwiLzDM4QYvXrzEN998hePxAY/v38sTrOAJVCFSYS+rhCrozWi/96FglQgUAKfGrTHzS+KSlKwCNWcrXyV4VBfGewlKUhYgmhKje/4MwTQSUMHOVkhSSgKYqvAVTWWRYvOeo5qhN6e9TKblNIsZy2ut11yv86FENUSE8/ksT5VFCwTfFZhkt9thHEcsizjqKSXMiznAOkA5Iae5mNwQegRep6wK0AvCUpLmxshIClVknXSLqLkAm1CBTjnDqw+Zc0bo3ErQXLPyzdclQvFt+nFEyozzNCHlJHhrXs/Jhy+upkTvw3xpXd6Kq314iAlc8asL/FF91m8T1Bb6EetgEa4XReEkK+G+7QTmbHtfhbDFY8yPKZQUtz5VYTOoCYnGfVqWwtoAPoRFWozHhNWA35wzOo32NpsNQKR0o1gS5SEEFWhR2zFKNDlNc/W9iDAvC5gqqp1SAscE1myA+RVl2iiAyMPSLC3u5tWE2ABdL5prf8VMM+dWaBh9P2C/2+Pu/h7b7VYS7x+ZG72KarVcHXlUraJ3vfrXAi4TQiq/lcf8IIAr1KDr6zdphho8ETon3D/nHJx36LogCfJiKjRctQNFnRK882oy9T4VQ7ILF4Q4hHLLhqyXFeyc5ODU0bT35boK4jrhTtnkFCpRmisoCsb+sBcHfl5wPh9xPo94enoCABwOe+ScscwLHLEEcgAcROCXnBFyxtBpXtTrmueMOTMCBXhICN9GjeRlEDmz+mu5DCyrv+ScQ5wXEfLAyOxAQTSu8wCTmtBiJllBVVlEQxdwczig77oCQzjykspaCRo3v9vilgDAnP8qVQp8qMbEh3qlkTEVOQtnIXQmOaeQFldSD0kL5hIKiNsETwjkinIIQGVWBk1bmCksgsBcMu6F5qOmxlYoAFDOAhvoU7YIudyf0Hfg1uxMqOC2qapxHEvKJ/MCRxL99X2PZVmQAXz3l76L7W6Hvu8xDAOICPMywxFEMFNEcB7IQo0Jviv3l1JG72tEKYi3JNNd8ICTxWURtGggeY6krgCrd78sC7Jvvqv/835NEqhRcvVbW63NBPjgcDw+4nQ+Fq3XLlQ5TiWBDT9rhbAK2MqXYwKYyuet5ms1YZ2Q5jvUwr+tjBX7LK6JZurtOS2DEOzmvfdAqimidtLJzuca+EFvoE1FfCxVtMJkIBDAkuKVWq+TYEK8LAsulwu894hLwjhWKs/zl5/AeY95WbBpzBIAeOcxL2ekFOGpwfHIwTWQSx86dM7XqFInsMAfihsVn22pxMwKVxhSnhFjBZzbScscRT/wWFI3Zg1WPxKuiaY3AuSV6WrH00BgkcIP538tMFfOfiNo1+7JB4fqe+b8X39GJNCGeaXtMTauwSaOgMKsaDGfVbSYuEAYdgLBlLyatVQnu9VicgKFLdSsxCponDJy45PZZIcQCi15ThFdEOrO5XIRnljfI3QB+8MW+8NWtdxUgo+UU2GODEMPy0AwMxInUBTNaGNn92wOMKBUGBfgGIhpKdE3q6kQn65OmPHmZPwqNSolRhcki9F3vboD6qYQaRbFY+i3+OTF38AnL77E+ekBcV7AuaZ9WqGTC1+9BzTjTh985gMJHtbM7fV32pfJRtL7XH+/vuXIQRA0Kr5mmUsb2JhSMZfXN2wn5LoExBZnXkVL8hChJMHXjqJgLiJUqZyb9eYF1vAg1lXTRKiu7+GIMG5G+L4rE9n3veTQYsZ0mRDnWfJqMYIoin/ne4S+QwaQmODJlYgveQY7oB/GmnBXYclJ7supwCQkpLxAsj1ONWyGDwExLgq3CByRUoIQHvXZjaER5Rpi7yzpHJWWI77qsBkx7kYB2ljZE+5b5qUZ3g8EEOpjkZNxX8EZRhn/NhVo0aplMFrTSwUYbrWtMNS9HOcdjGhARAhW9OFcLRYxE1rVuWJOJGYkmY8TPDzUbyOC7zpAheP64Y2BUXjntgLUue+HQc1pTY4DRoRj3N3fo+/7Alm8fPlSuGqKpR2fnoT/7wFwFEH2EoHFSOi6QaqN4DBPEX3vMPgRIYgzbmbffAl7z7RusmRxYrjgyoKySTVz07KCa8bCF1Np40uEoqVdZngn2jE44O72Bnf3z/H+3TdwLuF8OpUoeK196u9mps0KWaKckWR9N6aMRTy+RcCg6imWIKNCNvXztem2c9b78t6e00moYQxSO6hNZBeaNSrY6i2nmWqmwCtKb3+bT2eRZ05S9CGDxLAwKPhqFpm58M9Mk0kelTBNUkn08PAA5xxevXqF0/mEx8fHlXAQALCwBeY5YZ4t1eM1DydRo+XpTCicc1iWZeW02mdCRgwAPJwPxQ8j/cysgUXUtshYiZNGB7Jn+iDoySylaHqOzWaH29t7nM4XnM7nWlTzLZrneqxDEX6JCtfycYXzlQxChSJEoXgQBQDuo9dtQV+iSmhtn8nOFSx1A5QAtCRHS/Rj8ENzcos+PVCot2Zy7Hvt6hLnm5FyLnWXRIQlKzPTAVFNSFl1juD6Drf7Pe7u7tB1HTabDeZ5lrKwPoCYkaJo46enR3BO8MTwzqPf7tD3Uj4XQleSt1ZK1vUjYmIMvVREjX2HnBNc18N4UoBhXYDvenmOrMn2HOUeEbRsLQjAmsXPhKfCtEhpAdCtimMMpjBBIQJc8KAwgILD3f0dkCccnx4wz40mK35hDZpaQahCrGjYletTrksZUoMkaSmwCVa9LxG2gpCWqLposua85uYUQdRfi+P/sVcRDiVAtr6BPRgTVpScktgGIWaL+GraqgUq5fxipkOnEe2Syrk61TZW3zmOo2g3CoXEuMwzaBFsSnJwWUq5OldMlEEifT8WzeOVMdv3fVkIswKxobdJybAxawtnbBEx1zxrzhnzMotP2kaYTRK85ci1/Drnqt/rnMfNzQHeEU7HI87nCSnV2tUygWhA1mau2ijxW7WfsWOITARUgABmdXUowwDAKjfropbqDnzoM7b3FVrJM4FwqNqnlHa5dRhcByug04ruxDVV5BgIWpSaIQ7+0pgKmQiBNbxCCUbBCV78I3iHzWaD25sbEYJ5xmazwTRN2PKAFAVIJgjAGe2hg4frOjgX4JxH3w/FPBYh05+a3nLwvtK+r/3TVTTMAkzaeOac1R/0xSFnZoS+k6iL6+JaO+7VkxG/xoFI8sHPnj3Hy08/gwPjzTevSvFLIydQMGuV2lvJ0rUGU0ExAiSp9gKtPxehEY9OJMytzvWt2NrVde334qWGIGG6DVqrcUgwyCJwzjmNJmqkkWMC+yZoAABHWJRX3/pv9WahbFyhInddJ8wGtedOTZzlJU3rbLaDUEkWE1gHT4S+60Deo+t6bDc77HZ7DINosK7r64qzcL5UU6vAqzklZJCv97zWXlZwTCWT0Gl9pnfCXwMBoRuQEzRT4Ao7xLRrO6lVeOXvYRiwGXeIMeN0mUTw3Lp6i5P4hEytdjWhNU22Npf1OwZPAUBeaUTzK733SkbwV5GrCZwdw8XnLE90pdWcre6WT3UtqSlpYahGEQ5rPyArBcjOZWAmoJpRI9frouCcpepJtKRQfIvbQEDX9wihx/5wKMlvcawXnDTiOp2OhbfPKih91yt3bMQwbtCZloFgga2wk2kCu08SwfDewanpuzY7YgYdyAfxV3MC4BBCB8Fora2AlPxLMENF4LTSpEyWaDHJtIAYfR/w/JOX+M53fgm3t7eFN2eTN46jAONurUm+HZJAyVMSrn20NT/fBE3ID1Ug68sYwkl/lJ61huVWr2BIe9JcZGvPq2kQjeq8IvJYh8oldE65lLgVHyJrGZlzyE3hLjOv/BxmQtAolYhAwWO320ui2HuwM56Y8KsMchjHDZAYFAJCN2Cz3WG736MfNwhNpVTbDOZjfoP9G2NC1wvijmbhtK0YdNaEceBr8tjK/yxCJkIDzKJQYYh84/TrtQEx0SyTXFIyyjK152iFrQpFOcOV1llrnIparDXNt2khO74k2qmei1VpEonQGbGq/Z7JQTAVDAa8rWp8+CqOJtXV0GoyU/vgmoohImROYjaoIuJ2cYMQRGgVPjFB8KIhlmWGpQCtkYhzXTE9RMJNk4KSgO1uj34YETptRtKYyZbxYf+KaZe8pLKrwCzXNoLhNT5VJsK3KLzCLx+ZMEC5cRDAWWCUdelfTU4LxOB9gA8BKWdE1fgmeO0YtsHAh84366xp7tGV3z7y+jBIIPOTqH0WKsJVjnMEcC2NtJdZC1e0EEsNHtxagMQ0VKYBmsmyi9hKKhcg6aoTU1LU+mMOKMp1RUOSIN8WxqsZSnHBNAmfzDSZXV9oO4DvgvaT2ElS22mbhaZvxfX1bSGIeyCNUzgnEGoLpsy1dM9+Wqo4WPAg86ey4WGo1zOsL2l3IO+CNEtxYSVkOn1aGyla/MWLT/Ds2TNs98I6kcBE4BDRkKlkFa61kf0HCJjtwPBAYbm2gmSQxMcEzcxYKSRXig+3X2BVTKqFGXklPyW6bM0I5YpjGRZiAF7OuQBv8n0Rde+9ZSHqZLCE5ERr6KJw1UJAUL+mpUWXlBQzkqtRYdBCD6kDTBgG8VWGccB2u8M4bopmtOtYCN5Gy/ZZMT+Z4b2luDyQZdQyV46+PXtLtgTLMS54jfIU8sgZcB4hdGXcylhq8YhFd/V982tEu0pxshAYY0xSZdWQEHa7DWKcYXanNelF2OTpi0QQVR8UH3zTAoFrc9t8S8fww+NNXyrwTWvcLngX4ByrGVInNNZEd1nFrR1fXZzKzXu9UVYBu27Kcm2mgg/oukGqelB9DhvMw+FQwNPW+XYuIGcRni4M2Ix7eN+h6waE4BFCBYFNoIC6cJjFtEmyWlJhTIxM1echEuGjRlPIfVfcqEAcaFNwddHmzOh6mRTvG8YDoLBB9aOMnm7aRJD/rVZuyTh5cjhfhDfXadkfc0sBaubIgmg25aOmXxdvK0TFmuVatng9b+34F7NejoVaLNFiaMZex6LTiTDbzUiUV5PUYj8EiDNPVdOxmVGuk1HyelcPX6JKIvjg4TvBhZBZtVotUdtst1iWpTjwlhi3ye66Af2wUc6+aRwTRK9atNJb2meyMF20lSTEfR9kcTgCHEBtHwwogBklhUIWiV4NaOYoGQySQAgsTFxZzEHXp2jIGuGiOtdmGULAzc0dXrz4BI+P75HTjOPDUibashDQhR0Lx60FTGVGMymLyIT5Soiuo+dWAA2Qrn+jWLdGGmE20zBWNGSLUCdfzENhqLZZgManMtttDrG8XFFuNqF2bQfpSrN6eJ0kS36DCD44OCdcq2Hoiwa7JvsZJXq332Icd9jvD8W8mKl1hRRZ77elGZnglESyCrw53BZVrgZcJ420eMYWg0Ef1kdDil20i5ALYDjNe1K9H98hCfSKYB4HZ2SWCihycj/7/QGb7RYxRVymCXO8IGtbp0U77nCWcrp5vqy0DsoVFYss9ZtVk1UZuQ4YrrG3MhKNoLVvWzCxVkY236HFRExw6Ip810Zn1xBHkcEGd5LKzSxsFRaud9ty03tfqrOdZgvIAT54zTG6lUDYKrXBkc89NttDiTjtJUJpKZvaLbEV1FajMqO4CYD0YRNKtrJIc/VlWgG1CanZA8lbOgc41wHk4PX5KuxhAYN2/Cn4ogOz5E9hpg5OcL5hg5vDDabTE5bpjLic4X3QCnqJVqWQZirz8q2BlmljkQuYuhAsULoIrJABZYu0FsLOZHhD0aao1s857emmrbfCitGqDYK9qsPr/hVr+g8XQcyo7At7AG9FqDkjJQVxLROg55aIKYgQ2Dw0CPg8z9IWSrP5xV9yPcgHzPOCoe8KKwKotG4iIRR6XxF2G7gqjMZ7A8gRUloQgpa0eY/OBy170yip5ZvlrAxW7cnRCN4wDHDBwbluBUBn7fGR84LgAsDSU4J8EAOahfuVIXhj3wfc3NwjhB7zkopgAdAeHnLOlGJxyE0Urs14ealGZ+s7BxTtX/AvfOj0ryNPM1VXste+mpxtyW9Y7jE4r30asjAJmpteRQwa6dnKtldMCR0BUfEUdOKPUM5SgQwUjr8Uh4ipDZ1OICplyJoOExEG7VMh6aUB3TCUwtwKCGdYHaIIk5igGOOqpN6unXNG14tJY4ZGR7rokhAVyResB6WdZ5b+r6QNWCywIB8kfeVxU0SAAAAgAElEQVQlgux7YXNI9yFGigk5RYCCCBJLWiqlKM+SJUKTajRpZfXs2TPs9zfYbnfSmfsxIqUFm+0epxPAfMH5fAFlBgoiYEU70ABgbQZl/iud2twlA1aJDH+rQRDQpo/kxDW/e+WXFj6bNKwJLbc9G15CFu7nYt5aU2OrpLR2aoLa4KVwwySfAaFXp4QMIfoFLdAwjRCCFmGoiTNYwXAxsPDRfAjohwH9OKLvOvRDb0OnAmU+mWonApgTKFTBK/WVgjWA2WnXHmFzJmOOeAdmi6DMB2FF7F0xwdYnAiCxAKTWAFDhFuCYIIXJGlrJ9SQUh0AZDvBN2kcX3jAMONzc4HA4YIkXUMyY5iPiMmFaJtzutzidTghkRSUS3cN/zL9q3gNDu+qs3q8KQ90Fg+Gppfh86DK1EW6rkJgBVzpPO+n4xwC60MPDSejb+CA1oUyrNM0QahUQEWEx05kyXJIrFT+oQasBRoyzVPvoyokxFTOZc8bxeGxQbmCzkV4X2bA0R5hjlIGAQ4wL5lnap7eAq73a3miWAkuJpd9HAVolUOBkxTR19TJntKwFOYZLKsgGdxxHWGJbzLIKOIQCTgzkmMr1gFQiOSvqdeSECTxI77UYI6b5IkDxEvHZp5/i5uZGLEWJglvYqUW9muDLBM6t368vIzbac1/7dx/ayetz2Lgxs6SVWnLhtttIZj9luK4H0rLSWtcnLf6Zc0iwgUQ5JqeMxvyvXjFGDQCEFx85FuGw1qCDMi+89xg2G/gulIkzJqskkC26q8W0FfOqlCUocdK7ToQIQN/JphNgCwgYRAnkCch1wMqCW8EPNtjVJ+tChxRlIiwh7r2Hg91XbZDHnCCQJMFDesp6DkhRMiFjP4IQ8Fc/+6lE/46kMgqM49MJn336Qu6Jcw2QGnzs+kWAMjfWc7l+FhYLQJLua5UCAF1UH5lQGwlXmdEA4Nq26d577Kwt5tCLyiVhJJgW+9iPCal3Hh15OH3AnDMihAlrNyrXE4fZePfWA2yeLiVtZGtoWRYcj0eBNLoRm3G3EvIYI+bLpBhaKHSgFmiUgTHmgGorVsA5M+Z0KQyPZZlF63DCnGZkKO0pA1k13vViu6Yot8GFvW9RfNWqBq1oloClOV7Ki1RIkQMyo+t6PHvxAt//lb+FcdyBXNAF73A8X4Rp3AQ7BNtgQjZ3cFwxOACAd9IpGwDFDMq8us/2uaqWrXiZEDVbAqfAKG3A0JZZqtWrxQ3b7RYANOrTwlo4OKoA5rWUrl5ZuioXyIIAj8pHYgDeB93EQTIMcm2Z4JSl0UlSMxVCKNz+0+mE7f5Og4UAIJe2BLZil2UB0do/SCkVmCNl2Sug5eQDWCH2IjAijJaqccGjruJcFheg/P9QeWnMkk3IzAiharFpPorpKL1ixR8sfi1HOHYgHa+olU2h67U96TO8efMKp4cHxOUM52UxPz4+SgRvwlCg/pXIfLAAPKgENSsw1pHUG5Cl44C271k7BhKkYaUyK55Zd54J4ziu0i19P8BrysW5gOlyBsd5BWFY+qGtzTRTstqEINdWBlA/bhgGpcJUG55zRlwWZNUmhnNdsyfGcdRJovIdZmn3VAtRPiz1KvsLdHUFZmS4rPUJ0e5DBDIpxsMkGiVFE6y1Zqx7SGV4X8fhcrmgHwcscZIoXIOYTgtm6mQI+8N7D2L9UbMcgvLfgsd2s8f5dMG42eDlZ5/hm9eEx8f3uL+/w8O7N9LLbJ4+6DQONFqsrR0xYcwojOfy/UZTOzIsTAS3eqb6/fJlWmnLa5wutCsSAJwP2Iw7pJSw3QJPj+8xnY/IKa6iMxtoO6nhUtXRls0VpCUoF6wpJYb3WUiBDTfKJivnjGVaSqsEKR7usNvtVqaHcwVp2THGvgcjS2Bb2h7VxSM+lzShs6gXLoCRtQCiLhwBRxN6Ly2QMiVpLqx+VNtMpqVyt4vNFodBKa2w25jJ942NIbhaPbZCBKHr8Gu//uv4V//6LV69+iukRZrHPLu7xXR6wGYz4lHfa5+ZFEbSwHgdBNgcsmQeiOrnZBCF2HFJu7E6+xZum3nEt2nPKnCBHKHzlr902G52GDd9YxaAJ0dYLhfVbgnmS7RMjVbzyPcAcoxpyUIq1BL8EskayswVqmAGliUiLVH78UvkuT/cYrs7YKfm3NpGtry2lDJCkPSUDiOct4S2OLIxRd0JRTWwk7YG0hrK10FLSQqNUwK7APisQroWHhMu08rmYnRdhxzFR0SWjcNaQRTQOKl/Kh28ycn2M0iW0ZAUX+c8druxKIG7u1s8Pci1Ht69A1JE5we0O8C1RFJmlqZ1Sgtv9nsQ08hcqowKlZvEl+OcC4hehI9VaInhraUVJJfdClqrzQJzQoYVnoofsSy6Y0c/Yhw28I4wjyMulxNisytb65fVlUnaCC5imk8Az0gEJEh/L+9qIartRtY68eIWifk7nU6lM7ZsXzMU8p5MqgYkRBrtEFJUv4ehCeqEvh+gjb4Q0wznJckeXMC8XEoaCyQ5VRiUAOshbYvomqgJTNMF3gektKDvO4TQS2TmCIyo2r1mF0wDViFQ2jlkYy2ZxARiWYSk4/Xd7/4y/ta/9zX+dI6YzicQebz+6itsNzKGornjChivWhwgXtMVi5/MBsvIrnCZ7D0ICYBblVUamYJASHpGzYl8K/gfzuezSr+asyhFFPO8YBxH7T8RELoOA8ZyIkPRW2qOMSRsNzlQ1vZMTtp1s3Sfdj7BkTAScqMZsvYIiznJNjgxoR8H6dOv3LFe97Q8n8+a+5Q7Ej79OrqRJJWa86baygTl6ekJ/WCZhmaQyAGaMqpmuwY8a9gnA5RFE+UIRFk8/RCwLAkhdCUrUFNCdavFlCKIpRCGc0Z2snMJUVQqum69OG6w291oasnhcNji+PgeS5wUZ6xtutrImvS80AVzXT8LmI8lm4sV38sE1OhORYnUZtUmfeZzm9C2CgcAwuVyKkJiu3ewgrCX8xNCCNiMW0321obBVtJmKQgzlzlHhKBRGTn0XYclLmB28oQ5IUVp71RbK4k5y2kRDMbMoPO4u3sm2tRL0CD05Q7jKBiTI/H5pDq8ppfKas2VnSr37AA4bLcb3QjMwZO1UReN4IJHipX9Kg17SYthnDQCdur4luu5Mm6iyefC9QdM8xmo3RWBEDRC+4q43Ai39PgyQen7EZ++/BT7ww0up/f43vd+CT/76b9DzFGCJkB3PSl5CVCuPXizhoptUx0TNmHJAJ4ZFBnW1I2Za+ag+M4G8daggCFjQFeCVjTZdJFwGMyIYEzTRVUlCY3ZB1ymC4Z+wNAPyHGpoB8q+9Q5QuYoiPsyqXZKuqlolDIyJ2Bp3f4vrHwHyblFabgXZQIPhxvBxqhqJfH9rKSNwDlKV0KysjHbsCo3k1lbLxAcTqcTnJMMQxdqMDEMI5Y5fuDflIwFZzj4VRTNbLWXA0CKfWXVjU3Ng/lstiBli8QFxNrWqsyfOOIlgmXSApUBn3/+Ob7+6heYpsmc6hWyWmNKFRZeN0zJ1xFA0UcqHA2A3YpU8cXKe/VKlqUs52o0KTMjoKHvTMsMkDT2JSJk55F8B7csSDEK1147DdqE+SCDQ0RIOaLuvcMAazoIDKG3aC9VzrB9jlrzlWwFUhvFeTBVUzyOoxIOnSbXqURRBhhaAGKC1S6IlBKGYcASVSPr1i5934MBTJe5wgBY06OJSDtBWsoFxZckqpXodr5rLV+yI7421pPKJQdnW/MAyEmi8gKLaIbh/v4eLz/7HON2gy+++EK1kBTSiIBQ8ZFYgdiVIJFkMFrnqY3ufaPlWp+toP0F/jBdpoAyCIVCbMJKNZ0XrF3jOk2UAfWXwOYMJiSnBa+oobrPhj1B+z20N56QGIBGZlT4VA6cE1KMq8Z0tiJtqzzvBa/b7gTlt0Ajxhned5jnBd4DwQuTwfj45iuaYLW+lHMO0zTBaMuuk023GNA2CKl0+yayaLVJKZVd3arPY5kMu7++HyVIcITgrCyP1Xdbm3Q5vrYKsOS2nbdqbad7mTu8/PRz/PUXfwXZfxPwoattvWxhwKJBU0HVi4JmAdqmdq3T3qbMiOp8Fm1HpJ0b6/ull4UcuDLHjo1guCryFCKF7Hu4CEYWI+I8IcZFdvTVIlvZz3vGskwlNWOrt7RmN1dRL+ydVM8wJ3BOdWAUf6FmNFLOZTNV4Yd5jdgUpNXJysmKV7JCEjXSbYMT+1sAVOHhCyvX43w+q28hzq7swlE3tW81r3O+0H8cuaK9xcxFgTz02UsnIWe9Z2XhWbAl8IJsvmqdh6wARVI2spUiEWPoevzNv/mrWhUvfLXtdlu0kKhC8UWrECheRkItN+JDGWUd9zL/aMxkI5jt70LlVpmSyAFtzq21UAFYh7Orahz7jIHEtbe9jk5ZIUSGYuPDF6lWKCZGhJhJd0lrrm9gn++8Mm1lRY7jiGmaCsV6nmdsxhFSl7ku/Rfz3WJ4rgCzovqrX1QAS5CaWRMmE3ihhJv/VD8XDpz3TX8ymIYj5MRYENEFhxCqqS2usy1kzpLFSYuUAhY2Ra3ot/MzCz54e3uHn/3sZ5paE790v9+VDS5Mi/EHk9GUB5LBDlV+rlt12icWzJDha/SxL3Ld1BVrrQjUli4rAWnD0HIbTd9Rwa4YBvTajZOepzqOTgXw+rKyuZP0YZBaz5oPVCDUubKxvWlH2TQiYplnxLgICZBqBsLMkF2t9uyqC+dymYqGk4BAMhzLEgtVxzAxhiHvGUQBts9QSwywJV7frxkA0t6vy7IUYoBouooxRo6A43KsjX8brLS+3H6/xzgOZdyd1gM4cmVj+laL2PEFoVcznHKSvRmIyp7nzQElfSTAq/2vasdW2DJaIVVvrRGoUH+vN9hqtnYFrvCR9jjdHuWDtUOMUoxJ9QZUpwMcdfUFeKdVQMSIKaJ30s8CJI3vjC3y9PSE0HW4TBfxg6KE/hbh2f6b4FSyCpamGYYetTW6V0G3AhJLC2n1FEMCHA8Q+cbngQYlQSEd4bI5J71zQ9chOGHIsmZGxJci2bRCAUzOQrHOLEl7uyfRklyAVUb6cI2C4UOPYdzifDri7ds3YqLVFNpIswUA1jpC0cCiwTVsNJuRqSl709RAmbYipfK/GmHSSsEQc/G9s45VCQmupb88ThMMyCQ2UYaqfettbyAemn9FXqtTaslW+ztn8e88EYLiUM5LEYZttpVSwu3tLYgI0zRjf9hhibOW1AU4bz33A4LvwMoBM9qQ0Wn6fijPYzBMzhkGSK+eialoHKsSFxehrcZicNO23LQYKYU9eK/PImVxhh1KZlqO8Yr7eW9FJ5bH1HSV+soxRszzXAKOm9sbJSEwvvzyy6JhWE2hmEDSzEMbEaI2DgbUdzNZUStFVMilZn6LBmsUmZlZhiEObKZMtKFG/C7rAF4zKtqX/E2NILY6i8pF7fiPvVq/ixqhK5MFaejiQOAkNYt1o3WUpnc5Zwz9AO877LbSkGXcjGVLaNso3mg6tRVmjU4tV2rpsWVZZAvp5j7b7+bGoWXO2r7AgE47RnKAJedLrAUedcH1/QgBbT1CGOBDj9AN8G6Ac31x+E2zWhBQIIXMmKdJun6HDjkzXr58qQFQNYVyjkYiIIwK23ALzT21eU6DP3LOqz1N2zm0+4AplSZQK+aVa9gDwPZWqvSTjwmY3YDZ2mttJSGr4GD1b15/B6IdvJMw3ZxZkdkM5z26LiAuCeBYose4LOBhwNdff43vfOc7uFwumKYFn3/2HczzInABSbWTRZWAcOL6YUTX98gsTNh1Ul8E6ebmBk9PT8XnucbWzNSymmIp/tViYSfEQu8ZzndgOPTdAIKki5ZlAYPRdwO6vlPKD+D8IEIprSEBCCXboZJHq+kkkAvI2fzRRavfF8xzxHYrMAsvlRlry5hLYEZF6yBX/+t6rhlqUhlIxCpA60o0ZKUo6fMXlWOajVHwOtJgxNnFWom29z5mPlWK1p+35uNbvm8ApjmrAAozALBGI4QQdCsZPd/lcsHxeCytN6Py+bdb2YnksD9ontR46TZgKLv5Bj/A6h8rE0K+dT6fcblcStPkNulv38uNybS0VHUDCEwOcE7SRfBq2gSLm6YTYlxgEAFRRgiA8wxh587IeQawwBHKtj81RcaImRGz5FoFevG4e3aPzWaLV69fiSA5J1CGZkE8ETo46XZpvhih1DWguEGNL55Z9kVqp0w/K64S6nsy31xKIk2bAqrdliTM2/WKzR8VLBKdX4TRHNqVaeS2dOwqslGPwGr7Sq2nqtxVcUqnvokeG0LAsJFGKl999RUOBynoPZ3OIBIhZGYE1YTXjX+NuWpcMbvneV5wPl904lLx4Voz+m2mvy1IFsDVwyGDc0TOETFOmJcLpumsEfGCZZlwmZ4AygJELwvm5VLglJxZkt3xDCCVe0pJUnPESavUGX3fYb/b4z/8D34DwQekmFamrQRrGoFmQV81KOK6+0nzKjw31VjJsE6qi83wCfPzbH4dCJ6k3b4pjQRGdkB2qF19PjagK5PXaqnV/Zkl9gDSt+AoVfJlmzqpj7SIiiBgZj8MipZ7Kdl3hBhnZO1Wk3PG4XDA09MJIUx4/vwZHh4eMI4jzqcn2CaqhqdlizhLrCH3Zo30cmYpJ9Oqq8vlolVGlQhgfmAbKFgNp3dOMU81USSCEuMCzhGcGJwSljihyx7e9Zh0DwE4Qt+NcBCWcEoLUpoETOYMRwuksYzc/+V0whJnnM5HpBRxPs74V3/0f4GgjNuYChaac4YDVZIkObBmKqwdVpnXq1cp/iASv4tR5tRaK3xwVNacrnNgMmGtghjE91nn1q6FTrAnjVhY9/5hi2YFjuAPL90EDEDtNQrZPY09NpsD7m5vMQwjLsuM9w/vkXLGJ59+hvfvHwAi9H0nHDTncHNzg2laFONKmCahN1cHPup4qFb1knjuhx5x0d4RKjjTtKy0twi8kAitiupau9v3AGUSN+bB0lYxJ8QpYro8wrsgKaJiKgnDsFUWBiDaXarGyRPAVKPjnHFJp6Lp3j+8xTevv8ai5IObm3t0nUecGJ4cnr98hq9ev8ayLNhtd3h+/wyAaPplMQ2tOx53EkzkmAqV6VoTdpYlIZS9Tm3Sc/21QGit9QDpM0MgoVISF2Ozx3gjYEbfgTU2YzXWBrKyheQf98eMi28T8+zZc3Rdj/P5iOPxiL/+6hcAOfgu4Ob2Bp9++jmCk16vRB1ev36N7ei1P6yQ9V6+fIl377/GwwPwySef4NWrV3Ba/SwTJyYzJcG1s5rCzThgmuQ+eq1If/fuXUHXY4x4enrCdjuu2nDKudYN8YBaLOxcQAiidUGyQx05h8vlgv3trQQ8FErjY/NJC5uV9Z4hzV66Qbt/L4SHhwe8ffsW59OxuBaAw+3tvfzOwOV8wU+Pv8Av//IvIeeML//6S3zxxRfFr5NOiFLyRyQt63W160a4lYdmwhKjtEEw2k8bQUIF6xoybTFUKYlRzX93s/1dGUiDN9aiYpGlhdak1dKwCEYPJt8KFBXMZ1kW3N7e4nA4YJ5nvH//Fo+P7zBNZ91BLcMHiWKmacKbr1/jm69f49nz5/js8+/i/vlL3N/f4XyZsN/LHuPLMgutRid6t9thmSccj6eGBSLVT1JjKdHaze0t+i6g70eM44C+78uOJrvdBtvtviD3IdQ+ILZji5lJg0uqj2d+3oQUF5xPZ5yfztjfHjBspPs2uaBwR4UOCkN4BQPVegPnPDabLW72NzI233xTtE6MC96+eY3Hp0f8+g9/CALw87/6KeKylM5MNn/y7xp4MqAUgO6TtabSk0aJDhD6NlWqVdF4RQquXKtWLojg7253vyurqva7al/i9KOGpGyRZ/slwDY8ELUrJ3/+/Dm89zgej0KlVoapIerEHrIbvPllGSDpqvz+3Xv84ou/xs3NLW5v7/D97/8Knp6OePPmG+z3e9ze3uLNmzdwzuFwOODx8QHHp0cAFfdaloiYhNj44sULbDeb0kzFKqeISO/N0HFB3NuyN6kNrUGFJdmtuFiizSymaZ5wOT6hGwcMmw3GYaM9MkYAwDBUvr6ZqJp0D/ChQz+MpY0Xs/bQyBnbzRan0xmXyxkpLXj7zddw5PGnf/ZnWOYJgQhxiSpQNTXYQhBsbo9GhYCg/+VedJ6dr5uieucKRFGAW2ZYjUURgkbI2pe/u93/rqyedfXRStAg2hbFwa0SLF8QA+207nAcBTh9eHhYtZgUkLLRs5pechCcDETIWbRmhvDT3r17gzdvXmO33+P166/xt//2b+L9+/f4+c9/gc8//xt48+YbTNOE73znu+iHAfM8oQsBUSMuRx7b7R53d8/lvDmV5sAh9LIBa4wYhk1heVg0ahvQy9bT1iutmlvr1gh4xOWCGCdMpzOICIfbW4AY47BD1w0AdI8AqjQh6WYk/pnTnYFJiZQWceeU8PjwWFrc397c4Hh8wu6ww7/+l/8nhk2P4/ERvMwy/w30wSzQhoHEAMpWkqapDI5iZu2zC+3ZYQTPCnkY+i/B6VUPOxNd4prHVIXkb9VctgUO1xgZgVbN8T6QVhU676XlwOVyFgpNA2xW5xk1RVFiYvnXey9bBqaInBk5OaVuM77++mv8+7/2a/jxj3+MEHr89m//J9hstrhcztjvDzgcDpqXlCXX9z36YcQ4bjEMG3RhEH9H6zKdCwidFaQEbLdbPD4+qhkL+uNLFkHoQVLXYH31rQRwnhekZcJymZAzsNlssd3foBtGOHLowohhFKyu7wfllAlOxiQNZUiZIilleCddlRat1np4/yBFzpTx9u07AIzj6YSf//T/w9t33wC5MmRSkw4yANY0EKEgTTVKbP1vM5VqvURIFRVQbr+Bx60gty8u5xKBJUfwdzdiLk3STSjag0v+Ch/aXtJeWnKO1ERj1Vdp4QOwJVBbTZjVxNae+CBfsZqc4X2P169f4wc/+AG+971fxqtXXyHGiB/+8IcAgMfHB9zf3yFn0g2+ZIOGvh9U4HoMg2xlaL0tQif5Sds7oBS0KHXIgoO+78E5SstR9c1MCFOKOD09ImuE67seruuw3W4Rl4i+7zAOW5CTfmyAdOsmLUiOyyxlgNY3TAFfy6kejwJZKNcBr776Ep0nMDu8e/s1Hsy/bQK3MuEqYLaHaPs+NRJR57LpIWfaFiTt7plLXrOVj7U2QwkUHItFYlKczAa43MS1c9b8+WEmwAnD0TVOY6MNeaW1Kqq2GhDVbjlG9OOAlBLmWfaChKZ7xrHHi08+wePDA4L3+M53v4dXr1/hJz/5CQ6HPe7v7/H49ASpiL/Fu3dvEEJfiIkAME1zacFgq3EYBgTl3u92O3z11VcAS/9aoxrlnLDMM4a+R6c9yELwWOYF87xgmU+4nM7YHu4Q1JSmVN2GJUWMWjXf9z2YHChYxEagUCnUrG3hjcNPRHh8fIR3DsfTAzwxTqcnzAvhZz/9aWlQcz13IiBcdpHhRqCcqrZrZdJOuL0t0Xku1gyt49/ITdFsmcEkgBexlOiHFgv62AXl/aS+mLYSV0VFJmSwOvGmSqi95aK91vvyWOK6njBimYGuG9EFj2WRSGoct9jtdgCkUPabb77B669f4/nze9DtAT/7+c9we7vH+/fvsN/f4pe/9yu4vb3Fw8MDcmZsNmPpvTHPF+x2O0yXC+YpYxxHbMaxMDFevHiBd9+8LR3AnXNwTEDM6Dah0L5TSjg+PuFyesTx/XskIgw5wvktuqEvnLigHblzivBezKdXDI5DJSfID5cKrr7vcTmfkeMCoozz6YTpeML5fBZe2c0O4zjgzdv3FUpgmYGiW4zsQVQ3sWV12htFUQpcCpuiTJGaylqVtI5R168SPGRB+rNQcOEPu/F3vy1H2fpRBAh8QfWzahahuNm1/2VmtNZB2rnqNdrftUdaFt56TAtSygofiAN7OZ/w+PgOD49vcZkuWJYF3/veL4m/0w04Xy44nU64ubnD3d0dmKW/xzhucDgcAAjKv91uxeHX9u2mgb33iIsQI512CUIWIuRm3OrnImTLdMbTu7c4PR2xORzQDSO22x2cC9jttzAAehgGhK5H1w8qfIqGs6XYjDJuboRM6nQ+43R8BIgQ5wVvv3mL4+lJ22wlvP36FY7HR0B3lGHmZlOPOk9mKhmAbwDkNqMBQBWFzXULrTSTVGxpmTU50jUWSv1izqIBPzCX13nHD/OQKMLVKiytM/rosbXesUGOG4HMmsMEaW8G4kJKDJ1TQU04Hh8RZ+my44LHOIxgZvziF7/A559/B8OwVSIi4auvvsTd3R0Oh5uSjzSn3ftQNNf5fC69a5kZXeiwHUe8fnzEsESkGMHBYdwMSDlhmhRwvUw4PT3izTffaKRlNO1Om/QBISgDl4FRgwdyTgiQKrgSUFiqS1nnIByfnvDmm6+Rk9RVztOEuMyYLhfkvofXtltOAc6cU3VFqve9mjNSv0o6kiu151vy1UV+PoKdrgROXR1OXHw9+7ZX31JKGr4FumgF68P3U0F4bYOAlS9/da5vy41a5AJyYAj6zSQ0mGVZMAySSzwenzBNE6Z5klrJrgcUUT6fLvji51/g8fEB2+0e4yjFwO/fP0gvjf0ejhzO5wne99hsdri5vUPX94gpYZ6WgoY7J/uhbzcb6f8xTcrrkwwCmBHnC85Pj3j36jWOTycE3b9J+GKyglOccJkuSJnRDaNwx7yHJ9UmQInWBRLZlGBlWRY8PDzgdHzCNF3w+PCA8/mIfuhwe3OAd8oMQU2d2XxbbziTASMxgrnuyKxJchOKVrNRnRxYE+MSbV5/JrgXYJT3bP4fo5X40Dri9nJF33C1Z8oIbWLOqpWMZt3CHrRuwtICj62WI9LmvoqT2Xcla5UxzxP6occyZ7x//4RxHOBzxvl4KoPQdUm1gPhgz5+9EBD4dMTbt2+x2Wyw2W0xXRjYRAMAACAASURBVC4oG7eGgMPhgMtlQlwV8wZsthvE+YA337zG5fSEzThimaUvRF5mnC8nPLx9g3dv3oDh0fUDuq6XHeugfiY5jJstNts9hn5Qs6T4l42Rc3CsewfA/LIFaVmwGUfE7Rbv378vhAKQUpFYC7tZuV0aHBUyQLP8GY2P1rzaiHL9gQpR6Vq8Ljppj9XZhyhmLgwduzJJT1LdLEK/fG0eqWBYH8aGBOl10VKJr7VVG0xYaC6pCl/OVo5TX8HaJdm1pumC8+UJtzfPMQxeOyGmwngNIWAcc9lmhpnx+PiIYeiVuDjgdD5jS2K6LpezbqMTVNBuC8Mh5yTtEzrBzY4PHZ7ev8fQD+jCVFb9Mk94enjA+XTC7fPP4EIPBimo24sPuNlisztgs9kJ6OoJHLk8t9RpxtU4pZRwuVyQkyT/j8cT5nlG13d4fDzhyy+/xKBa03mUBoHWcrNObitEXAMDVN+zgGfG5bcx/wiyYNmD1haywhnm4ki9wjo1VdpfiEpdS3WVRhR6bRHs2nAfNTAkLW9fMxY+uFtYiVUd2MwMjgui9aJXsxUzMC8iTM4TLpczDocbEHV4eHgsLAnZFifCeunnzFiWGXFZcLhx8JsNuhBwOYtwySCydgZCCQxiipLO0Q6PNGTc3Nzh8eEBp8cnBN/p5BIulxNOxyeAgXG7xW5/UC0mK3oYR+wPtwjdRuGRiJRzSUY776/GqI22xUIsy4J5nqSvyDLjcr6As+R3N5sNOGfM86y+5GW1yLMKV3HdCYahoiqJxkyCm62eGmFqcTRu5KMIIn8ok00QZf8Gs9lESpVlgB2tJF5sNRVzVhx/mOBLSPOtDmR5iWSnXBO4bR7UFpNUMdamcd57pLjg4eE9AJaO0OOI4/EowGgnLZus96xp5fPppJx/MWeyJU3Gu7cPSCljt9tjWWYMvbA+mDM4ZilS6Trs9gfc3z/H5XxGill3skt4eP8el9MJN7fPcHt3i8PNDTYHiSqHccTh9g59LzlKzhGcZ+TM6Met7lQs+F8hHTT7VHnvcdKO313f43x6QpoXbDcbTJctlkUYvMu8aL3DiCd6Wk2+gbBOhaf4XwUjs+9qNOmoNrMoNcIqld+OWKBVba1glU/1mWRrH3PAnROMY/VFO9laWrFahR8PGGr0KL5OXR65nIP4Y6V0DsxRHFmy0jVhORARNhs5/2effYZpWtCFXqnbWvy7LBIxdh3gCDFlDIMrxSVm2t69eyvbTjOw324hIYwFNBk3fY+UIv76Zz8FUsKyJCyXC9K8YLfd4/75C2wON3ChwzBssNvtBMLwQTRJToXLXFoneF/6skI3tyAXVj1WrUp+vlwwT7IRxPH4iGm66BaFsrNeSgmn8wmWQpLuPS2o3sAKjda69pGLWInGWWk8RnVlrrVWUQp2jvK9KnhiLhthMA21NnltUFpvuNwaMYQRK0vm2qmvEi59tYr6BqTB2gcQCZV/QeLLSKX4IhgTEZ6ennBW87fbHXC4uVGz1+Ozzz/HZZImfpvNBptxRPBek9wD5mWBCx6sfo8g6+LU932HtETs9htpn5AZ/dhjv93h8viAaZkxn89gIuwOdxj3B/TjCOc9NpsdtrsbDP0oUSgyEjJ8FzTzoJVHQuQv40zEpfLbWnfJziUJ8zSLWZym4oeacMQ54u7uFm/evpLNOJpqMzQTfQ0rtfNSP7gSngKGXn2X1n53y/m341pGsb3CCphrfK+aZFVHEusGHVbQK8G9L8LxMZ+sOvK1Jx998J32Exsg2SHEOhXKKg+lt6xRffphwG/8xm9KS08VqNNJnGbvHPrdTgHRDqQFuDGKtksp4fh0xOl0xN3NrWi2mOGD7Iw7jjvc3j3H49u3eP/mLZgZh2fPsdlusRm3GIcNbm5vG0eeYWVyFsVaQxZQU7CDjMiyQSxUe/kQMC8LLtMFzjt0Qfb1bNtNzfMs/Lkl4hdffLESMEAS0wlqlRiyHQ7MfFbXZJ2K0oQ2Sb2SkT9bX5FJG01cuUQlnWRRKK+VSxGyazt6XYMpDrIMhpU7WSQi3fnwwWpoz/eBr9b4eoABkKbUuFRXmyEVOo3DMI5gRmFFbDYbjOOIt2/e4E/+5N/g/v4Znj1/gc1mi77rpJo7yF7eMWc8vnsHZsb9/T3GzYicMtIiG4TFeSnpk5xZ0ki+wxICdre3uHv+AvPxCegCdocbbA57RN2CL3SdBkTaToAIXd9LwxbloVnKhyATb8lmWKUVhHXBWgaYYsS8SG40KQO3FY6n0xHsMvreYZ4SUmwCMX1lw69MWaj/nJsArQhDgSFaa6IChuqft/ewNo3Q7bWpLKamdZRNtNOaOl4JRjV1GkGi+lAfN68fvlYarvy/lUqbIIs8paOOlK9JC9AUxVxst9uyEG5vb/Hll69wOOzw5u03SAkYxy1ubm4xaDt2R4RpmmRPJ23vdLlccHNzh6H3mBSpzxvRymJWR0H6M2PoByycsb3Z4/nLl5gzY9jvEboRz+6fY9xu4IMkxUMIGDf9mvVK1YTIaqojaC3MM0To5un/r+zNenVL0juvX8Qa32lPZ8p0le02nS64cKsNtGU3FgIkREstcYXgAyDRXHKHEHf+BH0D4o4b+gsg07gFTXULWYBayCqpJzurypWZdTJPnmkP77imiODiiYgVa+13n2pWaufZ+33XECviiWd+/k/H9u6B5nDkl998BRjKqhSC82k/19fX9H1P2xzJnKXte0LTL2lvLRwrEPE5IJXUzTDqalMCmmoyaqLfPVpbPBaaziYMKxz5/OT5YYxJ0kQSjpf+HWX2YwPhLBdjSpDOBYMgiMxgfYXdZpJOHJIbb4zlzZu3rJbrWBjS911Miy7LmsvLC19tnVFWC5xzPqt0wPQtzcGQ5YrSux9QAsacKYk3aiRp8LjTLFcbTvUOrGNzsaG+WLParFksF6xWS+lc54H6qqJEZWNhTnizUuuY9pw2KnNGwGSkQUbGYAeWqwXGdBwOe3b7PcuyIluvORwOrNYLTqdj7EkV1k6YgOTyj41l07lnoqifk2LnfpeVdskyj7lk0YCDWNmUujEAcu0jQs66R7V4gM/eVF6pZyTnWeOe+Y5IBzPfNfiJP8/5ZKdrrSl0Rms6yWD1le7WSieSH/7wN3j37h0oR98bfvSvfcHLl5/zwx/+kBcvXuJ8ZdJgpCQs89DnNzfPGAbD/e17Pt6+ZbO5gqTuUynBEBsRtMVvtlytOC2XdMagc2nVkxeFbxNURk5WFAUOqTay1pJriYTkfoPkWkdL0jqpa0QJV+v7jvuHW47HA0VVUZiCTBfs9we2D/e4TLFer+n6XohyGHwlVAx+ECu17BhLTu33YC3OIQjmR6o2zZd7GsOWI4SuQnFySmh59IeoKeeZcqFQnW2T8yKOMqHaPu2fGAb5iMBScpqJZsAnERKhDJTKefn8BR8/fvRVQTl/8Ad/wHffvuN3f/evc319xTBYvvnla06nryTPfjBsLi6FGyV1k8ZblHW9oFosePbspcQuffWOzjPysqQuSwrfzMKUBRlw2m3JMs2qrtlcXXNxc8NqvY5iOGTQWmulX4DOyHUGHh0xJELawUSrDW9sGWPpO9HD+rbjdNhz+/GjiErTczjsWGUllOIGMdbSNEfKouRopKKe1FtqR9AV0aG9mLaPiWO+HoH4JlzwzPrNC1VSuglHSE/P5yc/JgJkgMHiiGatjoO3aoQnEkIZ+w89RWDTz9Jyee3hRxUGiTN++PDOE3fJfn/kH/7DH7NcLrm9+0CWKYqi5NVnn/Pq1WcY4zjsd6wvLlmt1pRlQdM0UR9zzvkM2SVtfcJpJ8Ho45FSZ1S6BONwOijEmtK3AmyXK4p6wc3z5yw3F+RFQV0vYkldSMdGCzcUi3z0fYleliDm+Dl2dkTuaU4NXduSaU1zPCIOZMNAT60KMRqM4XRqafuOXOcYF/qCTudXQeyxeW5t5wSSisG56JzTxzxZ8ZGO598HQKdUe84SVKSWoIJAXMqnrITU6zOyPbVQ0pebvuT0ZUPSnrOW0ne8zXJHlkm/oT/8w7/JZrMi8+Gd+/v7WC52f3/H9c0NVzc3XF5eij/MjRCeFxcXaK05nU589/23HPsTTom1ul6vOTYN2+29RBl8TySVa7QCMxhy72Kxvp4zizUCIziK1prSuy10lpElKkLqQ0o3oRkMh/2et2/fMgw9i8WC5Uq6rzSHI9pHSA67Pc5ZTu0B5STc1tthann62G/0dSV4I/M1CFZlkIVxbUB6TgW/mH2sr8+Nw3RN5+udi06gAjlNBqCUio7CAHHpxJ8xmrUxAHEeyecp1jxR9JNBhoJXrXIUUOUF7WAZBsswtHz55ZcAscgjtk7Mc9qm4fbje+E6bcNiWUvPJSftdLqu4+JiTdu2PHv2bNQZ8pIsK1guxcemihLjIFeKHMXgwDUd9thQ+nTt3BNWURSx74D21qwjVJiPLpgsWZBAbMHBut/v+f77N+x3DxyOO5rTSWARnMNYwewoi4KLiwtUnnF3dyfwDVYiBsZ65Tuuj8ywVWA8xwx95Z1z+BqPGBbM3GOGYHGTZl4qaFVKPTp3Qi9naCCPH7jHOllKjeEn7lil4iDP0dGc9U5dGFMCnLhRvP6nfAMr0anwhF6w2+1YrVYR4Uc62hpevHhBVQoBBFA9MwzoRY32pW1lWVJVBXVd03uId0HgGWICZN/3FLnGdA6nMzCWw3bL9v1bOBzIVmsKB4VP2VFuzAsLPin8wtqAXasVQ99H5Tms3dAPHI9H3r9/x/v37zgeDz4w3rHf79FWesIHp7H058zZb3eiR6bZF951YZy0pkaBco5cJdIoegOShfDjDR878HBRTzOIc6JxLmZT+gl965hzshHekrgzzpbD4fOGmA7qnI434XDxtcZ7OxeUT+dLxRR5UVHa0vvFLri7u4sYrCsPvb5er2iO0mOgLEva04n3797yKvuc5aJmdbEi1BcEcZyXBUM/YJRglEkrRMtiUeOsoR96MqVo7h94+9Of0334SK2A0wnbdth+IC8LcTzm4nS1IAgODt8QQ/kExgFjXKSu0AvUDIa7+3t5p+bEYXfPw919zOI1w8But6UoCxSOsq5ojG++oRSD6Udry1p6z2msCuVuoD1wxcRfpqa/Kq1wxiVR6LG+EiX1Acq5uEmeUn/Cv2lIyTlHHqLykZjCQIPXn2AUT8lCB9tZ7nTWJfG0qPSXJUaRPM9jrwZvsXMonyUb8+U9t3LW0p4alstltOyur68pfBnbfrdjsVhwsbmiH6Cuc99lxaMPaU1V1oQmB8MgsO1d00hP877l+PCR73/xNd/85Cd88dmvoxcVtmno7u4oS+k5pWrjLUopGraDla54ynn4+VZqSK24Q3pjpAMwjqbpuL/9yMPdB+7ubrm7vaNrW9FFfU+jxXKBRkWYBG0MJx+bdV6UKKVQTlE4FcFdrPJoh747n/J6VQRLUS627fZTP6o8fqMov7ZPKjyJrhbpIqnfDTQQnbEST5vKWLGAxM80fuW8pfdJ+nnC4pCAcDooYeJjLFPr0C88yHkxhbvOt4EeBjItNY3VqqI9nsA4qucBhl2ILNPCCZrTnrzM0CzIsoLGDDhr0XlOrivQCmOFIxhrMV3L/v6B1z/7c777+c/45l/+OVdlQffsFawWHLb3/Pk/+wmf/9Zv8a///u+zXn/h1QUl/Q36AYWjGzoO2w90hwN9e2LoWtr+xLHt6TsweYF1mcRY25Zh6NBKewirls16zWl/jKB2ocNJ6Gh89HMUw2BBSQ/uKBi5GW4Ui8F9khhcIfXcr7y4Vmbr+ZRPMyxkKAJOXR+B2CKRSW3dWLFCGGSkKJU4a30fxJmiN1cK3YTTBVYZ7i6/T4PlAelwfGHnBJs/t+IiCJ2Gy7Kk6zrZ9Ywm86k5CkexFodB4WiPDR+GlmOz49Q2KKe5efaCerWiqteU1YJM5VjlaE4Nr7/+BW++/gqz2/JsWVHrgsV6Q1Ev6bqGm3XNzbKkcAZlvb7jdaauazgdHth/+I6vfvrPefv65wxdR6nFD9cNhu19z15VXL38jKpacmxa0bH8a5dFQds0NK0Uuzy7vqEoC6qy4nZ7R0jsVEpJFoyfVitMC+20XypR3qUWUs4LQMIZTPLawpqFloapz+wphT6sYkqIqRsr/J10JJheEW/qd8aoFap48lxEprI6wBr5p8ddNz7usSHg6dFbsp5YrSHLxXrsu4GiyH0mhnjwVZbhkGzYh4eeqpGyuPVyydCvcMby7Vdf8v3rn3F/+5E6L8mLmvXNcy6ubsgXG158/htcbS558fnnODLWV1fkyyXqeKS8vuFqfclivSErS26uLtlcrqmvLikuNlAW0tXXCYG9f/stb776kq/+2T+hKhSHu/cYNLbQoDKyIsPSsb3b0fSW569eMRjHbrtDKQQxsqromobr62vJji0KirxAQXQeC7wm3sEqs6gT5V5Eo5suGaPHQgjAT7pYVnJlUiAUCOdTDvWUbp4KUSXW5RPyz4k+NIrIVPzJS+mzrBRv4Xj0+MCqHw1i5HwqIcYoLhHlucgrwYhA0o5TjIosz2makw8LwWJZo5SjbY58ePNL7r7/Baf9vSzIckG3u+f97TuGAXRVU29e8vz5c/7w3/sPuLx6QbVYcvXiJae2ZXl5yavnr6g3S8o8p6wqLl7esLy8pt5comKPT8f+7o7/+x/9mPtv/4JV3tPj2D7cs3U5L68vWOYa6zS950RSFOIwfUcWikS04nQ80DYNRR/CVFLy1rYNQ98LIUS/WyAiyYgJmrPoack6IPqZTop3A9/IRCny9/CEOGvIla7RxGsAk9K7sLYp55u0vXnCAIkefO/pmBGhGndL8pDRZeEf6AloSswjoU10AiS8lOdhJhy51gxeHJSFpDZvNpuIsBMcocvFAuUcx/2e+7tb7t/+kuH0gOl7UBn7U0OeaapyiR0M7z9+oFcfePtmTVbk/Lv//t+iWix59vIV275noTOWVy+oNitKpSiLnGq1obq4olhtkmb3ho/v3vJPf/JnPKsadKE5tSfA8ZffvGO1XJJZi3KKbHGJbnbUVU17bNnuH6ICZYaeoe0EKVtrbGYpioyu7yT12hqs95HNJcPcMy+czbua/H94MZhiwdqg6CeGX7DKIgdL3R6MTEL4znlPQrh//ujbuOwpWw0P0zEhb/pC8t0505ZI0fHps6fM2bDzInPMRyJ0XysqyZQ1+OokAcALfipJ8jOcjh2H3Z43b14zHB/A9FjTg87YnlqeX12Q5QNt39O0LftW0K9/9uVf8G//3t9kWT9jc3mNdlBnOav1FYvlgiLPqIqccrmgvLgkq2uUIjYsNdZgzMB2t6dVlrY5Ua4qPm73FMsL8rqmMxl1dcnpzS2XVxX7/c6rBNLwwjqL04hhUo6Ae23TgBO9VKvpRo6LHtZJxw9jvv+o47sggoQBMNKPcqPREKIxcU1mrqrJ+s4419ytMRWXgYDTgcNYocSY7hEGoVRy/ZnDYWM9XuBqU3Y6Eu04ZpU8Q9oWt23L1WpDUeQcT0dBmvaWIkqyHJxzDH3H6XBgt9tyOh0ZmhbTd4BFF3Boe1a9Iy8Np94wDNB0bayLPOweuLm6pqqXVK8WVGXO5fqCuhKIzyLPycuCoq7QRRGbIjgcg3PcPH/Jm5+9IaOnOx3RTcvQD1xefY7OV2AVXSOo24fDnsP+QFmXaJ+eXtcLqrLEOseiqqXiXXt91DeISGHgx4UNKrOQTnAJCYcb1zVwtmjgxTK2QGzeGIg463IVmpgMPZV8KjUyx3VP1jQK3pRuxxMt3jyJBubIrbxnPgEcPhdITTx8BJ/LSMf2EdU/PuQNRqTGnLIsKDPpgdQ2jVRma03Xtmwfttze3nI8nei6jlPT0nQDx7anawesBWM11y//Kn/1R7/DyVf9hPEfDwf6XtrLlFVFsVigqoJ8WVOuFyI2lwtUnkVQXwm8Gy6ur/mtL36b3Wng9qFh1zje3+7ZLNcYu6AbNDrP+Bf/4p9LZfhux2B6SSfKcuqyRuc5WVmyXC0pFzXlopZMhqwg05mU3qm5g4EJR5p6B8YlkJ9QKebAWd9dzsYTx/WwnoEkBPMkIxmjOqlXIfw+0ckeXaxAjfTtjYCUy4SM/PPFI/N7OxUyD8K0pOVwakLA49SJ/y4tpLi4uIBemmEFDNmmOXI8njjs9zzcP3BqTnTdUeAtLTgn7atzrSmqFVcvfoOvv/qXDL4Bl1QBCdpi057IckXet1i3EqyyqqIMfjUkqE0AkvESaL1Z84Pf/E1WF9e8+e6AHQaavqOocv7ff/J/cXF1zXZ3z4d3b3nx4jnG9CyXK6q6pioFsM9lCucMmfK6sBKITmsMPY6ri0vpw0Q7yQmLxOXG+SRwncDFZht6HvKLn3mwlehUDT/ApHhEefGaOGXPMZpHOf7pMQYX3LjcbmZlBMvlE97Z1B0SPGaBiFLxKIPUj69VjqIqYk7YsqjReeEhB3Lvn+o4Hg/s9jt2+x1t12LtIP4/J4HiwTiyQnBZnev4xS9+gbMpFlnO8bTncNhirbSRdk6gp6y1WGPpkHQeB2CtVE8jISJjHZurK/7N3/t3ePcnf8zDfsdgLE174KuvfkqVS4/Oq6tLtEggiVhUNeViSb1Y4pwR67LtMNaPvR9oTidQisVqSV5WcNqPk2SdePbjShHHKPaEiwq9bxcRvaHzdUuV/LjGEIt/z+ndKTz/OTdHhI6aU3QgiuQPEY2JwyWYsFEvmw+WVEENiY+ph8w9Oi+8WIh/iYsiA1VEULk8LyiritVy5Yt2HW0res7Dwz1tJ5gXkufu6xKCra8y2q7lm69/QXs80htDXS/ZXF5y/eyGV69eje4TpSgrzWK5IityjDUS4/SIPDjJzbdGiLkfGozrqcslD9t7tvs9u8MRO0jYqm97bm5uKOsKnedcXV2xXK1QOqOsakDRtCdOxwPHw5Guk35Kdhjo2obe9KAVD4c9p/1WjATnXUhq9IuJ33yiXHsCJH6vZmsUFzVeor0hQTQS1BMO2XPrnh4T6zKlxDGsNPKywL4nro6E4pk9IFB3QDYcX2M62OnA5GmRVTsFThpjBeJru45FvYhJgsPQczgc6DoJzWjlkQt9y2iHKMYWx9D3NO39BIF6sV5Q+CxaEb0niiJnuZTuKCISBvrW0Fmpz6SqIv6G9UaItZY8K7h4fsVf/xt/gzfff49SOafjMaYjkUke3mK5JMsLcMSiE+csQ9czdD12MFFfE2IvaQ8HHu7u2axW3GstWLH4KQpiUYW1cnFzpzOrAWeJmCOTyAwuKEbEcFW6nv7fR7UDTK3dR66UlBjm0fN04RUjmJrczDfkenTu9MG5N83l0qAfTF9+dCw6nJM6S2nLF86T9jjGGJq24XQ88u7dO7KiIFMZRV7FFyyKQpL+PAZGABpGZbFeU6kxmbEoClbLjU8ZGhiGjiyTFoNaCSS8wvrGKwJUZ8xAN3T0wwDGd4/z6UhlKbj9Lz//NX733/o9lssl5CNi9sXlBZuLCxGTZYFzoogPduB4OtCcBBo0NCoz1sYClaIoMG3HerkYlQ7fv8jqqSRK8igeOUqzbIwznuM8aQzyXOb0nKDOZeekBDy6MM4Qixq19DhglcQvo+3qH2rPKPwBEnPU40ZuNuec4fcMcE7gBFACjtIPrZTzH0xMFvzyyy9R2lEvqli+b62hyHP6oZfmWn0vldiZJsvEaSsWoUzgarWiqguqsuDyUpTq1WrDZnPBcn3Bol5RVQt0kWOHHtf3GOvIbQ7OYLzrQQizBJWByuj6nl/74Q/44kc/4rPdZ+RagJaVUtS1QBo458jKjLzIGdqO7nhkaAXmvTc9RZFTlxVt30XO2w49l3lNWVUYO/hGrYmbIir5BkXA3ZiQ0CMjMV2zKWd7QmcjKPvT6+bFKZPY5VPBT6/d+oGHfth4fSwMZprZel4EPn2kFo7zD7BSCeHrFJVPw5FMU+erk6QgpCYvffWPESW6riq6tqMfOvI8k8yMvIgoOYOxZLmmLHNWq3XkMPv9js9/7Qes12vBtFivqJfLCE6sM4E2CKVhEu5CeiIhWR86K+J7V/UClRU8f/EqjknmRfSdYRioFzVVXcu4OmnqKmVyVuo5M82paWjbVoD8tJaEy75js7mgaY7e+iNo+H4+z0kWn+WctBdMCeT/13oF8lJT+o26qjfwHjtjOUMkDpQSgLbw/Xzgkwck7ou5eZyGmeS64JtKXlRrSfsNRO/1vNDjUfn43tBL5Y9kLuRgQZfyLBGVS5yzNL4V4PF45Oj1oiDe63rpRYeAm2w2G3GPXF2yubhitZKayrquKYoctKZXCrTg2o7IkFb6JumMoqjiGOt6xdXNMz6+f8sGadwqRcaZj1BI5XjXtrSNYJ9prSXbohbxfzqdUL4KPaAbZVnG6dSwWl3y7t2bZO2IndqCeiMAeTZahUoprEl0pYTzpFXlT3kcxs+DZTqSQPzOeR9bNsLW5yQcZE6xstBIOpmz0jwUJv4ylMch9bLbJlwxWJ9hYeUaF38CgcV7Od9VNtEDskwqyPO8RFmHGwRV0RjDYlFTVxXKCcla71Iwg2C9VtWCqlzgHBG//9QELC9fwFuMXeaur6+Fs1UVq81FrCYvcmlaX+TSogetsUrgMTWC25rlBQEfTSlBD8p0QVEtIlz6crn0fcSFk+VFgfE6nrNORH5dg5Kw2cXVJaYf2O+k3eJ2uyXPc/YPD1z7lkIxhjmz8FPfpdc6HonJqHdlCu3xXefEFfSzVDcLKk9IOp2cHwjOSphNa00eAGoVko+Uzah3EmaC2HRTCGn0M9sZ55rL5jjAcJ+gt87keJZpD1ziLS7rdQqd49Tgi2F9gwrlGAbn/VkCP3A6nVgul9w8f+Yry1uKU0me5RwOB8q8IisylptNxD8zFnRWsNlcsllfslpvyFVGmRViyuuMDCjz53l7wQAAIABJREFUQtB+nOB8WW1RWUbhe4fj58ci3PfofVl5VlAuar59/YbNZoN1jmW1oD2esE5qLkPfdGsceVWiy1Kq4ruO9cUldjDU1YLtfkdeFrHxxHyO07l3sR3J4yNVzK0JEuX8eROnb8IQzqpZSvhoyhnzwGVCC+GQPzYONHU1jNQclfiZ8zQd/NTJmrhE1LgF0oE653zrGnF5hE5zee4tmKwkd6GeUDGiCSnv1ReCK8uS0+kUA+fPX7ygPTWoTMeMja43bC4v0VrzzetfioGSZdS19EJaLBYUZSkdSLSEsIwZpH+UcmIVKk2eFRD7hWsvtyzN8ch+v4/Pe//+jViaKIq6Fl2ulWKSFLu/qhcs12ssju2wZX15QaYc94c70GJhCiRpEUW9c85XlT1WfZ7St6OijuMRBJRSE+711D2fcsIaZyff57gRFVkx6kfzgSnGbIPg01HRIJi+4LmXii9HqslNj+CykEEHApRcKqUyikIQpofBsqxFDFk7oJRD5QoGwZYIxbyC2y/+MV3kXK+W0i3DKUrvcmjbFuVguV6R5wncQFVRFYXgS1iLynMPGep7USpJQsxVJn2RUJKRiuF0OPDhwzse7m65vrmk8QmItx/vpa6Smq4bMB53Lcu075xiY/isWtQ8f/4CZy0fPn7k1ec/oDmeyPMtt7cf6bqOYRir+IMMC3pRIPhzHC79W5DLg/ya6tOTyM4jo+4x25tbmBHVZ/JARawhHIlLRasuqO7yIDxBTCuJ5w8NfPhcOUI6qJGww3fhRTxKo8DGAErMf2NZrzeAELu1BquUoP8MI1ic1pq8KLh5tqFtpJtJWQrM+nq95uc//5k4MwdDXVcitvJM+gkpyDOpotJZRuaEyzonuK9RT8GJnuYMtu85Hk98eP+eTDnubu99H6aSly9fev3Pcv9wJx3lnCMri8Rnt6Jc1BR1BUh2yW9dXrHb7rh8vuJweJCZtM6rrh4O1LfGdgGWKi6Fm+hT6dwrJRvDBoh8v7zKK1xzhhOiC7Em/ZxK5C3MlBLy1H8fW/oyFvY6Z6V6RQVROorO0dIYfS9nOVjQ1ZRGYRgLgRUuKn1T5+94r8cW7DAMLBYeCHggpr9Y6wTa0/uP2rZjtVqz2VyKTja03Nw8RyklZXDDwGq1xNiOspQGErkvb9NKyuek9jPzSX4+8yRTqDyj8NCczovIIO6lS55Yg0PbMpiBqq6iCNoftr5MTlKVClt4rIyCopTwWblYUJU1CrEyF4sVg7Uc90cPU3DyhC4IR87ZEAAIU+43+MhRUm6kUWCsz4iN7k7PE9RkvsNmVSpZ/4Ro09y/tBW2iuLSk905mY0SBD6HpPpor4elOltKHOdcIZPzwps8ITSjv8yOkYTUAnVO+TaF4gWv6xqXZThjqarST/RA73H5Q3PV3XbrcSuqGOsUF8GBsqxEr7GK6+sb4Xw+rSZ0oRE4euXLy4iuDxWrxIXItV8YB2JFOst2u+X6+tqLMem00pxO4MNFfouitNRjtn1PYSwlkqCIEzVltVrG/qHKeagp5yMReFcFespd3WPHa0qFYRXiPo9LMg0pzUXndF29RGJa7ebCekPM4HlSFwsma/g5M9on3mJOuMFtkTGl56nzMFig6fVKST78crmMVlXwfwWOkfrbAjaFlNf1HhLUxy8TAlVKMVjLan3J5vLKg6eU/p00QTyPKeSjdR2yhjMtYTPj+5y3fUffdRwPR+7u7iOcZ5aJx785nQTJZzBSmheIwoY+7tJnqe86mqalHwYRn0VO25xEF9aawVfAj2G68f3nBDb3f0UHbqYEdUiNgDCfwkVJ12jyuwdPDJAWkeNpD5sVrUcXFjwRVXGUMtUxOS15gFJTCv7VhDYPrsakk1FP0KNV69wIZeScpFsHQsmyjDIv6LqWLCsk7WfQOCuhlqIssdYx9ANDPrBab1j4Rl6996NdXIjP7dWrz6gqARkehkHgCooc64SDZglHj1htns05JyLc9J1HCDrgzEBZL7i5usRaK4DDnnv1fU/fCWhf7v10fS/PVM0p4vxLM9icIstoTtLyxxnDqW1Y5LlYurR+A+iR65OypunaZJmOXCp4MZUWnXYknKlUCrcI6zy3Ji3SMzTUe4y+Mq/4Kzw8AL4IdEYSQQ1TSMcvZTX4pqg6dV+4x6QWZPnsw/MkmRC1vIRwEeFGMtjj8YDWGZeXl2PQOMvFI48QoxRbSDfcoiglqK+170ISxOSJi6tL8iIXzAmdcXlxjXHSEXcYBvqupapr6sWSIpdAtgvjRGFdQEk3dE3LyfdB+v77N+y29yyWNb0v22vblvv7e5/BmzFYS9O19G1LboI3Pydv2iiOcaGOoaRrWprmRNe2klqEo+9ajBmt+nNKfdCV030exLNW05rKiGjrRq4NoiYEl0RM0grcypfjOZy05skyjDUxTTs8dlZIMqXUeDOXdtcJIjWR9yqw4PN+mblpO6U5L4aYsXNSbjbqZql7QmuNygVSyjkwRvxkg8nJs4xlvaCuKgY7UFYFh+MJa6SnZd91DMNAURR89tln4qsyA8fdFlNVNFpTVRVFWXrdR4g+05nMhzc2zGDY77Yc9zuatuX+/pb2JIS6qGvatvGcVwyLvhVcscEa+mEQ562xDF2HKXJsprF5gaksxlppj22ddAg+Hr2eJvFbrRXDzMKb6ExKalgneo6b/APMQoLB6PMKqQ26KL4uIL2XaP++nXUoNgnPG5lJPlnURHk7G7sKBBb/8j6ZWCowdUmcUxTHjo4TSpvoFlFsh+u1Bie9ubVSNE1D5qHU21NDUYRuuZIZUngHap7nLFaSbaq0ou8Ni8UiwpQvlkuf0p2R5wX73Y7Oh4CsNTx7/oLr568kZ0yNprkDqUxqLUPXcn93R9OcOB6lOMQO0kJHedSgtm3JskyI8XDAOsNgZMdL+EnT9y1dN7aQxjnv7pAGEf3QowgNvBTGGrKsGOdMpYW8yttZdmJvRcnkVaOg+Y8MIRGRCk8oclJqTYb11cEsJYhH76hKfWskAXJhmdLRQk9JYLyxynz5uwKnUWrw33oUwVmAbO5jic/RiiwdnH/haeUynigF91WmMyPPcuEuHhag8EB3YiGK78gOYBgoi4Le9AI1VVRYqwhhkrZtWdYluZPelA7D7e1HmVCluLhY45xYiWVf43VpjJXuwYICJG0Jj8eDIFS3HW3Txo1iuo7jYYsxA6dTw8PDPYfDDq0zBp+5myl88yuH8cAtxhhMP9B1LfUik9BT29A3TSzoLYqCQXmDRo2iLDzb+6N8LBUBx2PUvUJE0PrMjMC9wvIF6ZP5TSXK/ZQeLGMFWvJF4jtQI5HNyAnOEJko/BJOEU4TKlymTevPhZPmv0flcHJ+2CVyRtgF2jsZwzlSOV5EKy/PC5QX17I4HZ1pWS6X5IVMUVUt6LoG5wx9P7BaLSiKjLY5YbtBHKk2KNagdc7zFzdYaySdRiuqaikcxPWYQZqc9l1H355om5amldbNpu942G/puxbtHB9v31EUJU3TYEwffWh5XtD1Q/Aa4nyYqG0bdJ6j0BRFjnOGrrPs9wdpcpbnDMZQLxaYZojoR37WHhFL1EXma+OIoMjJxMd5njvk01qOqE6hJtelngEcETXokU6WFiKknweymHwe6TE8+HHxZ6rfhahC9P6nFuzsBcc0msSBqMUyK+vKZ2Es/LkiFoZh4Hg4kOca56SUTKHo+5ZTs+X27o7nz19we/eRqytpqtq1Ldv9lt3unsViyXK5pioXHI8n6vpItX2g71oWyw5rxL1gnaVrWo6HPcfT0ddP7mjbA19//XP2+61EHbDsDlvKYkGRFxwPB8DRtkcGU5CpgsFI92DnJH2pbzuUOuGsIysynJbwldKK9WbNYb/HOUdRlZitB/KzAiwzEpishYs72YtOiGtkgkhUkrsf0rGt8TpyoCq/9lrpGEmIDe9tYATTDI40RAlJtVIUZ1ZjNeiUSPzgXOKCcASFPdQhz8zelESTHRSsjriJVBoHnVwUrd5QpZTirIYWMFYPOGdojgfJO9M6wkc5Z6nrBUPf8O03X6OznNevv6KqJPFP5xrbdrx9+8ZzuBXmqsOuLtjvdxSlpPjUdc397R3ff/+Gqq64uBC3hOl77u8+cDwcuL+/ZX/c8v79G7ruJK1rEIeqXThaEJdFnmPsIBhmmXDiYRCv/TBY+sFQOCdgxdbiBoPKJH37/tTSto3g2TqJ6RoTeaFHDNfR7xj2sMCfK088YT29cTi3Sj1CQFDbwrqFXqLES0b6GIZh6kWwI0dzzo0dScZD/EEq4TbS4Iq4S7R3so1iTuR+GrkPx5yInwRnSYjTWhtjg4ELBujN1WoTuVjww/SdQGWuqhrtu4OE3P48z/jlN1/z7u171qs1h9MRlUnLHJRFO9jfP2CBrjtKxGAYMEo4Tp6XPtPB8u23v6QsS549e0ldS3Ow7f0tD/d3nE4H7h7uOOz29EPDMPTRUYxzHE8Hqqqi78Xo2O124KBciPO378VAKMvSB80zhr7HlUJ4+8Oe4/EoE5Up8lISMNEZDJKxq9Bjy6C5GhIozo1xSqle8MZYYmGm6zH6yZJqdESPdHZMuz7XYjz8/mSOvw3mrIeknLs1VLRm0nJ4dXagE0JidFeMg5kSZXp2+Lvve1bLDafTifXFhqZpKPOCssjoMYClaVv0oDHWsLnYcH1zydt33/H9999y/eyG3XbHZrPGOoNSjv12i8XQ9SIKu1646ul0ZLd/4OOH91hrOJ4O4CRjt+97vnv9NZvNpaA62oGH+zu6rqNrjlJNPvR0doiTH5TtsixjaKiqKu+E7ajyGrxDV5zAhedUjlNzoh0G30JRCppRmmpRe67uCcovuohJO5lPHVwSPgN1DAPJEbwcc33aU2dUoazPssVzRq3URLJEqzRIIGfI0NMucXLypM5lItfD5wK2m4MyOBfUu9F8TY9JBCHcM7g+xOyJ7FdcB36CnFQVye/yAm3fcXWxIDQc1Sj22x3WDgymR1pWi2iRziM9b99+z3q9Ybu7Y7/fcTzsyPKMuipomoN/d7EujYNTc2AYOpr26EFQPMYrTmCwnEVhGdqj73oyeMeoQdkEuz4Uz/gkg2EYIoGlDRm6vkGhqCspLGm7jrquxc0ClFWBVRJGW9RLmuaE1orVciUWvg5QEsR1JOFgo6Iv5rFkbPh5tWJtat+2O9KVcyLyAtPzPkrtjYigOkke2tPxHgdYNROXMYnN+/e1ClEAX4AQsIW8LFcJ1aYWzORBiesiOurGLO5EJ9CR0MYvfXhNKd9yz7E/HLnYaGkkUZaUdUHXWDBSwaS05uLyksViwffvBLZ86ASKabO5wjnY7ba0zX4U/fFQo2thGHBNurPDFnMe+txKMa0b431hUsOGxHlHpd/AwzAImMrge2nmgtajnODXSi6cJCQWuRS3OAVlLbUGImIztMpB+bRxI6qNsUOsaEqXYBRjEue0niOLH1JgHlLEzNATJDDD0F4w81VPwpntyCH9mwf1yvqXd85RaFEVzhb3aseMOkcLLgg85wzYDKVDhcr0OBtc9btMnMl+wR6dH1VGFEEHlDGYvqd3jtPBSeFtUdB4fH4BJcl58fwZL1684OPte95+/x1de8IaEU99d+J4OrBeX7Dfd964GBXY8GhjTLTGIGXOKu4x00+La5RSArfOqDiP3oFRd1Fa0/dNvCZwrcNxx3KxIq8qDs0J5xQXaoNTiv5hB8ButxMu/UyyOpTrcVZqHrRKE+cf85a5L8skhAKp159oMYbrQhc7zSgenQvtFGVe7JxiPOFansAni/mW6UyRQBQQiGIUhWPFt4urEl4s6CbRYAhca7zbuWHgCLDuEmBeLBR5pkT/6Toe7rYoJQXEVVVxc3PDer3hcNzz+vU3viOckUZXStH51J++v0MpF+GmwljHkIqbTJoeMZPiqK1yk/kI40Ux1kF47hWgnowxwrm887PrOyqko/Cp63BtxmUlqd+mN5Jl4hzWuyMWiwVa51xdXvCwvZsSzkyvTY2t+VoIs5FVTgt8nRsXPFwf3slZN0mrJuF+lsdAxlGHd4+/e7zQqTHAzM8VFYHRCkxR18JOUXh2m9zp3G5Lj7SeQHBiM/bHhs6nuGgVsCpyikzz/PlzFqslRVXy+rvv0JnUVpqhQynHMHTgDOvNkizkh50R7fEzR0QfjM5NraZczf8TutYG8Yl1qLAgRS4N5xHYzOA5D07Uru+wTvo9te2Rh4d7uqFHlZIa1HatwBEYy6pecHmxZrPZsNvvGUKjW2/yWWP9z8iV03dyzid3hjI5CxmZdPe2/kVgkvOvGN1OwdqPnNk7ZPXsObJ+IwPS52KUjw8Xf6Ll4n0x1uNNSNB41FvCAOOOcONncRCPnWPTBbcW54w008pyqYHMq4ly23U9KhNguqurK/b7LXkmBRf397e+PmBsNaMcUXeZlIAlOqXym2QcO37nhk3nCO6coEcqpSS5EfFFheul4FXjjOBcODumC4naIWAxINZn15+4f7gFrCRlOlHOV+sF6/Way6srrLW0XTPOl0rXZZQyzuIJTsUf5dHsJOFZxXLAOOfnLMzkkHUTdd8GI+5XHDpeOLtRuNn4XQC8m+JlRBC85B7nOMTZ41cQeHh2UZSJjqQEn8v7wayz6Dxjvd7Qtid2u3sOh3vevPkGZoq5UpLmEyp8Ju/uHj/3iVE9+i3kujkHBjA6EZmAweEyja4KBqSrnvHNHAJnCFZjnmu6/sT7D2/QueLZs2cUi4pyuWBxsWaxWHB/f8/pdPB6pPMWb+A00znXITnPje8l8+E3gE7dUlMQlrnP89w6nuNe8/N0+uHcnTElUplIpZikdkCqqofPpkrxtDD0Uzslfdnxvjqk3RSF37E+CdBjiD17dsN6veT1t7/ku+9ec3d7K7j6g+8WMgwM1sS6gLOTlIzJJsUx+J/5xClvWYnPbXJHjP87Q4FOuKhX9FOuGQqScU70NZUJfGfvePP2Daf2xOXFJRebDZnS7Pd7bu9u6fvOtzSUZRQ9yUsclY4mmGp4w8SXF44yBZ1nolfpqd4dUsnjXJxZw/lannPm5pOTgt7uhzfZs1HGzxfBL5ArgMFjLWTxBYPITAeUfiafh6kIdsN4HghUkzHS/AEUTTOQ59Lq78WLV7x48ZJvv/2Gd+/eYMxocaXvVpdVTNd+lEgZxuTHpXRi7s84dHBNjAGRaeImDpSR6wYc+CYXDgnuD2YAX9GjtEKFOlLjQFm6U0NZV6jMcTzs+OoXP2Vzfc1nr37AcrEmz3LJ7GjbOI5QPR/aM4eEgaByhPx750SnDrHFsKGDby8YAUqpMe/fGwMpQ5nYg7O1RIWsmnHNn0S/9pd783i6NSTa7+LCqDDvEwTa4KSdWjvnFjgxSJMjBGwH2vbIarVhGHy1UpaxXqz4/Nc+58Xz5xyOW15/+zXGdhLATvCzwq4L3Xmf2nmPiOmTIjMd5uw8NQLxBnz8kHbdNI3sbC0GwEioI5fQmcRks6JAKuRb7u/e83B3z3p9wc3Nc5qTOKDHAXt9y6+t85p66qOM6tvoZo8iNFqCSmGHscAl3n68eHKPcQrG9XWJGyS8UwwrqUCBo96OKLVjUnZw4IVqbe0VQOVz3zOVCbdxEhZB20RZHh86Z7XnxehoLGgUz66usUp7/LCSly9ecX1zye7hjp/+9M9p2iN2MNgZF5tYOVo/Qo5Ox4Wa+oHi50yJMA2djdkKYS5kGQYcuVMUWcaAnbRmthA97LJBBTlJa/E3SYmbIysrHBpjeqzpMNtOIKb6o+dSwfzVPp1aQnzBIInz7adfnO0eXCWxPqPbxhMYQGhqrj0IzrljHkIEJj7WQEp5aAjlPPmlojLlMKluL/9qr6N5i8ULfUn/CBM6srewiIqpbyYw4GARzRmDtVLZ07cDl8+fsVzWlHlNvVjw7etvef36LzkethJMDpZgckRfzoy7hX8nnPUJO2QSmmFqEEm4iWT3y+dGQeYcWZ4lld7MGumGe4zzgJKGrcoKBKizjsVqzWAFdfFweEBlo9XrjWyi+1dJXcTEpaASdICEe8ZNrBRE+M5kUyk+SWBnK5tcQmhKKCk/nk5c+HL+cL9MjUSR3FW+tkwJIjQYcBZrBGbKKcETk401uuqiqqnSyVWM8UsIYhLCLrNgBw6nPepO050EN/bu7iMPu1uG/pSgND6ekLkYTI/UIAmOZOHACRmMdEXgEtGHl/aV8Up1ODNTYkG2fSfEPL6dF2nEs128N15fk3sEmKfmGDI4BixGqr5DKMuOye9WhVqI8d2VUpErhdlWWlwqcV7sjPMpFR3o8aJw7kRXOz+v8XPg1HbkbTdMb5hO+hPHxHUBkaOl3FBeViZOB9EQCPWMCEr/nj/fDgNNe6BpD2gPu2TsAMpghl7EN0nG5r+qPjUbw1wXUeF/bnp+fI5XiAOXSLmzvHe6UCp+HgnKz4UKYmN86iP+cTqdgFm1djSQxhT2MH+Td1PEtJw0EP4py/Dc4VIjINHnHh3RioLddo/6a3/td9z7d9+B7/wBoAKYmkoGHDhMsksmd0smSOkkVMGUnY4GqVT/BLl8ziR2Tgp1M6XJihxZJBtZdRC9+B7cqSiYT/ZcwZ1P9OS5KQOPYxxFUzw/6KR+XlJis4zvpJWPa0Yuo2JBTchscCGd3Y5pU+cWMxBZ+lkMwk+IIaxZoh5YEZc6jHc215NVDGOdcK4wx4nide5QgFEobVldvUT/0R/9kedAkm8UQiBeY/cPCH+HO0giY6pgBt+W0inRnXkRJ5wver7PHClxWGsZrPFB8BB77JOe3MGnF1b/X21Hzr3c8Wd+TpTxyeufG6/yxBmI2evkyvnFjVPqU5Ydo6XnnOf63veGwC/JfRN/5ExEhR8TfG3p3MkficEhYHeBRub3COeE93j0roE7RDr49KGUIssz/uv/5r9FdV3j/spv/joO6V4bM3oYHxQGdt7jL19GjqBtmG0/y1NO4bUNJIKQiV0wY+/nxJ3oCY9FqR5pC+vxW89NQuQqcz3s3POS1nqPRDqjjylwssn3UYebX5feXlYy3DvF309vpxMuGfTBxyhIT6sGKUGNitvj8+JcBO75ieNXbeD0yIqcP/1//inaGMvDw97HHT91ozF294j6k3PkJ2P0k83wZtXkH3+PqXh79ORgYifoNOO5oZJdPVrY9Pq5u+RTz5tfMx/nJz+b6XXhcEh1kFNMuM5kDPML45g//T5PjS2It5GLMoYxSRQdP+ZfRWDhub9K5xWJYPjw8Tb0Zs/5n/7e35PGBcE3FexsNR381Nj0u9CzdNBeVCrEBB1BctMGEzKA0aM052JPWYlaa5RT6CRxRJ6qQz+J8cMnJki+dhOdazIxyXNVYvafu96F7zzUeNRZA+efZZomdx45RphjRtGqJs9/rEvG+ROFLt7/3GHdY+swUlvgBwoPjXp+3ubqxKeel85dnuf8d//D/yjrNgyD2x/2vHr1kpurDcqNuPtxcsOYfF5TKmak6AQgA9Wf3f1Tl0XYP9mox50Z5Pw+WmuUR42xyscCtaZwAhI87x4bxzcnEjW+07miliA2Ytbn7JTUskw3iAo6mX9Fh4tl/UqldYvyb7rxglsDiI5apZQEvpWUqU3m3DlC1nuKpv6rvAL+JL87x039FP5++H4yN7PP5vOrMylmef3dW/7iL79juVyglVIs6gX/6Mf/mDzPI4TQ9GHTB04WUzLw4jnzh0s671ykjO87f4Hz8zLdRcGPEx2QZyYjXZQ50Qbl+9wx4WBndm/qFpiOL5mCZLHTBXTpZel4kudrYERzmeJUPFIZmKsuqdR5LFLDs0KbnLnxM5mfT6gKT/0uFxvINP/z3//fqBcLlPKyR2vNF198Qdv00/OTyfG3TNj2mHgorzvG0gIRyAAE/nJCaH5FRjcIj154TjRxssPETgc6cTjOOVpYnOm4plGAcJz77Nx9g0UZudjkmeO7PtWHaLqQ09oGZZ2EOhJ15ewmPKOrpeMLP3P3jMuUB/R7vPmAR5LsKc527rldZ7jfNvz2j/6NKCm0UgKAdn19zT/+P/+UrKwIzQykBxGoYAkqifAbM8SNqHyWpVYFipwAPRR3nxKnqbAHhUOjEWytwDVmI47IfCNxmshhFeJzyoA8gl2Jr11bb20mOz/VZwDxdH9CpXCM6M2oMen83CKLWZOEspwDZ3HKRE6DIjZOGPUxvKhTk7CWiGoHGnSehLvU+H3kkOPHca4DZPx0OmdqByp4qyLHns9Reo+nJIwi2QxOnPNWOerlgv/l7/8D1uvNONZQffLHf/zH/PjH/we///t/CDpjcAMOSV1On+McSWzMxgcmj37yJUczOdzrU9YdXlwBZL47riccK2nGzkbjPyrdYRhzEZmKr5C6HbqzzYcfcdfcWJ3zqQlP58Z7wcaXiCJrfDERm6HHUwhKu3HsWsX3BaK3fjo/0zjvfGxPGVBBDeCJ78M5n7JeRY+TphlKCTh/lgvMwt/5L/8rfv3X/0rcPEoplPWwe6EIdbfb87/+yT/g7/wX/zmff3ZD15xQaDJdxBePCruC0ecV9pYTX9n49sF9Sywp9Tj4RMV/KpY9aLsPgQQEv0EakeqxDF9uH6p0xiiD1Uwcneeq2s8eilGEzDrUTquyZ0cCY+nSsT1aQ+UzZp1wMT+HgUPLTDzmRIGxphJC+05vKrG2J/rfI72aR5+nx68yGs4TuWyGvKx4/+EDf/fv/vf8h3/rb1N5qNSU6wIC85jnBVdXF/xn/+l/wp/92U/44rd/h/cftuz30s9bdJuJ+hrmLnn440HFFw07yCu1cunTfhe5JnDHQCwu4S7nOao8ZpzMlP3PRcHcQIg3SM+ZjelJc16F/6mz8Fti1QqBqSc5yePFn4835RIp4YXKqLR24VPK/fz+c3fFOZ0unGeMYX888v5uyw9+4wv+5H//U/6jv/0fe2UfGL3MW1veAAAAB0lEQVRw/H+b2Lc3Rg46LgAAAABJRU5ErkJggg==" y="-21.499943"/>
</g>
<g id="text_1">
<!-- Train Image -->
<defs>
<path d="M -0.296875 72.90625
L 61.375 72.90625
L 61.375 64.59375
L 35.5 64.59375
L 35.5 0
L 25.59375 0
L 25.59375 64.59375
L -0.296875 64.59375
z
" id="DejaVuSans-84"/>
<path d="M 41.109375 46.296875
Q 39.59375 47.171875 37.8125 47.578125
Q 36.03125 48 33.890625 48
Q 26.265625 48 22.1875 43.046875
Q 18.109375 38.09375 18.109375 28.8125
L 18.109375 0
L 9.078125 0
L 9.078125 54.6875
L 18.109375 54.6875
L 18.109375 46.1875
Q 20.953125 51.171875 25.484375 53.578125
Q 30.03125 56 36.53125 56
Q 37.453125 56 38.578125 55.875
Q 39.703125 55.765625 41.0625 55.515625
z
" id="DejaVuSans-114"/>
<path d="M 34.28125 27.484375
Q 23.390625 27.484375 19.1875 25
Q 14.984375 22.515625 14.984375 16.5
Q 14.984375 11.71875 18.140625 8.90625
Q 21.296875 6.109375 26.703125 6.109375
Q 34.1875 6.109375 38.703125 11.40625
Q 43.21875 16.703125 43.21875 25.484375
L 43.21875 27.484375
z
M 52.203125 31.203125
L 52.203125 0
L 43.21875 0
L 43.21875 8.296875
Q 40.140625 3.328125 35.546875 0.953125
Q 30.953125 -1.421875 24.3125 -1.421875
Q 15.921875 -1.421875 10.953125 3.296875
Q 6 8.015625 6 15.921875
Q 6 25.140625 12.171875 29.828125
Q 18.359375 34.515625 30.609375 34.515625
L 43.21875 34.515625
L 43.21875 35.40625
Q 43.21875 41.609375 39.140625 45
Q 35.0625 48.390625 27.6875 48.390625
Q 23 48.390625 18.546875 47.265625
Q 14.109375 46.140625 10.015625 43.890625
L 10.015625 52.203125
Q 14.9375 54.109375 19.578125 55.046875
Q 24.21875 56 28.609375 56
Q 40.484375 56 46.34375 49.84375
Q 52.203125 43.703125 52.203125 31.203125
z
" id="DejaVuSans-97"/>
<path d="M 9.421875 54.6875
L 18.40625 54.6875
L 18.40625 0
L 9.421875 0
z
M 9.421875 75.984375
L 18.40625 75.984375
L 18.40625 64.59375
L 9.421875 64.59375
z
" id="DejaVuSans-105"/>
<path d="M 54.890625 33.015625
L 54.890625 0
L 45.90625 0
L 45.90625 32.71875
Q 45.90625 40.484375 42.875 44.328125
Q 39.84375 48.1875 33.796875 48.1875
Q 26.515625 48.1875 22.3125 43.546875
Q 18.109375 38.921875 18.109375 30.90625
L 18.109375 0
L 9.078125 0
L 9.078125 54.6875
L 18.109375 54.6875
L 18.109375 46.1875
Q 21.34375 51.125 25.703125 53.5625
Q 30.078125 56 35.796875 56
Q 45.21875 56 50.046875 50.171875
Q 54.890625 44.34375 54.890625 33.015625
z
" id="DejaVuSans-110"/>
<path id="DejaVuSans-32"/>
<path d="M 9.8125 72.90625
L 19.671875 72.90625
L 19.671875 0
L 9.8125 0
z
" id="DejaVuSans-73"/>
<path d="M 52 44.1875
Q 55.375 50.25 60.0625 53.125
Q 64.75 56 71.09375 56
Q 79.640625 56 84.28125 50.015625
Q 88.921875 44.046875 88.921875 33.015625
L 88.921875 0
L 79.890625 0
L 79.890625 32.71875
Q 79.890625 40.578125 77.09375 44.375
Q 74.3125 48.1875 68.609375 48.1875
Q 61.625 48.1875 57.5625 43.546875
Q 53.515625 38.921875 53.515625 30.90625
L 53.515625 0
L 44.484375 0
L 44.484375 32.71875
Q 44.484375 40.625 41.703125 44.40625
Q 38.921875 48.1875 33.109375 48.1875
Q 26.21875 48.1875 22.15625 43.53125
Q 18.109375 38.875 18.109375 30.90625
L 18.109375 0
L 9.078125 0
L 9.078125 54.6875
L 18.109375 54.6875
L 18.109375 46.1875
Q 21.1875 51.21875 25.484375 53.609375
Q 29.78125 56 35.6875 56
Q 41.65625 56 45.828125 52.96875
Q 50 49.953125 52 44.1875
z
" id="DejaVuSans-109"/>
<path d="M 45.40625 27.984375
Q 45.40625 37.75 41.375 43.109375
Q 37.359375 48.484375 30.078125 48.484375
Q 22.859375 48.484375 18.828125 43.109375
Q 14.796875 37.75 14.796875 27.984375
Q 14.796875 18.265625 18.828125 12.890625
Q 22.859375 7.515625 30.078125 7.515625
Q 37.359375 7.515625 41.375 12.890625
Q 45.40625 18.265625 45.40625 27.984375
z
M 54.390625 6.78125
Q 54.390625 -7.171875 48.1875 -13.984375
Q 42 -20.796875 29.203125 -20.796875
Q 24.46875 -20.796875 20.265625 -20.09375
Q 16.0625 -19.390625 12.109375 -17.921875
L 12.109375 -9.1875
Q 16.0625 -11.328125 19.921875 -12.34375
Q 23.78125 -13.375 27.78125 -13.375
Q 36.625 -13.375 41.015625 -8.765625
Q 45.40625 -4.15625 45.40625 5.171875
L 45.40625 9.625
Q 42.625 4.78125 38.28125 2.390625
Q 33.9375 0 27.875 0
Q 17.828125 0 11.671875 7.65625
Q 5.515625 15.328125 5.515625 27.984375
Q 5.515625 40.671875 11.671875 48.328125
Q 17.828125 56 27.875 56
Q 33.9375 56 38.28125 53.609375
Q 42.625 51.21875 45.40625 46.390625
L 45.40625 54.6875
L 54.390625 54.6875
z
" id="DejaVuSans-103"/>
<path d="M 56.203125 29.59375
L 56.203125 25.203125
L 14.890625 25.203125
Q 15.484375 15.921875 20.484375 11.0625
Q 25.484375 6.203125 34.421875 6.203125
Q 39.59375 6.203125 44.453125 7.46875
Q 49.3125 8.734375 54.109375 11.28125
L 54.109375 2.78125
Q 49.265625 0.734375 44.1875 -0.34375
Q 39.109375 -1.421875 33.890625 -1.421875
Q 20.796875 -1.421875 13.15625 6.1875
Q 5.515625 13.8125 5.515625 26.8125
Q 5.515625 40.234375 12.765625 48.109375
Q 20.015625 56 32.328125 56
Q 43.359375 56 49.78125 48.890625
Q 56.203125 41.796875 56.203125 29.59375
z
M 47.21875 32.234375
Q 47.125 39.59375 43.09375 43.984375
Q 39.0625 48.390625 32.421875 48.390625
Q 24.90625 48.390625 20.390625 44.140625
Q 15.875 39.890625 15.1875 32.171875
z
" id="DejaVuSans-101"/>
</defs>
<g transform="translate(48.199347 16.318125)scale(0.12 -0.12)">
<use xlink:href="#DejaVuSans-84"/>
<use x="46.333984" xlink:href="#DejaVuSans-114"/>
<use x="87.447266" xlink:href="#DejaVuSans-97"/>
<use x="148.726562" xlink:href="#DejaVuSans-105"/>
<use x="176.509766" xlink:href="#DejaVuSans-110"/>
<use x="239.888672" xlink:href="#DejaVuSans-32"/>
<use x="271.675781" xlink:href="#DejaVuSans-73"/>
<use x="301.167969" xlink:href="#DejaVuSans-109"/>
<use x="398.580078" xlink:href="#DejaVuSans-97"/>
<use x="459.859375" xlink:href="#DejaVuSans-103"/>
<use x="523.335938" xlink:href="#DejaVuSans-101"/>
</g>
</g>
</g>
<g id="axes_2">
<g clip-path="url(#pf02e2d733d)">
<image height="153" id="imageb081ed1ee7" transform="scale(1 -1)translate(0 -153)" width="153" x="189.818182" xlink:href="data:image/png;base64,
iVBORw0KGgoAAAANSUhEUgAAAJkAAACZCAYAAAA8XJi6AAAABHNCSVQICAgIfAhkiAAADEVJREFUeJzt3V9MW2UfB/Bv2xUKsm44Fkt0Gpdtzo3ojM4MnasJ4mYUsmxqvMBsmYlx80+miboLY7hxGnahhkVNVDSMhKibYEC2MmAONkWQbR3IYIVRkilsDAoUO1tOe96LveN9m7bQQp/zPE/5fRKS9ZyTnm/gu/Ocnp4/usLCQhUAkpOTsW/fPoTz2Wef4dVXXw07j8xOTk4O6urqQqZXVlbCbrdzSMSOnncAEmzr1q1YuXIl7xhxNVWypKQknjlIAtMDgNlsxltvvRV2Aa/XC7fbrWmo+WBychLj4+O8Y2hCn56ejjfffDPszOvXr+Pzzz+PuK9GZq+xsREvvfRS2HlpaWkwGAwaJ2JH/8Ybb4Sd4fF48O2330YsIGEnPz8ft99+O+8YcRN2x9/j8aC0tBR79uzROg9JQGFL1tPTg927d2udhSQoOoQhqOXLl8NkMvGOERdUMkFZrVYsWrSId4y4CCnZ6OgoSkpKeGQhCSqkZCMjI/j00095ZJl32tvbceTIEd4xmAsqmdvtxnvvvccry7zT3d2No0eP8o7BXFDJPB4PysvLeWUhCYp2/AlzVDLOqqqqcPDgQd4xmKKScXb16lU4nU7eMZiikhHmqGQC27lzJxYvXsw7xpxRyQRmMpmg0+l4x5gzKhlhjkpGmAsqmd/v55VjXgsEAlBVlXcMZqZKNjY2llBnY8rk448/xgcffBB23oIFCzROE380XApuz549SEtL4x1jTqhkhDkqGWGOSiaBjIwMqY+X6QFAVVU0NzfzzjKv9ff3Y2BgIOy8HTt2SH2Fvx4AfD4ftmzZwjvLvPbVV1+hoqKCdwwmaLiUxJo1a6QdMqlkAmlsbMTFixfDzsvPz4deL+efS87UCeq7775DS0sL7xhxRyWTSE5ODu8Is0Ilk0h2draU+2VUMsEUFxfjzJkz0y7z3HPPYfv27RolmjsqmWBaWlrw999/R5xfUFCANWvWICsrCy+88IKGyWaPSiaZ5cuXT/377rvv5pgkelQyiRmNRuzatYt3jBlRyQS0a9euqA5l6HQ6LFmyRINEc0MlE9DQ0BC8Xi/vGHFDJZNcamqq8LddpZIlANHP0KCSCcpqteL8+fNRLbto0SKh7/E7VbJEum98Ioj16iWRvwnQAzce3jUyMgKTyTT1I/ommATT6XTCbiimtmRmsxnXr1+f+jl58iQWL16M1NRUnvnmtfHxcQQCgaiWzcjIwM6dO9kGmqWI+2QbNmyAy+XCoUOHYDabtcxE/uuxxx5DX18f7xhzNuOVo9u2bYPH40FhYeHUtMHBQfzzzz8sc5EEEtXlyQUFBSgoKJh6/c477+D48eNRrUBRFHR0dMwuHUFnZyfuuusuqa8k16mMb8IwMTGBp556Cv/++y/++OMPlqtKWAMDA7BYLDMud/nyZXz99dcaJIqN3uFwwOFwoLe3l8kK0tLS0NTUhJ9++glPP/00NmzYwGQ9iayurg6Kosy4XEpKipD3M9HdfAa5wWBAfn4+gBthWT2iuKurC7t378Yvv/zC5P0T1ejoaFSPwent7UVZWZkGiaI39enS7/ejoqICFRUVqKqqwoULF5iscPXq1SguLsYTTzzB5P2JeMIewnC73bDZbGhsbERXV1fcV5qVlYUDBw5g8+bNcX9vIh7D448/XhhuhtfrhdPpxODgIEZHR6GqKm699da4rdhisSArKwtOp5PZ/mAimZiYwJNPPjnjtZculyvq7zy1MuPn4uHhYfz222+4dOkSOjs7Y3pzvV6PZ555JuL8devW4dFHH4XNZovpfeejgwcP4sCBA1Ieyog68ZUrV3DlypWY3lyn00FRFGzdujXiMs8++yxaWlpQXV0d03sTeUQcLuNlaGgI/f396Ovrw+rVq0PmL126FOvXr4fD4UBPTw/LKNL79ddf8eKLL057xkVKSgoMBgP6+/s1TDY95tveQCCAvr4+GAwGXLt2DRaLJWQIXblyJTIzM1lHkd6JEydmPAUoJSUFS5cu1ShRdDQ7adHv9+Ovv/7CuXPnwg6N+/fvR25urlZxpLV27VreEWKm+Zmxfr8fbrc7ZLrFYpH+Brxa6O7unnGZe+65R6j7ZnA5/bqnpwc1NTU8Vj0vGI1GZGdnw2q18o4CgFPJAoEA2traUFtbGzS9vLxcmF+MyKI5v89gMGDTpk145JFHNEg0PW4XkgQCgZAnoCQnJ0t7ozctRXsun16vF+L3yTVBuE9KIvxSSHxx/Yu2trbixIkTQdPq6urw0EMPcUqUeER4ZpN831GQqKiqilOnTuH06dO8o4hZsoULF0Kv10d9pc58E+lpvqqqTt1Dw263o6GhQctYEXEvmc/nw+TkJIxG49S0hoYGWK1WnDp1iooWxvDwcNBrVVUxMTGBgYEBlJeXc0oVGfeSNTc3w2w2Izs7O2j6yZMnsWzZMly+fJlTMnmMj4/jk08+4R0jIvooR5gTomQulwsTExO8YxBGhChZa2srHA5HyPTc3NygfTUC5OXl8Y4QMyFKFklJSQkWLlzIO4ZQZHzIlzAl6+vrg8vlCpn+8ssvS3nKMfkfYUrW3t6OwcHBkOkffvghkpOTOSQi8SJMyQDg3LlzGBkZ4R1DWEVFRVJ+tytU4osXL2JsbIx3DGG9/vrrIef3+3y+kFOmREM7OxJTFAWHDx8O+8lcJEJtyQCgtrY25GuTmpoaYW9VyUsgEEBZWZnwBQMELNng4GDIgxI2bdok9I13eVBVVajL3qYjXMlI4qGSEeaELFlpaSkdykggQpbM6/WGnEc2NjZG+2WSErJk4dDzBOQlTcmIvKhkhDkqGWGOSkaYo5IR5qhkhDkqGWGOSkaYo5IR5qhkhDkqGWGOSkaYo5IR5qhkhDkqGWGOSkaYo5IR5qhkhDkqGWGOSkaYo5JJymAw4O233+YdIypUMonJcnNAKhlhjkomkcnJSd4RZoVKJhGz2Rzy+EYZUMkIc1QywhyVjDBHJSPMUckIc1QywhyVjDBHJSPMUckIc1QywpwcX+PP0d69e/Hggw/yjhGipaUFxcXFvGMwl/Al27t3L959911YLBbeUULk5OQgNzcXFRUV+Oabb3jHYSahS/baa69h3759uO2223hHCSszMxN5eXl44IEHoCgKDh06xDsSEwm7T/bKK6/g/fffF7Zg/++OO+5AUVERnn/+ed5RmEjILdmOHTuwf/9+pKen844SNYvFgiVLlvCOwURClWzLli0oKSnBLbfcArPZzDtOzD766CNcvXoVR44c4R0lroQdLr/88ku43e6ol9+4cSN++OEHZGZmSlkw4MZJiaWlpdi8eTPvKHEl7JbM5/NBVdWoll23bh3q6+uRlJQUcZmysjI4nc44pYuPe++9F9u3bw+alpqaiqqqKvj9fqxfvx4dHR1T81wul5QPlxW2ZLHQ6/XTFuz7779Hb2+vhomi09HRgaSkJOTl5QVNNxqNMBqNIQ+1N5lMWsaLG2GHy2itWLECbW1tEedXVlbiwoULGiaKzXRba7vdjhUrVkj/dDzpS5acnBx2uqqqOHbsGOx2u8aJYnP27FnYbLaIZXM4HAgEAsjIyAiZJ8tFJVIPl6tWrQraZwFuPABeURQ0NTXh999/55QsNs3NzUhKSoLVag0ZIm8aGhoKeu33+1FUVKRFvDmTtmTp6eno7u4Omd7e3o7KykoOieamsbERRqMR2dnZUu7cT0eq4fLOO+8EAOh0OixbtixkvqIo8Hg8WseKm/r6erS2tkJRFN5R4kqqLZnT6cTatWthMpnC7uz39vaitraWQ7L4sdlsWLBgAe6//34YjcaIy127dk3DVHMjVckA4M8//ww73ev1Ynh4WOM0bPz888/Q6XS47777whZNVVV88cUXHJLNjtDDZX9/f8gD78Px+Xxoa2vD8ePHNUiljerqapw/fz7s0Hnp0iUOiWZP6JL9+OOP8Hq9My43PDycUAW7qbq6GmfOnEFnZ2fQf7aysjKOqWIn/HBpt9vx8MMPR/xo7/V60dXVpXEq7Rw9ehTAjS//p9tHE5nwJbPZbJicnMTGjRtDjnz7fD40NTXh9OnTnNJp59ixY7wjzJrQw+VNDQ0NIUfEFUVBfX39vCiY7ITfkt1UU1MT9DoQCODs2bOc0pBYSFOy6b4EJ2KTYrgkcqOSEeaoZIQ5KhlhjkpGmKOSEeaoZIQ5Khlh7j+IobnQcdL/mQAAAABJRU5ErkJggg==" y="-21.499943"/>
</g>
<g id="text_2">
<!-- Label -->
<defs>
<path d="M 9.8125 72.90625
L 19.671875 72.90625
L 19.671875 8.296875
L 55.171875 8.296875
L 55.171875 0
L 9.8125 0
z
" id="DejaVuSans-76"/>
<path d="M 48.6875 27.296875
Q 48.6875 37.203125 44.609375 42.84375
Q 40.53125 48.484375 33.40625 48.484375
Q 26.265625 48.484375 22.1875 42.84375
Q 18.109375 37.203125 18.109375 27.296875
Q 18.109375 17.390625 22.1875 11.75
Q 26.265625 6.109375 33.40625 6.109375
Q 40.53125 6.109375 44.609375 11.75
Q 48.6875 17.390625 48.6875 27.296875
z
M 18.109375 46.390625
Q 20.953125 51.265625 25.265625 53.625
Q 29.59375 56 35.59375 56
Q 45.5625 56 51.78125 48.09375
Q 58.015625 40.1875 58.015625 27.296875
Q 58.015625 14.40625 51.78125 6.484375
Q 45.5625 -1.421875 35.59375 -1.421875
Q 29.59375 -1.421875 25.265625 0.953125
Q 20.953125 3.328125 18.109375 8.203125
L 18.109375 0
L 9.078125 0
L 9.078125 75.984375
L 18.109375 75.984375
z
" id="DejaVuSans-98"/>
<path d="M 9.421875 75.984375
L 18.40625 75.984375
L 18.40625 0
L 9.421875 0
z
" id="DejaVuSans-108"/>
</defs>
<g transform="translate(249.721278 16.318125)scale(0.12 -0.12)">
<use xlink:href="#DejaVuSans-76"/>
<use x="55.712891" xlink:href="#DejaVuSans-97"/>
<use x="116.992188" xlink:href="#DejaVuSans-98"/>
<use x="180.46875" xlink:href="#DejaVuSans-101"/>
<use x="241.992188" xlink:href="#DejaVuSans-108"/>
</g>
</g>
</g>
</g>
<defs>
<clipPath id="p58ad9a7e6d">
<rect height="152.181818" width="152.181818" x="7.2" y="22.318125"/>
</clipPath>
<clipPath id="pf02e2d733d">
<rect height="152.181818" width="152.181818" x="189.818182" y="22.318125"/>
</clipPath>
</defs>
</svg>
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# MNIST数据集使用LeNet进行图像分类\n",
"本示例教程演示如何在MNIST数据集上用LeNet进行图像分类。\n",
"手写数字的MNIST数据集,包含60,000个用于训练的示例和10,000个用于测试的示例。这些数字已经过尺寸标准化并位于图像中心,图像是固定大小(28x28像素),其值为0到1。该数据集的官方地址为:http://yann.lecun.com/exdb/mnist/"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 环境\n",
"本教程基于paddle-2.0-beta编写,如果您的环境不是本版本,请先安装paddle-2.0-beta版本。"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"2.0.0-beta0\n"
]
}
],
"source": [
"import paddle\n",
"print(paddle.__version__)\n",
"paddle.disable_static()\n",
"# 开启动态图"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 加载数据集\n",
"我们使用飞桨自带的paddle.dataset完成mnist数据集的加载。"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"download training data and load training data\n",
"load finished\n"
]
}
],
"source": [
"print('download training data and load training data')\n",
"train_dataset = paddle.vision.datasets.MNIST(mode='train')\n",
"test_dataset = paddle.vision.datasets.MNIST(mode='test')\n",
"print('load finished')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"取训练集中的一条数据看一下。"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"train_data0 label is: [5]\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAI4AAACOCAYAAADn/TAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAIY0lEQVR4nO3dXWhUZxoH8P/jaPxav7KREtNgiooQFvwg1l1cNOr6sQUN3ixR0VUK9cKPXTBYs17ohReLwl5ovCmuZMU1y+IaWpdC0GIuxCJJMLhJa6oWtSl+FVEXvdDK24s5nc5zapKTZ86cOTPz/4Hk/M8xc17w8Z13zpl5RpxzIBquEbkeAOUnFg6ZsHDIhIVDJiwcMmHhkElGhSMiq0WkT0RuisjesAZF8SfW6zgikgDwFYAVAPoBdABY75z7IrzhUVyNzOB33wVw0zn3NQCIyL8A1AEYsHDKyspcVVVVBqekqHV1dX3nnJvq359J4VQA+CYt9wNYONgvVFVVobOzM4NTUtRE5M6b9md9cSwiH4hIp4h0Pnr0KNuno4hkUjjfAqhMy297+xTn3EfOuRrnXM3UqT+b8ShPZVI4HQBmicg7IlICoB7AJ+EMi+LOvMZxzn0vIjsAtAFIADjhnOsNbWQUa5ksjuGc+xTApyGNhfIIrxyTCQuHTFg4ZMLCIRMWDpmwcMiEhUMmLBwyYeGQCQuHTFg4ZMLCIZOMbnIWk9evX6v89OnTwL/b1NSk8osXL1Tu6+tT+dixYyo3NDSo3NLSovKYMWNU3rv3p88N7N+/P/A4h4MzDpmwcMiEhUMmRbPGuXv3rsovX75U+fLlyypfunRJ5SdPnqh85syZ0MZWWVmp8s6dO1VubW1VecKECSrPmTNH5SVLloQ2toFwxiETFg6ZsHDIpGDXOFevXlV52bJlKg/nOkzYEomEygcPHlR5/PjxKm/cuFHladOmqTxlyhSVZ8+enekQh8QZh0xYOGTCwiGTgl3jTJ8+XeWysjKVw1zjLFyom3T41xwXL15UuaSkROVNmzaFNpaocMYhExYOmbBwyKRg1zilpaUqHz58WOVz586pPG/ePJV37do16OPPnTs3tX3hwgV1zH8dpqenR+UjR44M+tj5gDMOmQxZOCJyQkQeikhP2r5SETkvIje8n1MGewwqPEFmnGYAq3379gL4zDk3C8BnXqYiEqjPsYhUAfivc+5XXu4DUOucuyci5QDanXND3iCpqalxcek6+uzZM5X973HZtm2bysePH1f51KlTqe0NGzaEPLr4EJEu51yNf791jfOWc+6et30fwFvmkVFeynhx7JJT1oDTFtvVFiZr4TzwnqLg/Xw40F9ku9rCZL2O8wmAPwL4q/fz49BGFJGJEycOenzSpEmDHk9f89TX16tjI0YU/lWOIC/HWwB8DmC2iPSLyPtIFswKEbkB4HdepiIy5IzjnFs/wKHlIY+F8kjhz6mUFQV7rypTBw4cULmrq0vl9vb21Lb/XtXKlSuzNazY4IxDJiwcMmHhkIn5Ozkt4nSvarhu3bql8vz581PbkydPVseWLl2qck2NvtWzfft2lUUkjCFmRdj3qqjIsXDIhC/HA5oxY4bKzc3Nqe2tW7eqYydPnhw0P3/+XOXNmzerXF5ebh1mZDjjkAkLh0xYOGTCNY7RunXrUtszZ85Ux3bv3q2y/5ZEY2Ojynfu6O+E37dvn8oVFRXmcWYLZxwyYeGQCQuHTHjLIQv8rW39HzfesmWLyv5/g+XL9Xvkzp8/H97ghom3HChULBwyYeGQCdc4OTB69GiVX716pfKoUaNUbmtrU7m2tjYr43oTrnEoVCwcMmHhkAnvVYXg2rVrKvu/kqijo0Nl/5rGr7q6WuXFixdnMLrs4IxDJiwcMmHhkAnXOAH5v+L56NGjqe2zZ8+qY/fv3x/WY48cqf8Z/O85jmPblPiNiPJCkP44lSJyUUS+EJFeEfmTt58ta4tYkBnnewC7nXPVAH4NYLuIVIMta4takMZK9wDc87b/LyJfAqgAUAeg1vtr/wDQDuDDrIwyAv51yenTp1VuampS+fbt2+ZzLViwQGX/e4zXrl1rfuyoDGuN4/U7ngfgCtiytqgFLhwR+QWA/wD4s3NOdZcerGUt29UWpkCFIyKjkCyafzrnfnztGahlLdvVFqYh1ziS7MHxdwBfOuf+lnYor1rWPnjwQOXe3l6Vd+zYofL169fN5/J/1eKePXtUrqurUzmO12mGEuQC4CIAmwD8T0S6vX1/QbJg/u21r70D4A/ZGSLFUZBXVZcADNT5hy1ri1T+zZEUCwVzr+rx48cq+782qLu7W2V/a7bhWrRoUWrb/1nxVatWqTx27NiMzhVHnHHIhIVDJiwcMsmrNc6VK1dS24cOHVLH/O/r7e/vz+hc48aNU9n/ddLp95f8XxddDDjjkAkLh0zy6qmqtbX1jdtB+D9ysmbNGpUTiYTKDQ0NKvu7pxc7zjhkwsIhExYOmbDNCQ2KbU4oVCwcMmHhkAkLh0xYOGTCwiETFg6ZsHDIhIVDJiwcMmHhkEmk96pE5BGSn/osA/BdZCcenriOLVfjmu6c+9mH/iMtnNRJRTrfdOMsDuI6triNi09VZMLCIZNcFc5HOTpvEHEdW6zGlZM1DuU/PlWRSaSFIyKrRaRPRG6KSE7b24rICRF5KCI9afti0bs5H3pLR1Y4IpIAcAzA7wFUA1jv9UvOlWYAq3374tK7Of69pZ1zkfwB8BsAbWm5EUBjVOcfYExVAHrSch+Acm+7HEBfLseXNq6PAayI0/iifKqqAPBNWu739sVJ7Ho3x7W3NBfHA3DJ/9Y5fclp7S0dhSgL51sAlWn5bW9fnATq3RyFTHpLRyHKwukAMEtE3hGREgD1SPZKjpMfezcDOezdHKC3NJDr3tIRL/LeA/AVgFsA9uV4wdmC5JebvEJyvfU+gF8i+WrlBoALAEpzNLbfIvk0dA1At/fnvbiMzznHK8dkw8UxmbBwyISFQyYsHDJh4ZAJC4dMWDhkwsIhkx8AyyZIbO5tLBIAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 144x144 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"train_data0, train_label_0 = train_dataset[0][0],train_dataset[0][1]\n",
"train_data0 = train_data0.reshape([28,28])\n",
"plt.figure(figsize=(2,2))\n",
"plt.imshow(train_data0, cmap=plt.cm.binary)\n",
"print('train_data0 label is: ' + str(train_label_0))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 组网\n",
"用paddle.nn下的API,如`Conv2d`、`MaxPool2d`、`Linear`完成LeNet的构建。"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"import paddle\n",
"import paddle.nn.functional as F\n",
"class LeNet(paddle.nn.Layer):\n",
" def __init__(self):\n",
" super(LeNet, self).__init__()\n",
" self.conv1 = paddle.nn.Conv2d(in_channels=1, out_channels=6, kernel_size=5, stride=1, padding=2)\n",
" self.max_pool1 = paddle.nn.MaxPool2d(kernel_size=2, stride=2)\n",
" self.conv2 = paddle.nn.Conv2d(in_channels=6, out_channels=16, kernel_size=5, stride=1)\n",
" self.max_pool2 = paddle.nn.MaxPool2d(kernel_size=2, stride=2)\n",
" self.linear1 = paddle.nn.Linear(in_features=16*5*5, out_features=120)\n",
" self.linear2 = paddle.nn.Linear(in_features=120, out_features=84)\n",
" self.linear3 = paddle.nn.Linear(in_features=84, out_features=10)\n",
"\n",
" def forward(self, x):\n",
" x = self.conv1(x)\n",
" x = F.relu(x)\n",
" x = self.max_pool1(x)\n",
" x = F.relu(x)\n",
" x = self.conv2(x)\n",
" x = self.max_pool2(x)\n",
" x = paddle.flatten(x, start_axis=1,stop_axis=-1)\n",
" x = self.linear1(x)\n",
" x = F.relu(x)\n",
" x = self.linear2(x)\n",
" x = F.relu(x)\n",
" x = self.linear3(x)\n",
" x = F.softmax(x)\n",
" return x"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 训练方式一\n",
"组网后,开始对模型进行训练,先构建`train_loader`,加载训练数据,然后定义`train`函数,设置好损失函数后,按batch加载数据,完成模型的训练。"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch: 0, batch_id: 0, loss is: [2.3037894], acc is: [0.140625]\n",
"epoch: 0, batch_id: 100, loss is: [1.6175328], acc is: [0.9375]\n",
"epoch: 0, batch_id: 200, loss is: [1.5388051], acc is: [0.96875]\n",
"epoch: 0, batch_id: 300, loss is: [1.5251061], acc is: [0.96875]\n",
"epoch: 0, batch_id: 400, loss is: [1.4678856], acc is: [1.]\n",
"epoch: 0, batch_id: 500, loss is: [1.4944503], acc is: [0.984375]\n",
"epoch: 0, batch_id: 600, loss is: [1.5365536], acc is: [0.96875]\n",
"epoch: 0, batch_id: 700, loss is: [1.4885054], acc is: [0.984375]\n",
"epoch: 0, batch_id: 800, loss is: [1.4872254], acc is: [0.984375]\n",
"epoch: 0, batch_id: 900, loss is: [1.4884174], acc is: [0.984375]\n",
"epoch: 1, batch_id: 0, loss is: [1.4776722], acc is: [1.]\n",
"epoch: 1, batch_id: 100, loss is: [1.4751343], acc is: [1.]\n",
"epoch: 1, batch_id: 200, loss is: [1.4772581], acc is: [1.]\n",
"epoch: 1, batch_id: 300, loss is: [1.4918218], acc is: [0.984375]\n",
"epoch: 1, batch_id: 400, loss is: [1.5038397], acc is: [0.96875]\n",
"epoch: 1, batch_id: 500, loss is: [1.5088196], acc is: [0.96875]\n",
"epoch: 1, batch_id: 600, loss is: [1.4961376], acc is: [0.984375]\n",
"epoch: 1, batch_id: 700, loss is: [1.4755756], acc is: [1.]\n",
"epoch: 1, batch_id: 800, loss is: [1.4921497], acc is: [0.984375]\n",
"epoch: 1, batch_id: 900, loss is: [1.4944404], acc is: [1.]\n"
]
}
],
"source": [
"import paddle\n",
"train_loader = paddle.io.DataLoader(train_dataset, places=paddle.CPUPlace(), batch_size=64, shuffle=True)\n",
"# 加载训练集 batch_size 设为 64\n",
"def train(model):\n",
" model.train()\n",
" epochs = 2\n",
" optim = paddle.optimizer.Adam(learning_rate=0.001, parameters=model.parameters())\n",
" # 用Adam作为优化函数\n",
" for epoch in range(epochs):\n",
" for batch_id, data in enumerate(train_loader()):\n",
" x_data = data[0]\n",
" y_data = data[1]\n",
" predicts = model(x_data)\n",
" loss = paddle.nn.functional.cross_entropy(predicts, y_data)\n",
" # 计算损失\n",
" acc = paddle.metric.accuracy(predicts, y_data, k=2)\n",
" avg_loss = paddle.mean(loss)\n",
" avg_acc = paddle.mean(acc)\n",
" avg_loss.backward()\n",
" if batch_id % 100 == 0:\n",
" print(\"epoch: {}, batch_id: {}, loss is: {}, acc is: {}\".format(epoch, batch_id, avg_loss.numpy(), avg_acc.numpy()))\n",
" optim.step()\n",
" optim.clear_grad()\n",
"model = LeNet()\n",
"train(model)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 对模型进行验证\n",
"训练完成后,需要验证模型的效果,此时,加载测试数据集,然后用训练好的模对测试集进行预测,计算损失与精度。"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"batch_id: 0, loss is: [1.4915928], acc is: [1.]\n",
"batch_id: 20, loss is: [1.4818308], acc is: [1.]\n",
"batch_id: 40, loss is: [1.5006062], acc is: [0.984375]\n",
"batch_id: 60, loss is: [1.521233], acc is: [1.]\n",
"batch_id: 80, loss is: [1.4772738], acc is: [1.]\n",
"batch_id: 100, loss is: [1.4755945], acc is: [1.]\n",
"batch_id: 120, loss is: [1.4746133], acc is: [1.]\n",
"batch_id: 140, loss is: [1.4786345], acc is: [1.]\n"
]
}
],
"source": [
"import paddle\n",
"test_loader = paddle.io.DataLoader(test_dataset, places=paddle.CPUPlace(), batch_size=64)\n",
"# 加载测试数据集\n",
"def test(model):\n",
" model.eval()\n",
" batch_size = 64\n",
" for batch_id, data in enumerate(test_loader()):\n",
" x_data = data[0]\n",
" y_data = data[1]\n",
" predicts = model(x_data)\n",
" # 获取预测结果\n",
" loss = paddle.nn.functional.cross_entropy(predicts, y_data)\n",
" acc = paddle.metric.accuracy(predicts, y_data, k=2)\n",
" avg_loss = paddle.mean(loss)\n",
" avg_acc = paddle.mean(acc)\n",
" avg_loss.backward()\n",
" if batch_id % 20 == 0:\n",
" print(\"batch_id: {}, loss is: {}, acc is: {}\".format(batch_id, avg_loss.numpy(), avg_acc.numpy()))\n",
"test(model)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 训练方式一结束\n",
"以上就是训练方式一,通过这种方式,可以清楚的看到训练和测试中的每一步过程。但是,这种方式句法比较复杂。因此,我们提供了训练方式二,能够更加快速、高效的完成模型的训练与测试。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3.训练方式二\n",
"通过paddle提供的`Model` 构建实例,使用封装好的训练与测试接口,快速完成模型训练与测试。"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"import paddle\n",
"from paddle.static import InputSpec\n",
"from paddle.metric import Accuracy\n",
"inputs = InputSpec([None, 784], 'float32', 'x')\n",
"labels = InputSpec([None, 10], 'float32', 'x')\n",
"model = paddle.Model(LeNet(), inputs, labels)\n",
"optim = paddle.optimizer.Adam(learning_rate=0.001, parameters=model.parameters())\n",
"\n",
"model.prepare(\n",
" optim,\n",
" paddle.nn.loss.CrossEntropyLoss(),\n",
" Accuracy(topk=(1, 2))\n",
" )"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 使用model.fit来训练模型"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/2\n",
"step 200/938 - loss: 1.5219 - acc_top1: 0.9829 - acc_top2: 0.9965 - 14ms/step\n",
"step 400/938 - loss: 1.4765 - acc_top1: 0.9825 - acc_top2: 0.9958 - 13ms/step\n",
"step 600/938 - loss: 1.4624 - acc_top1: 0.9823 - acc_top2: 0.9953 - 13ms/step\n",
"step 800/938 - loss: 1.4768 - acc_top1: 0.9829 - acc_top2: 0.9955 - 13ms/step\n",
"step 938/938 - loss: 1.4612 - acc_top1: 0.9836 - acc_top2: 0.9956 - 13ms/step\n",
"Epoch 2/2\n",
"step 200/938 - loss: 1.4705 - acc_top1: 0.9834 - acc_top2: 0.9959 - 13ms/step\n",
"step 400/938 - loss: 1.4620 - acc_top1: 0.9833 - acc_top2: 0.9960 - 13ms/step\n",
"step 600/938 - loss: 1.4613 - acc_top1: 0.9830 - acc_top2: 0.9960 - 13ms/step\n",
"step 800/938 - loss: 1.4763 - acc_top1: 0.9831 - acc_top2: 0.9960 - 13ms/step\n",
"step 938/938 - loss: 1.4924 - acc_top1: 0.9834 - acc_top2: 0.9959 - 13ms/step\n"
]
}
],
"source": [
"model.fit(train_dataset,\n",
" epochs=2,\n",
" batch_size=64,\n",
" log_freq=200\n",
" )"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 使用model.evaluate来预测模型"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Eval begin...\n",
"step 20/157 - loss: 1.5246 - acc_top1: 0.9773 - acc_top2: 0.9969 - 6ms/step\n",
"step 40/157 - loss: 1.4622 - acc_top1: 0.9758 - acc_top2: 0.9961 - 6ms/step\n",
"step 60/157 - loss: 1.5241 - acc_top1: 0.9763 - acc_top2: 0.9951 - 6ms/step\n",
"step 80/157 - loss: 1.4612 - acc_top1: 0.9787 - acc_top2: 0.9959 - 6ms/step\n",
"step 100/157 - loss: 1.4612 - acc_top1: 0.9823 - acc_top2: 0.9967 - 5ms/step\n",
"step 120/157 - loss: 1.4612 - acc_top1: 0.9835 - acc_top2: 0.9966 - 5ms/step\n",
"step 140/157 - loss: 1.4612 - acc_top1: 0.9844 - acc_top2: 0.9969 - 5ms/step\n",
"step 157/157 - loss: 1.4612 - acc_top1: 0.9838 - acc_top2: 0.9966 - 5ms/step\n",
"Eval samples: 10000\n"
]
},
{
"data": {
"text/plain": [
"{'loss': [1.4611504], 'acc_top1': 0.9838, 'acc_top2': 0.9966}"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.evaluate(test_dataset, log_freq=20, batch_size=64)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 训练方式二结束\n",
"以上就是训练方式二,可以快速、高效的完成网络模型训练与预测。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 总结\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"以上就是用LeNet对手写数字数据及MNIST进行分类。本示例提供了两种训练模型的方式,一种可以快速完成模型的组建与预测,非常适合新手用户上手。另一种则需要多个步骤来完成模型的训练,适合进阶用户使用。"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.3"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
...@@ -7,18 +7,19 @@ MNIST数据集使用LeNet进行图像分类 ...@@ -7,18 +7,19 @@ MNIST数据集使用LeNet进行图像分类
环境 环境
---- ----
本教程基于paddle-develop编写,如果您的环境不是本版本,请先安装paddle-develop版本。 本教程基于paddle-2.0-beta编写,如果您的环境不是本版本,请先安装paddle-2.0-beta版本。
.. code:: ipython3 .. code:: ipython3
import paddle import paddle
print(paddle.__version__) print(paddle.__version__)
paddle.disable_static() paddle.disable_static()
# 开启动态图
.. parsed-literal:: .. parsed-literal::
0.0.0 2.0.0-beta0
加载数据集 加载数据集
...@@ -34,12 +35,6 @@ MNIST数据集使用LeNet进行图像分类 ...@@ -34,12 +35,6 @@ MNIST数据集使用LeNet进行图像分类
print('load finished') print('load finished')
.. parsed-literal::
/Library/Python/3.7/site-packages/ipykernel/ipkernel.py:287: DeprecationWarning: `should_run_async` will not call `transform_cell` automatically in the future. Please pass the result to `transformed_cell` argument and any exception that happen during thetransform in `preprocessing_exc_tuple` in IPython 7.17 and above.
and should_run_async(code)
.. parsed-literal:: .. parsed-literal::
download training data and load training data download training data and load training data
...@@ -64,12 +59,14 @@ MNIST数据集使用LeNet进行图像分类 ...@@ -64,12 +59,14 @@ MNIST数据集使用LeNet进行图像分类
train_data0 label is: [5] train_data0 label is: [5]
.. image:: https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/paddle/user_guides/cv_case/image_classification/image/cifar.png?raw=true
2.组网 .. image:: https://github.com/PaddlePaddle/FluidDoc/tree/develop/doc/paddle/tutorial/cv_case/image_segmentation/mnist_lenet_classification_files/mnist_lenet_classification_001.png?raw=true
------
paddle.nn下的API,如\ ``Conv2d``\ \ ``Pool2D``\ \ ``Linead``\ 完成LeNet的构建。 组网
----
paddle.nn下的API,如\ ``Conv2d``\ \ ``MaxPool2d``\ \ ``Linear``\ 完成LeNet的构建。
.. code:: ipython3 .. code:: ipython3
...@@ -93,7 +90,7 @@ MNIST数据集使用LeNet进行图像分类 ...@@ -93,7 +90,7 @@ MNIST数据集使用LeNet进行图像分类
x = F.relu(x) x = F.relu(x)
x = self.conv2(x) x = self.conv2(x)
x = self.max_pool2(x) x = self.max_pool2(x)
x = paddle.reshape(x, shape=[-1, 16*5*5]) x = paddle.flatten(x, start_axis=1,stop_axis=-1)
x = self.linear1(x) x = self.linear1(x)
x = F.relu(x) x = F.relu(x)
x = self.linear2(x) x = self.linear2(x)
...@@ -102,22 +99,15 @@ MNIST数据集使用LeNet进行图像分类 ...@@ -102,22 +99,15 @@ MNIST数据集使用LeNet进行图像分类
x = F.softmax(x) x = F.softmax(x)
return x return x
训练方式一
.. parsed-literal:: ----------
/Library/Python/3.7/site-packages/ipykernel/ipkernel.py:287: DeprecationWarning: `should_run_async` will not call `transform_cell` automatically in the future. Please pass the result to `transformed_cell` argument and any exception that happen during thetransform in `preprocessing_exc_tuple` in IPython 7.17 and above.
and should_run_async(code)
3.训练方式一
------------
组网后,开始对模型进行训练,先构建\ ``train_loader``\ ,加载训练数据,然后定义\ ``train``\ 函数,设置好损失函数后,按batch加载数据,完成模型的训练。 组网后,开始对模型进行训练,先构建\ ``train_loader``\ ,加载训练数据,然后定义\ ``train``\ 函数,设置好损失函数后,按batch加载数据,完成模型的训练。
.. code:: ipython3 .. code:: ipython3
import paddle import paddle
train_loader = paddle.io.DataLoader(train_dataset, places=paddle.CPUPlace(), batch_size=64) train_loader = paddle.io.DataLoader(train_dataset, places=paddle.CPUPlace(), batch_size=64, shuffle=True)
# 加载训练集 batch_size 设为 64 # 加载训练集 batch_size 设为 64
def train(model): def train(model):
model.train() model.train()
...@@ -137,34 +127,34 @@ MNIST数据集使用LeNet进行图像分类 ...@@ -137,34 +127,34 @@ MNIST数据集使用LeNet进行图像分类
avg_loss.backward() avg_loss.backward()
if batch_id % 100 == 0: if batch_id % 100 == 0:
print("epoch: {}, batch_id: {}, loss is: {}, acc is: {}".format(epoch, batch_id, avg_loss.numpy(), avg_acc.numpy())) print("epoch: {}, batch_id: {}, loss is: {}, acc is: {}".format(epoch, batch_id, avg_loss.numpy(), avg_acc.numpy()))
optim.minimize(avg_loss) optim.step()
model.clear_gradients() optim.clear_grad()
model = LeNet() model = LeNet()
train(model) train(model)
.. parsed-literal:: .. parsed-literal::
epoch: 0, batch_id: 0, loss is: [2.3062382], acc is: [0.109375] epoch: 0, batch_id: 0, loss is: [2.3037894], acc is: [0.140625]
epoch: 0, batch_id: 100, loss is: [1.6826601], acc is: [0.84375] epoch: 0, batch_id: 100, loss is: [1.6175328], acc is: [0.9375]
epoch: 0, batch_id: 200, loss is: [1.685574], acc is: [0.796875] epoch: 0, batch_id: 200, loss is: [1.5388051], acc is: [0.96875]
epoch: 0, batch_id: 300, loss is: [1.5752499], acc is: [0.96875] epoch: 0, batch_id: 300, loss is: [1.5251061], acc is: [0.96875]
epoch: 0, batch_id: 400, loss is: [1.5006541], acc is: [1.] epoch: 0, batch_id: 400, loss is: [1.4678856], acc is: [1.]
epoch: 0, batch_id: 500, loss is: [1.5343401], acc is: [0.984375] epoch: 0, batch_id: 500, loss is: [1.4944503], acc is: [0.984375]
epoch: 0, batch_id: 600, loss is: [1.4875913], acc is: [0.984375] epoch: 0, batch_id: 600, loss is: [1.5365536], acc is: [0.96875]
epoch: 0, batch_id: 700, loss is: [1.5139006], acc is: [0.984375] epoch: 0, batch_id: 700, loss is: [1.4885054], acc is: [0.984375]
epoch: 0, batch_id: 800, loss is: [1.5227785], acc is: [0.984375] epoch: 0, batch_id: 800, loss is: [1.4872254], acc is: [0.984375]
epoch: 0, batch_id: 900, loss is: [1.4938308], acc is: [1.] epoch: 0, batch_id: 900, loss is: [1.4884174], acc is: [0.984375]
epoch: 1, batch_id: 0, loss is: [1.4826943], acc is: [0.984375] epoch: 1, batch_id: 0, loss is: [1.4776722], acc is: [1.]
epoch: 1, batch_id: 100, loss is: [1.4852213], acc is: [0.984375] epoch: 1, batch_id: 100, loss is: [1.4751343], acc is: [1.]
epoch: 1, batch_id: 200, loss is: [1.5008337], acc is: [1.] epoch: 1, batch_id: 200, loss is: [1.4772581], acc is: [1.]
epoch: 1, batch_id: 300, loss is: [1.505826], acc is: [1.] epoch: 1, batch_id: 300, loss is: [1.4918218], acc is: [0.984375]
epoch: 1, batch_id: 400, loss is: [1.4768786], acc is: [1.] epoch: 1, batch_id: 400, loss is: [1.5038397], acc is: [0.96875]
epoch: 1, batch_id: 500, loss is: [1.4950027], acc is: [0.984375] epoch: 1, batch_id: 500, loss is: [1.5088196], acc is: [0.96875]
epoch: 1, batch_id: 600, loss is: [1.4762383], acc is: [0.984375] epoch: 1, batch_id: 600, loss is: [1.4961376], acc is: [0.984375]
epoch: 1, batch_id: 700, loss is: [1.5276604], acc is: [0.96875] epoch: 1, batch_id: 700, loss is: [1.4755756], acc is: [1.]
epoch: 1, batch_id: 800, loss is: [1.4897399], acc is: [1.] epoch: 1, batch_id: 800, loss is: [1.4921497], acc is: [0.984375]
epoch: 1, batch_id: 900, loss is: [1.4927337], acc is: [1.] epoch: 1, batch_id: 900, loss is: [1.4944404], acc is: [1.]
对模型进行验证 对模型进行验证
...@@ -180,7 +170,7 @@ MNIST数据集使用LeNet进行图像分类 ...@@ -180,7 +170,7 @@ MNIST数据集使用LeNet进行图像分类
def test(model): def test(model):
model.eval() model.eval()
batch_size = 64 batch_size = 64
for batch_id, data in enumerate(train_loader()): for batch_id, data in enumerate(test_loader()):
x_data = data[0] x_data = data[0]
y_data = data[1] y_data = data[1]
predicts = model(x_data) predicts = model(x_data)
...@@ -190,23 +180,21 @@ MNIST数据集使用LeNet进行图像分类 ...@@ -190,23 +180,21 @@ MNIST数据集使用LeNet进行图像分类
avg_loss = paddle.mean(loss) avg_loss = paddle.mean(loss)
avg_acc = paddle.mean(acc) avg_acc = paddle.mean(acc)
avg_loss.backward() avg_loss.backward()
if batch_id % 100 == 0: if batch_id % 20 == 0:
print("batch_id: {}, loss is: {}, acc is: {}".format(batch_id, avg_loss.numpy(), avg_acc.numpy())) print("batch_id: {}, loss is: {}, acc is: {}".format(batch_id, avg_loss.numpy(), avg_acc.numpy()))
test(model) test(model)
.. parsed-literal:: .. parsed-literal::
batch_id: 0, loss is: [1.4630548], acc is: [1.] batch_id: 0, loss is: [1.4915928], acc is: [1.]
batch_id: 100, loss is: [1.4789999], acc is: [0.984375] batch_id: 20, loss is: [1.4818308], acc is: [1.]
batch_id: 200, loss is: [1.4621592], acc is: [1.] batch_id: 40, loss is: [1.5006062], acc is: [0.984375]
batch_id: 300, loss is: [1.486401], acc is: [1.] batch_id: 60, loss is: [1.521233], acc is: [1.]
batch_id: 400, loss is: [1.4767764], acc is: [1.] batch_id: 80, loss is: [1.4772738], acc is: [1.]
batch_id: 500, loss is: [1.4987783], acc is: [0.984375] batch_id: 100, loss is: [1.4755945], acc is: [1.]
batch_id: 600, loss is: [1.4767168], acc is: [1.] batch_id: 120, loss is: [1.4746133], acc is: [1.]
batch_id: 700, loss is: [1.4876428], acc is: [0.984375] batch_id: 140, loss is: [1.4786345], acc is: [1.]
batch_id: 800, loss is: [1.4924926], acc is: [0.984375]
batch_id: 900, loss is: [1.4799261], acc is: [1.]
训练方式一结束 训练方式一结束
...@@ -244,214 +232,24 @@ MNIST数据集使用LeNet进行图像分类 ...@@ -244,214 +232,24 @@ MNIST数据集使用LeNet进行图像分类
model.fit(train_dataset, model.fit(train_dataset,
epochs=2, epochs=2,
batch_size=64, batch_size=64,
save_dir='mnist_checkpoint') log_freq=200
)
.. parsed-literal:: .. parsed-literal::
Epoch 1/2 Epoch 1/2
step 10/938 - loss: 2.2252 - acc_top1: 0.2547 - acc_top2: 0.4234 - 16ms/step step 200/938 - loss: 1.5219 - acc_top1: 0.9829 - acc_top2: 0.9965 - 14ms/step
step 400/938 - loss: 1.4765 - acc_top1: 0.9825 - acc_top2: 0.9958 - 13ms/step
step 600/938 - loss: 1.4624 - acc_top1: 0.9823 - acc_top2: 0.9953 - 13ms/step
.. parsed-literal:: step 800/938 - loss: 1.4768 - acc_top1: 0.9829 - acc_top2: 0.9955 - 13ms/step
step 938/938 - loss: 1.4612 - acc_top1: 0.9836 - acc_top2: 0.9956 - 13ms/step
/Library/Python/3.7/site-packages/paddle/fluid/layers/utils.py:76: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
return (isinstance(seq, collections.Sequence) and
.. parsed-literal::
step 20/938 - loss: 1.9721 - acc_top1: 0.3664 - acc_top2: 0.5164 - 15ms/step
step 30/938 - loss: 1.8697 - acc_top1: 0.4464 - acc_top2: 0.5651 - 15ms/step
step 40/938 - loss: 1.8475 - acc_top1: 0.4859 - acc_top2: 0.5898 - 15ms/step
step 50/938 - loss: 1.8683 - acc_top1: 0.5256 - acc_top2: 0.6156 - 14ms/step
step 60/938 - loss: 1.8091 - acc_top1: 0.5437 - acc_top2: 0.6237 - 14ms/step
step 70/938 - loss: 1.7934 - acc_top1: 0.5607 - acc_top2: 0.6335 - 14ms/step
step 80/938 - loss: 1.7796 - acc_top1: 0.5760 - acc_top2: 0.6418 - 14ms/step
step 90/938 - loss: 1.8004 - acc_top1: 0.5868 - acc_top2: 0.6476 - 14ms/step
step 100/938 - loss: 1.7650 - acc_top1: 0.5972 - acc_top2: 0.6536 - 14ms/step
step 110/938 - loss: 1.7839 - acc_top1: 0.6033 - acc_top2: 0.6570 - 14ms/step
step 120/938 - loss: 1.8094 - acc_top1: 0.6087 - acc_top2: 0.6592 - 14ms/step
step 130/938 - loss: 1.8125 - acc_top1: 0.6153 - acc_top2: 0.6638 - 14ms/step
step 140/938 - loss: 1.7318 - acc_top1: 0.6217 - acc_top2: 0.6673 - 14ms/step
step 150/938 - loss: 1.8209 - acc_top1: 0.6267 - acc_top2: 0.6702 - 14ms/step
step 160/938 - loss: 1.7661 - acc_top1: 0.6308 - acc_top2: 0.6725 - 14ms/step
step 170/938 - loss: 1.7099 - acc_top1: 0.6341 - acc_top2: 0.6741 - 14ms/step
step 180/938 - loss: 1.8059 - acc_top1: 0.6363 - acc_top2: 0.6753 - 14ms/step
step 190/938 - loss: 1.7681 - acc_top1: 0.6400 - acc_top2: 0.6779 - 14ms/step
step 200/938 - loss: 1.8631 - acc_top1: 0.6430 - acc_top2: 0.6826 - 14ms/step
step 210/938 - loss: 1.6808 - acc_top1: 0.6479 - acc_top2: 0.6879 - 14ms/step
step 220/938 - loss: 1.5447 - acc_top1: 0.6558 - acc_top2: 0.6965 - 14ms/step
step 230/938 - loss: 1.6170 - acc_top1: 0.6641 - acc_top2: 0.7051 - 14ms/step
step 240/938 - loss: 1.6190 - acc_top1: 0.6719 - acc_top2: 0.7134 - 14ms/step
step 250/938 - loss: 1.5698 - acc_top1: 0.6794 - acc_top2: 0.7209 - 14ms/step
step 260/938 - loss: 1.6071 - acc_top1: 0.6869 - acc_top2: 0.7284 - 14ms/step
step 270/938 - loss: 1.5507 - acc_top1: 0.6939 - acc_top2: 0.7364 - 14ms/step
step 280/938 - loss: 1.5286 - acc_top1: 0.7023 - acc_top2: 0.7451 - 14ms/step
step 290/938 - loss: 1.5740 - acc_top1: 0.7098 - acc_top2: 0.7532 - 14ms/step
step 300/938 - loss: 1.5179 - acc_top1: 0.7172 - acc_top2: 0.7608 - 14ms/step
step 310/938 - loss: 1.5325 - acc_top1: 0.7240 - acc_top2: 0.7677 - 14ms/step
step 320/938 - loss: 1.4961 - acc_top1: 0.7305 - acc_top2: 0.7744 - 14ms/step
step 330/938 - loss: 1.5420 - acc_top1: 0.7369 - acc_top2: 0.7804 - 14ms/step
step 340/938 - loss: 1.5652 - acc_top1: 0.7427 - acc_top2: 0.7861 - 14ms/step
step 350/938 - loss: 1.5122 - acc_top1: 0.7484 - acc_top2: 0.7918 - 14ms/step
step 360/938 - loss: 1.5308 - acc_top1: 0.7544 - acc_top2: 0.7972 - 14ms/step
step 370/938 - loss: 1.5354 - acc_top1: 0.7596 - acc_top2: 0.8023 - 14ms/step
step 380/938 - loss: 1.5433 - acc_top1: 0.7645 - acc_top2: 0.8073 - 14ms/step
step 390/938 - loss: 1.5341 - acc_top1: 0.7693 - acc_top2: 0.8119 - 14ms/step
step 400/938 - loss: 1.4826 - acc_top1: 0.7740 - acc_top2: 0.8163 - 14ms/step
step 410/938 - loss: 1.4995 - acc_top1: 0.7785 - acc_top2: 0.8205 - 14ms/step
step 420/938 - loss: 1.5057 - acc_top1: 0.7827 - acc_top2: 0.8244 - 14ms/step
step 430/938 - loss: 1.4927 - acc_top1: 0.7866 - acc_top2: 0.8282 - 14ms/step
step 440/938 - loss: 1.5281 - acc_top1: 0.7902 - acc_top2: 0.8316 - 14ms/step
step 450/938 - loss: 1.5060 - acc_top1: 0.7936 - acc_top2: 0.8347 - 14ms/step
step 460/938 - loss: 1.5135 - acc_top1: 0.7968 - acc_top2: 0.8380 - 14ms/step
step 470/938 - loss: 1.5206 - acc_top1: 0.8004 - acc_top2: 0.8411 - 14ms/step
step 480/938 - loss: 1.4963 - acc_top1: 0.8039 - acc_top2: 0.8441 - 14ms/step
step 490/938 - loss: 1.4984 - acc_top1: 0.8071 - acc_top2: 0.8470 - 14ms/step
step 500/938 - loss: 1.4947 - acc_top1: 0.8101 - acc_top2: 0.8498 - 14ms/step
step 510/938 - loss: 1.4639 - acc_top1: 0.8130 - acc_top2: 0.8524 - 14ms/step
step 520/938 - loss: 1.4781 - acc_top1: 0.8158 - acc_top2: 0.8549 - 14ms/step
step 530/938 - loss: 1.4806 - acc_top1: 0.8187 - acc_top2: 0.8575 - 14ms/step
step 540/938 - loss: 1.4830 - acc_top1: 0.8214 - acc_top2: 0.8600 - 14ms/step
step 550/938 - loss: 1.4852 - acc_top1: 0.8239 - acc_top2: 0.8623 - 14ms/step
step 560/938 - loss: 1.5302 - acc_top1: 0.8263 - acc_top2: 0.8645 - 14ms/step
step 570/938 - loss: 1.5520 - acc_top1: 0.8286 - acc_top2: 0.8667 - 14ms/step
step 580/938 - loss: 1.4897 - acc_top1: 0.8305 - acc_top2: 0.8687 - 14ms/step
step 590/938 - loss: 1.4857 - acc_top1: 0.8328 - acc_top2: 0.8707 - 14ms/step
step 600/938 - loss: 1.5081 - acc_top1: 0.8351 - acc_top2: 0.8727 - 14ms/step
step 610/938 - loss: 1.5013 - acc_top1: 0.8373 - acc_top2: 0.8746 - 14ms/step
step 620/938 - loss: 1.4949 - acc_top1: 0.8395 - acc_top2: 0.8764 - 14ms/step
step 630/938 - loss: 1.4971 - acc_top1: 0.8412 - acc_top2: 0.8781 - 14ms/step
step 640/938 - loss: 1.4869 - acc_top1: 0.8434 - acc_top2: 0.8800 - 14ms/step
step 650/938 - loss: 1.5202 - acc_top1: 0.8450 - acc_top2: 0.8815 - 14ms/step
step 660/938 - loss: 1.5002 - acc_top1: 0.8468 - acc_top2: 0.8832 - 14ms/step
step 670/938 - loss: 1.5178 - acc_top1: 0.8487 - acc_top2: 0.8848 - 14ms/step
step 680/938 - loss: 1.4939 - acc_top1: 0.8504 - acc_top2: 0.8864 - 14ms/step
step 690/938 - loss: 1.4650 - acc_top1: 0.8520 - acc_top2: 0.8878 - 14ms/step
step 700/938 - loss: 1.4934 - acc_top1: 0.8537 - acc_top2: 0.8892 - 14ms/step
step 710/938 - loss: 1.5473 - acc_top1: 0.8552 - acc_top2: 0.8905 - 14ms/step
step 720/938 - loss: 1.4956 - acc_top1: 0.8568 - acc_top2: 0.8918 - 14ms/step
step 730/938 - loss: 1.4644 - acc_top1: 0.8583 - acc_top2: 0.8932 - 14ms/step
step 740/938 - loss: 1.4868 - acc_top1: 0.8598 - acc_top2: 0.8946 - 14ms/step
step 750/938 - loss: 1.5142 - acc_top1: 0.8613 - acc_top2: 0.8959 - 14ms/step
step 760/938 - loss: 1.4656 - acc_top1: 0.8628 - acc_top2: 0.8971 - 14ms/step
step 770/938 - loss: 1.5005 - acc_top1: 0.8641 - acc_top2: 0.8983 - 14ms/step
step 780/938 - loss: 1.5557 - acc_top1: 0.8653 - acc_top2: 0.8994 - 14ms/step
step 790/938 - loss: 1.4687 - acc_top1: 0.8666 - acc_top2: 0.9006 - 14ms/step
step 800/938 - loss: 1.4686 - acc_top1: 0.8680 - acc_top2: 0.9017 - 14ms/step
step 810/938 - loss: 1.5202 - acc_top1: 0.8693 - acc_top2: 0.9028 - 14ms/step
step 820/938 - loss: 1.4773 - acc_top1: 0.8705 - acc_top2: 0.9038 - 14ms/step
step 830/938 - loss: 1.4838 - acc_top1: 0.8717 - acc_top2: 0.9049 - 14ms/step
step 840/938 - loss: 1.4726 - acc_top1: 0.8728 - acc_top2: 0.9059 - 14ms/step
step 850/938 - loss: 1.4734 - acc_top1: 0.8741 - acc_top2: 0.9069 - 14ms/step
step 860/938 - loss: 1.4627 - acc_top1: 0.8752 - acc_top2: 0.9078 - 14ms/step
step 870/938 - loss: 1.4872 - acc_top1: 0.8763 - acc_top2: 0.9088 - 14ms/step
step 880/938 - loss: 1.4916 - acc_top1: 0.8773 - acc_top2: 0.9096 - 14ms/step
step 890/938 - loss: 1.4818 - acc_top1: 0.8784 - acc_top2: 0.9105 - 14ms/step
step 900/938 - loss: 1.4967 - acc_top1: 0.8794 - acc_top2: 0.9114 - 14ms/step
step 910/938 - loss: 1.4614 - acc_top1: 0.8804 - acc_top2: 0.9123 - 14ms/step
step 920/938 - loss: 1.4819 - acc_top1: 0.8815 - acc_top2: 0.9132 - 14ms/step
step 930/938 - loss: 1.5114 - acc_top1: 0.8824 - acc_top2: 0.9140 - 14ms/step
step 938/938 - loss: 1.4621 - acc_top1: 0.8832 - acc_top2: 0.9146 - 14ms/step
save checkpoint at /Users/chenlong/online_repo/book/paddle2.0_docs/image_classification/mnist_checkpoint/0
Epoch 2/2 Epoch 2/2
step 10/938 - loss: 1.5033 - acc_top1: 0.9734 - acc_top2: 0.9906 - 15ms/step step 200/938 - loss: 1.4705 - acc_top1: 0.9834 - acc_top2: 0.9959 - 13ms/step
step 20/938 - loss: 1.4812 - acc_top1: 0.9734 - acc_top2: 0.9906 - 14ms/step step 400/938 - loss: 1.4620 - acc_top1: 0.9833 - acc_top2: 0.9960 - 13ms/step
step 30/938 - loss: 1.4623 - acc_top1: 0.9714 - acc_top2: 0.9911 - 14ms/step step 600/938 - loss: 1.4613 - acc_top1: 0.9830 - acc_top2: 0.9960 - 13ms/step
step 40/938 - loss: 1.4775 - acc_top1: 0.9711 - acc_top2: 0.9918 - 14ms/step step 800/938 - loss: 1.4763 - acc_top1: 0.9831 - acc_top2: 0.9960 - 13ms/step
step 50/938 - loss: 1.4857 - acc_top1: 0.9712 - acc_top2: 0.9922 - 14ms/step step 938/938 - loss: 1.4924 - acc_top1: 0.9834 - acc_top2: 0.9959 - 13ms/step
step 60/938 - loss: 1.4895 - acc_top1: 0.9695 - acc_top2: 0.9904 - 14ms/step
step 70/938 - loss: 1.4746 - acc_top1: 0.9708 - acc_top2: 0.9908 - 14ms/step
step 80/938 - loss: 1.4945 - acc_top1: 0.9719 - acc_top2: 0.9912 - 14ms/step
step 90/938 - loss: 1.4644 - acc_top1: 0.9722 - acc_top2: 0.9911 - 14ms/step
step 100/938 - loss: 1.4727 - acc_top1: 0.9722 - acc_top2: 0.9912 - 14ms/step
step 110/938 - loss: 1.4634 - acc_top1: 0.9720 - acc_top2: 0.9915 - 14ms/step
step 120/938 - loss: 1.4856 - acc_top1: 0.9730 - acc_top2: 0.9915 - 14ms/step
step 130/938 - loss: 1.4778 - acc_top1: 0.9736 - acc_top2: 0.9916 - 14ms/step
step 140/938 - loss: 1.4949 - acc_top1: 0.9730 - acc_top2: 0.9914 - 14ms/step
step 150/938 - loss: 1.4836 - acc_top1: 0.9726 - acc_top2: 0.9914 - 14ms/step
step 160/938 - loss: 1.5430 - acc_top1: 0.9725 - acc_top2: 0.9917 - 14ms/step
step 170/938 - loss: 1.4882 - acc_top1: 0.9722 - acc_top2: 0.9916 - 14ms/step
step 180/938 - loss: 1.4777 - acc_top1: 0.9721 - acc_top2: 0.9919 - 14ms/step
step 190/938 - loss: 1.4816 - acc_top1: 0.9723 - acc_top2: 0.9920 - 14ms/step
step 200/938 - loss: 1.4916 - acc_top1: 0.9730 - acc_top2: 0.9923 - 14ms/step
step 210/938 - loss: 1.5290 - acc_top1: 0.9734 - acc_top2: 0.9923 - 14ms/step
step 220/938 - loss: 1.5006 - acc_top1: 0.9736 - acc_top2: 0.9923 - 14ms/step
step 230/938 - loss: 1.5103 - acc_top1: 0.9737 - acc_top2: 0.9923 - 14ms/step
step 240/938 - loss: 1.4905 - acc_top1: 0.9733 - acc_top2: 0.9920 - 14ms/step
step 250/938 - loss: 1.5066 - acc_top1: 0.9734 - acc_top2: 0.9920 - 14ms/step
step 260/938 - loss: 1.4846 - acc_top1: 0.9736 - acc_top2: 0.9920 - 14ms/step
step 270/938 - loss: 1.4717 - acc_top1: 0.9738 - acc_top2: 0.9921 - 14ms/step
step 280/938 - loss: 1.4648 - acc_top1: 0.9742 - acc_top2: 0.9921 - 14ms/step
step 290/938 - loss: 1.4657 - acc_top1: 0.9745 - acc_top2: 0.9921 - 14ms/step
step 300/938 - loss: 1.4630 - acc_top1: 0.9744 - acc_top2: 0.9920 - 14ms/step
step 310/938 - loss: 1.5053 - acc_top1: 0.9742 - acc_top2: 0.9918 - 14ms/step
step 320/938 - loss: 1.4843 - acc_top1: 0.9745 - acc_top2: 0.9919 - 14ms/step
step 330/938 - loss: 1.4915 - acc_top1: 0.9745 - acc_top2: 0.9919 - 14ms/step
step 340/938 - loss: 1.5146 - acc_top1: 0.9745 - acc_top2: 0.9918 - 14ms/step
step 350/938 - loss: 1.4768 - acc_top1: 0.9742 - acc_top2: 0.9916 - 14ms/step
step 360/938 - loss: 1.4827 - acc_top1: 0.9743 - acc_top2: 0.9918 - 14ms/step
step 370/938 - loss: 1.5097 - acc_top1: 0.9740 - acc_top2: 0.9917 - 14ms/step
step 380/938 - loss: 1.5225 - acc_top1: 0.9739 - acc_top2: 0.9916 - 14ms/step
step 390/938 - loss: 1.4701 - acc_top1: 0.9740 - acc_top2: 0.9917 - 14ms/step
step 400/938 - loss: 1.4986 - acc_top1: 0.9741 - acc_top2: 0.9920 - 14ms/step
step 410/938 - loss: 1.5210 - acc_top1: 0.9740 - acc_top2: 0.9918 - 14ms/step
step 420/938 - loss: 1.4799 - acc_top1: 0.9740 - acc_top2: 0.9917 - 14ms/step
step 430/938 - loss: 1.4845 - acc_top1: 0.9744 - acc_top2: 0.9919 - 14ms/step
step 440/938 - loss: 1.4773 - acc_top1: 0.9741 - acc_top2: 0.9918 - 14ms/step
step 450/938 - loss: 1.4719 - acc_top1: 0.9743 - acc_top2: 0.9918 - 14ms/step
step 460/938 - loss: 1.4773 - acc_top1: 0.9742 - acc_top2: 0.9918 - 14ms/step
step 470/938 - loss: 1.4944 - acc_top1: 0.9741 - acc_top2: 0.9918 - 14ms/step
step 480/938 - loss: 1.4793 - acc_top1: 0.9743 - acc_top2: 0.9919 - 14ms/step
step 490/938 - loss: 1.4625 - acc_top1: 0.9746 - acc_top2: 0.9920 - 14ms/step
step 500/938 - loss: 1.4829 - acc_top1: 0.9745 - acc_top2: 0.9921 - 14ms/step
step 510/938 - loss: 1.4659 - acc_top1: 0.9747 - acc_top2: 0.9921 - 14ms/step
step 520/938 - loss: 1.4862 - acc_top1: 0.9743 - acc_top2: 0.9921 - 14ms/step
step 530/938 - loss: 1.5039 - acc_top1: 0.9742 - acc_top2: 0.9921 - 14ms/step
step 540/938 - loss: 1.5070 - acc_top1: 0.9740 - acc_top2: 0.9921 - 14ms/step
step 550/938 - loss: 1.5033 - acc_top1: 0.9740 - acc_top2: 0.9922 - 14ms/step
step 560/938 - loss: 1.4846 - acc_top1: 0.9741 - acc_top2: 0.9921 - 14ms/step
step 570/938 - loss: 1.4613 - acc_top1: 0.9741 - acc_top2: 0.9921 - 14ms/step
step 580/938 - loss: 1.4616 - acc_top1: 0.9743 - acc_top2: 0.9921 - 14ms/step
step 590/938 - loss: 1.4801 - acc_top1: 0.9745 - acc_top2: 0.9921 - 14ms/step
step 600/938 - loss: 1.4772 - acc_top1: 0.9746 - acc_top2: 0.9921 - 14ms/step
step 610/938 - loss: 1.4612 - acc_top1: 0.9746 - acc_top2: 0.9921 - 14ms/step
step 620/938 - loss: 1.4951 - acc_top1: 0.9746 - acc_top2: 0.9922 - 14ms/step
step 630/938 - loss: 1.4755 - acc_top1: 0.9747 - acc_top2: 0.9923 - 14ms/step
step 640/938 - loss: 1.5296 - acc_top1: 0.9749 - acc_top2: 0.9924 - 14ms/step
step 650/938 - loss: 1.5054 - acc_top1: 0.9748 - acc_top2: 0.9924 - 14ms/step
step 660/938 - loss: 1.4775 - acc_top1: 0.9749 - acc_top2: 0.9925 - 14ms/step
step 670/938 - loss: 1.4829 - acc_top1: 0.9749 - acc_top2: 0.9925 - 14ms/step
step 680/938 - loss: 1.4612 - acc_top1: 0.9750 - acc_top2: 0.9926 - 14ms/step
step 690/938 - loss: 1.4869 - acc_top1: 0.9751 - acc_top2: 0.9926 - 14ms/step
step 700/938 - loss: 1.4612 - acc_top1: 0.9752 - acc_top2: 0.9927 - 14ms/step
step 710/938 - loss: 1.5235 - acc_top1: 0.9752 - acc_top2: 0.9927 - 14ms/step
step 720/938 - loss: 1.5317 - acc_top1: 0.9752 - acc_top2: 0.9926 - 14ms/step
step 730/938 - loss: 1.4898 - acc_top1: 0.9751 - acc_top2: 0.9926 - 14ms/step
step 740/938 - loss: 1.4612 - acc_top1: 0.9753 - acc_top2: 0.9926 - 14ms/step
step 750/938 - loss: 1.4935 - acc_top1: 0.9752 - acc_top2: 0.9926 - 14ms/step
step 760/938 - loss: 1.5140 - acc_top1: 0.9749 - acc_top2: 0.9926 - 14ms/step
step 770/938 - loss: 1.4883 - acc_top1: 0.9748 - acc_top2: 0.9925 - 14ms/step
step 780/938 - loss: 1.4759 - acc_top1: 0.9748 - acc_top2: 0.9926 - 14ms/step
step 790/938 - loss: 1.4773 - acc_top1: 0.9750 - acc_top2: 0.9926 - 14ms/step
step 800/938 - loss: 1.4766 - acc_top1: 0.9750 - acc_top2: 0.9926 - 14ms/step
step 810/938 - loss: 1.5058 - acc_top1: 0.9750 - acc_top2: 0.9927 - 14ms/step
step 820/938 - loss: 1.4867 - acc_top1: 0.9749 - acc_top2: 0.9927 - 14ms/step
step 830/938 - loss: 1.4766 - acc_top1: 0.9748 - acc_top2: 0.9927 - 14ms/step
step 840/938 - loss: 1.4680 - acc_top1: 0.9747 - acc_top2: 0.9927 - 14ms/step
step 850/938 - loss: 1.4628 - acc_top1: 0.9746 - acc_top2: 0.9927 - 14ms/step
step 860/938 - loss: 1.5035 - acc_top1: 0.9747 - acc_top2: 0.9928 - 14ms/step
step 870/938 - loss: 1.4857 - acc_top1: 0.9748 - acc_top2: 0.9928 - 14ms/step
step 880/938 - loss: 1.4767 - acc_top1: 0.9748 - acc_top2: 0.9927 - 14ms/step
step 890/938 - loss: 1.4612 - acc_top1: 0.9750 - acc_top2: 0.9928 - 14ms/step
step 900/938 - loss: 1.4620 - acc_top1: 0.9751 - acc_top2: 0.9928 - 14ms/step
step 910/938 - loss: 1.4621 - acc_top1: 0.9751 - acc_top2: 0.9928 - 14ms/step
step 920/938 - loss: 1.4768 - acc_top1: 0.9751 - acc_top2: 0.9927 - 14ms/step
step 930/938 - loss: 1.4806 - acc_top1: 0.9752 - acc_top2: 0.9928 - 14ms/step
step 938/938 - loss: 1.4910 - acc_top1: 0.9752 - acc_top2: 0.9928 - 14ms/step
save checkpoint at /Users/chenlong/online_repo/book/paddle2.0_docs/image_classification/mnist_checkpoint/1
save checkpoint at /Users/chenlong/online_repo/book/paddle2.0_docs/image_classification/mnist_checkpoint/final
使用model.evaluate来预测模型 使用model.evaluate来预测模型
...@@ -459,28 +257,20 @@ MNIST数据集使用LeNet进行图像分类 ...@@ -459,28 +257,20 @@ MNIST数据集使用LeNet进行图像分类
.. code:: ipython3 .. code:: ipython3
model.evaluate(test_dataset, batch_size=64) model.evaluate(test_dataset, log_freq=20, batch_size=64)
.. parsed-literal:: .. parsed-literal::
Eval begin... Eval begin...
step 10/157 - loss: 1.5014 - acc_top1: 0.9766 - acc_top2: 0.9953 - 6ms/step step 20/157 - loss: 1.5246 - acc_top1: 0.9773 - acc_top2: 0.9969 - 6ms/step
step 20/157 - loss: 1.5239 - acc_top1: 0.9742 - acc_top2: 0.9922 - 6ms/step step 40/157 - loss: 1.4622 - acc_top1: 0.9758 - acc_top2: 0.9961 - 6ms/step
step 30/157 - loss: 1.4926 - acc_top1: 0.9740 - acc_top2: 0.9932 - 6ms/step step 60/157 - loss: 1.5241 - acc_top1: 0.9763 - acc_top2: 0.9951 - 6ms/step
step 40/157 - loss: 1.4612 - acc_top1: 0.9734 - acc_top2: 0.9938 - 6ms/step step 80/157 - loss: 1.4612 - acc_top1: 0.9787 - acc_top2: 0.9959 - 6ms/step
step 50/157 - loss: 1.4612 - acc_top1: 0.9719 - acc_top2: 0.9938 - 6ms/step step 100/157 - loss: 1.4612 - acc_top1: 0.9823 - acc_top2: 0.9967 - 5ms/step
step 60/157 - loss: 1.5114 - acc_top1: 0.9721 - acc_top2: 0.9938 - 6ms/step step 120/157 - loss: 1.4612 - acc_top1: 0.9835 - acc_top2: 0.9966 - 5ms/step
step 70/157 - loss: 1.4793 - acc_top1: 0.9696 - acc_top2: 0.9935 - 6ms/step step 140/157 - loss: 1.4612 - acc_top1: 0.9844 - acc_top2: 0.9969 - 5ms/step
step 80/157 - loss: 1.4736 - acc_top1: 0.9695 - acc_top2: 0.9932 - 6ms/step step 157/157 - loss: 1.4612 - acc_top1: 0.9838 - acc_top2: 0.9966 - 5ms/step
step 90/157 - loss: 1.4892 - acc_top1: 0.9720 - acc_top2: 0.9939 - 6ms/step
step 100/157 - loss: 1.4623 - acc_top1: 0.9738 - acc_top2: 0.9941 - 6ms/step
step 110/157 - loss: 1.4612 - acc_top1: 0.9737 - acc_top2: 0.9939 - 6ms/step
step 120/157 - loss: 1.4612 - acc_top1: 0.9746 - acc_top2: 0.9939 - 6ms/step
step 130/157 - loss: 1.4703 - acc_top1: 0.9757 - acc_top2: 0.9942 - 6ms/step
step 140/157 - loss: 1.4612 - acc_top1: 0.9771 - acc_top2: 0.9946 - 6ms/step
step 150/157 - loss: 1.4748 - acc_top1: 0.9782 - acc_top2: 0.9950 - 6ms/step
step 157/157 - loss: 1.4612 - acc_top1: 0.9770 - acc_top2: 0.9949 - 6ms/step
Eval samples: 10000 Eval samples: 10000
...@@ -488,7 +278,7 @@ MNIST数据集使用LeNet进行图像分类 ...@@ -488,7 +278,7 @@ MNIST数据集使用LeNet进行图像分类
.. parsed-literal:: .. parsed-literal::
{'loss': [1.4611504], 'acc_top1': 0.977, 'acc_top2': 0.9949} {'loss': [1.4611504], 'acc_top1': 0.9838, 'acc_top2': 0.9966}
......
...@@ -9,31 +9,28 @@ ...@@ -9,31 +9,28 @@
"本示例教程演示如何在IMDB数据集上用简单的BOW网络完成文本分类的任务。\n", "本示例教程演示如何在IMDB数据集上用简单的BOW网络完成文本分类的任务。\n",
"\n", "\n",
"IMDB数据集是一个对电影评论标注为正向评论与负向评论的数据集,共有25000条文本数据作为训练集,25000条文本数据作为测试集。\n", "IMDB数据集是一个对电影评论标注为正向评论与负向评论的数据集,共有25000条文本数据作为训练集,25000条文本数据作为测试集。\n",
"该数据集的官方地址为: http://ai.stanford.edu/~amaas/data/sentiment/\n", "该数据集的官方地址为: http://ai.stanford.edu/~amaas/data/sentiment/"
"\n",
"- Warning: `paddle.dataset.imdb`先在是一个非常粗野的实现,后续需要有替代的方案。"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 环境设置\n", "## 环境设置\n",
"\n", "\n",
"本示例基于飞桨开源框架2.0版本。" "本示例基于飞桨开源框架2.0版本。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 4, "execution_count": 2,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"0.0.0\n", "2.0.0-beta0\n"
"264e76cae6861ad9b1d4bcd8c3212f7a78c01e4d\n"
] ]
} }
], ],
...@@ -42,22 +39,21 @@ ...@@ -42,22 +39,21 @@
"import numpy as np\n", "import numpy as np\n",
"\n", "\n",
"paddle.disable_static()\n", "paddle.disable_static()\n",
"print(paddle.__version__)\n", "print(paddle.__version__)"
"print(paddle.__git_commit__)\n"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 加载数据\n", "## 加载数据\n",
"\n", "\n",
"我们会使用`paddle.dataset`完成数据下载,构建字典和准备数据读取器。在飞桨2.0版本中,推荐使用padding的方式来对同一个batch中长度不一的数据进行补齐,所以在字典中,我们还会添加一个特殊的`<pad>`词,用来在后续对batch中较短的句子进行填充。" "我们会使用`paddle.dataset`完成数据下载,构建字典和准备数据读取器。在飞桨2.0版本中,推荐使用padding的方式来对同一个batch中长度不一的数据进行补齐,所以在字典中,我们还会添加一个特殊的`<pad>`词,用来在后续对batch中较短的句子进行填充。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 5, "execution_count": 3,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -78,7 +74,7 @@ ...@@ -78,7 +74,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 6, "execution_count": 4,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -119,14 +115,14 @@ ...@@ -119,14 +115,14 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 参数设置\n", "## 参数设置\n",
"\n", "\n",
"在这里我们设置一下词表大小,`embedding`的大小,batch_size,等等" "在这里我们设置一下词表大小,`embedding`的大小,batch_size,等等"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 7, "execution_count": 22,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
...@@ -157,7 +153,7 @@ ...@@ -157,7 +153,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 8, "execution_count": 23,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -183,14 +179,14 @@ ...@@ -183,14 +179,14 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 用padding的方式对齐数据\n", "## 用padding的方式对齐数据\n",
"\n", "\n",
"文本数据中,每一句话的长度都是不一样的,为了方便后续的神经网络的计算,常见的处理方式是把数据集中的数据都统一成同样长度的数据。这包括:对于较长的数据进行截断处理,对于较短的数据用特殊的词`<pad>`进行填充。接下来的代码会对数据集中的数据进行这样的处理。" "文本数据中,每一句话的长度都是不一样的,为了方便后续的神经网络的计算,常见的处理方式是把数据集中的数据都统一成同样长度的数据。这包括:对于较长的数据进行截断处理,对于较短的数据用特殊的词`<pad>`进行填充。接下来的代码会对数据集中的数据进行这样的处理。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 9, "execution_count": 24,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -234,14 +230,14 @@ ...@@ -234,14 +230,14 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 组建网络\n", "## 组建网络\n",
"\n", "\n",
"本示例中,我们将会使用一个不考虑词的顺序的BOW的网络,在查找到每个词对应的embedding后,简单的取平均,作为一个句子的表示。然后用`Linear`进行线性变换。为了防止过拟合,我们还使用了`Dropout`。" "本示例中,我们将会使用一个不考虑词的顺序的BOW的网络,在查找到每个词对应的embedding后,简单的取平均,作为一个句子的表示。然后用`Linear`进行线性变换。为了防止过拟合,我们还使用了`Dropout`。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 11, "execution_count": 25,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
...@@ -264,24 +260,24 @@ ...@@ -264,24 +260,24 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 开始模型的训练\n" "## 开始模型的训练\n"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 13, "execution_count": 26,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"epoch: 0, batch_id: 0, loss is: [0.6926701]\n", "epoch: 0, batch_id: 0, loss is: [0.6918494]\n",
"epoch: 0, batch_id: 500, loss is: [0.41248566]\n", "epoch: 0, batch_id: 500, loss is: [0.33142853]\n",
"[validation] accuracy/loss: 0.8505121469497681/0.3615057170391083\n", "[validation] accuracy/loss: 0.8506321907043457/0.3620821535587311\n",
"epoch: 1, batch_id: 0, loss is: [0.29521096]\n", "epoch: 1, batch_id: 0, loss is: [0.37161]\n",
"epoch: 1, batch_id: 500, loss is: [0.2916747]\n", "epoch: 1, batch_id: 500, loss is: [0.2296829]\n",
"[validation] accuracy/loss: 0.86475670337677/0.3259459137916565\n" "[validation] accuracy/loss: 0.8622759580612183/0.3286365270614624\n"
] ]
} }
], ],
...@@ -311,8 +307,8 @@ ...@@ -311,8 +307,8 @@
" if batch_id % 500 == 0:\n", " if batch_id % 500 == 0:\n",
" print(\"epoch: {}, batch_id: {}, loss is: {}\".format(epoch, batch_id, avg_loss.numpy()))\n", " print(\"epoch: {}, batch_id: {}, loss is: {}\".format(epoch, batch_id, avg_loss.numpy()))\n",
" avg_loss.backward()\n", " avg_loss.backward()\n",
" opt.minimize(avg_loss)\n", " opt.step()\n",
" model.clear_gradients()\n", " opt.clear_grad()\n",
"\n", "\n",
" # evaluate model after one epoch\n", " # evaluate model after one epoch\n",
" model.eval()\n", " model.eval()\n",
...@@ -345,17 +341,10 @@ ...@@ -345,17 +341,10 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# The End\n", "## The End\n",
"\n", "\n",
"可以看到,在这个数据集上,经过两轮的迭代可以得到86%左右的准确率。你也可以通过调整网络结构和超参数,来获得更好的效果。" "可以看到,在这个数据集上,经过两轮的迭代可以得到86%左右的准确率。你也可以通过调整网络结构和超参数,来获得更好的效果。"
] ]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
} }
], ],
"metadata": { "metadata": {
...@@ -369,8 +358,20 @@ ...@@ -369,8 +358,20 @@
"display_name": "Python 3", "display_name": "Python 3",
"language": "python", "language": "python",
"name": "python3" "name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.3"
} }
}, },
"nbformat": 4, "nbformat": 4,
"nbformat_minor": 1 "nbformat_minor": 4
} }
...@@ -6,29 +6,23 @@ IMDB 数据集使用BOW网络的文本分类 ...@@ -6,29 +6,23 @@ IMDB 数据集使用BOW网络的文本分类
IMDB数据集是一个对电影评论标注为正向评论与负向评论的数据集,共有25000条文本数据作为训练集,25000条文本数据作为测试集。 IMDB数据集是一个对电影评论标注为正向评论与负向评论的数据集,共有25000条文本数据作为训练集,25000条文本数据作为测试集。
该数据集的官方地址为: http://ai.stanford.edu/~amaas/data/sentiment/ 该数据集的官方地址为: http://ai.stanford.edu/~amaas/data/sentiment/
- Warning:
``paddle.dataset.imdb``\ 先在是一个非常粗野的实现,后续需要有替代的方案。
环境设置 环境设置
-------- --------
本示例基于飞桨开源框架2.0版本。 本示例基于飞桨开源框架2.0版本。
.. code:: .. code:: ipython3
import paddle import paddle
import numpy as np import numpy as np
paddle.disable_static() paddle.disable_static()
print(paddle.__version__) print(paddle.__version__)
print(paddle.__git_commit__)
.. parsed-literal:: .. parsed-literal::
0.0.0 2.0.0-beta0
264e76cae6861ad9b1d4bcd8c3212f7a78c01e4d
加载数据 加载数据
...@@ -36,7 +30,7 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数 ...@@ -36,7 +30,7 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数
我们会使用\ ``paddle.dataset``\ 完成数据下载,构建字典和准备数据读取器。在飞桨2.0版本中,推荐使用padding的方式来对同一个batch中长度不一的数据进行补齐,所以在字典中,我们还会添加一个特殊的\ ``<pad>``\ 词,用来在后续对batch中较短的句子进行填充。 我们会使用\ ``paddle.dataset``\ 完成数据下载,构建字典和准备数据读取器。在飞桨2.0版本中,推荐使用padding的方式来对同一个batch中长度不一的数据进行补齐,所以在字典中,我们还会添加一个特殊的\ ``<pad>``\ 词,用来在后续对batch中较短的句子进行填充。
.. code:: .. code:: ipython3
print("Loading IMDB word dict....") print("Loading IMDB word dict....")
word_dict = paddle.dataset.imdb.word_dict() word_dict = paddle.dataset.imdb.word_dict()
...@@ -51,7 +45,7 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数 ...@@ -51,7 +45,7 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数
Loading IMDB word dict.... Loading IMDB word dict....
.. code:: .. code:: ipython3
# add a pad token to the dict for later padding the sequence # add a pad token to the dict for later padding the sequence
word_dict['<pad>'] = len(word_dict) word_dict['<pad>'] = len(word_dict)
...@@ -88,7 +82,7 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数 ...@@ -88,7 +82,7 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数
在这里我们设置一下词表大小,\ ``embedding``\ 的大小,batch_size,等等 在这里我们设置一下词表大小,\ ``embedding``\ 的大小,batch_size,等等
.. code:: .. code:: ipython3
vocab_size = len(word_dict) vocab_size = len(word_dict)
emb_size = 256 emb_size = 256
...@@ -109,7 +103,7 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数 ...@@ -109,7 +103,7 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数
在这里,取出一条数据打印出来看看,可以对数据有一个初步直观的印象。 在这里,取出一条数据打印出来看看,可以对数据有一个初步直观的印象。
.. code:: .. code:: ipython3
# 取出来第一条数据看看样子。 # 取出来第一条数据看看样子。
sent, label = next(train_reader()) sent, label = next(train_reader())
...@@ -127,11 +121,11 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数 ...@@ -127,11 +121,11 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数
padding的方式对齐数据 padding的方式对齐数据
---------------------------- -----------------------
文本数据中,每一句话的长度都是不一样的,为了方便后续的神经网络的计算,常见的处理方式是把数据集中的数据都统一成同样长度的数据。这包括:对于较长的数据进行截断处理,对于较短的数据用特殊的词\ ``<pad>``\ 进行填充。接下来的代码会对数据集中的数据进行这样的处理。 文本数据中,每一句话的长度都是不一样的,为了方便后续的神经网络的计算,常见的处理方式是把数据集中的数据都统一成同样长度的数据。这包括:对于较长的数据进行截断处理,对于较短的数据用特殊的词\ ``<pad>``\ 进行填充。接下来的代码会对数据集中的数据进行这样的处理。
.. code:: .. code:: ipython3
def create_padded_dataset(reader): def create_padded_dataset(reader):
padded_sents = [] padded_sents = []
...@@ -172,7 +166,7 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数 ...@@ -172,7 +166,7 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数
本示例中,我们将会使用一个不考虑词的顺序的BOW的网络,在查找到每个词对应的embedding后,简单的取平均,作为一个句子的表示。然后用\ ``Linear``\ 进行线性变换。为了防止过拟合,我们还使用了\ ``Dropout``\ 本示例中,我们将会使用一个不考虑词的顺序的BOW的网络,在查找到每个词对应的embedding后,简单的取平均,作为一个句子的表示。然后用\ ``Linear``\ 进行线性变换。为了防止过拟合,我们还使用了\ ``Dropout``\
.. code:: .. code:: ipython3
class MyNet(paddle.nn.Layer): class MyNet(paddle.nn.Layer):
def __init__(self): def __init__(self):
...@@ -191,7 +185,7 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数 ...@@ -191,7 +185,7 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数
开始模型的训练 开始模型的训练
-------------- --------------
.. code:: .. code:: ipython3
def train(model): def train(model):
model.train() model.train()
...@@ -218,8 +212,8 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数 ...@@ -218,8 +212,8 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数
if batch_id % 500 == 0: if batch_id % 500 == 0:
print("epoch: {}, batch_id: {}, loss is: {}".format(epoch, batch_id, avg_loss.numpy())) print("epoch: {}, batch_id: {}, loss is: {}".format(epoch, batch_id, avg_loss.numpy()))
avg_loss.backward() avg_loss.backward()
opt.minimize(avg_loss) opt.step()
model.clear_gradients() opt.clear_grad()
# evaluate model after one epoch # evaluate model after one epoch
model.eval() model.eval()
...@@ -250,16 +244,15 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数 ...@@ -250,16 +244,15 @@ IMDB数据集是一个对电影评论标注为正向评论与负向评论的数
.. parsed-literal:: .. parsed-literal::
epoch: 0, batch_id: 0, loss is: [0.6926701] epoch: 0, batch_id: 0, loss is: [0.6918494]
epoch: 0, batch_id: 500, loss is: [0.41248566] epoch: 0, batch_id: 500, loss is: [0.33142853]
[validation] accuracy/loss: 0.8505121469497681/0.3615057170391083 [validation] accuracy/loss: 0.8506321907043457/0.3620821535587311
epoch: 1, batch_id: 0, loss is: [0.29521096] epoch: 1, batch_id: 0, loss is: [0.37161]
epoch: 1, batch_id: 500, loss is: [0.2916747] epoch: 1, batch_id: 500, loss is: [0.2296829]
[validation] accuracy/loss: 0.86475670337677/0.3259459137916565 [validation] accuracy/loss: 0.8622759580612183/0.3286365270614624
The End The End
-------- -------
可以看到,在这个数据集上,经过两轮的迭代可以得到86%左右的准确率。你也可以通过调整网络结构和超参数,来获得更好的效果。 可以看到,在这个数据集上,经过两轮的迭代可以得到86%左右的准确率。你也可以通过调整网络结构和超参数,来获得更好的效果。
...@@ -16,21 +16,21 @@ ...@@ -16,21 +16,21 @@
"metadata": {}, "metadata": {},
"source": [ "source": [
"## 环境\n", "## 环境\n",
"本教程基于paddle-develop编写,如果您的环境不是本版本,请先安装paddle-develop。" "本教程基于paddle-2.0-beta编写,如果您的环境不是本版本,请先安装paddle-2.0-beta。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 23, "execution_count": 1,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"data": { "data": {
"text/plain": [ "text/plain": [
"'0.0.0'" "'2.0.0-beta0'"
] ]
}, },
"execution_count": 23, "execution_count": 1,
"metadata": {}, "metadata": {},
"output_type": "execute_result" "output_type": "execute_result"
} }
...@@ -51,23 +51,22 @@ ...@@ -51,23 +51,22 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 24, "execution_count": 3,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"--2020-09-09 14:58:26-- https://ocw.mit.edu/ans7870/6/6.006/s08/lecturenotes/files/t8.shakespeare.txt\n", "--2020-09-12 13:49:29-- https://ocw.mit.edu/ans7870/6/6.006/s08/lecturenotes/files/t8.shakespeare.txt\n",
"正在解析主机 ocw.mit.edu (ocw.mit.edu)... 151.101.110.133\n", "正在连接 172.19.57.45:3128... 已连接。\n",
"正在连接 ocw.mit.edu (ocw.mit.edu)|151.101.110.133|:443... 已连接。\n", "已发出 Proxy 请求,正在等待回应... 200 OK\n",
"已发出 HTTP 请求,正在等待回应... 200 OK\n",
"长度:5458199 (5.2M) [text/plain]\n", "长度:5458199 (5.2M) [text/plain]\n",
"正在保存至: “t8.shakespeare.txt”\n", "正在保存至: “t8.shakespeare.txt”\n",
"\n", "\n",
"t8.shakespeare.txt 100%[===================>] 5.21M 94.1KB/s 用时 70s \n", "t8.shakespeare.txt 100%[===================>] 5.21M 2.01MB/s 用时 2.6s \n",
"\n", "\n",
"2020-09-09 14:59:38 (75.7 KB/s) - 已保存 “t8.shakespeare.txt” [5458199/5458199])\n", "2020-09-12 13:49:33 (2.01 MB/s) - 已保存 “t8.shakespeare.txt” [5458199/5458199])\n",
"\n" "\n"
] ]
} }
...@@ -197,7 +196,7 @@ ...@@ -197,7 +196,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 12, "execution_count": 9,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
...@@ -233,12 +232,13 @@ ...@@ -233,12 +232,13 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 13, "execution_count": 30,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"import paddle\n", "import paddle\n",
"import numpy as np\n", "import numpy as np\n",
"import paddle.nn.functional as F\n",
"hidden_size = 1024\n", "hidden_size = 1024\n",
"class NGramModel(paddle.nn.Layer):\n", "class NGramModel(paddle.nn.Layer):\n",
" def __init__(self, vocab_size, embedding_dim, context_size):\n", " def __init__(self, vocab_size, embedding_dim, context_size):\n",
...@@ -251,7 +251,7 @@ ...@@ -251,7 +251,7 @@
" x = self.embedding(x)\n", " x = self.embedding(x)\n",
" x = paddle.reshape(x, [-1, context_size * embedding_dim])\n", " x = paddle.reshape(x, [-1, context_size * embedding_dim])\n",
" x = self.linear1(x)\n", " x = self.linear1(x)\n",
" x = paddle.nn.functional.relu(x)\n", " x = F.relu(x)\n",
" x = self.linear2(x)\n", " x = self.linear2(x)\n",
" return x" " return x"
] ]
...@@ -265,33 +265,34 @@ ...@@ -265,33 +265,34 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 19, "execution_count": 31,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"epoch: 0, batch_id: 0, loss is: [10.252193]\n", "epoch: 0, batch_id: 0, loss is: [10.252176]\n",
"epoch: 0, batch_id: 500, loss is: [6.894636]\n", "epoch: 0, batch_id: 500, loss is: [6.6429553]\n",
"epoch: 0, batch_id: 1000, loss is: [6.849346]\n", "epoch: 0, batch_id: 1000, loss is: [6.801544]\n",
"epoch: 0, batch_id: 1500, loss is: [6.931605]\n", "epoch: 0, batch_id: 1500, loss is: [6.7114644]\n",
"epoch: 0, batch_id: 2000, loss is: [6.6860313]\n", "epoch: 0, batch_id: 2000, loss is: [6.628998]\n",
"epoch: 0, batch_id: 2500, loss is: [6.2472367]\n", "epoch: 0, batch_id: 2500, loss is: [6.511376]\n",
"epoch: 0, batch_id: 3000, loss is: [6.8818874]\n", "epoch: 0, batch_id: 3000, loss is: [6.878798]\n",
"epoch: 0, batch_id: 3500, loss is: [6.941615]\n", "epoch: 0, batch_id: 3500, loss is: [6.8752203]\n",
"epoch: 1, batch_id: 0, loss is: [6.3628616]\n", "epoch: 1, batch_id: 0, loss is: [6.5908413]\n",
"epoch: 1, batch_id: 500, loss is: [6.2065206]\n", "epoch: 1, batch_id: 500, loss is: [6.9765778]\n",
"epoch: 1, batch_id: 1000, loss is: [6.5334334]\n", "epoch: 1, batch_id: 1000, loss is: [6.603841]\n",
"epoch: 1, batch_id: 1500, loss is: [6.5788]\n", "epoch: 1, batch_id: 1500, loss is: [6.9935036]\n",
"epoch: 1, batch_id: 2000, loss is: [6.352103]\n", "epoch: 1, batch_id: 2000, loss is: [6.751287]\n",
"epoch: 1, batch_id: 2500, loss is: [6.6272373]\n", "epoch: 1, batch_id: 2500, loss is: [7.1222277]\n",
"epoch: 1, batch_id: 3000, loss is: [6.801074]\n", "epoch: 1, batch_id: 3000, loss is: [6.6431484]\n",
"epoch: 1, batch_id: 3500, loss is: [6.2274427]\n" "epoch: 1, batch_id: 3500, loss is: [6.6024966]\n"
] ]
} }
], ],
"source": [ "source": [
"import paddle.nn.functional as F\n",
"vocab_size = len(vocab)\n", "vocab_size = len(vocab)\n",
"epochs = 2\n", "epochs = 2\n",
"losses = []\n", "losses = []\n",
...@@ -303,15 +304,15 @@ ...@@ -303,15 +304,15 @@
" x_data = data[0]\n", " x_data = data[0]\n",
" y_data = data[1]\n", " y_data = data[1]\n",
" predicts = model(x_data)\n", " predicts = model(x_data)\n",
" y_data = paddle.reshape(y_data, ([-1, 1]))\n", " y_data = paddle.reshape(y_data, shape=[-1, 1])\n",
" loss = paddle.nn.functional.softmax_with_cross_entropy(predicts, y_data)\n", " loss = F.softmax_with_cross_entropy(predicts, y_data)\n",
" avg_loss = paddle.mean(loss)\n", " avg_loss = paddle.mean(loss)\n",
" avg_loss.backward()\n", " avg_loss.backward()\n",
" if batch_id % 500 == 0:\n", " if batch_id % 500 == 0:\n",
" losses.append(avg_loss.numpy())\n", " losses.append(avg_loss.numpy())\n",
" print(\"epoch: {}, batch_id: {}, loss is: {}\".format(epoch, batch_id, avg_loss.numpy())) \n", " print(\"epoch: {}, batch_id: {}, loss is: {}\".format(epoch, batch_id, avg_loss.numpy())) \n",
" optim.minimize(avg_loss)\n", " optim.step()\n",
" model.clear_gradients()\n", " optim.clear_grad()\n",
"model = NGramModel(vocab_size, embedding_dim, context_size)\n", "model = NGramModel(vocab_size, embedding_dim, context_size)\n",
"train(model)" "train(model)"
] ]
...@@ -326,22 +327,22 @@ ...@@ -326,22 +327,22 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 20, "execution_count": 32,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"data": { "data": {
"text/plain": [ "text/plain": [
"[<matplotlib.lines.Line2D at 0x14e27b3c8>]" "[<matplotlib.lines.Line2D at 0x15c295cc0>]"
] ]
}, },
"execution_count": 20, "execution_count": 32,
"metadata": {}, "metadata": {},
"output_type": "execute_result" "output_type": "execute_result"
}, },
{ {
"data": { "data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dd3xV9f3H8dcnBMgAkgCBQIKAgogiIKQoKo46ceHAqq1W/dX5k4qttdUO7c8uOy2Oaqm1rioqLqyjOKqixRH2Hg4gYYWVQPb4/P64NxpiEMjNzb2c+34+Hnnk3nNP7vkw8s7J93zO92vujoiIBFdSrAsQEZHoUtCLiAScgl5EJOAU9CIiAaegFxEJuORYF9BU9+7dvV+/frEuQ0RknzJr1qxN7p7d3GtxF/T9+vWjoKAg1mWIiOxTzGzVrl7T0I2ISMAp6EVEAk5BLyIScAp6EZGAU9CLiAScgl5EJOAU9CIiAReYoC/aVsEfpy9j1eayWJciIhJXAhP0JeU13P3mShatLY11KSIicSUwQZ+blQpA4dbyGFciIhJfAhP0Gant6ZySTOHWiliXIiISVwIT9AB5WWkUKehFRHYSsKBP1Rm9iEgTgQr63MxUCreWowXPRUS+EKigz8tKpay6jpKKmliXIiISNwIW9GkAGr4REWkkYEGvFksRkaZ2G/Rm9qCZbTSzhY22dTWz18xsRfhz1i6+9tLwPivM7NLWLLw5XwS9zuhFRBrsyRn9Q8CpTbbdDLzh7gOBN8LPd2JmXYHbgMOBUcBtu/qB0FoyUtvTqaN66UVEGttt0Lv7O8CWJpvHAQ+HHz8MnN3Ml54CvObuW9x9K/AaX/6B0arMLNx5o6AXEWnQ0jH6nu6+Lvx4PdCzmX1ygTWNnheGt0VVXlYqRdsU9CIiDSK+GOuhpvWIGtfN7CozKzCzguLi4ojqCd00pYuxIiINWhr0G8ysF0D488Zm9ikC+jR6nhfe9iXuPtnd8909Pzs7u4UlheRmpbK9sla99CIiYS0N+mlAQxfNpcALzezzb+BkM8sKX4Q9Obwtqhp66TXnjYhIyJ60Vz4BzAQGmVmhmX0HuAM4ycxWACeGn2Nm+Wb2AIC7bwF+AXwU/rg9vC2q1EsvIrKz5N3t4O4X7eKlE5rZtwC4otHzB4EHW1xdC+RmqpdeRKSxQN0ZC9A1vQOp7dup80ZEJCxwQW9m6rwREWkkcEEPoc4bDd2IiIQEMuh105SIyBcCGvRpbCuvYXuleulFRAIZ9A2dNzqrFxEJaNA39NLrpikRkcAGvVaaEhFpEMig796pAx2TkzR0IyJCQIPezMItluqlFxEJZNBDaPhGQzciIgEO+tzMVF2MFREhwEGfl5XK5rJqyqtrY12KiEhMBTroQS2WIiKBD/pCdd6ISIILcNCrl15EBCIMejObaGYLzWyRmd3QzOvHmVmJmc0Nf9wayfH2RnanjnRol6QWSxFJeLtdYWpXzGwIcCUwCqgGXjWzf7n7yia7znD3MyKosUWSkozemSkaoxeRhBfJGf1g4AN3L3f3WuBt4NzWKat1qJdeRCSyoF8IjDGzbmaWBpwG9Glmv9FmNs/MXjGzQ5p7IzO7yswKzKyguLg4gpJ2lqcFSEREWh707r4E+C0wHXgVmAvUNdltNtDX3YcBdwPP7+K9Jrt7vrvnZ2dnt7SkL8nNTGXTjioqa5qWJSKSOCK6GOvuf3f3ke5+DLAVWN7k9VJ33xF+/DLQ3sy6R3LMvZHXVfPSi4hE2nXTI/x5P0Lj8483eT3HzCz8eFT4eJsjOebeUIuliEgEXTdhz5hZN6AGuM7dt5nZNQDufj8wHrjWzGqBCuBCd/cIj7nHPl9pSkEvIgksoqB39zHNbLu/0eN7gHsiOUYkenZJITnJ1EsvIgktsHfGArRLMnpnqvNGRBJboIMewtMV62KsiCSwwAd9nlaaEpEElwBBn8aG0iqqatVLLyKJKfBBnxuernjdtsoYVyIiEhuBD/rP56XXBVkRSVAJFPQapxeRxBT4oM/pkkK7JFPnjYgkrMAHfXK7JHK6pGjoRkQSVuCDHtRiKSKJLSGCPjcrVfPdiEjCSoigz8tKY31pJdW19bEuRUSkzSVI0KdS77C+RL30IpJ4EiPow9MVF27TOL2IJJ7ECHotQCIiCSzSFaYmmtlCM1tkZjc087qZ2V1mttLM5pvZiEiO11I5GSkkmYJeRBJTi4PezIYAVwKjgGHAGWY2oMluY4GB4Y+rgPtaerxIdEhOomeXFHXeiEhCiuSMfjDwgbuXu3st8DahdWMbGwc84iHvA5lm1iuCY7aYeulFJFFFEvQLgTFm1s3M0oDTgD5N9skF1jR6XhjethMzu8rMCsysoLi4OIKSdi0vK01DNyKSkFoc9O6+BPgtMB14FZgLtGjSd3ef7O757p6fnZ3d0pK+Um5mKutLK6mtUy+9iCSWiC7Guvvf3X2kux8DbAWWN9mliJ3P8vPC29pcXlYqdfXO+lL10otIYom066ZH+PN+hMbnH2+yyzTg2+HumyOAEndfF8kxW0otliKSqJIj/PpnzKwbUANc5+7bzOwaAHe/H3iZ0Nj9SqAcuDzC47VYw0pT6rwRkUQTUdC7+5hmtt3f6LED10VyjNbSOzMF0Bm9iCSehLgzFqBjcjt6dumoFksRSTgJE/QQ6rzRSlMikmgSKujVSy8iiSjBgj6VtdsqqKv3WJciItJmEiroc7NSqa13Nm5XL72IJI6ECnr10otIIkqwoA8vQKLOGxFJIAkV9LmZumlKRBJPQgV9Svt2dO/UUUM3IpJQEirooWFeegW9iCSOhAv63CzdNCUiiSXhgj4vK5WirRXUq5deRBJEAgZ9GtV19RTvqIp1KSIibSLxgj6zocVSwzcikhgSL+jVSy8iCSbhgj43S2f0IpJYIl1K8HtmtsjMFprZE2aW0uT1y8ys2Mzmhj+uiKzcyKV1SKZregd13ohIwmhx0JtZLnA9kO/uQ4B2wIXN7Pqkuw8PfzzQ0uO1JvXSi0giiXToJhlINbNkIA1YG3lJ0RcKeo3Ri0hiaHHQu3sR8AdgNbAOKHH36c3sep6ZzTezqWbWp7n3MrOrzKzAzAqKi4tbWtIey80M9dKHlrQVEQm2SIZusoBxQH+gN5BuZhc32e1FoJ+7DwVeAx5u7r3cfbK757t7fnZ2dktL2mN5WWlU1dazaUd11I8lIhJrkQzdnAh86u7F7l4DPAsc2XgHd9/s7g13Jj0AjIzgeK2mocVSF2RFJBFEEvSrgSPMLM3MDDgBWNJ4BzPr1ejpWU1fj5Vc9dKLSAJJbukXuvsHZjYVmA3UAnOAyWZ2O1Dg7tOA683srPDrW4DLIi85crm6O1ZEEkiLgx7A3W8Dbmuy+dZGr98C3BLJMaKhc0p7MtPaawESEUkICXdnbIPcTLVYikhiSNig101TIpIoEjboczPTKNqmXnoRCb6EDfq8rFTKq+vYWl4T61JERKIqoYMe1GIpIsGXsEHf0EuvzhsRCbqEDfq8rDRAvfQiEnwJG/QZqe3pnJKsoRsRCbyEDXoIz2Kp+W5EJOASOujzstI0dCMigZfgQR+6aUq99CISZAkf9DuqaimtqI11KSIiUZPwQQ+wRhdkRSTAEjzo1WIpIsGX0EHfMC+9Om9EJMgiCnoz+56ZLTKzhWb2hJmlNHm9o5k9aWYrzewDM+sXyfFaW2Zae9I7tFMvvYgEWiSLg+cC1wP57j4EaAdc2GS37wBb3X0AcCfw25YeLxrMTC2WIhJ4kQ7dJAOpZpYMpAFrm7w+Dng4/HgqcEJ4fdm4kZuVqvluRCTQWhz07l4E/IHQIuHrgBJ3n95kt1xgTXj/WqAE6Nb0vczsKjMrMLOC4uLilpbUIqFeeg3diEhwRTJ0k0XojL0/0BtIN7OLW/Je7j7Z3fPdPT87O7ulJbVIXlYqpZW1lFZqXnoRCaZIhm5OBD5192J3rwGeBY5ssk8R0AcgPLyTAWyO4JitLjcz1GKp4RsRCapIgn41cISZpYXH3U8AljTZZxpwafjxeOBNj7P5Br5YgERBLyLBFMkY/QeELrDOBhaE32uymd1uZmeFd/s70M3MVgLfB26OsN5Wp5WmRCTokiP5Yne/DbityeZbG71eCZwfyTGirWt6B1LaJ2noRkQCK6HvjAX10otI8CV80EO4xXKbhm5EJJgU9IRXmtIZvYgElIKe0CyWW8tr2FGleelFJHgU9HzReaOzehEJIgU9ofluAIo0Ti8iAaSgRzdNiUiwKeiB7E4d6ZicpKAXkUBS0BPqpVfnjYgElYI+LFfTFYtIQCnow3R3rIgElYI+LC8rlc1l1VRU18W6FBGRVqWgD8tTi6WIBJSCPqwh6Ndo+EZEAkZBH6aVpkQkqBT0YT06d6R9O9MFWREJnEgWBx9kZnMbfZSa2Q1N9jnOzEoa7XPrrt4v1pKSQr30arEUkaBp8QpT7r4MGA5gZu0ILQT+XDO7znD3M1p6nLaUm5VK0Tad0YtIsLTW0M0JwMfuvqqV3i8m8jLVSy8iwdNaQX8h8MQuXhttZvPM7BUzO6S5HczsKjMrMLOC4uLiVipp7+VlpVK8vYrKGvXSi0hwRBz0ZtYBOAt4upmXZwN93X0YcDfwfHPv4e6T3T3f3fOzs7MjLanFGqYrXqvhGxEJkNY4ox8LzHb3DU1fcPdSd98Rfvwy0N7MurfCMaMiLyvUYqnhGxEJktYI+ovYxbCNmeWYmYUfjwofb3MrHDMqNC+9iARRi7tuAMwsHTgJuLrRtmsA3P1+YDxwrZnVAhXAhe7ukRwzmnp2SSE5yTQNgogESkRB7+5lQLcm2+5v9Pge4J5IjtGW2iUZvTJTdEYvIoGiO2ObUIuliASNgr6J3CytNCUiwaKgbyIvK5UN2yupqlUvvYgEg4K+ibysNNxh3bbKWJciItIqFPRN5GY2LECi4RsRCQYFfRNf9NKrxVJEgkFB30SvjBTaJWleehEJDgV9E8ntksjpkqLOGxEJDAV9M3KzUnVGLyKBoaBvRp4WIBGRAFHQNyMvM5V1JRXU1NXHuhQRkYgp6JuRl5VGvcP6EvXSi8i+T0HfDE1XLCJBoqBvRq566UUkQBT0zeiVkYqZzuhFJBhaHPRmNsjM5jb6KDWzG5rsY2Z2l5mtNLP5ZjYi8pKjr0NyuJdenTciEgAtXnjE3ZcBwwHMrB1QBDzXZLexwMDwx+HAfeHPcS83M1VDNyISCK01dHMC8LG7r2qyfRzwiIe8D2SaWa9WOmZU5emmKREJiNYK+gtpfoHwXGBNo+eF4W07MbOrzKzAzAqKi4tbqaTI5GWlsb6kklr10ovIPi7ioDezDsBZwNMtfQ93n+zu+e6en52dHWlJrSI3K5XaemfD9qpYlyIiEpGIFgcPGwvMdvcNzbxWBPRp9DwvvC3ufd5Lv6X88znq483G0kqKd1RxcK8umFmsy9knVNbU8dnmMj7bVMZnm8v5bFMZG0orueW0wRzYs3OsyxOJitYI+otoftgGYBowwcymELoIW+Lu61rhmFGXl5UGxMcCJO7OhtIqFhSVsKCohIXhj43h3zZG79+N/xt3iIIqrLKmjtVbyvl0U0Ogl/HZpnI+21zGuiZ3O3fv1IHKmnomPD6baROOJqV9uxhVLRI9EQW9maUDJwFXN9p2DYC73w+8DJwGrATKgcsjOV5b6pWRAsAbSzfSNb0DvTJSyclIoUtKclTPnt2dtSWVLCgsYdHaL4J9045qAJIMDsjuxNEDujMkN4N6d+5+cyVjJ83g0tH9uOGkgXRJaR+1+uJFVW0da7aU8+mm0Fn5p+Gz9FWby1lbUoH7F/t2Te9Av25pjD6gG/27pdO3ezr9u6XTr3sanVPa8/byYi598EN+/fISbh83JHZ/KJEoMW/8HREH8vPzvaCgINZlAHD6XTNYtLZ0p21pHdqRk5FCr4wUcrqkhj43PM9IoVdGKllp7ffoh4G7U7i1goXhM/UFRSUsWlvKlrJQqLdLMgb26MSQ3AyG9O7CoXkZDO7VhbQOO/983lJWzR+mL+OJD1fTLb0DPzz1IMaPyCMpKXjDOf9ZtpHbXljEmq3lO4V5Zlp7+nVLp3/3dPqFQzz0OZ2M1N3/4PvlvxbzwLuf8vdL8zlhcM8o/glEosPMZrl7frOvKeh3rbaung3bq1hfUsG6kkrWl1Q2+lzB+pJKNmyvoq5+57/DhhuudvoB0CWFnIxU6ur9i+GXtSVsK68BIDnJOLBnZ4bkduHQ3AyG5IZCfW+GEhYWlXDrCwuZvXobw/tkcvu4Qxial9mqfyextKWsmpP+9DYZae05Y2hv+ofDvH/3dDLTOkT03lW1dZxz739ZX1rJqxPH0KNLSitVLdI2FPRRVFfvbNpRFf4B0MwPhNIKNpRUUd2oTbN9O2NQTufPA31I7wwG5XRulfHh+nrnuTlF/OaVpWwuq+KC/D7cdMogunXqGPF7x9rEKXN4ecE6Xvzu0RyU06XV33/lxu2ccfe7fK1fVx6+fFQgfyOS4PqqoG+Ni7EJrV2S0bNLCj27pECf5s+e3Z0tZdWfXwg8sGdnOiRHZ5qhpCTjvJF5nHxITya9voKH/vsZLy9Yx40nD+Jbh+9Hcrt9c3qj1xdv4IW5a7nhxIFRCXmAAT06c+sZh/Dj5xbw4HufcsWY/aNyHJG2tm9+1+9jzIxunTqGzt5zM6IW8o11TmnPT884mFdvGMPQvExum7aIM+5+lw8+2Rz1Y7e2kooafvL8Ag7K6cz/Hjcgqse6aFQfTjmkJ799dSkLi0qieizZNxVvr+KeN1dw+l0zeHfFpliXs0c0dJMA3J1XF67nly8toWhbBWcN682PTxtMTsa+MQ79o6nzeXrWGp6/7qg2ueawtayasZNmkNaxHf/67tFfuvgticfdmb16K4/MXMXLC9ZRU+d0TkmmY3ISr0w8huzOsR8a/aqhG53RJwAzY+yhvXj9+8dy/QkDeXXRer7+x7f4y1srqaqti3V5X2nGimKeLFjDVccc0GYXlrPSO/Cnbwzj001l/OJfi9vkmBKfKqrrmPLhak6/613Ou28mby7ZyMVH9OWNG4/lmWuPZHtlLTc+PY/6+vg6YW5KZ/QJaPXmcn7x0mJeW7yB/t3TufXMgzl+UI9Yl/UlZVW1nHznO3RMTuLliWPa/Gam3766lPve+pj7Lx7BqUP2ibn4pJV8tqmMR99fxdMFayitrOWgnM5cMrovZw/PJb3jF7/hPfr+Kn72/EJ+ctpgrjwmttd0dDFWdrJftzT+9u183lq2kdtfXMzl//iIEwf34GdnHEzfbumxLu9zv//3MtaWVPD01aNjcsfq9048kPdWbuJHzyxgWJ9MemXE51QY0jrq6p23lm3kkZmreHt5MclJxqlDcvj26H58rV9Ws/fGXHz4fsxYXszv/r2UI/bvxqF5GTGofPd0Rp/gqmvrefC9T7nrjRXU1jtXH7M/1x0/IOZTAXz02RbOv38mlx3Zj5+fdUjM6vh0Uxmn3zWDoXkZ/POKI2inlsvA2VpWzZMFa3js/VUUbq2gR+eOfOvwvlw0qs8e3U/RcE0ntUPomk7jM/62pD562a31JZX85pUlvDB3LWMGdueBS/PpmBybsK+sqeO0STOorqvn3zccE7NvnAZPF6zhpqnz+eGpg6Le9SNtZ37hNh6ZuYpp89ZSXVvP4f278u3R/Tj5kJ6038s25Jkfb+abD7zPeSPy+MP5w6JU8VfT0I3sVk5GCpMuPIyjB3Tnpqnzuf6JOdz7zREx6bu/8/XlfLKpjH9ecXjMQx5g/Mg83lpezJ+mL+fIA7ozfBf3S0j8q6yp46X563jk/VXMW7ONtA7t+EZ+Hpcc0Y9BOS2fFHD0Ad2YcPwA7n5zJcccmM1Zw3q3YtWRi/13kcSV8/P7UFZVy89fXMxNU+fzx/OHtekdovPWbONv73zChV/rw1EDurfZcb+KmfHrsw9l7uptTJwyh5euH0OnOPgBJHtu844q/jbjU54qWMOWsmoOyE7n/846hHNH5NK5lSYBnHjCQN5buYmfPLuAw/pk0qdrWqu8b2tQe6V8yWVH9eemUwbx3Jwibp22kLYa3quuredHz8ynR+cUfnz64DY55p7KSGvPnRcMZ82Wcn4+bVGsy5G98NL8dZx05ztMfudjvtYvi39ecTivf/9YLj2yX6uFPEByuyQmXXgYANdPmUNNHK1Op6CXZv3vcQdwzbEH8Nj7q7nj1aVtEvZ/eWslS9dv51fnDInLqZZH9e/KhK8PZOqsQqbNWxvrcuJKvF3rg9AdrNc+NovrHp9NXlYqr0w8hr9eks9RA7pHbarxPl3T+PW5hzJn9TYmvb4iKsdoCf3+Kc0yM3506iDKqmr569uf0LljMhO+PjBqx1u6vpR73lzJ2cN7x/U0wdd/fQDvrijmJ8/F36/n0VJVW8fG0tDEfQ2ztn4xaV9oMr9NO6o5ekB3bh57EIN7RWcuoj3l7kybt5afT1tEWVUdPzr1IK4c07/NrjedOaw37ywv5t63VnLUgO6MPqBbmxz3q0TUdWNmmcADwBDAgf9x95mNXj8OeAH4NLzpWXe//aveU1038aW+3vnB0/N4dk4Rt515MJcf1b/Vj1FbV8+59/2Xoq0VvPb9Y+maHtmUw9G2Zks5YyfN4KCczky56oh9dqI4CN35ub60mQAvqWR9aWhbw6I3jXXumExOo7UY0jsm8+zsIkorazhvRB7fP+lAesdgCc6N2yv56XMLmb54A8P7ZPL78UMZGIOV18qqajnz7ncpr67jlYljyGqD/9PR7LqZBLzq7uPDi4Q3d3ozw93PiPA4EiNJScbvxg+lvLqO/3txMekdkvnG1/rs/gv3wgPvfsr8whLu/eaIuA95CP16/qtzhjBxylzu/c/HTDwxer/pRMM7y4v5/b+XsWZr+efrITSWmdaenC6hAD80N3OnxXV6ZYRmam1ubPuGEw7kL2+t5B///YwX563l8qP6c+1xB+zRwi+Rcneen1vEz6ctpqKmjh+fdhDfOXr/mN33kN4xmbsuOoxz/vIeP3xmPpMvGRnTdZ1bfEZvZhnAXGB/38WbhM/of7A3Qa8z+vhUVVvHlY/M4t0Vxdx10WGcMbR12sc+Lt7B2EkzOH5QNvdfHNtvhr31vSfn8sLcIp66ejT5/brGupzdqq937nv7Y/4wfRn7d0/nqAHdPw/wnl1Cq6PldEkhtUNk908Ubavgj9OX8dycIjJS2zPh+AFcMrpv1O7L2FBayU+eW8DrSzYysm8Wvxs/lAOyO0XlWHvrgRmf8MuXlvCLs4dwyRF9o3qsqNwwZWbDgcnAYmAYMAuY6O5ljfY5DngGKATWEgr9r2xZUNDHr4rqOi598ENmr97K376dz/EHRTY/Tn29c8HkmSzfsIPXvnfMPreq0/bKGk67awb19fDKDWPi8gJyg+2VNdz41DymL97AuOG9uePcoREH+u4sWlvCHa8sZcaKTeRlpXLTKYM4c2jvVmvXdXeemV3E7S8uoqq2nptOGcTlR/WPq7uX6+udyx76iA8+2cy0CUdH1Ku/O9GavTIZGAHc5+6HAWXAzU32mQ30dfdhwN3A87so8CozKzCzguLi4ghKkmhK7dCOBy7LZ3CvLlzz2CxmfhzZ3PaPvr+Kjz7bys/OOHifC3kIzfk/6cLDWF8aGheOx84TgBUbtjPunvd4c+lGbjvzYP58wfCohzzAIb0zePQ7h/Pod0bRJaU9E6fMZdy97/HfjyOfw31dSQWXP/QRP3h6HoNyOvPqDcdwxZjYDdXsSlKS8cfzh9E5JZnvPjGbyprYzBYbSdAXAoXu/kH4+VRCwf85dy919x3hxy8D7c3sS3fBuPtkd8939/zs7OwISpJo65LSnof/ZxT7dU3jioc/Ys7qrS16nzVbyvntq0s59sBszhuR28pVtp0R+2VxwwkDmTZvLc/NKYp1OV/y0vx1jLv3PUora3n8yiO4/Kj+bT48NmZgNv/67tHcecEwtpRV882/fcDl//iQpetL9/q93J2nPlrDyX96hw8+2cJtZx7Mk1eNpn/3+JmMr6nszh354zeGs3zDDn710pKY1NDioHf39cAaMxsU3nQCoWGcz5lZjoX/V5nZqPDx9r0ljmQnXdM78M8rDqd7545c9o+PWLJu775h3Z1bnl2AAb8+99B9aly+Of97/ABG9evKz55fyKrNZbv/gjZQW1fPr19ewnWPz+agnM68dP3RjOofu+sISUnGOYfl8caNx/KT0wYza9VWxk6awU1Pz2NdScUevUfRtgou/cdH/PCZ+Rzcuwuv3jCGy4/qv0+s7XvsgdlcOaY/j76/iumL1rf58SNtrxxOqL2yA/AJcDlwAYC7329mE4BrgVqgAvi+u//3q95TY/T7jjVbyjn//pnU1tfz1NWj2X8PL4A9+dFqfvTMAn559hAujvIFqrZStK2CsX9+h/7ZnZh6zei9nhSrNW3eUcWEx+cw85PNfHt0X356+sFtsnzl3thWXs1f3vqYh977DDP4ztH9uea4A5q9zuHuTPloDb96aQn17tw89iAuPrzvPhHwjVXX1nPufe9RuLWCVyaOafVprzV7pUTNyo07uOCvM+mYnMRT14wmL+urbyBaX1LJSXe+zSG9u/D4FUfsc9+sX+Wl+eu47vHZTDh+AD84ZdDuvyAK5q3ZxrWPzWJzWTW/PudQzhuZF5M69tSaLeX86bXlPDeniKy09nz36wO5+Ii+n/9gKtxazs3PLODdlZsYvX83fjd+6D59k9onxTs44+53ozLttYJeomrx2lIunDyTrukdeOqa0fTo3PyFVXfnykcKeHflJv59wzFxtchJa/nh1Hk8PauQP18wnNMP7dWmN1M98eFqbnthET26dOT+i0cyJDc+F8FozsKiUIfOuys3sV/XNH5wyiBKK2r4zcuhMe1bThvMN0ftF4gTg4Zpr286ZRDXHd96014r6CXqZq3ayiV//4A+WWk8efURZKZ9+canF+YWMXHKXH56+mCuGBPbZdeipayqlnP+8h7LN+ygR+eOnDMil/NH5jGgR/Ta6ipr6vj5tEVM+WgNxxyYzaQLhrfJnZjR8M7yYjJIGyYAAAdTSURBVH7zytLPr/scPaA7d5x36G5/U9yXuDvXT5nLywvW8dTVoxnZN6tV3ldBL23ivZWbuPyhjxic05nHrjh8p7snN+2o4qQ/vU2/7ulMvebIuGuDa03VtfW8uXQjU2cV8p9lG6mrd4b3yWT8yDzOHNa7Ve8UXbutgmsfm8W8whImHD+A75104D7/d1tf77w4fy317pw9PHefv1jfnNLKGk6bNAOAlye2zj0YCnppM68v3sA1j81iRN8sHr581Of92hMen830RRt46fqjYzL3SKwUb6/ihblFPF1QyLIN2+mQnMQph+QwfmQeRw/oHlEo/3flJiY8MYea2nr++I1hnHxITitWLtE2a9VWvvHXmZx2aC/uunB4xD/QFPTSpqbNW8vEKXM49sBsJl+Sz3+WbeTqR2fxg5MPjOoMmPHM3VlYVMrUWWt4Yd5atpXXkNMlhXNH5DJ+ZN4edyw1vNffZnzCHa8s5YDsTvz1kpF79fUSP+55cwV/mL6c348fyvn5kc0hpaCXNjflw9Xc/OwCTj64J3PWbCO7U0demHBUTNsO40VVbR1vLAkN7by1bCP1DiP7ZjF+ZB6nD+31lb/G76iq5UdT5/PSgnWcfmgvfjd+aFwstygtU1fvfOuB95lfWMK/vnt0RD+wFfQSEw0TOrVLMl647qh9qgukrWwsreS5OUU8PauQlRt3kNI+iVMPyWH8yD4ceUC3nbpMPi7ewdWPzuKT4h3cMnYwV4xp+7tcpfWtK6lg7KQZ5GWl8uy1R7X4ngcFvcTMlA9X0yE5iXNHxHc/d6y5O/MKS5g6aw3T5q6ltLKW3hkpnDcyj/NG5LFsw3ZufGoeHZKTuOeiwzgyTtbTldYxfdF6rnp0FleO6c9PTj+4Re+hoBfZh1TW1PH6kg08XVDIjBXF1Ie/RYf1yeS+b42IyYIeEn2/emkxA3p04oKv7deir1fQi+yj1peEhnbq6uu58pj9ozanu+z7ornClIhEUU5GCtced0Csy5B9nFogREQCTkEvIhJwCnoRkYBT0IuIBJyCXkQk4BT0IiIBp6AXEQk4Bb2ISMDF3Z2xZlYMrIrgLboDm1qpnGiI9/og/muM9/og/muM9/pANe6tvu6e3dwLcRf0kTKzgl3dBhwP4r0+iP8a470+iP8a470+UI2tSUM3IiIBp6AXEQm4IAb95FgXsBvxXh/Ef43xXh/Ef43xXh+oxlYTuDF6ERHZWRDP6EVEpBEFvYhIwAUm6M3sVDNbZmYrzezmWNfTlJn1MbP/mNliM1tkZhNjXVNzzKydmc0xs3/FupbmmFmmmU01s6VmtsTMRse6psbM7Hvhf9+FZvaEmaXEQU0PmtlGM1vYaFtXM3vNzFaEP2fFYY2/D/87zzez58wsM57qa/TajWbmZha3C/kGIujNrB1wLzAWOBi4yMxatsJu9NQCN7r7wcARwHVxWCPARGBJrIv4CpOAV939IGAYcVSrmeUC1wP57j4EaAdcGNuqAHgIOLXJtpuBN9x9IPBG+HksPcSXa3wNGOLuQ4HlwC1tXVQjD/Hl+jCzPsDJwOq2LmhvBCLogVHASnf/xN2rgSnAuBjXtBN3X+fus8OPtxMKqNzYVrUzM8sDTgceiHUtzTGzDOAY4O8A7l7t7ttiW9WXJAOpZpYMpAFrY1wP7v4OsKXJ5nHAw+HHDwNnt2lRTTRXo7tPd/fa8NP3gbw2L+yLWpr7OwS4E/ghENddLUEJ+lxgTaPnhcRZiDZmZv2Aw4APYlvJl/yZ0H/a+lgXsgv9gWLgH+HhpQfMLD3WRTVw9yLgD4TO7tYBJe4+PbZV7VJPd18Xfrwe6BnLYvbA/wCvxLqIxsxsHFDk7vNiXcvuBCXo9xlm1gl4BrjB3UtjXU8DMzsD2Ojus2Jdy1dIBkYA97n7YUAZsR9y+Fx4nHscoR9IvYF0M7s4tlXtnod6rOP2jNTMfkJo6POfsa6lgZmlAT8Gbo11LXsiKEFfBPRp9DwvvC2umFl7QiH/T3d/Ntb1NHEUcJaZfUZo6OvrZvZYbEv6kkKg0N0bfhOaSij448WJwKfuXuzuNcCzwJExrmlXNphZL4Dw540xrqdZZnYZcAbwLY+vm34OIPQDfV74eyYPmG1mOTGtaheCEvQfAQPNrL+ZdSB0AWxajGvaiZkZobHlJe7+p1jX05S73+Luee7ej9Df35vuHldno+6+HlhjZoPCm04AFsewpKZWA0eYWVr43/sE4uhicRPTgEvDjy8FXohhLc0ys1MJDSWe5e7lsa6nMXdf4O493L1f+HumEBgR/j8adwIR9OELNhOAfxP6xnrK3RfFtqovOQq4hNCZ8tzwx2mxLmof9F3gn2Y2HxgO/DrG9Xwu/JvGVGA2sIDQ91fMb5E3syeAmcAgMys0s+8AdwAnmdkKQr+J3BGHNd4DdAZeC3+/3B9n9e0zNAWCiEjABeKMXkREdk1BLyIScAp6EZGAU9CLiAScgl5EJOAU9CIiAaegFxEJuP8HvHiKw1jJ554AAAAASUVORK5CYII=\n", "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXxU9b3/8dcn67AlYQlJ2F0QRVDQCNHq7aK2ar2irW3BfQG0tdVut623rf1da3vb6217a/XqRbAuCNparbZalaqtVmUJm4KIguwSCBASAknI8vn9MRMaYlCYmeQMZ97PxyOPzJxzMufDkvec+ZzvOV9zd0REJLwygi5AREQ6l4JeRCTkFPQiIiGnoBcRCTkFvYhIyGUFXUB7/fr182HDhgVdhojIYWXhwoXb3L2wo3UpF/TDhg2jvLw86DJERA4rZrbuQOvUuhERCTkFvYhIyCnoRURCTkEvIhJyCnoRkZBT0IuIhJyCXkQk5EIT9Jt21vHL51eydtvuoEsREUkpoQn6nXv2cseLq1ixuSboUkREUkpogr44LwLA5ur6gCsREUktoQn6Pj1yyMnMYEuNgl5EpK3QBL2ZUZSfS4WCXkRkP6EJeoi2byrUuhER2U+ogr4oL6IjehGRdkIV9CX50SN6dw+6FBGRlBGqoC/Ki9DQ1EJ1XWPQpYiIpIxQBX1xfnSIpdo3IiL/FK6gj42l1wlZEZF/+sigN7P7zGyrmS1rs6yPmc0xs3dj33sf4GevjG3zrpldmczCO7LviF5BLyKyz8Ec0d8PnNNu2feAF9x9OPBC7Pl+zKwP8CNgPDAO+NGB3hCSpX8vtW5ERNr7yKB395eBHe0WTwAeiD1+ALiwgx/9DDDH3Xe4exUwhw++YSRVTlYG/Xrm6OpYEZE24u3RF7n75tjjCqCog20GAhvaPN8YW9apinTRlIjIfhI+GevRQesJDVw3s6lmVm5m5ZWVlQnVU5If0Y3NRETaiDfot5hZCUDs+9YOttkEDG7zfFBs2Qe4+zR3L3X30sLCwjhLiirKi6h1IyLSRrxB/xTQOormSuDJDrZ5Dvi0mfWOnYT9dGxZpyrOi1C1p5H6xubO3pWIyGHhYIZXzgZeB0aY2UYzuxb4GXC2mb0LnBV7jpmVmtl0AHffAfwYWBD7ujW2rFMVxYZYbq1p6OxdiYgcFrI+agN3n3SAVWd2sG05MLnN8/uA++KuLg4l+a0TkNQxpG/3rty1iEhKCtWVsdDm6lj16UVEgBAGfWvrRidkRUSiQhf0vXKz6J6TSUW1evQiIhDCoDczijXEUkRkn9AFPURvbra5ui7oMkREUkI4gz4vwhYNrxQRAUIa9EX50dZNS4umFBQRCWXQF+dFaGpxtu/eG3QpIiKBC2fQawISEZF9whn0umhKRGSfcAa9JgkXEdknlEHfr2cumRnGFrVuRETCGfSZGUb/XrmagEREhJAGPWgCEhGRVqEN+uK8iHr0IiKEOejzI+rRi4gQ8qDf1dBEbUNT0KWIiAQqvEGfp4umREQgwaA3s5vMbJmZLTezr3ew/hNmVm1mS2JftySyv0NRlKcJSERE4CDmjD0QMxsFTAHGAXuBZ83sz+6+qt2mr7j7+QnUGBfdBkFEJCqRI/rjgHnuvsfdm4C/A59LTlmJ020QRESiEgn6ZcAZZtbXzLoD5wGDO9juVDNbamZ/MbPjO3ohM5tqZuVmVl5ZWZlASf/ULSeT/G7ZOqIXkbQXd+vG3VeY2c+B54HdwBKgud1mi4Ch7l5rZucBfwSGd/Ba04BpAKWlpUm7ibzG0ouIJHgy1t1nuPvJ7v4vQBXwTrv1Ne5eG3v8DJBtZv0S2eehaJ2AREQknSU66qZ/7PsQov35We3WF5uZxR6Pi+1veyL7PBTFeblq3YhI2ou7dRPzBzPrCzQCN7j7TjO7HsDd7wEuBr5sZk1AHTDR3btsfr/i/G5U1jbQ2NxCdmZoLxkQEflQCQW9u5/RwbJ72jy+E7gzkX0kojgvgjtU7mpgQEG3oMoQEQlUqA9zi/NzAQ2xFJH0Fuqg33d1rPr0IpLGQh30JfnRdo0mIBGRdBbqoO/dPZucrAwNsRSRtBbqoDczivJy1aMXkbQW6qCH2NWxat2ISBoLf9Dnd1PrRkTSWviDPi+XzdX1dOF1WiIiKSX0QV+UF6GhqYXqusagSxERCUTog37fBCRq34hImgp90JdopikRSXOhD/oiTRIuImku9EHfv5daNyKS3kIf9DlZGfTrmaMhliKStkIf9BBt36h1IyLpKi2CviQ/ohubiUjaSougL8rT3LEikr4SnTP2JjNbZmbLzezrHaw3M7vDzFaZ2RtmdlIi+4tXcV6Eqj2N1Dc2B7F7EZFAxR30ZjYKmAKMA04Ezjezo9ttdi4wPPY1Fbg73v0loig2ln5rTUMQuxcRCVQiR/THAfPcfY+7NwF/Bz7XbpsJwIMeNRcoMLOSBPYZlxJdHSsiaSyRoF8GnGFmfc2sO3AeMLjdNgOBDW2eb4wt24+ZTTWzcjMrr6ysTKCkjhXHLpraXF2X9NcWEUl1cQe9u68Afg48DzwLLAHiaoK7+zR3L3X30sLCwnhLOqDW1o1OyIpIOkroZKy7z3D3k939X4Aq4J12m2xi/6P8QbFlXapXbhbdczKpqFaPXkTST6KjbvrHvg8h2p+f1W6Tp4ArYqNvyoBqd9+cyD7jYWYU52uIpYikp6wEf/4PZtYXaARucPedZnY9gLvfAzxDtHe/CtgDXJ3g/uJWnBdRj15E0lJCQe/uZ3Sw7J42jx24IZF9JEtxXoR5a3YEXYaISJdLiytjIXpCdktNPS0tmlJQRNJL2gR9SX6EphZn++69QZciItKl0iboNQGJiKSrtAn61oumdHWsiKSb9Al63QZBRNJU2gR9v565ZGYYW9S6EZE0kzZBn5lh9O+VqwlIRCTtpE3QgyYgEZH0lFZBX5wXUY9eRNJOegV9fkQ9ehFJO2kX9LsamqhtaAq6FBGRLpNeQa+LpkQkDaVV0LdeHasTsiKSTtIq6PddNKUjehFJI+kV9LoNgoikobQK+m45meR3y9YRvYiklbQKetBYehFJP2kX9EWaO1ZE0kyik4N/w8yWm9kyM5ttZpF2668ys0ozWxL7mpxYuYkryYuodSMiaSXuoDezgcCNQKm7jwIygYkdbPqou4+JfU2Pd3/JUpQfobK2gcbmlqBLERHpEom2brKAbmaWBXQH3k+8pM5VnBfBHSp3NQRdiohIl4g76N19E/DfwHpgM1Dt7s93sOnnzewNM3vMzAZ39FpmNtXMys2svLKyMt6SDkpxfi6gIZYikj4Sad30BiYARwADgB5mdlm7zf4EDHP3E4A5wAMdvZa7T3P3UncvLSwsjLekg1Kc1w1ANzcTkbSRSOvmLGCNu1e6eyPwOHBa2w3cfbu7t/ZIpgMnJ7C/pNCUgiKSbhIJ+vVAmZl1NzMDzgRWtN3AzEraPL2g/fog9O6eTU5WhkbeiEjayIr3B919npk9BiwCmoDFwDQzuxUod/engBvN7ILY+h3AVYmXnBgzoygvV0f0IpI24g56AHf/EfCjdotvabP+ZuDmRPbRGYo1ll5E0kjaXRkLUJzfTVfHikjaSM+gz8tlc3U97h50KSIinS4tg74oL0JDUwvVdY1BlyIi0unSMug1xFJE0klaBn2JZpoSkTSSlkFfpEnCRSSNpGXQ9++l1o2IpI+0DPqcrAz69czREEsRSQtpGfQQPSGr1o2IpIP0Dfq8CBU1uie9iIRf2gZ9UV6Eiuq6oMsQEel0aRv0xXkRqvY0Ut/YHHQpIiKdKn2DPjaWfqvaNyIScmkf9BpiKSJhl75BH7toarP69CIScmkb9EWxI3qNpReRsEvboO+Vm0WPnEwqqtWjF5FwSyjozewbZrbczJaZ2Wwzi7Rbn2tmj5rZKjObZ2bDEtlfMpkZRfkRHdGLSOjFHfRmNhC4ESh191FAJjCx3WbXAlXufjTwK+Dn8e6vMxTnRdSjF5HQS7R1kwV0M7MsoDvwfrv1E4AHYo8fA840M0twn0lTnBdhi4ZXikjIxR307r4J+G9gPbAZqHb359ttNhDYENu+CagG+rZ/LTObamblZlZeWVkZb0mHrLV109KiKQVFJLwSad30JnrEfgQwAOhhZpfF81ruPs3dS929tLCwMN6SDllJfoSmFmf77r1dtk8Rka6WSOvmLGCNu1e6eyPwOHBau202AYMBYu2dfGB7AvtMKk1AIiLpIJGgXw+UmVn3WN/9TGBFu22eAq6MPb4YeNHdU6ZP0nrRlK6OFZEwS6RHP4/oCdZFwJux15pmZrea2QWxzWYAfc1sFfBN4HsJ1ptUug2CiKSDrER+2N1/BPyo3eJb2qyvB76QyD46U7+euWRmGFvUuhGREEvbK2MBMjOM/r1ydUQvIqGW1kEPrROQKOhFJLzSPuijUwoq6EUkvBT0+RH16EUk1BT0+RF2NTRR29AUdCkiIp1CQa+LpkQk5NI+6FuvjtXtikUkrNI+6EvydUQvIuGW9kGvq2NFJOzSPugj2Znkd8vWEb2IhFbaBz1oLL2IhJuCnthYegW9iISUgp7YEb1aNyISUgp6olMKVtY20NjcEnQpIiJJp6AnekTvDpW7NFG4iISPgh4ozs8FNMRSRMJJQQ8U53UD0M3NRCSUFPTooikRCbe4g97MRpjZkjZfNWb29XbbfMLMqttsc8uBXi9Ivbtnk5OVoZE3IhJKcc8Z6+4rgTEAZpYJbAKe6GDTV9z9/Hj30xXMjKI8TSkoIuGUrNbNmcBqd1+XpNfrciV53XRELyKhlKygnwjMPsC6U81sqZn9xcyO72gDM5tqZuVmVl5ZWZmkkg5Nka6OFZGQSjjozSwHuAD4fQerFwFD3f1E4DfAHzt6DXef5u6l7l5aWFiYaElxKc7LZXN1Pe4eyP5FRDpLMo7ozwUWufuW9ivcvcbda2OPnwGyzaxfEvaZdEV5ERqaWqiuawy6FBGRpEpG0E/iAG0bMys2M4s9Hhfb3/Yk7DPpSvKjY+l1QlZEwiahoDezHsDZwONtll1vZtfHnl4MLDOzpcAdwERP0d7IvqtjdUJWREIm7uGVAO6+G+jbbtk9bR7fCdyZyD66SpEmCReRkNKVsTH9e+nqWBEJJwV9TE5WBv165mqIpYiEjoK+jeL8XLVuRCR0FPRtROeO1T3pRSRcFPRtFOVFqKiuC7oMEZGkUtC3UZwXoWpPI/WNzUGXIiKSNAr6NlrvS79V7RsRCREFfRuagEREwkhB30Zx7KKpzerTi0iIKOjbKIod0WssvYiEiYK+jV65WfTIyaSiWj16EQkPBX0bZqYJSEQkdBT07RTnRdSjFwmhdB42raBvpzgvwhYNrxQJjeYW5+bH3+CkH89hyYadQZcTCAV9O8Wx1k1LS0reNl9EDkFTcwvf+t0SZs/fQIYZ1z+0kMpd6Xcgp6Bvpzg/QlOLs3333qBLEZEE7G1q4cZHFvPHJe/zb58ZwaPXlbGzbi83PLyIvU0tQZfXpRT07bROQKITsiKHr/rGZr7y8EKeebOCH3z2OG745NEcPyCfn3/+BOav3cFtT78VdIldKu6gN7MRZrakzVeNmX293TZmZneY2Soze8PMTkq85M71z4umFPQih6O6vc1MebCcv67Yyo8vHMXkM47ct27CmIFMOeMIHnx9Hb9bsCHAKrtW3FMJuvtKYAyAmWUCm4An2m12LjA89jUeuDv2PWWV6DYIIoet3Q1NXPvAAuat2cF/ff4EvnjK4A9s891zjmXF5l384I/LGF7Uk7FDegdQaddKVuvmTGC1u69rt3wC8KBHzQUKzKwkSfvsFH175pKZYWzREb3IQXvr/Rqq6xoDraGmvpHLZ8xjwdoq/udLYzoMeYCszAx+M2ksRfm5XD9zIVt3hf93PVlBPxGY3cHygUDbz0cbY8v2Y2ZTzazczMorKyuTVFJ8MjOM/r1ydUQvKWNj1R7mvrc96DIO6HcLNnDeHa9w5i/+zlNL38e960es7dyzl8umz+PNTdXcOWksE8Z8IGb207tHDv93WSnVdY18ZWb4T84mHPRmlgNcAPw+3tdw92nuXurupYWFhYmWlLDoBCQKegneqq27uPCu15g4bS4Pvb426HI+4LGFG/nu429w2lF9GVAQ4cbZi7n6/gVs2LGny2rYVtvAxGlzeXvzLu657GTOHX1wTYORA/L4r4tPpHxdFbf+eXknVxmsZBzRnwsscvctHazbBLT9/DQotiylleRHdEQvgVu1dRcTp80D4F+OKeSHTy5n+ivvBVzVPz2xeCP/9thSTjuqL/dddQpPfOVj3HL+SOav2cGnf/Uy9778Hk3NnXukvKWmnonT5rJ2+25mXFXKmccVHdLPX3DiAK77+JHMnLueR+av76Qqg5eMoJ9Ex20bgKeAK2Kjb8qAanffnIR9dqqivIh69BKoVVtr94X8I1PLmHFlKZ8dXcJtT6/grpdWBVwdPLX0fb71u6WUHdGX6VecQiQ7k8wM45rTj2DONz/OaUf15SfPrGDCXa/y5sbqTqlh0846vvR/r7N5Zx33Xz2OM4bH1w34zmeO5Yzh/bjlyeUsWl+V5CpTQ0JBb2Y9gLOBx9ssu97Mro89fQZ4D1gF3At8JZH9dZXi/Ai7GpqobWgKuhRJQ6sra5l071zAeWTqeI7u35PszAx+PXEMF44ZwO3PreRXc94JpBcO8PQbm/nGo0soHdaHGVeV0i0nc7/1Awu6Mf3KUv730pOo3NXAhLv+wa1/eovdSfx9Wr99D1+853W21+7lwWvHU3Zk37hfKzPD+M2ksRTnR7j+oYVsDeGn+biHVwK4+26gb7tl97R57MANiewjCK1j6Suq6zm6f8+Aq5G29ja1sHTjTlpanMwMw8zIsOgva4bFvjIg0zpYlxF9vm+72LqcrAy65yT0q5A0qytrmTRtLu7O7CllHN2/1751WZkZ/OKLY6Kh/8K77G1u4TufGYGZdVl9zy7bzI2PLGbs4AJ+e9UpB/x7MzPOG13C6cP78V/Pvs19r67hueUV3Drh+ENur7S3urKWS++dR31TM7OmlDF6UH5CrwdQ0D2HaVeczEV3vcaXH17E7Cll5GSF53rS1PjfnWLaXh2roE8dtQ1NXHt/dIx0MpnBlacO4+bzjiU3K/Ojf6CTvBcL+eYWZ/bUMoYX9frANpkZxs8/fwI5WRnc/bfVNDS28MPzj+uSsH9+eQVfnbWYEwflc/814+iR+9HxkRfJ5rYLR3PhmIHc/PibXPtAOZ8dXcKP/nUk/WO/Z4diZcUuLp0+b98b4XElefH8UTp0bHEet3/hBL46azH/70/L+elFo5P22kFT0Hdg30VT6tOnjOo9jVz52/m8uamaWyccz1GFPWlxp7nFcWff45bY4+gXtLR88HFz7LnHfmZlxS7uf20t5et2cOekkxjWr0eX//nWbNvNpHv/GfLHdBDyrTIyjNsuHEVOVgb3vbqGvc3N3HrBKDIyOi/sX1ixhRtmLeL4gdGQ73kQId9W6bA+PH3jGUx7eTV3vLiKl9+t5LvnHMsl44YcdN3LNlVz+Yx5ZGdmMGvq/p92kuX8Ewaw/P0a7v7bakYPzGfSuCFJ30cQFPQd0CThqWV7bQOXz5jPqq213H3pSXz6+OKk7+OTx/bnO4+9wfm/+Qc//dxoLjhxQNL3cSBrtu1m4rTXaWp2Zk358JBvZWbccv5IcrMyuefvq2lscn76udFkdkLYv/T2Vr48cxHHleTx4DXjyItkx/U6OVkZfPVTwzlvdAnff2IZP/jjMp5YvIn//Nzoj/wzL15fxZX3zadXJJuHJ4/v1Dfjb396BG+9X8MtTy7jmKJenDz08L9yNjxNqCSKZGeS3y076Tc2q29sTstbpCZiS009X5o2l9WVtdx7ZWmnhDzAZ44v5pmbzuCYop7cOHsxNz/+BnV7O3+iirXbdjNp2lwaYyE/ovjgj1LNjO+eM4IbzxzOo+Ub+PbvlyZ9OOPf36nkupkLGV7Uk4euGU9+t/hCvq0jC3sya8p4br/4BFZX1vLZO17hF8+vPODEIAvW7uDyGfMp6J7Do9eVdfonrswM446JYxlQ0I0vz1wYihscWlBn7g+ktLTUy8vLgy6Dz/zqZYb07c69V5Qm/FqrK2t5eO56Hlu4gZr6Jgb36cYpQ/tQOqwPpwzrzVGFPTv1Y/fhamPVHi6dPo9tuxqYcdUpCY2sOFiNzS38cs473P231RxT1JO7Ljmpw155MqzbvpuJ0+bS0NTCrCnjObY4/n7zXS+t4vbnVvLZ0SX8z8ToCdtE/ePdbVz7wIJoME8eT+8eOQm/Znvbaxu47ekVPLF4E0f068FPLhrFaUf127f+1VXbmPxAOSUFEWZNLtv3absrrKzYxUX/+yojinvxyNSyQM/fHAwzW+juHQaWgv4ArrxvPlV79vLUV0+P6+ebmlv464otPDR3Ha+u2k52pnHuqBJGDcxj8fqdLFi7g2210XveF3TPpnRo733BP2pgfsr/p+psa7bt5tJ751Lb0MQD14zr8htP/f2dSr756BJ2723i1gmj+MLJg5J6wrM15OsboyNHknFScfor73Hb0ys4e2QRd14yNqH/Q6+t3sY19y9gWN8ezJpSRp9OCPm2Xnm3ku8/sYz1O/bwhZMH8e/nHceSjTu57qGFHNG3BzMnj6ewV26n1tCRZ97czFceXsSkcYP5z8+d0OX7PxQK+jh897E3eGnlVuZ//6xD+rktNfXMnr+eR+ZvoKKmnoEF3bhk/BC+WDp4v/+o7s7a7XtYsHYH5Wt3UL62ive27QYgNyuDEwcXcMqwaPifNKR3Uj4yHy7e2RIdWdHc4jx07TiOH5D48Ll4bK2p56ZHlvD6e9u5cMwAbrto9CGfhOzI+u17mDjtdeoam3l4chkjByRv5MiDr6/llieX84kRhdxz2clEsg897Oe+t52rf7uAwX26MXtKGX17dk3A1u1t5o4X32Xay++R3y2bXfWNHFPUi4euHd/pbzQf5vbn3uaul1bzk4tGcen4oYHV8VEU9HH45Zx3+M2L7/LObed+5Mdgd+f11duZOW8dzy3fQnOL8/FjCrm8bCifPLb/QZ8g21bbQPnaKsrX7mDBuiqWb6qmqcUxgxFFvThlWB9Kh/XmlGF9GFDQLRl/zJSz38iKKeM7ZWTFoWhuce58cRW/fuEdhvbtwZ2XjE3ojac15Pc0NjMrySHfavb89fz7E2/ysaP6Me2Kkw/pGoEFa3dw5X3zKcmP8MjUUwM5il6xuYYf/nEZmRnGtCtKAz/IaW5xrn1gAa+u2sbsKWWUDusTaD0HoqCPw6x50V+W1773qQOGanVdI48v2sjMuetYXbmbgu7ZfKl0MJeMH8LQvomfMNqzt4klG3ZSvraKBWt3sGhdFbtjJwgHFnSjNHbEP3ZwASOKeyWlLxukheuquOq388mLZDNryvik/B0my9z3tnPTI4up2t3ID84/jsvLhh5yK2fDjj1MnDaX3XubeHjy+E79pPLYwo1857GllA7rw31XnXJQn0QWrqviihnzKMqL8MjUsrjGuYdVdV0jE+78B7v3NvPnr52+71qbVKKgj8NLb2/l6vsX8PhXTuOkdv3hZZuqmTl3HU8ueZ+6xmbGDingsvFD+ewJJXF9VD5YTc0tvF2xa98R/4I1O9gaG8UTyc5g1IB8xgwuYMyQAsYMLmBgQbcuvWoyEa+tjp50K8qLMHPyeAam4CeW7bUNfPv3S3lpZSXnHF/Mzy8+4aCPNltDvrYhGvKjBnZ+O+qppe/zjUeXcMKgfO6/etyH1rp4fRWXz5hPv545PDL11C496Xm4eGfLLi6661WGF/Xi0etS7+Ssgj4Ob71fw3l3vMLdl57EuaNLqG9s5uk3NvPQ3HUs2bCTSHYGF44ZyGVlQ7vkl7Yj7s7GqjoWb9jJkvU7WbKhimXv1+y7t3a/nrmMGVzA2FjwnzAon15xjoHuTC+t3Mr1Dy1kaN/uzJw8nv69UjdkWlqcGf9Yw8+ffZvi/Ai/mTT2I08UBxHyrZ5dtpmvzV7MscV5PHTtOAq6f7DX/cbGnVw6fR69Y8MXS/JT7002VTy7bDPXz1zEl0oH87PPj06pAykFfRx27N7LST+ew5QzjiDDjN+Vb6BqTyNHFvbgsvFD+fzJgwLvHXZkb1MLb1fUsKQ1/Dfu5L3K6EleMzi6sOd+R/0jinqRFWDLpzWIRhT34sFrgj3pdigWr6/ia7MXU1Fdz3fOGcHk04/scIjsxqpoyNfUNTJrSlkgBwUvrNjCl2cu4qj+PZl57bj9Tq4u21TNJffOJa9bNo9ed2pKfpJKNb94fiW/eXEVP75wFJeXpc7JWQV9HNydET98lr1NLWRmGJ8eWcTlZUM59ai+KfUufjCq9zSydOPOaPjHvnbsjg7t7JadyeiB+YwZUsCJg6JH/111ovePizfxrd8v5cRB+fz2I1oLqai6rpHv/eEN/rKsgk+OKOQXXxyz3xtV25B/eHJybr4Vr5ffqWTKg+UM6dOdh6dEPzUtf7+aS+6dR8/cLB6ZWsbgPt0Dq+9w0tLiTH6wnJffqeT04f0oyY9QlBehOC9CUX70e3FehILu2V2aFQr6ON39t9U0NDUz8ZQhoepZujsbdtSxeEPVvuBf3qblc2RhD84+roizRxYxdkjvTrmsvnVkSNkRfZl+ZelB3SArFbk7M+eu48d/XkHvHtn8euJYyo7sy6addUyc9jrVexqZOXk8JwwqCLpUXlu9jWvvL6ckP8IP/3Uk33x0CZHsTB6deipD+irkD0V1XSP/8dRy3tm6i4rqBrbvbqB9lOZmZbR7A8iNPo+9GRTFvpJ1l0wFvXykvU0trNhcw8J1Vby0citz39tOY7PTt0cOnzq2P2eNLOKM4f2Scjvf+/6xhlv//BafHFHI3XGO9U41y9+v5muzFrN2+26u+/hRPP3GZqr27OXhFAn5VgvW7uDq3y6gtqGJ4tjomiBu4hY2jc0tbN3VQEV1HRXVDVTU1LOlpp6K6vr9Hjd0MDdtv545+94Qzhjej6s+dkRcNSjo5ZDV1Dfy95WV/HXFFl56eys19fCw+7EAAAbPSURBVE3kZmVw+tH9OGtkEWce1z+uk6atl+qfO6qYX08cG6p7ftc2NPHD2I26ekWymHnteE4cnDoh32rJhp3c/bdVfPecYzmyULfh7iruTnVdIxWtbwDt3gQqaho49ci+3PKvI+N6fQW9JKSxuYUFa3YwZ8UW5ry1hY1VdQCMGVzA2SOjLZ7h/Xt+aD/S3bn9uZX8799Wc9HYgdx+8QmBngTuLO7OCyu2MqRv94O6C6VIsnRa0JtZATAdGAU4cI27v95m/SeAJ4E1sUWPu/utH/aaCvrU5u6s3LKLOcu38NcVW1gamw90aN/unHVcEWcdV8Qpw3rvF+Luzn/86S3uf20tk8YN4ScXdu6900XSUWcG/QPAK+4+3cxygO7uvrPN+k8A33b38w/2NRX0h5eK6npeeDt6pP/aqu3sbW4hv1s2nzq2P2ePLOL04f346dMreGTBBq752BFdNhuSSLr5sKCP+8yameUD/wJcBeDue4G98b6eHJ6K8yNcOn4ol44fyu6GJl55t5Ln39rCi29v5YnFmzADd/jap47mm2cfo5AXCUAiQyiOACqB35rZicBC4KbYhOFtnWpmS4H3iR7dL09gn5LCeuRmcc6oEs4ZVUJTcwsL11Xx4sqtHFXYky+WDg66PJG0FXfrxsxKgbnAx9x9npn9Gqhx9x+22SYPaHH3WjM7D/i1uw/v4LWmAlMBhgwZcvK6deviqklEJF19WOsmkWEPG4GN7j4v9vwx4KS2G7h7jbvXxh4/A2SbWT/acfdp7l7q7qWFhYUJlCQiIu3FHfTuXgFsMLMRsUVnAm+13cbMii3WlDWzcbH9bY93nyIicugSvczxa8DDsRE37wFXm9n1AO5+D3Ax8GUzawLqgImeagP3RURCThdMiYiEQGf16EVE5DCgoBcRCTkFvYhIyCnoRURCLuVOxppZJZDIFVP9gG1JKqczpHp9kPo1pnp9kPo1pnp9oBoP1VB37/BCpJQL+kSZWfmBzjynglSvD1K/xlSvD1K/xlSvD1RjMql1IyIScgp6EZGQC2PQTwu6gI+Q6vVB6teY6vVB6teY6vWBakya0PXoRURkf2E8ohcRkTYU9CIiIReaoDezc8xspZmtMrPvBV1Pe2Y22MxeMrO3zGy5md0UdE0dMbNMM1tsZn8OupaOmFmBmT1mZm+b2QozOzXomtoys2/E/n2XmdlsM4ukQE33mdlWM1vWZlkfM5tjZu/GvvdOwRpvj/07v2FmT5hZQSrV12bdt8zMO5prI1WEIujNLBO4CzgXGAlMMrORwVb1AU3At9x9JFAG3JCCNQLcBKwIuogP8WvgWXc/FjiRFKrVzAYCNwKl7j4KyAQmBlsVAPcD57Rb9j3ghdiMby/Engfpfj5Y4xxglLufALwD3NzVRbVxPx+sDzMbDHwaWN/VBR2KUAQ9MA5Y5e7vxSYpfwSYEHBN+3H3ze6+KPZ4F9GAGhhsVfszs0HAZ4HpQdfSkTYT0s+A6IT07r4z2Ko+IAvoZmZZQHeicyUHyt1fBna0WzwBeCD2+AHgwi4tqp2OanT35929KfZ0LjCoywv7Zy0d/R0C/Ar4DpDSo1rCEvQDgQ1tnm8kxUK0LTMbBowF5n34ll3uf4j+p20JupADaDsh/WIzm25mPYIuqpW7bwL+m+jR3Wag2t2fD7aqAypy982xxxVAUZDFHIRrgL8EXURbZjYB2OTuS4Ou5aOEJegPG2bWE/gD8HV3rwm6nlZmdj6w1d0XBl3Lh8giOi/x3e4+FthN8C2HfWJ97glE35AGAD3M7LJgq/posVnfUvaI1My+T7T1+XDQtbQys+7AvwO3BF3LwQhL0G8CBrd5Pii2LKWYWTbRkH/Y3R8Pup52PgZcYGZriba+PmVmM4Mt6QM+ckL6gJ0FrHH3SndvBB4HTgu4pgPZYmYlALHvWwOup0NmdhVwPnBpik1DehTRN/Slsd+ZQcAiMysOtKoDCEvQLwCGm9kRsflrJwJPBVzTfmKTpM8AVrj7L4Oupz13v9ndB7n7MKJ/fy+6e0odjR7MhPQBWw+UmVn32L/3maTQyeJ2ngKujD2+EngywFo6ZGbnEG0lXuDue4Kupy13f9Pd+7v7sNjvzEbgpNj/0ZQTiqCPnbD5KvAc0V+s37n78mCr+oCPAZcTPVJeEvs6L+iiDkOtE9K/AYwBfhpwPfvEPmk8BiwC3iT6+xX4JfJmNht4HRhhZhvN7FrgZ8DZZvYu0U8iP0vBGu8EegFzYr8v96RYfYcN3QJBRCTkQnFELyIiB6agFxEJOQW9iEjIKehFREJOQS8iEnIKehGRkFPQi4iE3P8HF9ly3Z4MnywAAAAASUVORK5CYII=\n",
"text/plain": [ "text/plain": [
"<Figure size 432x288 with 1 Axes>" "<Figure size 432x288 with 1 Axes>"
] ]
...@@ -371,7 +372,7 @@ ...@@ -371,7 +372,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 22, "execution_count": 36,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
......
...@@ -10,7 +10,7 @@ trigram,以此类推。实际应用通常采用 bigram 和 trigram 进行计 ...@@ -10,7 +10,7 @@ trigram,以此类推。实际应用通常采用 bigram 和 trigram 进行计
环境 环境
---- ----
本教程基于paddle-develop编写,如果您的环境不是本版本,请先安装paddle-develop 本教程基于paddle-2.0-beta编写,如果您的环境不是本版本,请先安装paddle-2.0-beta
.. code:: ipython3 .. code:: ipython3
...@@ -22,7 +22,7 @@ trigram,以此类推。实际应用通常采用 bigram 和 trigram 进行计 ...@@ -22,7 +22,7 @@ trigram,以此类推。实际应用通常采用 bigram 和 trigram 进行计
.. parsed-literal:: .. parsed-literal::
'0.0.0' '2.0.0-beta0'
...@@ -39,16 +39,15 @@ context_size设为2,意味着是trigram。embedding_dim设为256。 ...@@ -39,16 +39,15 @@ context_size设为2,意味着是trigram。embedding_dim设为256。
.. parsed-literal:: .. parsed-literal::
--2020-09-09 14:58:26-- https://ocw.mit.edu/ans7870/6/6.006/s08/lecturenotes/files/t8.shakespeare.txt --2020-09-12 13:49:29-- https://ocw.mit.edu/ans7870/6/6.006/s08/lecturenotes/files/t8.shakespeare.txt
正在解析主机 ocw.mit.edu (ocw.mit.edu)... 151.101.110.133 正在连接 172.19.57.45:3128... 已连接。
正在连接 ocw.mit.edu (ocw.mit.edu)|151.101.110.133|:443... 已连接。 已发出 Proxy 请求,正在等待回应... 200 OK
已发出 HTTP 请求,正在等待回应... 200 OK
长度:5458199 (5.2M) [text/plain] 长度:5458199 (5.2M) [text/plain]
正在保存至: t8.shakespeare.txt 正在保存至: t8.shakespeare.txt
t8.shakespeare.txt 100%[===================>] 5.21M 94.1KB/s 用时 70s t8.shakespeare.txt 100%[===================>] 5.21M 2.01MB/s 用时 2.6s
2020-09-09 14:59:38 (75.7 KB/s) - 已保存 t8.shakespeare.txt [5458199/5458199]) 2020-09-12 13:49:33 (2.01 MB/s) - 已保存 t8.shakespeare.txt [5458199/5458199])
...@@ -164,6 +163,7 @@ context_size设为2,意味着是trigram。embedding_dim设为256。 ...@@ -164,6 +163,7 @@ context_size设为2,意味着是trigram。embedding_dim设为256。
import paddle import paddle
import numpy as np import numpy as np
import paddle.nn.functional as F
hidden_size = 1024 hidden_size = 1024
class NGramModel(paddle.nn.Layer): class NGramModel(paddle.nn.Layer):
def __init__(self, vocab_size, embedding_dim, context_size): def __init__(self, vocab_size, embedding_dim, context_size):
...@@ -176,7 +176,7 @@ context_size设为2,意味着是trigram。embedding_dim设为256。 ...@@ -176,7 +176,7 @@ context_size设为2,意味着是trigram。embedding_dim设为256。
x = self.embedding(x) x = self.embedding(x)
x = paddle.reshape(x, [-1, context_size * embedding_dim]) x = paddle.reshape(x, [-1, context_size * embedding_dim])
x = self.linear1(x) x = self.linear1(x)
x = paddle.nn.functional.relu(x) x = F.relu(x)
x = self.linear2(x) x = self.linear2(x)
return x return x
...@@ -185,6 +185,7 @@ context_size设为2,意味着是trigram。embedding_dim设为256。 ...@@ -185,6 +185,7 @@ context_size设为2,意味着是trigram。embedding_dim设为256。
.. code:: ipython3 .. code:: ipython3
import paddle.nn.functional as F
vocab_size = len(vocab) vocab_size = len(vocab)
epochs = 2 epochs = 2
losses = [] losses = []
...@@ -196,37 +197,37 @@ context_size设为2,意味着是trigram。embedding_dim设为256。 ...@@ -196,37 +197,37 @@ context_size设为2,意味着是trigram。embedding_dim设为256。
x_data = data[0] x_data = data[0]
y_data = data[1] y_data = data[1]
predicts = model(x_data) predicts = model(x_data)
y_data = paddle.reshape(y_data, ([-1, 1])) y_data = paddle.reshape(y_data, shape=[-1, 1])
loss = paddle.nn.functional.softmax_with_cross_entropy(predicts, y_data) loss = F.softmax_with_cross_entropy(predicts, y_data)
avg_loss = paddle.mean(loss) avg_loss = paddle.mean(loss)
avg_loss.backward() avg_loss.backward()
if batch_id % 500 == 0: if batch_id % 500 == 0:
losses.append(avg_loss.numpy()) losses.append(avg_loss.numpy())
print("epoch: {}, batch_id: {}, loss is: {}".format(epoch, batch_id, avg_loss.numpy())) print("epoch: {}, batch_id: {}, loss is: {}".format(epoch, batch_id, avg_loss.numpy()))
optim.minimize(avg_loss) optim.step()
model.clear_gradients() optim.clear_grad()
model = NGramModel(vocab_size, embedding_dim, context_size) model = NGramModel(vocab_size, embedding_dim, context_size)
train(model) train(model)
.. parsed-literal:: .. parsed-literal::
epoch: 0, batch_id: 0, loss is: [10.252193] epoch: 0, batch_id: 0, loss is: [10.252176]
epoch: 0, batch_id: 500, loss is: [6.894636] epoch: 0, batch_id: 500, loss is: [6.6429553]
epoch: 0, batch_id: 1000, loss is: [6.849346] epoch: 0, batch_id: 1000, loss is: [6.801544]
epoch: 0, batch_id: 1500, loss is: [6.931605] epoch: 0, batch_id: 1500, loss is: [6.7114644]
epoch: 0, batch_id: 2000, loss is: [6.6860313] epoch: 0, batch_id: 2000, loss is: [6.628998]
epoch: 0, batch_id: 2500, loss is: [6.2472367] epoch: 0, batch_id: 2500, loss is: [6.511376]
epoch: 0, batch_id: 3000, loss is: [6.8818874] epoch: 0, batch_id: 3000, loss is: [6.878798]
epoch: 0, batch_id: 3500, loss is: [6.941615] epoch: 0, batch_id: 3500, loss is: [6.8752203]
epoch: 1, batch_id: 0, loss is: [6.3628616] epoch: 1, batch_id: 0, loss is: [6.5908413]
epoch: 1, batch_id: 500, loss is: [6.2065206] epoch: 1, batch_id: 500, loss is: [6.9765778]
epoch: 1, batch_id: 1000, loss is: [6.5334334] epoch: 1, batch_id: 1000, loss is: [6.603841]
epoch: 1, batch_id: 1500, loss is: [6.5788] epoch: 1, batch_id: 1500, loss is: [6.9935036]
epoch: 1, batch_id: 2000, loss is: [6.352103] epoch: 1, batch_id: 2000, loss is: [6.751287]
epoch: 1, batch_id: 2500, loss is: [6.6272373] epoch: 1, batch_id: 2500, loss is: [7.1222277]
epoch: 1, batch_id: 3000, loss is: [6.801074] epoch: 1, batch_id: 3000, loss is: [6.6431484]
epoch: 1, batch_id: 3500, loss is: [6.2274427] epoch: 1, batch_id: 3500, loss is: [6.6024966]
打印loss下降曲线 打印loss下降曲线
...@@ -248,12 +249,12 @@ context_size设为2,意味着是trigram。embedding_dim设为256。 ...@@ -248,12 +249,12 @@ context_size设为2,意味着是trigram。embedding_dim设为256。
.. parsed-literal:: .. parsed-literal::
[<matplotlib.lines.Line2D at 0x14e27b3c8>] [<matplotlib.lines.Line2D at 0x15c295cc0>]
.. image:: n_gram_model_files/n_gram_model_19_1.png .. image:: https://github.com/PaddlePaddle/FluidDoc/tree/develop/doc/paddle/tutorial/nlp_case/n_gram_model/n_gram_model_files/n_gram_model_001.png?raw=true
预测 预测
......
...@@ -13,7 +13,7 @@ ...@@ -13,7 +13,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 环境设置\n", "## 环境设置\n",
"\n", "\n",
"本示例教程基于飞桨2.0-beta版本。" "本示例教程基于飞桨2.0-beta版本。"
] ]
...@@ -27,8 +27,7 @@ ...@@ -27,8 +27,7 @@
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"0.0.0\n", "2.0.0-beta0\n"
"89af2088b6e74bdfeef2d4d78e08461ed2aafee5\n"
] ]
} }
], ],
...@@ -39,15 +38,14 @@ ...@@ -39,15 +38,14 @@
"import numpy as np\n", "import numpy as np\n",
"\n", "\n",
"paddle.disable_static()\n", "paddle.disable_static()\n",
"print(paddle.__version__)\n", "print(paddle.__version__)"
"print(paddle.__git_commit__)"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 下载数据集\n", "## 下载数据集\n",
"\n", "\n",
"我们将使用 [http://www.manythings.org/anki/](http://www.manythings.org/anki/) 提供的中英文的英汉句对作为数据集,来完成本任务。该数据集含有23610个中英文双语的句对。" "我们将使用 [http://www.manythings.org/anki/](http://www.manythings.org/anki/) 提供的中英文的英汉句对作为数据集,来完成本任务。该数据集含有23610个中英文双语的句对。"
] ]
...@@ -61,16 +59,16 @@ ...@@ -61,16 +59,16 @@
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"--2020-09-04 16:13:35-- https://www.manythings.org/anki/cmn-eng.zip\n", "--2020-09-10 16:17:25-- https://www.manythings.org/anki/cmn-eng.zip\n",
"Resolving www.manythings.org (www.manythings.org)... 104.24.109.196, 172.67.173.198, 2606:4700:3037::6818:6cc4, ...\n", "Resolving www.manythings.org (www.manythings.org)... 2606:4700:3033::6818:6dc4, 2606:4700:3036::ac43:adc6, 2606:4700:3037::6818:6cc4, ...\n",
"Connecting to www.manythings.org (www.manythings.org)|104.24.109.196|:443... connected.\n", "Connecting to www.manythings.org (www.manythings.org)|2606:4700:3033::6818:6dc4|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n", "HTTP request sent, awaiting response... 200 OK\n",
"Length: 1030722 (1007K) [application/zip]\n", "Length: 1030722 (1007K) [application/zip]\n",
"Saving to: ‘cmn-eng.zip’\n", "Saving to: ‘cmn-eng.zip’\n",
"\n", "\n",
"cmn-eng.zip 100%[===================>] 1007K 520KB/s in 1.9s \n", "cmn-eng.zip 100%[===================>] 1007K 91.2KB/s in 11s \n",
"\n", "\n",
"2020-09-04 16:13:38 (520 KB/s) - ‘cmn-eng.zip’ saved [1030722/1030722]\n", "2020-09-10 16:17:38 (91.2 KB/s) - ‘cmn-eng.zip’ saved [1030722/1030722]\n",
"\n", "\n",
"Archive: cmn-eng.zip\n", "Archive: cmn-eng.zip\n",
" inflating: cmn.txt \n", " inflating: cmn.txt \n",
...@@ -91,7 +89,7 @@ ...@@ -91,7 +89,7 @@
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
" 23610 cmn.txt\r\n" " 23610 cmn.txt\n"
] ]
} }
], ],
...@@ -103,7 +101,7 @@ ...@@ -103,7 +101,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 构建双语句对的数据结构\n", "## 构建双语句对的数据结构\n",
"\n", "\n",
"接下来我们通过处理下载下来的双语句对的文本文件,将双语句对读入到python的数据结构中。这里做了如下的处理。\n", "接下来我们通过处理下载下来的双语句对的文本文件,将双语句对读入到python的数据结构中。这里做了如下的处理。\n",
"\n", "\n",
...@@ -169,7 +167,7 @@ ...@@ -169,7 +167,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 创建词表\n", "## 创建词表\n",
"\n", "\n",
"接下来我们分别创建中英文的词表,这两份词表会用来将英文和中文的句子转换为词的ID构成的序列。词表中还加入了如下三个特殊的词:\n", "接下来我们分别创建中英文的词表,这两份词表会用来将英文和中文的句子转换为词的ID构成的序列。词表中还加入了如下三个特殊的词:\n",
"- `<pad>`: 用来对较短的句子进行填充。\n", "- `<pad>`: 用来对较短的句子进行填充。\n",
...@@ -220,7 +218,7 @@ ...@@ -220,7 +218,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 创建padding过的数据集\n", "## 创建padding过的数据集\n",
"\n", "\n",
"接下来根据词表,我们将会创建一份实际的用于训练的用numpy array组织起来的数据集。\n", "接下来根据词表,我们将会创建一份实际的用于训练的用numpy array组织起来的数据集。\n",
"- 所有的句子都通过`<pad>`补充成为了长度相同的句子。\n", "- 所有的句子都通过`<pad>`补充成为了长度相同的句子。\n",
...@@ -271,7 +269,7 @@ ...@@ -271,7 +269,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 创建网络\n", "## 创建网络\n",
"\n", "\n",
"我们将会创建一个Encoder-AttentionDecoder架构的模型结构用来完成机器翻译任务。\n", "我们将会创建一个Encoder-AttentionDecoder架构的模型结构用来完成机器翻译任务。\n",
"首先我们将设置一些必要的网络结构中用到的参数。" "首先我们将设置一些必要的网络结构中用到的参数。"
...@@ -296,7 +294,7 @@ ...@@ -296,7 +294,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# Encoder部分\n", "## Encoder部分\n",
"\n", "\n",
"在编码器的部分,我们通过查找完Embedding之后接一个LSTM的方式构建一个对源语言编码的网络。飞桨的RNN系列的API,除了LSTM之外,还提供了SimleRNN, GRU供使用,同时,还可以使用反向RNN,双向RNN,多层RNN等形式。也可以通过`dropout`参数设置是否对多层RNN的中间层进行`dropout`处理,来防止过拟合。\n", "在编码器的部分,我们通过查找完Embedding之后接一个LSTM的方式构建一个对源语言编码的网络。飞桨的RNN系列的API,除了LSTM之外,还提供了SimleRNN, GRU供使用,同时,还可以使用反向RNN,双向RNN,多层RNN等形式。也可以通过`dropout`参数设置是否对多层RNN的中间层进行`dropout`处理,来防止过拟合。\n",
"\n", "\n",
...@@ -328,7 +326,7 @@ ...@@ -328,7 +326,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# AttentionDecoder部分\n", "## AttentionDecoder部分\n",
"\n", "\n",
"在解码器部分,我们通过一个带有注意力机制的LSTM来完成解码。\n", "在解码器部分,我们通过一个带有注意力机制的LSTM来完成解码。\n",
"\n", "\n",
...@@ -402,7 +400,7 @@ ...@@ -402,7 +400,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 训练模型\n", "## 训练模型\n",
"\n", "\n",
"接下来我们开始训练模型。\n", "接下来我们开始训练模型。\n",
"\n", "\n",
...@@ -421,65 +419,65 @@ ...@@ -421,65 +419,65 @@
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"epoch:0\n", "epoch:0\n",
"iter 0, loss:[7.6194725]\n", "iter 0, loss:[7.620109]\n",
"iter 200, loss:[3.4147663]\n", "iter 200, loss:[2.9760551]\n",
"epoch:1\n", "epoch:1\n",
"iter 0, loss:[3.0931656]\n", "iter 0, loss:[2.9679596]\n",
"iter 200, loss:[2.7543137]\n", "iter 200, loss:[3.161064]\n",
"epoch:2\n", "epoch:2\n",
"iter 0, loss:[2.8413522]\n", "iter 0, loss:[2.7516625]\n",
"iter 200, loss:[2.340513]\n", "iter 200, loss:[2.9755423]\n",
"epoch:3\n", "epoch:3\n",
"iter 0, loss:[2.597812]\n", "iter 0, loss:[2.7249248]\n",
"iter 200, loss:[2.5552855]\n", "iter 200, loss:[2.3419888]\n",
"epoch:4\n", "epoch:4\n",
"iter 0, loss:[2.0783448]\n", "iter 0, loss:[2.3236473]\n",
"iter 200, loss:[2.4544785]\n", "iter 200, loss:[2.3453429]\n",
"epoch:5\n", "epoch:5\n",
"iter 0, loss:[1.8709135]\n", "iter 0, loss:[2.1926975]\n",
"iter 200, loss:[1.8736631]\n", "iter 200, loss:[2.1977856]\n",
"epoch:6\n", "epoch:6\n",
"iter 0, loss:[1.9589291]\n", "iter 0, loss:[2.014393]\n",
"iter 200, loss:[2.119414]\n", "iter 200, loss:[2.1863418]\n",
"epoch:7\n", "epoch:7\n",
"iter 0, loss:[1.5829577]\n", "iter 0, loss:[1.8619595]\n",
"iter 200, loss:[1.6002902]\n", "iter 200, loss:[1.8904227]\n",
"epoch:8\n", "epoch:8\n",
"iter 0, loss:[1.6022769]\n", "iter 0, loss:[1.5901132]\n",
"iter 200, loss:[1.52694]\n", "iter 200, loss:[1.7812968]\n",
"epoch:9\n", "epoch:9\n",
"iter 0, loss:[1.3616685]\n", "iter 0, loss:[1.341565]\n",
"iter 200, loss:[1.5420443]\n", "iter 200, loss:[1.4957166]\n",
"epoch:10\n", "epoch:10\n",
"iter 0, loss:[1.0397792]\n", "iter 0, loss:[1.2202356]\n",
"iter 200, loss:[1.2458231]\n", "iter 200, loss:[1.3485341]\n",
"epoch:11\n", "epoch:11\n",
"iter 0, loss:[1.2107158]\n", "iter 0, loss:[1.1035374]\n",
"iter 200, loss:[1.426417]\n", "iter 200, loss:[1.2871654]\n",
"epoch:12\n", "epoch:12\n",
"iter 0, loss:[1.1840894]\n", "iter 0, loss:[1.194801]\n",
"iter 200, loss:[1.0999664]\n", "iter 200, loss:[1.0479954]\n",
"epoch:13\n", "epoch:13\n",
"iter 0, loss:[1.0968472]\n", "iter 0, loss:[1.0022258]\n",
"iter 200, loss:[0.8149167]\n", "iter 200, loss:[1.0899843]\n",
"epoch:14\n", "epoch:14\n",
"iter 0, loss:[0.95585203]\n", "iter 0, loss:[0.93466896]\n",
"iter 200, loss:[1.0070628]\n", "iter 200, loss:[0.99347967]\n",
"epoch:15\n", "epoch:15\n",
"iter 0, loss:[0.89463925]\n", "iter 0, loss:[0.83665943]\n",
"iter 200, loss:[0.8288595]\n", "iter 200, loss:[0.9594004]\n",
"epoch:16\n", "epoch:16\n",
"iter 0, loss:[0.5672495]\n", "iter 0, loss:[0.78929776]\n",
"iter 200, loss:[0.7317069]\n", "iter 200, loss:[0.945769]\n",
"epoch:17\n", "epoch:17\n",
"iter 0, loss:[0.76785177]\n", "iter 0, loss:[0.62574965]\n",
"iter 200, loss:[0.5319323]\n", "iter 200, loss:[0.6308163]\n",
"epoch:18\n", "epoch:18\n",
"iter 0, loss:[0.5250005]\n", "iter 0, loss:[0.63433456]\n",
"iter 200, loss:[0.4182841]\n", "iter 200, loss:[0.6287957]\n",
"epoch:19\n", "epoch:19\n",
"iter 0, loss:[0.52320284]\n", "iter 0, loss:[0.54270047]\n",
"iter 200, loss:[0.47618982]\n" "iter 200, loss:[0.72688276]\n"
] ]
} }
], ],
...@@ -527,16 +525,15 @@ ...@@ -527,16 +525,15 @@
" print(\"iter {}, loss:{}\".format(iteration, loss.numpy()))\n", " print(\"iter {}, loss:{}\".format(iteration, loss.numpy()))\n",
"\n", "\n",
" loss.backward()\n", " loss.backward()\n",
" opt.minimize(loss)\n", " opt.step()\n",
" encoder.clear_gradients()\n", " opt.clear_grad()"
" atten_decoder.clear_gradients()"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 使用模型进行机器翻译\n", "## 使用模型进行机器翻译\n",
"\n", "\n",
"根据你所使用的计算设备的不同,上面的训练过程可能需要不等的时间。(在一台Mac笔记本上,大约耗时15~20分钟)\n", "根据你所使用的计算设备的不同,上面的训练过程可能需要不等的时间。(在一台Mac笔记本上,大约耗时15~20分钟)\n",
"完成上面的模型训练之后,我们可以得到一个能够从英文翻译成中文的机器翻译模型。接下来我们通过一个greedy search来实现使用该模型完成实际的机器翻译。(实际的任务中,你可能需要用beam search算法来提升效果)" "完成上面的模型训练之后,我们可以得到一个能够从英文翻译成中文的机器翻译模型。接下来我们通过一个greedy search来实现使用该模型完成实际的机器翻译。(实际的任务中,你可能需要用beam search算法来提升效果)"
...@@ -544,43 +541,43 @@ ...@@ -544,43 +541,43 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 18, "execution_count": 12,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"i agree with him\n", "i want to study french\n",
"true: 我同意他。\n", "true: 我要学法语。\n",
"pred: 我同意他。\n", "pred: 我要学法语。\n",
"i think i ll take a bath tonight\n", "i didn t know that he was there\n",
"true: 我想我今晚會洗澡。\n", "true: 我不知道他在那裡。\n",
"pred: 我想我今晚會洗澡。\n", "pred: 我不知道他在那裡。\n",
"he asked for a drink of water\n", "i called tom\n",
"true: 他要了水喝。\n", "true: 我給湯姆打了電話。\n",
"pred: 他喝了一杯水。\n", "pred: 我看見湯姆了。\n",
"i began running\n", "he is getting along with his employees\n",
"true: 我開始跑。\n", "true: 他和他的員工相處。\n",
"pred: 我開始跑。\n", "pred: 他和他的員工相處。\n",
"i m sick\n", "we raced toward the fire\n",
"true: 我生病了。\n", "true: 我們急忙跑向火。\n",
"pred: 我生病了。\n", "pred: 我們住在美國。\n",
"you had better go to the dentist s\n", "i ran away in a hurry\n",
"true: 你最好去看牙醫。\n", "true: 我趕快跑走了。\n",
"pred: 你最好去看牙醫。\n", "pred: 我在班里是最高。\n",
"we went for a walk in the forest\n", "he cut the envelope open\n",
"true: 我们去了林中散步。\n", "true: 他裁開了那個信封。\n",
"pred: 我們去公园散步。\n", "pred: 他裁開了信封。\n",
"you ve arrived very early\n", "he s shorter than tom\n",
"true: 你來得很早。\n", "true: 他比湯姆矮。\n",
"pred: 你去早个。\n", "pred: 他比湯姆矮。\n",
"he pretended not to be listening\n", "i ve just started playing tennis\n",
"true: 他裝作沒在聽。\n", "true: 我剛開始打網球。\n",
"pred: 他假装聽到它。\n", "pred: 我剛去打網球。\n",
"he always wanted to study japanese\n", "i need to go home\n",
"true: 他一直想學日語。\n", "true: 我该回家了。\n",
"pred: 他一直想學日語。\n" "pred: 我该回家了。\n"
] ]
} }
], ],
...@@ -628,17 +625,10 @@ ...@@ -628,17 +625,10 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# The End\n", "## The End\n",
"\n", "\n",
"你还可以通过变换网络结构,调整数据集,尝试不同的参数的方式来进一步提升本示例当中的机器翻译的效果。同时,也可以尝试在其他的类似的任务中用飞桨来完成实际的实践。" "你还可以通过变换网络结构,调整数据集,尝试不同的参数的方式来进一步提升本示例当中的机器翻译的效果。同时,也可以尝试在其他的类似的任务中用飞桨来完成实际的实践。"
] ]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
} }
], ],
"metadata": { "metadata": {
...@@ -657,7 +647,7 @@ ...@@ -657,7 +647,7 @@
"name": "python", "name": "python",
"nbconvert_exporter": "python", "nbconvert_exporter": "python",
"pygments_lexer": "ipython3", "pygments_lexer": "ipython3",
"version": "3.7.7" "version": "3.7.3"
} }
}, },
"nbformat": 4, "nbformat": 4,
......
...@@ -4,7 +4,7 @@ ...@@ -4,7 +4,7 @@
本示例教程介绍如何使用飞桨完成一个机器翻译任务。我们将会使用飞桨提供的LSTM的API,组建一个\ ``sequence to sequence with attention``\ 的机器翻译的模型,并在示例的数据集上完成从英文翻译成中文的机器翻译。 本示例教程介绍如何使用飞桨完成一个机器翻译任务。我们将会使用飞桨提供的LSTM的API,组建一个\ ``sequence to sequence with attention``\ 的机器翻译的模型,并在示例的数据集上完成从英文翻译成中文的机器翻译。
环境设置 环境设置
--------- --------
本示例教程基于飞桨2.0-beta版本。 本示例教程基于飞桨2.0-beta版本。
...@@ -17,17 +17,15 @@ ...@@ -17,17 +17,15 @@
paddle.disable_static() paddle.disable_static()
print(paddle.__version__) print(paddle.__version__)
print(paddle.__git_commit__)
.. parsed-literal:: .. parsed-literal::
0.0.0 2.0.0-beta0
89af2088b6e74bdfeef2d4d78e08461ed2aafee5
下载数据集 下载数据集
------------ ----------
我们将使用 http://www.manythings.org/anki/ 我们将使用 http://www.manythings.org/anki/
提供的中英文的英汉句对作为数据集,来完成本任务。该数据集含有23610个中英文双语的句对。 提供的中英文的英汉句对作为数据集,来完成本任务。该数据集含有23610个中英文双语的句对。
...@@ -39,16 +37,16 @@ ...@@ -39,16 +37,16 @@
.. parsed-literal:: .. parsed-literal::
--2020-09-04 16:13:35-- https://www.manythings.org/anki/cmn-eng.zip --2020-09-10 16:17:25-- https://www.manythings.org/anki/cmn-eng.zip
Resolving www.manythings.org (www.manythings.org)... 104.24.109.196, 172.67.173.198, 2606:4700:3037::6818:6cc4, ... Resolving www.manythings.org (www.manythings.org)... 2606:4700:3033::6818:6dc4, 2606:4700:3036::ac43:adc6, 2606:4700:3037::6818:6cc4, ...
Connecting to www.manythings.org (www.manythings.org)|104.24.109.196|:443... connected. Connecting to www.manythings.org (www.manythings.org)|2606:4700:3033::6818:6dc4|:443... connected.
HTTP request sent, awaiting response... 200 OK HTTP request sent, awaiting response... 200 OK
Length: 1030722 (1007K) [application/zip] Length: 1030722 (1007K) [application/zip]
Saving to: ‘cmn-eng.zip’ Saving to: ‘cmn-eng.zip’
cmn-eng.zip 100%[===================>] 1007K 520KB/s in 1.9s cmn-eng.zip 100%[===================>] 1007K 91.2KB/s in 11s
2020-09-04 16:13:38 (520 KB/s) - ‘cmn-eng.zip’ saved [1030722/1030722] 2020-09-10 16:17:38 (91.2 KB/s) - ‘cmn-eng.zip’ saved [1030722/1030722]
Archive: cmn-eng.zip Archive: cmn-eng.zip
inflating: cmn.txt inflating: cmn.txt
...@@ -62,11 +60,11 @@ ...@@ -62,11 +60,11 @@
.. parsed-literal:: .. parsed-literal::
23610 cmn.txt 23610 cmn.txt
构建双语句对的数据结构 构建双语句对的数据结构
------------------------- ----------------------
接下来我们通过处理下载下来的双语句对的文本文件,将双语句对读入到python的数据结构中。这里做了如下的处理。 接下来我们通过处理下载下来的双语句对的文本文件,将双语句对读入到python的数据结构中。这里做了如下的处理。
...@@ -116,7 +114,7 @@ ...@@ -116,7 +114,7 @@
创建词表 创建词表
---------- --------
接下来我们分别创建中英文的词表,这两份词表会用来将英文和中文的句子转换为词的ID构成的序列。词表中还加入了如下三个特殊的词: 接下来我们分别创建中英文的词表,这两份词表会用来将英文和中文的句子转换为词的ID构成的序列。词表中还加入了如下三个特殊的词:
- ``<pad>``: 用来对较短的句子进行填充。 - ``<bos>``: “begin of - ``<pad>``: 用来对较短的句子进行填充。 - ``<bos>``: “begin of
...@@ -157,7 +155,7 @@ Note: ...@@ -157,7 +155,7 @@ Note:
创建padding过的数据集 创建padding过的数据集
----------------------------- ---------------------
接下来根据词表,我们将会创建一份实际的用于训练的用numpy 接下来根据词表,我们将会创建一份实际的用于训练的用numpy
array组织起来的数据集。 - array组织起来的数据集。 -
...@@ -198,7 +196,7 @@ array组织起来的数据集。 - ...@@ -198,7 +196,7 @@ array组织起来的数据集。 -
创建网络 创建网络
--------- --------
我们将会创建一个Encoder-AttentionDecoder架构的模型结构用来完成机器翻译任务。 我们将会创建一个Encoder-AttentionDecoder架构的模型结构用来完成机器翻译任务。
首先我们将设置一些必要的网络结构中用到的参数。 首先我们将设置一些必要的网络结构中用到的参数。
...@@ -214,7 +212,7 @@ array组织起来的数据集。 - ...@@ -214,7 +212,7 @@ array组织起来的数据集。 -
batch_size = 16 batch_size = 16
Encoder部分 Encoder部分
---------------- -----------
在编码器的部分,我们通过查找完Embedding之后接一个LSTM的方式构建一个对源语言编码的网络。飞桨的RNN系列的API,除了LSTM之外,还提供了SimleRNN, 在编码器的部分,我们通过查找完Embedding之后接一个LSTM的方式构建一个对源语言编码的网络。飞桨的RNN系列的API,除了LSTM之外,还提供了SimleRNN,
GRU供使用,同时,还可以使用反向RNN,双向RNN,多层RNN等形式。也可以通过\ ``dropout``\ 参数设置是否对多层RNN的中间层进行\ ``dropout``\ 处理,来防止过拟合。 GRU供使用,同时,还可以使用反向RNN,双向RNN,多层RNN等形式。也可以通过\ ``dropout``\ 参数设置是否对多层RNN的中间层进行\ ``dropout``\ 处理,来防止过拟合。
...@@ -239,7 +237,7 @@ LSTMCell等API更灵活的创建单步的RNN计算,甚至通过继承RNNCellBa ...@@ -239,7 +237,7 @@ LSTMCell等API更灵活的创建单步的RNN计算,甚至通过继承RNNCellBa
return x return x
AttentionDecoder部分 AttentionDecoder部分
------------------------ --------------------
在解码器部分,我们通过一个带有注意力机制的LSTM来完成解码。 在解码器部分,我们通过一个带有注意力机制的LSTM来完成解码。
...@@ -358,77 +356,76 @@ AttentionDecoder部分 ...@@ -358,77 +356,76 @@ AttentionDecoder部分
print("iter {}, loss:{}".format(iteration, loss.numpy())) print("iter {}, loss:{}".format(iteration, loss.numpy()))
loss.backward() loss.backward()
opt.minimize(loss) opt.step()
encoder.clear_gradients() opt.clear_grad()
atten_decoder.clear_gradients()
.. parsed-literal:: .. parsed-literal::
epoch:0 epoch:0
iter 0, loss:[7.6194725] iter 0, loss:[7.620109]
iter 200, loss:[3.4147663] iter 200, loss:[2.9760551]
epoch:1 epoch:1
iter 0, loss:[3.0931656] iter 0, loss:[2.9679596]
iter 200, loss:[2.7543137] iter 200, loss:[3.161064]
epoch:2 epoch:2
iter 0, loss:[2.8413522] iter 0, loss:[2.7516625]
iter 200, loss:[2.340513] iter 200, loss:[2.9755423]
epoch:3 epoch:3
iter 0, loss:[2.597812] iter 0, loss:[2.7249248]
iter 200, loss:[2.5552855] iter 200, loss:[2.3419888]
epoch:4 epoch:4
iter 0, loss:[2.0783448] iter 0, loss:[2.3236473]
iter 200, loss:[2.4544785] iter 200, loss:[2.3453429]
epoch:5 epoch:5
iter 0, loss:[1.8709135] iter 0, loss:[2.1926975]
iter 200, loss:[1.8736631] iter 200, loss:[2.1977856]
epoch:6 epoch:6
iter 0, loss:[1.9589291] iter 0, loss:[2.014393]
iter 200, loss:[2.119414] iter 200, loss:[2.1863418]
epoch:7 epoch:7
iter 0, loss:[1.5829577] iter 0, loss:[1.8619595]
iter 200, loss:[1.6002902] iter 200, loss:[1.8904227]
epoch:8 epoch:8
iter 0, loss:[1.6022769] iter 0, loss:[1.5901132]
iter 200, loss:[1.52694] iter 200, loss:[1.7812968]
epoch:9 epoch:9
iter 0, loss:[1.3616685] iter 0, loss:[1.341565]
iter 200, loss:[1.5420443] iter 200, loss:[1.4957166]
epoch:10 epoch:10
iter 0, loss:[1.0397792] iter 0, loss:[1.2202356]
iter 200, loss:[1.2458231] iter 200, loss:[1.3485341]
epoch:11 epoch:11
iter 0, loss:[1.2107158] iter 0, loss:[1.1035374]
iter 200, loss:[1.426417] iter 200, loss:[1.2871654]
epoch:12 epoch:12
iter 0, loss:[1.1840894] iter 0, loss:[1.194801]
iter 200, loss:[1.0999664] iter 200, loss:[1.0479954]
epoch:13 epoch:13
iter 0, loss:[1.0968472] iter 0, loss:[1.0022258]
iter 200, loss:[0.8149167] iter 200, loss:[1.0899843]
epoch:14 epoch:14
iter 0, loss:[0.95585203] iter 0, loss:[0.93466896]
iter 200, loss:[1.0070628] iter 200, loss:[0.99347967]
epoch:15 epoch:15
iter 0, loss:[0.89463925] iter 0, loss:[0.83665943]
iter 200, loss:[0.8288595] iter 200, loss:[0.9594004]
epoch:16 epoch:16
iter 0, loss:[0.5672495] iter 0, loss:[0.78929776]
iter 200, loss:[0.7317069] iter 200, loss:[0.945769]
epoch:17 epoch:17
iter 0, loss:[0.76785177] iter 0, loss:[0.62574965]
iter 200, loss:[0.5319323] iter 200, loss:[0.6308163]
epoch:18 epoch:18
iter 0, loss:[0.5250005] iter 0, loss:[0.63433456]
iter 200, loss:[0.4182841] iter 200, loss:[0.6287957]
epoch:19 epoch:19
iter 0, loss:[0.52320284] iter 0, loss:[0.54270047]
iter 200, loss:[0.47618982] iter 200, loss:[0.72688276]
使用模型进行机器翻译 使用模型进行机器翻译
----------------------- --------------------
根据你所使用的计算设备的不同,上面的训练过程可能需要不等的时间。(在一台Mac笔记本上,大约耗时15~20分钟) 根据你所使用的计算设备的不同,上面的训练过程可能需要不等的时间。(在一台Mac笔记本上,大约耗时15~20分钟)
完成上面的模型训练之后,我们可以得到一个能够从英文翻译成中文的机器翻译模型。接下来我们通过一个greedy 完成上面的模型训练之后,我们可以得到一个能够从英文翻译成中文的机器翻译模型。接下来我们通过一个greedy
...@@ -478,40 +475,39 @@ search算法来提升效果) ...@@ -478,40 +475,39 @@ search算法来提升效果)
.. parsed-literal:: .. parsed-literal::
i agree with him i want to study french
true: 我同意他 true: 我要学法语
pred: 我同意他 pred: 我要学法语
i think i ll take a bath tonight i didn t know that he was there
true: 我想我今晚會洗澡 true: 我不知道他在那裡
pred: 我想我今晚會洗澡 pred: 我不知道他在那裡
he asked for a drink of water i called tom
true: 他要了水喝 true: 我給湯姆打了電話
pred: 他喝了一杯水 pred: 我看見湯姆了
i began running he is getting along with his employees
true: 我開始跑 true: 他和他的員工相處
pred: 我開始跑 pred: 他和他的員工相處
i m sick we raced toward the fire
true: 我生病了 true: 我們急忙跑向火
pred: 我生病了 pred: 我們住在美國
you had better go to the dentist s i ran away in a hurry
true: 你最好去看牙醫 true: 我趕快跑走了
pred: 你最好去看牙醫 pred: 我在班里是最高
we went for a walk in the forest he cut the envelope open
true: 我们去了林中散步 true: 他裁開了那個信封
pred: 我們去公园散步 pred: 他裁開了信封
you ve arrived very early he s shorter than tom
true: 你來得很早 true: 他比湯姆矮
pred: 你去早个 pred: 他比湯姆矮
he pretended not to be listening i ve just started playing tennis
true: 他裝作沒在聽 true: 我剛開始打網球
pred: 他假装聽到它 pred: 我剛去打網球
he always wanted to study japanese i need to go home
true: 他一直想學日語 true: 我该回家了
pred: 他一直想學日語 pred: 我该回家了
The End The End
------- -------
你还可以通过变换网络结构,调整数据集,尝试不同的参数的方式来进一步提升本示例当中的机器翻译的效果。同时,也可以尝试在其他的类似的任务中用飞桨来完成实际的实践。 你还可以通过变换网络结构,调整数据集,尝试不同的参数的方式来进一步提升本示例当中的机器翻译的效果。同时,也可以尝试在其他的类似的任务中用飞桨来完成实际的实践。
...@@ -15,7 +15,7 @@ ...@@ -15,7 +15,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 设置环境\n", "## 设置环境\n",
"\n", "\n",
"我们将使用飞桨2.0beta版本,并确认已经开启了动态图模式。" "我们将使用飞桨2.0beta版本,并确认已经开启了动态图模式。"
] ]
...@@ -29,8 +29,7 @@ ...@@ -29,8 +29,7 @@
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"0.0.0\n", "2.0.0-beta0\n"
"89af2088b6e74bdfeef2d4d78e08461ed2aafee5\n"
] ]
} }
], ],
...@@ -40,15 +39,14 @@ ...@@ -40,15 +39,14 @@
"import numpy as np\n", "import numpy as np\n",
"\n", "\n",
"paddle.disable_static()\n", "paddle.disable_static()\n",
"print(paddle.__version__)\n", "print(paddle.__version__)"
"print(paddle.__git_commit__)\n"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 基本用法\n", "## 基本用法\n",
"\n", "\n",
"在动态图模式下,您可以直接运行一个飞桨提供的API,它会立刻返回结果到python。不再需要首先创建一个计算图,然后再给定数据去运行。" "在动态图模式下,您可以直接运行一个飞桨提供的API,它会立刻返回结果到python。不再需要首先创建一个计算图,然后再给定数据去运行。"
] ]
...@@ -62,16 +60,16 @@ ...@@ -62,16 +60,16 @@
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"[[-0.49341336 -0.8112665 ]\n", "[[ 1.5645729 -0.74514765]\n",
" [ 0.8929015 0.24661176]\n", " [-0.01248 0.68240154]\n",
" [-0.64440054 -0.7945008 ]\n", " [ 0.11316949 -1.6579045 ]\n",
" [-0.07345356 1.3641853 ]]\n", " [-0.1425675 -1.0153968 ]]\n",
"[1. 2.]\n", "[1. 2.]\n",
"[[0.5065867 1.1887336 ]\n", "[[2.5645728 1.2548523 ]\n",
" [1.8929014 2.2466118 ]\n", " [0.98752 2.6824017 ]\n",
" [0.35559946 1.2054992 ]\n", " [1.1131694 0.3420955 ]\n",
" [0.92654645 3.3641853 ]]\n", " [0.8574325 0.98460317]]\n",
"[-2.1159463 1.386125 -2.2334023 2.654917 ]\n" "[ 0.07427764 1.352323 -3.2026396 -2.173361 ]\n"
] ]
} }
], ],
...@@ -93,14 +91,14 @@ ...@@ -93,14 +91,14 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 使用python的控制流\n", "## 使用python的控制流\n",
"\n", "\n",
"动态图模式下,您可以使用python的条件判断和循环,这类控制语句来执行神经网络的计算。(不再需要`cond`, `loop`这类OP)\n" "动态图模式下,您可以使用python的条件判断和循环,这类控制语句来执行神经网络的计算。(不再需要`cond`, `loop`这类OP)\n"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 4, "execution_count": 5,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -108,12 +106,12 @@ ...@@ -108,12 +106,12 @@
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"0 +> [5 6 7]\n", "0 +> [5 6 7]\n",
"1 +> [5 7 9]\n", "1 -> [-3 -3 -3]\n",
"2 +> [ 5 9 15]\n", "2 +> [ 5 9 15]\n",
"3 -> [-3 3 21]\n", "3 -> [-3 3 21]\n",
"4 -> [-3 11 75]\n", "4 +> [ 5 21 87]\n",
"5 +> [ 5 37 249]\n", "5 -> [ -3 27 237]\n",
"6 +> [ 5 69 735]\n", "6 -> [ -3 59 723]\n",
"7 -> [ -3 123 2181]\n", "7 -> [ -3 123 2181]\n",
"8 +> [ 5 261 6567]\n", "8 +> [ 5 261 6567]\n",
"9 +> [ 5 517 19689]\n" "9 +> [ 5 517 19689]\n"
...@@ -138,7 +136,7 @@ ...@@ -138,7 +136,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 构建更加灵活的网络:控制流\n", "## 构建更加灵活的网络:控制流\n",
"\n", "\n",
"- 使用动态图可以用来创建更加灵活的网络,比如根据控制流选择不同的分支网络,和方便的构建权重共享的网络。接下来我们来看一个具体的例子,在这个例子中,第二个线性变换只有0.5的可能性会运行。\n", "- 使用动态图可以用来创建更加灵活的网络,比如根据控制流选择不同的分支网络,和方便的构建权重共享的网络。接下来我们来看一个具体的例子,在这个例子中,第二个线性变换只有0.5的可能性会运行。\n",
"- 在sequence to sequence with attention的机器翻译的示例中,你会看到更实际的使用动态图构建RNN类的网络带来的灵活性。\n" "- 在sequence to sequence with attention的机器翻译的示例中,你会看到更实际的使用动态图构建RNN类的网络带来的灵活性。\n"
...@@ -146,7 +144,7 @@ ...@@ -146,7 +144,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 5, "execution_count": 6,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
...@@ -172,28 +170,28 @@ ...@@ -172,28 +170,28 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 5, "execution_count": 7,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"0 [2.0915627]\n", "0 [1.3384138]\n",
"200 [0.67530334]\n", "200 [0.7855983]\n",
"400 [0.52042854]\n", "400 [0.59084535]\n",
"600 [0.28010666]\n", "600 [0.30849028]\n",
"800 [0.09739777]\n", "800 [0.26992702]\n",
"1000 [0.09307177]\n", "1000 [0.03990713]\n",
"1200 [0.04252927]\n", "1200 [0.07111286]\n",
"1400 [0.03095707]\n", "1400 [0.01177792]\n",
"1600 [0.03022156]\n", "1600 [0.03160322]\n",
"1800 [0.01616007]\n", "1800 [0.02757282]\n",
"2000 [0.01069116]\n", "2000 [0.00916022]\n",
"2200 [0.0055158]\n", "2200 [0.00217024]\n",
"2400 [0.00195092]\n", "2400 [0.00186833]\n",
"2600 [0.00101116]\n", "2600 [0.00101926]\n",
"2800 [0.00192219]\n" "2800 [0.0009654]\n"
] ]
} }
], ],
...@@ -220,39 +218,39 @@ ...@@ -220,39 +218,39 @@
" print(t, loss.numpy())\n", " print(t, loss.numpy())\n",
"\n", "\n",
" loss.backward()\n", " loss.backward()\n",
" optimizer.minimize(loss)\n", " optimizer.step()\n",
" model.clear_gradients()" " optimizer.clear_grad()"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 构建更加灵活的网络:共享权重\n", "## 构建更加灵活的网络:共享权重\n",
"\n", "\n",
"- 使用动态图还可以更加方便的创建共享权重的网络,下面的示例展示了一个共享了权重的简单的AutoEncoder的示例。\n", "- 使用动态图还可以更加方便的创建共享权重的网络,下面的示例展示了一个共享了权重的简单的AutoEncoder。\n",
"- 你也可以参考图像搜索的示例看到共享参数权重的更实际的使用。" "- 你也可以参考图像搜索的示例看到共享参数权重的更实际的使用。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 7, "execution_count": 8,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"step: 0, loss: [0.37666085]\n", "step: 0, loss: [0.33474904]\n",
"step: 1, loss: [0.3063845]\n", "step: 1, loss: [0.31669515]\n",
"step: 2, loss: [0.2647248]\n", "step: 2, loss: [0.29729688]\n",
"step: 3, loss: [0.23831272]\n", "step: 3, loss: [0.27288628]\n",
"step: 4, loss: [0.21714918]\n", "step: 4, loss: [0.24694422]\n",
"step: 5, loss: [0.1955545]\n", "step: 5, loss: [0.2203041]\n",
"step: 6, loss: [0.17261818]\n", "step: 6, loss: [0.19171436]\n",
"step: 7, loss: [0.15009595]\n", "step: 7, loss: [0.16213782]\n",
"step: 8, loss: [0.13051331]\n", "step: 8, loss: [0.13443354]\n",
"step: 9, loss: [0.11537809]\n" "step: 9, loss: [0.11170781]\n"
] ]
} }
], ],
...@@ -270,15 +268,15 @@ ...@@ -270,15 +268,15 @@
" loss = loss_fn(outputs, inputs)\n", " loss = loss_fn(outputs, inputs)\n",
" loss.backward()\n", " loss.backward()\n",
" print(\"step: {}, loss: {}\".format(i, loss.numpy()))\n", " print(\"step: {}, loss: {}\".format(i, loss.numpy()))\n",
" optimizer.minimize(loss)\n", " optimizer.step()\n",
" linear.clear_gradients()" " optimizer.clear_grad()"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# The end\n", "## The end\n",
"\n", "\n",
"可以看到使用动态图带来了更灵活易用的方式来组网和训练。" "可以看到使用动态图带来了更灵活易用的方式来组网和训练。"
] ]
...@@ -300,7 +298,7 @@ ...@@ -300,7 +298,7 @@
"name": "python", "name": "python",
"nbconvert_exporter": "python", "nbconvert_exporter": "python",
"pygments_lexer": "ipython3", "pygments_lexer": "ipython3",
"version": "3.7.7" "version": "3.7.3"
} }
}, },
"nbformat": 4, "nbformat": 4,
......
...@@ -18,14 +18,11 @@ ...@@ -18,14 +18,11 @@
paddle.disable_static() paddle.disable_static()
print(paddle.__version__) print(paddle.__version__)
print(paddle.__git_commit__)
.. parsed-literal:: .. parsed-literal::
0.0.0 2.0.0-beta0
89af2088b6e74bdfeef2d4d78e08461ed2aafee5
基本用法 基本用法
...@@ -50,16 +47,16 @@ ...@@ -50,16 +47,16 @@
.. parsed-literal:: .. parsed-literal::
[[-0.49341336 -0.8112665 ] [[ 1.5645729 -0.74514765]
[ 0.8929015 0.24661176] [-0.01248 0.68240154]
[-0.64440054 -0.7945008 ] [ 0.11316949 -1.6579045 ]
[-0.07345356 1.3641853 ]] [-0.1425675 -1.0153968 ]]
[1. 2.] [1. 2.]
[[0.5065867 1.1887336 ] [[2.5645728 1.2548523 ]
[1.8929014 2.2466118 ] [0.98752 2.6824017 ]
[0.35559946 1.2054992 ] [1.1131694 0.3420955 ]
[0.92654645 3.3641853 ]] [0.8574325 0.98460317]]
[-2.1159463 1.386125 -2.2334023 2.654917 ] [ 0.07427764 1.352323 -3.2026396 -2.173361 ]
使用python的控制流 使用python的控制流
...@@ -87,19 +84,19 @@ ...@@ -87,19 +84,19 @@
.. parsed-literal:: .. parsed-literal::
0 +> [5 6 7] 0 +> [5 6 7]
1 +> [5 7 9] 1 -> [-3 -3 -3]
2 +> [ 5 9 15] 2 +> [ 5 9 15]
3 -> [-3 3 21] 3 -> [-3 3 21]
4 -> [-3 11 75] 4 +> [ 5 21 87]
5 +> [ 5 37 249] 5 -> [ -3 27 237]
6 +> [ 5 69 735] 6 -> [ -3 59 723]
7 -> [ -3 123 2181] 7 -> [ -3 123 2181]
8 +> [ 5 261 6567] 8 +> [ 5 261 6567]
9 +> [ 5 517 19689] 9 +> [ 5 517 19689]
构建更加灵活的网络:控制流 构建更加灵活的网络:控制流
------------------------------- --------------------------
- 使用动态图可以用来创建更加灵活的网络,比如根据控制流选择不同的分支网络,和方便的构建权重共享的网络。接下来我们来看一个具体的例子,在这个例子中,第二个线性变换只有0.5的可能性会运行。 - 使用动态图可以用来创建更加灵活的网络,比如根据控制流选择不同的分支网络,和方便的构建权重共享的网络。接下来我们来看一个具体的例子,在这个例子中,第二个线性变换只有0.5的可能性会运行。
- sequence to sequence with - sequence to sequence with
...@@ -150,33 +147,33 @@ ...@@ -150,33 +147,33 @@
print(t, loss.numpy()) print(t, loss.numpy())
loss.backward() loss.backward()
optimizer.minimize(loss) optimizer.step()
model.clear_gradients() optimizer.clear_grad()
.. parsed-literal:: .. parsed-literal::
0 [2.0915627] 0 [1.3384138]
200 [0.67530334] 200 [0.7855983]
400 [0.52042854] 400 [0.59084535]
600 [0.28010666] 600 [0.30849028]
800 [0.09739777] 800 [0.26992702]
1000 [0.09307177] 1000 [0.03990713]
1200 [0.04252927] 1200 [0.07111286]
1400 [0.03095707] 1400 [0.01177792]
1600 [0.03022156] 1600 [0.03160322]
1800 [0.01616007] 1800 [0.02757282]
2000 [0.01069116] 2000 [0.00916022]
2200 [0.0055158] 2200 [0.00217024]
2400 [0.00195092] 2400 [0.00186833]
2600 [0.00101116] 2600 [0.00101926]
2800 [0.00192219] 2800 [0.0009654]
构建更加灵活的网络:共享权重 构建更加灵活的网络:共享权重
--------------------------------- ----------------------------
- 使用动态图还可以更加方便的创建共享权重的网络,下面的示例展示了一个共享了权重的简单的AutoEncoder的示例 - 使用动态图还可以更加方便的创建共享权重的网络,下面的示例展示了一个共享了权重的简单的AutoEncoder
- 你也可以参考图像搜索的示例看到共享参数权重的更实际的使用。 - 你也可以参考图像搜索的示例看到共享参数权重的更实际的使用。
.. code:: ipython3 .. code:: ipython3
...@@ -194,25 +191,25 @@ ...@@ -194,25 +191,25 @@
loss = loss_fn(outputs, inputs) loss = loss_fn(outputs, inputs)
loss.backward() loss.backward()
print("step: {}, loss: {}".format(i, loss.numpy())) print("step: {}, loss: {}".format(i, loss.numpy()))
optimizer.minimize(loss) optimizer.step()
linear.clear_gradients() optimizer.clear_grad()
.. parsed-literal:: .. parsed-literal::
step: 0, loss: [0.37666085] step: 0, loss: [0.33474904]
step: 1, loss: [0.3063845] step: 1, loss: [0.31669515]
step: 2, loss: [0.2647248] step: 2, loss: [0.29729688]
step: 3, loss: [0.23831272] step: 3, loss: [0.27288628]
step: 4, loss: [0.21714918] step: 4, loss: [0.24694422]
step: 5, loss: [0.1955545] step: 5, loss: [0.2203041]
step: 6, loss: [0.17261818] step: 6, loss: [0.19171436]
step: 7, loss: [0.15009595] step: 7, loss: [0.16213782]
step: 8, loss: [0.13051331] step: 8, loss: [0.13443354]
step: 9, loss: [0.11537809] step: 9, loss: [0.11170781]
The end The end
-------- -------
可以看到使用动态图带来了更灵活易用的方式来组网和训练。 可以看到使用动态图带来了更灵活易用的方式来组网和训练。
...@@ -31,16 +31,16 @@ ...@@ -31,16 +31,16 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 4, "execution_count": 1,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"data": { "data": {
"text/plain": [ "text/plain": [
"'0.0.0'" "'2.0.0-beta0'"
] ]
}, },
"execution_count": 4, "execution_count": 1,
"metadata": {}, "metadata": {},
"output_type": "execute_result" "output_type": "execute_result"
} }
......
...@@ -26,7 +26,7 @@ ...@@ -26,7 +26,7 @@
.. parsed-literal:: .. parsed-literal::
'0.0.0' '2.0.0-beta0'
......
...@@ -18,7 +18,7 @@ ...@@ -18,7 +18,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 普通程序跟机器学习程序的逻辑区别\n", "## 普通程序跟机器学习程序的逻辑区别\n",
"\n", "\n",
"作为一名开发者,你最熟悉的开始学习一门编程语言,或者一个深度学习框架的方式,可能是通过一个hello, world程序。\n", "作为一名开发者,你最熟悉的开始学习一门编程语言,或者一个深度学习框架的方式,可能是通过一个hello, world程序。\n",
"\n", "\n",
...@@ -37,7 +37,7 @@ ...@@ -37,7 +37,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 24, "execution_count": 22,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -80,7 +80,7 @@ ...@@ -80,7 +80,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 导入飞桨\n", "## 导入飞桨\n",
"\n", "\n",
"为了能够使用飞桨,我们需要先用python的`import`语句导入飞桨`paddle`。\n", "为了能够使用飞桨,我们需要先用python的`import`语句导入飞桨`paddle`。\n",
"同时,为了能够更好的对数组进行计算和处理,我们也还需要导入`numpy`。\n", "同时,为了能够更好的对数组进行计算和处理,我们也还需要导入`numpy`。\n",
...@@ -90,28 +90,28 @@ ...@@ -90,28 +90,28 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 25, "execution_count": 3,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"paddle version 0.0.0\n" "paddle 2.0.0-beta0\n"
] ]
} }
], ],
"source": [ "source": [
"import paddle\n", "import paddle\n",
"paddle.disable_static()\n", "paddle.disable_static()\n",
"print(\"paddle version \" + paddle.__version__)" "print(\"paddle \" + paddle.__version__)"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 准备数据\n", "## 准备数据\n",
"\n", "\n",
"在这个机器学习任务中,我们已经知道了乘客的行驶里程`distance_travelled`,和对应的,这些乘客的总费用`total_fee`。\n", "在这个机器学习任务中,我们已经知道了乘客的行驶里程`distance_travelled`,和对应的,这些乘客的总费用`total_fee`。\n",
"通常情况下,在机器学习任务中,像`distance_travelled`这样的输入值,一般被称为`x`(或者特征`feature`),像`total_fee`这样的输出值,一般被称为`y`(或者标签`label`)。\n", "通常情况下,在机器学习任务中,像`distance_travelled`这样的输入值,一般被称为`x`(或者特征`feature`),像`total_fee`这样的输出值,一般被称为`y`(或者标签`label`)。\n",
...@@ -121,7 +121,7 @@ ...@@ -121,7 +121,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 26, "execution_count": 4,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
...@@ -133,7 +133,7 @@ ...@@ -133,7 +133,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 用飞桨定义模型的计算\n", "## 用飞桨定义模型的计算\n",
"\n", "\n",
"使用飞桨定义模型的计算的过程,本质上,是我们用python,通过飞桨提供的API,来告诉飞桨我们的计算规则的过程。回顾一下,我们想要通过飞桨用机器学习方法,从数据当中学习出来如下公式当中的`w`和`b`。这样在未来,给定`x`时就可以估算出来`y`值(估算出来的`y`记为`y_predict`)\n", "使用飞桨定义模型的计算的过程,本质上,是我们用python,通过飞桨提供的API,来告诉飞桨我们的计算规则的过程。回顾一下,我们想要通过飞桨用机器学习方法,从数据当中学习出来如下公式当中的`w`和`b`。这样在未来,给定`x`时就可以估算出来`y`值(估算出来的`y`记为`y_predict`)\n",
"\n", "\n",
...@@ -150,7 +150,7 @@ ...@@ -150,7 +150,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 27, "execution_count": 5,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
...@@ -161,21 +161,21 @@ ...@@ -161,21 +161,21 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 准备好运行飞桨\n", "## 准备好运行飞桨\n",
"\n", "\n",
"机器(计算机)在一开始的时候会随便猜`w`和`b`,我们先看看机器猜的怎么样。你应该可以看到,这时候的`w`是一个随机值,`b`是0.0,这是飞桨的初始化策略,也是这个领域常用的初始化策略。(如果你愿意,也可以采用其他的初始化的方式,今后你也会看到,选择不同的初始化策略也是对于做好深度学习任务来说很重要的一点)。" "机器(计算机)在一开始的时候会随便猜`w`和`b`,我们先看看机器猜的怎么样。你应该可以看到,这时候的`w`是一个随机值,`b`是0.0,这是飞桨的初始化策略,也是这个领域常用的初始化策略。(如果你愿意,也可以采用其他的初始化的方式,今后你也会看到,选择不同的初始化策略也是对于做好深度学习任务来说很重要的一点)。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 28, "execution_count": 6,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"w before optimize: -1.7107375860214233\n", "w before optimize: -1.696260690689087\n",
"b before optimize: 0.0\n" "b before optimize: 0.0\n"
] ]
} }
...@@ -192,7 +192,7 @@ ...@@ -192,7 +192,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 告诉飞桨怎么样学习\n", "## 告诉飞桨怎么样学习\n",
"\n", "\n",
"前面我们定义好了神经网络(尽管是一个最简单的神经网络),我们还需要告诉飞桨,怎么样去**学习**,从而能得到参数`w`和`b`。\n", "前面我们定义好了神经网络(尽管是一个最简单的神经网络),我们还需要告诉飞桨,怎么样去**学习**,从而能得到参数`w`和`b`。\n",
"\n", "\n",
...@@ -205,7 +205,7 @@ ...@@ -205,7 +205,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 29, "execution_count": 7,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
...@@ -217,26 +217,26 @@ ...@@ -217,26 +217,26 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 运行优化算法\n", "## 运行优化算法\n",
"\n", "\n",
"接下来,我们让飞桨运行一下这个优化算法,这会是一个前面介绍过的逐步调整参数的过程,你应该可以看到loss值(衡量`y`和`y_predict`的差距的`loss`)在不断的降低。" "接下来,我们让飞桨运行一下这个优化算法,这会是一个前面介绍过的逐步调整参数的过程,你应该可以看到loss值(衡量`y`和`y_predict`的差距的`loss`)在不断的降低。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 30, "execution_count": 8,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"epoch 0 loss [2107.3943]\n", "epoch 0 loss [2094.069]\n",
"epoch 1000 loss [7.8432994]\n", "epoch 1000 loss [7.8451133]\n",
"epoch 2000 loss [1.7537074]\n", "epoch 2000 loss [1.7541145]\n",
"epoch 3000 loss [0.39211753]\n", "epoch 3000 loss [0.39221546]\n",
"epoch 4000 loss [0.08767726]\n", "epoch 4000 loss [0.08769739]\n",
"finished training, loss [0.01963376]\n" "finished training, loss [0.0196382]\n"
] ]
} }
], ],
...@@ -246,8 +246,8 @@ ...@@ -246,8 +246,8 @@
" y_predict = linear(x_data)\n", " y_predict = linear(x_data)\n",
" loss = mse_loss(y_predict, y_data)\n", " loss = mse_loss(y_predict, y_data)\n",
" loss.backward()\n", " loss.backward()\n",
" sgd_optimizer.minimize(loss)\n", " sgd_optimizer.step()\n",
" linear.clear_gradients()\n", " sgd_optimizer.clear_grad()\n",
" \n", " \n",
" if i%1000 == 0:\n", " if i%1000 == 0:\n",
" print(\"epoch {} loss {}\".format(i, loss.numpy()))\n", " print(\"epoch {} loss {}\".format(i, loss.numpy()))\n",
...@@ -259,22 +259,22 @@ ...@@ -259,22 +259,22 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# 机器学习出来的参数\n", "## 机器学习出来的参数\n",
"\n", "\n",
"经过了这样的对参数`w`和`b`的调整(**学习**),我们再通过下面的程序,来看看现在的参数变成了多少。你应该会发现`w`变成了很接近2.0的一个值,`b`变成了接近10.0的一个值。虽然并不是正好的2和10,但却是从数据当中学习出来的还不错的模型的参数,可以在未来的时候,用从这批数据当中学习到的参数来预估了。(如果你愿意,也可以通过让机器多学习一段时间,从而得到更加接近2.0和10.0的参数值。)" "经过了这样的对参数`w`和`b`的调整(**学习**),我们再通过下面的程序,来看看现在的参数变成了多少。你应该会发现`w`变成了很接近2.0的一个值,`b`变成了接近10.0的一个值。虽然并不是正好的2和10,但却是从数据当中学习出来的还不错的模型的参数,可以在未来的时候,用从这批数据当中学习到的参数来预估了。(如果你愿意,也可以通过让机器多学习一段时间,从而得到更加接近2.0和10.0的参数值。)"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 31, "execution_count": 9,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"w after optimize: 2.017843246459961\n", "w after optimize: 2.0178451538085938\n",
"b after optimize: 9.771851539611816\n" "b after optimize: 9.771825790405273\n"
] ]
} }
], ],
...@@ -290,14 +290,14 @@ ...@@ -290,14 +290,14 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# hello paddle\n", "## hello paddle\n",
"\n", "\n",
"通过这个小示例,希望你已经初步了解了飞桨,能在接下来随着对飞桨的更多学习,来解决实际遇到的问题。" "通过这个小示例,希望你已经初步了解了飞桨,能在接下来随着对飞桨的更多学习,来解决实际遇到的问题。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 32, "execution_count": 10,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
...@@ -335,9 +335,9 @@ ...@@ -335,9 +335,9 @@
"name": "python", "name": "python",
"nbconvert_exporter": "python", "nbconvert_exporter": "python",
"pygments_lexer": "ipython3", "pygments_lexer": "ipython3",
"version": "3.7.7" "version": "3.7.3"
} }
}, },
"nbformat": 4, "nbformat": 4,
"nbformat_minor": 1 "nbformat_minor": 4
} }
...@@ -51,7 +51,7 @@ world程序。 ...@@ -51,7 +51,7 @@ world程序。
接下来,我们看看用飞桨如何实现这个hello, world级别的机器学习程序。 接下来,我们看看用飞桨如何实现这个hello, world级别的机器学习程序。
导入飞桨 导入飞桨
--------- --------
为了能够使用飞桨,我们需要先用python的\ ``import``\ 语句导入飞桨\ ``paddle``\ 。 为了能够使用飞桨,我们需要先用python的\ ``import``\ 语句导入飞桨\ ``paddle``\ 。
同时,为了能够更好的对数组进行计算和处理,我们也还需要导入\ ``numpy``\ 。 同时,为了能够更好的对数组进行计算和处理,我们也还需要导入\ ``numpy``\ 。
...@@ -62,16 +62,16 @@ world程序。 ...@@ -62,16 +62,16 @@ world程序。
import paddle import paddle
paddle.disable_static() paddle.disable_static()
print("paddle version " + paddle.__version__) print("paddle " + paddle.__version__)
.. parsed-literal:: .. parsed-literal::
paddle version 0.0.0 paddle 2.0.0-beta0
准备数据 准备数据
--------- --------
在这个机器学习任务中,我们已经知道了乘客的行驶里程\ ``distance_travelled``\ ,和对应的,这些乘客的总费用\ ``total_fee``\ 。 在这个机器学习任务中,我们已经知道了乘客的行驶里程\ ``distance_travelled``\ ,和对应的,这些乘客的总费用\ ``total_fee``\ 。
通常情况下,在机器学习任务中,像\ ``distance_travelled``\ 这样的输入值,一般被称为\ ``x``\ (或者特征\ ``feature``\ ),像\ ``total_fee``\ 这样的输出值,一般被称为\ ``y``\ (或者标签\ ``label``)。 通常情况下,在机器学习任务中,像\ ``distance_travelled``\ 这样的输入值,一般被称为\ ``x``\ (或者特征\ ``feature``\ ),像\ ``total_fee``\ 这样的输出值,一般被称为\ ``y``\ (或者标签\ ``label``)。
...@@ -104,7 +104,7 @@ world程序。 ...@@ -104,7 +104,7 @@ world程序。
linear = paddle.nn.Linear(in_features=1, out_features=1) linear = paddle.nn.Linear(in_features=1, out_features=1)
准备好运行飞桨 准备好运行飞桨
---------------- --------------
机器(计算机)在一开始的时候会随便猜\ ``w``\ 和\ ``b``\ ,我们先看看机器猜的怎么样。你应该可以看到,这时候的\ ``w``\ 是一个随机值,\ ``b``\ 是0.0,这是飞桨的初始化策略,也是这个领域常用的初始化策略。(如果你愿意,也可以采用其他的初始化的方式,今后你也会看到,选择不同的初始化策略也是对于做好深度学习任务来说很重要的一点)。 机器(计算机)在一开始的时候会随便猜\ ``w``\ 和\ ``b``\ ,我们先看看机器猜的怎么样。你应该可以看到,这时候的\ ``w``\ 是一个随机值,\ ``b``\ 是0.0,这是飞桨的初始化策略,也是这个领域常用的初始化策略。(如果你愿意,也可以采用其他的初始化的方式,今后你也会看到,选择不同的初始化策略也是对于做好深度学习任务来说很重要的一点)。
...@@ -119,12 +119,12 @@ world程序。 ...@@ -119,12 +119,12 @@ world程序。
.. parsed-literal:: .. parsed-literal::
w before optimize: -1.7107375860214233 w before optimize: -1.696260690689087
b before optimize: 0.0 b before optimize: 0.0
告诉飞桨怎么样学习 告诉飞桨怎么样学习
-------------------- ------------------
前面我们定义好了神经网络(尽管是一个最简单的神经网络),我们还需要告诉飞桨,怎么样去\ **学习**\ ,从而能得到参数\ ``w``\ 和\ ``b``\ 。 前面我们定义好了神经网络(尽管是一个最简单的神经网络),我们还需要告诉飞桨,怎么样去\ **学习**\ ,从而能得到参数\ ``w``\ 和\ ``b``\ 。
...@@ -143,7 +143,7 @@ descent)作为优化算法(传给\ ``paddle.optimizer.SGD``\ 的参数\ ``lear ...@@ -143,7 +143,7 @@ descent)作为优化算法(传给\ ``paddle.optimizer.SGD``\ 的参数\ ``lear
sgd_optimizer = paddle.optimizer.SGD(learning_rate=0.001, parameters = linear.parameters()) sgd_optimizer = paddle.optimizer.SGD(learning_rate=0.001, parameters = linear.parameters())
运行优化算法 运行优化算法
--------------- ------------
接下来,我们让飞桨运行一下这个优化算法,这会是一个前面介绍过的逐步调整参数的过程,你应该可以看到loss值(衡量\ ``y``\ 和\ ``y_predict``\ 的差距的\ ``loss``)在不断的降低。 接下来,我们让飞桨运行一下这个优化算法,这会是一个前面介绍过的逐步调整参数的过程,你应该可以看到loss值(衡量\ ``y``\ 和\ ``y_predict``\ 的差距的\ ``loss``)在不断的降低。
...@@ -154,8 +154,8 @@ descent)作为优化算法(传给\ ``paddle.optimizer.SGD``\ 的参数\ ``lear ...@@ -154,8 +154,8 @@ descent)作为优化算法(传给\ ``paddle.optimizer.SGD``\ 的参数\ ``lear
y_predict = linear(x_data) y_predict = linear(x_data)
loss = mse_loss(y_predict, y_data) loss = mse_loss(y_predict, y_data)
loss.backward() loss.backward()
sgd_optimizer.minimize(loss) sgd_optimizer.step()
linear.clear_gradients() sgd_optimizer.clear_grad()
if i%1000 == 0: if i%1000 == 0:
print("epoch {} loss {}".format(i, loss.numpy())) print("epoch {} loss {}".format(i, loss.numpy()))
...@@ -165,16 +165,16 @@ descent)作为优化算法(传给\ ``paddle.optimizer.SGD``\ 的参数\ ``lear ...@@ -165,16 +165,16 @@ descent)作为优化算法(传给\ ``paddle.optimizer.SGD``\ 的参数\ ``lear
.. parsed-literal:: .. parsed-literal::
epoch 0 loss [2107.3943] epoch 0 loss [2094.069]
epoch 1000 loss [7.8432994] epoch 1000 loss [7.8451133]
epoch 2000 loss [1.7537074] epoch 2000 loss [1.7541145]
epoch 3000 loss [0.39211753] epoch 3000 loss [0.39221546]
epoch 4000 loss [0.08767726] epoch 4000 loss [0.08769739]
finished training, loss [0.01963376] finished training, loss [0.0196382]
机器学习出来的参数 机器学习出来的参数
------------------- ------------------
经过了这样的对参数\ ``w``\ 和\ ``b``\ 的调整(\ **学习**),我们再通过下面的程序,来看看现在的参数变成了多少。你应该会发现\ ``w``\ 变成了很接近2.0的一个值,\ ``b``\ 变成了接近10.0的一个值。虽然并不是正好的2和10,但却是从数据当中学习出来的还不错的模型的参数,可以在未来的时候,用从这批数据当中学习到的参数来预估了。(如果你愿意,也可以通过让机器多学习一段时间,从而得到更加接近2.0和10.0的参数值。) 经过了这样的对参数\ ``w``\ 和\ ``b``\ 的调整(\ **学习**),我们再通过下面的程序,来看看现在的参数变成了多少。你应该会发现\ ``w``\ 变成了很接近2.0的一个值,\ ``b``\ 变成了接近10.0的一个值。虽然并不是正好的2和10,但却是从数据当中学习出来的还不错的模型的参数,可以在未来的时候,用从这批数据当中学习到的参数来预估了。(如果你愿意,也可以通过让机器多学习一段时间,从而得到更加接近2.0和10.0的参数值。)
...@@ -190,12 +190,12 @@ descent)作为优化算法(传给\ ``paddle.optimizer.SGD``\ 的参数\ ``lear ...@@ -190,12 +190,12 @@ descent)作为优化算法(传给\ ``paddle.optimizer.SGD``\ 的参数\ ``lear
.. parsed-literal:: .. parsed-literal::
w after optimize: 2.017843246459961 w after optimize: 2.0178451538085938
b after optimize: 9.771851539611816 b after optimize: 9.771825790405273
hello paddle hello paddle
--------------- ------------
通过这个小示例,希望你已经初步了解了飞桨,能在接下来随着对飞桨的更多学习,来解决实际遇到的问题。 通过这个小示例,希望你已经初步了解了飞桨,能在接下来随着对飞桨的更多学习,来解决实际遇到的问题。
......
{ {
"metadata": {
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4-final"
},
"orig_nbformat": 2,
"kernelspec": {
"name": "python37464bitc4da1ac836094043840bff631bedbf7f",
"display_name": "Python 3.7.4 64-bit"
}
},
"nbformat": 4,
"nbformat_minor": 2,
"cells": [ "cells": [
{ {
"cell_type": "markdown", "cell_type": "markdown",
...@@ -57,16 +36,18 @@ ...@@ -57,16 +36,18 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 4, "execution_count": 1,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"output_type": "execute_result",
"data": { "data": {
"text/plain": "'0.0.0'" "text/plain": [
"'2.0.0-beta0'"
]
}, },
"execution_count": 1,
"metadata": {}, "metadata": {},
"execution_count": 4 "output_type": "execute_result"
} }
], ],
"source": [ "source": [
...@@ -91,10 +72,7 @@ ...@@ -91,10 +72,7 @@
"* 如何进行模型的组网。\n", "* 如何进行模型的组网。\n",
"* 高层API进行模型训练的相关API使用。\n", "* 高层API进行模型训练的相关API使用。\n",
"* 如何在fit接口满足需求的时候进行自定义,使用基础API来完成训练。\n", "* 如何在fit接口满足需求的时候进行自定义,使用基础API来完成训练。\n",
"* 如何使用多卡来加速训练。\n", "* 如何使用多卡来加速训练。"
"\n",
"其他端到端的示例教程:\n",
"* TBD"
] ]
}, },
{ {
...@@ -112,22 +90,23 @@ ...@@ -112,22 +90,23 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 17, "execution_count": 5,
"metadata": { "metadata": {
"tags": [] "tags": []
}, },
"outputs": [ "outputs": [
{ {
"output_type": "execute_result", "name": "stdout",
"data": { "output_type": "stream",
"text/plain": "['DatasetFolder',\n 'ImageFolder',\n 'MNIST',\n 'Flowers',\n 'Cifar10',\n 'Cifar100',\n 'VOC2012']" "text": [
}, "视觉相关数据集: ['DatasetFolder', 'ImageFolder', 'MNIST', 'Flowers', 'Cifar10', 'Cifar100', 'VOC2012']\n",
"metadata": {}, "自然语言相关数据集: ['Conll05st', 'Imdb', 'Imikolov', 'Movielens', 'MovieReviews', 'UCIHousing', 'WMT14', 'WMT16']\n"
"execution_count": 17 ]
} }
], ],
"source": [ "source": [
"paddle.vision.datasets.__all__" "print('视觉相关数据集:', paddle.vision.datasets.__all__)\n",
"print('自然语言相关数据集:', paddle.text.datasets.__all__)"
] ]
}, },
{ {
...@@ -143,7 +122,7 @@ ...@@ -143,7 +122,7 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"# 测试数据集\n", "# 训练数据集\n",
"train_dataset = vision.datasets.MNIST(mode='train')\n", "train_dataset = vision.datasets.MNIST(mode='train')\n",
"\n", "\n",
"# 验证数据集\n", "# 验证数据集\n",
...@@ -167,9 +146,20 @@ ...@@ -167,9 +146,20 @@
}, },
"outputs": [ "outputs": [
{ {
"output_type": "stream",
"name": "stdout", "name": "stdout",
"text": "=============train dataset=============\ntraindata1 label1\ntraindata2 label2\ntraindata3 label3\ntraindata4 label4\n=============evaluation dataset=============\ntestdata1 label1\ntestdata2 label2\ntestdata3 label3\ntestdata4 label4\n" "output_type": "stream",
"text": [
"=============train dataset=============\n",
"traindata1 label1\n",
"traindata2 label2\n",
"traindata3 label3\n",
"traindata4 label4\n",
"=============evaluation dataset=============\n",
"testdata1 label1\n",
"testdata2 label2\n",
"testdata3 label3\n",
"testdata4 label4\n"
]
} }
], ],
"source": [ "source": [
...@@ -458,9 +448,9 @@ ...@@ -458,9 +448,9 @@
"source": [ "source": [
"## 5. 模型训练\n", "## 5. 模型训练\n",
"\n", "\n",
"使用`paddle.Model`封装成模型类后进行训练非常的简洁方便,我们可以直接通过调用`Model.fit`就可以完成训练过程。\n", "网络结构通过`paddle.Model`接口封装成模型类后进行执行操作非常的简洁方便,可以直接通过调用`Model.fit`就可以完成训练过程。\n",
"\n", "\n",
"使用`Model.fit`接口启动训练前,我们先通过`Model.prepare`接口来对训练进行提前的配置准备工作,包括设置模型优化器,Loss计算方法,精度计算方法等。\n", "使用`Model.fit`接口启动训练前,我们先通过`Model.prepare`接口来对训练进行提前的配置准备工作,包括设置模型优化器,Loss计算方法,精度计算方法等。\n",
"\n" "\n"
] ]
}, },
...@@ -553,13 +543,269 @@ ...@@ -553,13 +543,269 @@
"python -m paddle.distributed.launch train.py" "python -m paddle.distributed.launch train.py"
] ]
}, },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 5.3 自定义Loss\n",
"\n",
"有时我们会遇到特定任务的Loss计算方式在框架既有的Loss接口中不存在,或算法不符合自己的需求,那么期望能够自己来进行Loss的自定义,我们这里就会讲解介绍一下如何进行Loss的自定义操作,首先来看下面的代码:\n",
"\n",
"```python\n",
"class SelfDefineLoss(paddle.nn.Layer):\n",
" \"\"\"\n",
" 1. 继承paddle.nn.Layer\n",
" \"\"\"\n",
" def __init__(self):\n",
" \"\"\"\n",
" 2. 构造函数根据自己的实际算法需求和使用需求进行参数定义即可\n",
" \"\"\"\n",
" super(SelfDefineLoss, self).__init__()\n",
"\n",
" def forward(self, input, label):\n",
" \"\"\"\n",
" 3. 实现forward函数,forward在调用时会传递两个参数:input和label\n",
" - input:单个或批次训练数据经过模型前向计算输出结果\n",
" - label:单个或批次训练数据对应的标签数据\n",
"\n",
" 接口返回值是一个Tensor,根据自定义的逻辑加和或计算均值后的损失\n",
" \"\"\"\n",
" # 使用Paddle中相关API自定义的计算逻辑\n",
" # output = xxxxx\n",
" # return output\n",
"```\n",
"\n",
"那么了解完代码层面如果编写自定义代码后我们看一个实际的例子,下面是在图像分割示例代码中写的一个自定义Loss,当时主要是想使用自定义的softmax计算维度。\n",
"\n",
"```python\n",
"class SoftmaxWithCrossEntropy(paddle.nn.Layer):\n",
" def __init__(self):\n",
" super(SoftmaxWithCrossEntropy, self).__init__()\n",
"\n",
" def forward(self, input, label):\n",
" loss = F.softmax_with_cross_entropy(input, \n",
" label, \n",
" return_softmax=False,\n",
" axis=1)\n",
" return paddle.mean(loss)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 5.4 自定义Metric\n",
"\n",
"和Loss一样,如果遇到一些想要做个性化实现的操作时,我们也可以来通过框架完成自定义的评估计算方法,具体的实现方式如下:\n",
"\n",
"```python\n",
"class SelfDefineMetric(paddle.metric.Metric):\n",
" \"\"\"\n",
" 1. 继承paddle.metric.Metric\n",
" \"\"\"\n",
" def __init__(self):\n",
" \"\"\"\n",
" 2. 构造函数实现,自定义参数即可\n",
" \"\"\"\n",
" super(SelfDefineMetric, self).__init__()\n",
"\n",
" def name(self):\n",
" \"\"\"\n",
" 3. 实现name方法,返回定义的评估指标名字\n",
" \"\"\"\n",
" return '自定义评价指标的名字'\n",
"\n",
" def compute(self, ...)\n",
" \"\"\"\n",
" 4. 本步骤可以省略,实现compute方法,这个方法主要用于`update`的加速,可以在这个方法中调用一些paddle实现好的Tensor计算API,编译到模型网络中一起使用低层C++ OP计算。\n",
" \"\"\"\n",
"\n",
" return 自己想要返回的数据,会做为update的参数传入。\n",
"\n",
" def update(self, ...):\n",
" \"\"\"\n",
" 5. 实现update方法,用于单个batch训练时进行评估指标计算。\n",
" - 当`compute`类函数未实现时,会将模型的计算输出和标签数据的展平作为`update`的参数传入。\n",
" - 当`compute`类函数做了实现时,会将compute的返回结果作为`update`的参数传入。\n",
" \"\"\"\n",
" return acc value\n",
" \n",
" def accumulate(self):\n",
" \"\"\"\n",
" 6. 实现accumulate方法,返回历史batch训练积累后计算得到的评价指标值。\n",
" 每次`update`调用时进行数据积累,`accumulate`计算时对积累的所有数据进行计算并返回。\n",
" 结算结果会在`fit`接口的训练日志中呈现。\n",
" \"\"\"\n",
" # 利用update中积累的成员变量数据进行计算后返回\n",
" return accumulated acc value\n",
"\n",
" def reset(self):\n",
" \"\"\"\n",
" 7. 实现reset方法,每个Epoch结束后进行评估指标的重置,这样下个Epoch可以重新进行计算。\n",
" \"\"\"\n",
" # do reset action\n",
"```\n",
"\n",
"我们看一个框架中的具体例子,这个是框架中已提供的一个评估指标计算接口,这里就是按照上述说明中的实现方法进行了相关类继承和成员函数实现。\n",
"\n",
"```python\n",
"from paddle.metric import Metric\n",
"\n",
"\n",
"class Precision(Metric):\n",
" \"\"\"\n",
" Precision (also called positive predictive value) is the fraction of\n",
" relevant instances among the retrieved instances. Refer to\n",
" https://en.wikipedia.org/wiki/Evaluation_of_binary_classifiers\n",
"\n",
" Noted that this class manages the precision score only for binary\n",
" classification task.\n",
" \n",
" ......\n",
"\n",
" \"\"\"\n",
"\n",
" def __init__(self, name='precision', *args, **kwargs):\n",
" super(Precision, self).__init__(*args, **kwargs)\n",
" self.tp = 0 # true positive\n",
" self.fp = 0 # false positive\n",
" self._name = name\n",
"\n",
" def update(self, preds, labels):\n",
" \"\"\"\n",
" Update the states based on the current mini-batch prediction results.\n",
"\n",
" Args:\n",
" preds (numpy.ndarray): The prediction result, usually the output\n",
" of two-class sigmoid function. It should be a vector (column\n",
" vector or row vector) with data type: 'float64' or 'float32'.\n",
" labels (numpy.ndarray): The ground truth (labels),\n",
" the shape should keep the same as preds.\n",
" The data type is 'int32' or 'int64'.\n",
" \"\"\"\n",
" if isinstance(preds, paddle.Tensor):\n",
" preds = preds.numpy()\n",
" elif not _is_numpy_(preds):\n",
" raise ValueError(\"The 'preds' must be a numpy ndarray or Tensor.\")\n",
"\n",
" if isinstance(labels, paddle.Tensor):\n",
" labels = labels.numpy()\n",
" elif not _is_numpy_(labels):\n",
" raise ValueError(\"The 'labels' must be a numpy ndarray or Tensor.\")\n",
"\n",
" sample_num = labels.shape[0]\n",
" preds = np.floor(preds + 0.5).astype(\"int32\")\n",
"\n",
" for i in range(sample_num):\n",
" pred = preds[i]\n",
" label = labels[i]\n",
" if pred == 1:\n",
" if pred == label:\n",
" self.tp += 1\n",
" else:\n",
" self.fp += 1\n",
"\n",
" def reset(self):\n",
" \"\"\"\n",
" Resets all of the metric state.\n",
" \"\"\"\n",
" self.tp = 0\n",
" self.fp = 0\n",
"\n",
" def accumulate(self):\n",
" \"\"\"\n",
" Calculate the final precision.\n",
"\n",
" Returns:\n",
" A scaler float: results of the calculated precision.\n",
" \"\"\"\n",
" ap = self.tp + self.fp\n",
" return float(self.tp) / ap if ap != 0 else .0\n",
"\n",
" def name(self):\n",
" \"\"\"\n",
" Returns metric name\n",
" \"\"\"\n",
" return self._name\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 5.5 自定义Callback\n",
"\n",
"`fit`接口的callback参数支持我们传一个Callback类实例,用来在每轮训练和每个batch训练前后进行调用,可以通过callback收集到训练过程中的一些数据和参数,或者实现一些自定义操作。\n",
"\n",
"```python\n",
"class SelfDefineCallback(paddle.callbacks.Callback):\n",
" \"\"\"\n",
" 1. 继承paddle.callbacks.Callback\n",
" 2. 按照自己的需求实现以下类成员方法:\n",
" def on_train_begin(self, logs=None) 训练开始前,`Model.fit`接口中调用\n",
" def on_train_end(self, logs=None) 训练结束后,`Model.fit`接口中调用\n",
" def on_eval_begin(self, logs=None) 评估开始前,`Model.evaluate`接口调用\n",
" def on_eval_end(self, logs=None) 评估结束后,`Model.evaluate`接口调用\n",
" def on_test_begin(self, logs=None) 预测测试开始前,`Model.predict`接口中调用\n",
" def on_test_end(self, logs=None) 预测测试结束后,`Model.predict`接口中调用 \n",
" def on_epoch_begin(self, epoch, logs=None) 每轮训练开始前,`Model.fit`接口中调用 \n",
" def on_epoch_end(self, epoch, logs=None) 每轮训练结束后,`Model.fit`接口中调用 \n",
" def on_train_batch_begin(self, step, logs=None) 单个Batch训练开始前,`Model.fit`和`Model.train_batch`接口中调用\n",
" def on_train_batch_end(self, step, logs=None) 单个Batch训练结束后,`Model.fit`和`Model.train_batch`接口中调用\n",
" def on_eval_batch_begin(self, step, logs=None) 单个Batch评估开始前,`Model.evalute`和`Model.eval_batch`接口中调用\n",
" def on_eval_batch_end(self, step, logs=None) 单个Batch评估结束后,`Model.evalute`和`Model.eval_batch`接口中调用\n",
" def on_test_batch_begin(self, step, logs=None) 单个Batch预测测试开始前,`Model.predict`和`Model.test_batch`接口中调用\n",
" def on_test_batch_end(self, step, logs=None) 单个Batch预测测试结束后,`Model.predict`和`Model.test_batch`接口中调用\n",
" \"\"\"\n",
" def __init__(self):\n",
" super(SelfDefineCallback, self).__init__()\n",
"\n",
" 按照需求定义自己的类成员方法\n",
"```\n",
"\n",
"我们看一个框架中的实际例子,这是一个框架自带的ModelCheckpoint回调函数,方便用户在fit训练模型时自动存储每轮训练得到的模型。\n",
"\n",
"```python\n",
"class ModelCheckpoint(Callback):\n",
" def __init__(self, save_freq=1, save_dir=None):\n",
" self.save_freq = save_freq\n",
" self.save_dir = save_dir\n",
"\n",
" def on_epoch_begin(self, epoch=None, logs=None):\n",
" self.epoch = epoch\n",
"\n",
" def _is_save(self):\n",
" return self.model and self.save_dir and ParallelEnv().local_rank == 0\n",
"\n",
" def on_epoch_end(self, epoch, logs=None):\n",
" if self._is_save() and self.epoch % self.save_freq == 0:\n",
" path = '{}/{}'.format(self.save_dir, epoch)\n",
" print('save checkpoint at {}'.format(os.path.abspath(path)))\n",
" self.model.save(path)\n",
"\n",
" def on_train_end(self, logs=None):\n",
" if self._is_save():\n",
" path = '{}/final'.format(self.save_dir)\n",
" print('save checkpoint at {}'.format(os.path.abspath(path)))\n",
" self.model.save(path)\n",
"\n",
"```"
]
},
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"## 6. 模型评估\n", "## 6. 模型评估\n",
"\n", "\n",
"对于训练好的模型进行评估操作可以使用`evaluate`接口来实现。" "对于训练好的模型进行评估操作可以使用`evaluate`接口来实现,事先定义好用于评估使用的数据集后,可以简单的调用`evaluate`接口即可完成模型评估操作,结束后根据prepare中loss和metric的定义来进行相关评估结果计算返回。\n",
"\n",
"返回格式是一个字典:\n",
"* 只包含loss,`{'loss': xxx}`\n",
"* 包含loss和一个评估指标,`{'loss': xxx, 'metric name': xxx}`\n",
"* 包含loss和多个评估指标,`{'loss': xxx, 'metric name': xxx, 'metric name': xxx}`"
] ]
}, },
{ {
...@@ -577,7 +823,13 @@ ...@@ -577,7 +823,13 @@
"source": [ "source": [
"## 7. 模型预测\n", "## 7. 模型预测\n",
"\n", "\n",
"高层API中提供`predict`接口,支持用户使用测试数据来完成模型的预测。" "高层API中提供了`predict`接口来方便用户对训练好的模型进行预测验证,只需要基于训练好的模型将需要进行预测测试的数据放到接口中进行计算即可,接口会将经过模型计算得到的预测结果进行返回。\n",
"\n",
"返回格式是一个list,元素数目对应模型的输出数目:\n",
"* 模型是单一输出:[(numpy_ndarray_1, numpy_ndarray_2, ..., numpy_ndarray_n)]\n",
"* 模型是多输出:[(numpy_ndarray_1, numpy_ndarray_2, ..., numpy_ndarray_n), (numpy_ndarray_1, numpy_ndarray_2, ..., numpy_ndarray_n), ...]\n",
"\n",
"numpy_ndarray_n是对应原始数据经过模型计算后得到的预测数据,数目对应预测数据集的数目。"
] ]
}, },
{ {
...@@ -589,6 +841,23 @@ ...@@ -589,6 +841,23 @@
"pred_result = model.predict(val_dataset)" "pred_result = model.predict(val_dataset)"
] ]
}, },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 7.1 使用多卡进行预测\n",
"\n",
"有时我们需要进行预测验证的数据较多,单卡无法满足我们的时间诉求,那么`predict`接口也为用户支持实现了使用多卡模式来运行。\n",
"\n",
"使用起来也是超级简单,无需修改代码程序,只需要使用launch来启动对应的预测脚本即可。\n",
"\n",
"```bash\n",
"$ python3 -m paddle.distributed.launch infer.py\n",
"```\n",
"\n",
"infer.py里面就是包含model.predict的代码程序。"
]
},
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
...@@ -597,7 +866,7 @@ ...@@ -597,7 +866,7 @@
"\n", "\n",
"### 8.1 模型存储\n", "### 8.1 模型存储\n",
"\n", "\n",
"模型训练和验证达到我们的预期后,可以使用`save`接口来将我们的模型保存下来,用于后续模型的Fine-tuning或推理部署。" "模型训练和验证达到我们的预期后,可以使用`save`接口来将我们的模型保存下来,用于后续模型的Fine-tuning(接口参数training=True)或推理部署(接口参数training=False)。"
] ]
}, },
{ {
...@@ -619,5 +888,26 @@ ...@@ -619,5 +888,26 @@
"有了用于推理部署的模型,就可以使用推理部署框架来完成预测服务部署,具体可以参见:[预测部署](https://www.paddlepaddle.org.cn/documentation/docs/zh/advanced_guide/inference_deployment/index_cn.html), 包括服务端部署、移动端部署和模型压缩。" "有了用于推理部署的模型,就可以使用推理部署框架来完成预测服务部署,具体可以参见:[预测部署](https://www.paddlepaddle.org.cn/documentation/docs/zh/advanced_guide/inference_deployment/index_cn.html), 包括服务端部署、移动端部署和模型压缩。"
] ]
} }
] ],
} "metadata": {
\ No newline at end of file "kernelspec": {
"display_name": "Python 3.7.4 64-bit",
"language": "python",
"name": "python37464bitc4da1ac836094043840bff631bedbf7f"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
...@@ -45,7 +45,7 @@ paddle即可使用相关高层API,如:paddle.Model、视觉领域paddle.visi ...@@ -45,7 +45,7 @@ paddle即可使用相关高层API,如:paddle.Model、视觉领域paddle.visi
.. parsed-literal:: .. parsed-literal::
'0.0.0' '2.0.0-beta0'
...@@ -62,8 +62,6 @@ paddle即可使用相关高层API,如:paddle.Model、视觉领域paddle.visi ...@@ -62,8 +62,6 @@ paddle即可使用相关高层API,如:paddle.Model、视觉领域paddle.visi
- 如何在fit接口满足需求的时候进行自定义,使用基础API来完成训练。 - 如何在fit接口满足需求的时候进行自定义,使用基础API来完成训练。
- 如何使用多卡来加速训练。 - 如何使用多卡来加速训练。
其他端到端的示例教程: \* TBD
3. 数据集定义、加载和数据预处理 3. 数据集定义、加载和数据预处理
------------------------------- -------------------------------
...@@ -76,28 +74,21 @@ paddle即可使用相关高层API,如:paddle.Model、视觉领域paddle.visi ...@@ -76,28 +74,21 @@ paddle即可使用相关高层API,如:paddle.Model、视觉领域paddle.visi
.. code:: ipython3 .. code:: ipython3
paddle.vision.datasets.__all__ print('视觉相关数据集:', paddle.vision.datasets.__all__)
print('自然语言相关数据集:', paddle.text.datasets.__all__)
.. parsed-literal:: .. parsed-literal::
['DatasetFolder', 视觉相关数据集: ['DatasetFolder', 'ImageFolder', 'MNIST', 'Flowers', 'Cifar10', 'Cifar100', 'VOC2012']
'ImageFolder', 自然语言相关数据集: ['Conll05st', 'Imdb', 'Imikolov', 'Movielens', 'MovieReviews', 'UCIHousing', 'WMT14', 'WMT16']
'MNIST',
'Flowers',
'Cifar10',
'Cifar100',
'VOC2012']
这里我们是加载一个手写数字识别的数据集,用\ ``mode``\ 来标识是训练数据还是测试数据集。数据集接口会自动从远端下载数据集到本机缓存目录\ ``~/.cache/paddle/dataset``\ 这里我们是加载一个手写数字识别的数据集,用\ ``mode``\ 来标识是训练数据还是测试数据集。数据集接口会自动从远端下载数据集到本机缓存目录\ ``~/.cache/paddle/dataset``\
.. code:: ipython3 .. code:: ipython3
# 测试数据集 # 训练数据集
train_dataset = vision.datasets.MNIST(mode='train') train_dataset = vision.datasets.MNIST(mode='train')
# 验证数据集 # 验证数据集
...@@ -340,9 +331,9 @@ paddle即可使用相关高层API,如:paddle.Model、视觉领域paddle.visi ...@@ -340,9 +331,9 @@ paddle即可使用相关高层API,如:paddle.Model、视觉领域paddle.visi
5. 模型训练 5. 模型训练
----------- -----------
使用\ ``paddle.Model``\ 封装成模型类后进行训练非常的简洁方便,我们可以直接通过调用\ ``Model.fit``\ 就可以完成训练过程。 网络结构通过\ ``paddle.Model``\ 接口封装成模型类后进行执行操作非常的简洁方便,可以直接通过调用\ ``Model.fit``\ 就可以完成训练过程。
使用\ ``Model.fit``\ 接口启动训练前,我们先通过\ ``Model.prepare``\ 接口来对训练进行提前的配置准备工作,包括设置模型优化器,Loss计算方法,精度计算方法等。 使用\ ``Model.fit``\ 接口启动训练前,我们先通过\ ``Model.prepare``\ 接口来对训练进行提前的配置准备工作,包括设置模型优化器,Loss计算方法,精度计算方法等。
.. code:: ipython3 .. code:: ipython3
...@@ -398,10 +389,252 @@ paddle即可使用相关高层API,如:paddle.Model、视觉领域paddle.visi ...@@ -398,10 +389,252 @@ paddle即可使用相关高层API,如:paddle.Model、视觉领域paddle.visi
# train.py里面包含的就是单机单卡代码 # train.py里面包含的就是单机单卡代码
python -m paddle.distributed.launch train.py python -m paddle.distributed.launch train.py
5.3 自定义Loss
~~~~~~~~~~~~~~
有时我们会遇到特定任务的Loss计算方式在框架既有的Loss接口中不存在,或算法不符合自己的需求,那么期望能够自己来进行Loss的自定义,我们这里就会讲解介绍一下如何进行Loss的自定义操作,首先来看下面的代码:
.. code:: python
class SelfDefineLoss(paddle.nn.Layer):
"""
1. 继承paddle.nn.Layer
"""
def __init__(self):
"""
2. 构造函数根据自己的实际算法需求和使用需求进行参数定义即可
"""
super(SelfDefineLoss, self).__init__()
def forward(self, input, label):
"""
3. 实现forward函数,forward在调用时会传递两个参数:input和label
- input:单个或批次训练数据经过模型前向计算输出结果
- label:单个或批次训练数据对应的标签数据
接口返回值是一个Tensor,根据自定义的逻辑加和或计算均值后的损失
"""
# 使用Paddle中相关API自定义的计算逻辑
# output = xxxxx
# return output
那么了解完代码层面如果编写自定义代码后我们看一个实际的例子,下面是在图像分割示例代码中写的一个自定义Loss,当时主要是想使用自定义的softmax计算维度。
.. code:: python
class SoftmaxWithCrossEntropy(paddle.nn.Layer):
def __init__(self):
super(SoftmaxWithCrossEntropy, self).__init__()
def forward(self, input, label):
loss = F.softmax_with_cross_entropy(input,
label,
return_softmax=False,
axis=1)
return paddle.mean(loss)
5.4 自定义Metric
~~~~~~~~~~~~~~~~
Loss一样,如果遇到一些想要做个性化实现的操作时,我们也可以来通过框架完成自定义的评估计算方法,具体的实现方式如下:
.. code:: python
class SelfDefineMetric(paddle.metric.Metric):
"""
1. 继承paddle.metric.Metric
"""
def __init__(self):
"""
2. 构造函数实现,自定义参数即可
"""
super(SelfDefineMetric, self).__init__()
def name(self):
"""
3. 实现name方法,返回定义的评估指标名字
"""
return '自定义评价指标的名字'
def compute(self, ...)
"""
4. 本步骤可以省略,实现compute方法,这个方法主要用于`update`的加速,可以在这个方法中调用一些paddle实现好的Tensor计算API,编译到模型网络中一起使用低层C++ OP计算。
"""
return 自己想要返回的数据,会做为update的参数传入。
def update(self, ...):
"""
5. 实现update方法,用于单个batch训练时进行评估指标计算。
- 当`compute`类函数未实现时,会将模型的计算输出和标签数据的展平作为`update`的参数传入。
- 当`compute`类函数做了实现时,会将compute的返回结果作为`update`的参数传入。
"""
return acc value
def accumulate(self):
"""
6. 实现accumulate方法,返回历史batch训练积累后计算得到的评价指标值。
每次`update`调用时进行数据积累,`accumulate`计算时对积累的所有数据进行计算并返回。
结算结果会在`fit`接口的训练日志中呈现。
"""
# 利用update中积累的成员变量数据进行计算后返回
return accumulated acc value
def reset(self):
"""
7. 实现reset方法,每个Epoch结束后进行评估指标的重置,这样下个Epoch可以重新进行计算。
"""
# do reset action
我们看一个框架中的具体例子,这个是框架中已提供的一个评估指标计算接口,这里就是按照上述说明中的实现方法进行了相关类继承和成员函数实现。
.. code:: python
from paddle.metric import Metric
class Precision(Metric):
"""
Precision (also called positive predictive value) is the fraction of
relevant instances among the retrieved instances. Refer to
https://en.wikipedia.org/wiki/Evaluation_of_binary_classifiers
Noted that this class manages the precision score only for binary
classification task.
......
"""
def __init__(self, name='precision', *args, **kwargs):
super(Precision, self).__init__(*args, **kwargs)
self.tp = 0 # true positive
self.fp = 0 # false positive
self._name = name
def update(self, preds, labels):
"""
Update the states based on the current mini-batch prediction results.
Args:
preds (numpy.ndarray): The prediction result, usually the output
of two-class sigmoid function. It should be a vector (column
vector or row vector) with data type: 'float64' or 'float32'.
labels (numpy.ndarray): The ground truth (labels),
the shape should keep the same as preds.
The data type is 'int32' or 'int64'.
"""
if isinstance(preds, paddle.Tensor):
preds = preds.numpy()
elif not _is_numpy_(preds):
raise ValueError("The 'preds' must be a numpy ndarray or Tensor.")
if isinstance(labels, paddle.Tensor):
labels = labels.numpy()
elif not _is_numpy_(labels):
raise ValueError("The 'labels' must be a numpy ndarray or Tensor.")
sample_num = labels.shape[0]
preds = np.floor(preds + 0.5).astype("int32")
for i in range(sample_num):
pred = preds[i]
label = labels[i]
if pred == 1:
if pred == label:
self.tp += 1
else:
self.fp += 1
def reset(self):
"""
Resets all of the metric state.
"""
self.tp = 0
self.fp = 0
def accumulate(self):
"""
Calculate the final precision.
Returns:
A scaler float: results of the calculated precision.
"""
ap = self.tp + self.fp
return float(self.tp) / ap if ap != 0 else .0
def name(self):
"""
Returns metric name
"""
return self._name
5.5 自定义Callback
~~~~~~~~~~~~~~~~~~
``fit``\ 接口的callback参数支持我们传一个Callback类实例,用来在每轮训练和每个batch训练前后进行调用,可以通过callback收集到训练过程中的一些数据和参数,或者实现一些自定义操作。
.. code:: python
class SelfDefineCallback(paddle.callbacks.Callback):
"""
1. 继承paddle.callbacks.Callback
2. 按照自己的需求实现以下类成员方法:
def on_train_begin(self, logs=None) 训练开始前,`Model.fit`接口中调用
def on_train_end(self, logs=None) 训练结束后,`Model.fit`接口中调用
def on_eval_begin(self, logs=None) 评估开始前,`Model.evaluate`接口调用
def on_eval_end(self, logs=None) 评估结束后,`Model.evaluate`接口调用
def on_test_begin(self, logs=None) 预测测试开始前,`Model.predict`接口中调用
def on_test_end(self, logs=None) 预测测试结束后,`Model.predict`接口中调用
def on_epoch_begin(self, epoch, logs=None) 每轮训练开始前,`Model.fit`接口中调用
def on_epoch_end(self, epoch, logs=None) 每轮训练结束后,`Model.fit`接口中调用
def on_train_batch_begin(self, step, logs=None) 单个Batch训练开始前,`Model.fit`和`Model.train_batch`接口中调用
def on_train_batch_end(self, step, logs=None) 单个Batch训练结束后,`Model.fit`和`Model.train_batch`接口中调用
def on_eval_batch_begin(self, step, logs=None) 单个Batch评估开始前,`Model.evalute`和`Model.eval_batch`接口中调用
def on_eval_batch_end(self, step, logs=None) 单个Batch评估结束后,`Model.evalute`和`Model.eval_batch`接口中调用
def on_test_batch_begin(self, step, logs=None) 单个Batch预测测试开始前,`Model.predict`和`Model.test_batch`接口中调用
def on_test_batch_end(self, step, logs=None) 单个Batch预测测试结束后,`Model.predict`和`Model.test_batch`接口中调用
"""
def __init__(self):
super(SelfDefineCallback, self).__init__()
按照需求定义自己的类成员方法
我们看一个框架中的实际例子,这是一个框架自带的ModelCheckpoint回调函数,方便用户在fit训练模型时自动存储每轮训练得到的模型。
.. code:: python
class ModelCheckpoint(Callback):
def __init__(self, save_freq=1, save_dir=None):
self.save_freq = save_freq
self.save_dir = save_dir
def on_epoch_begin(self, epoch=None, logs=None):
self.epoch = epoch
def _is_save(self):
return self.model and self.save_dir and ParallelEnv().local_rank == 0
def on_epoch_end(self, epoch, logs=None):
if self._is_save() and self.epoch % self.save_freq == 0:
path = '{}/{}'.format(self.save_dir, epoch)
print('save checkpoint at {}'.format(os.path.abspath(path)))
self.model.save(path)
def on_train_end(self, logs=None):
if self._is_save():
path = '{}/final'.format(self.save_dir)
print('save checkpoint at {}'.format(os.path.abspath(path)))
self.model.save(path)
6. 模型评估 6. 模型评估
----------- -----------
对于训练好的模型进行评估操作可以使用\ ``evaluate``\ 接口来实现。 对于训练好的模型进行评估操作可以使用\ ``evaluate``\ 接口来实现,事先定义好用于评估使用的数据集后,可以简单的调用\ ``evaluate``\ 接口即可完成模型评估操作,结束后根据preparelossmetric的定义来进行相关评估结果计算返回。
返回格式是一个字典: \* 只包含loss\ ``{'loss': xxx}`` \*
包含loss和一个评估指标,\ ``{'loss': xxx, 'metric name': xxx}`` \*
包含loss和多个评估指标,\ ``{'loss': xxx, 'metric name': xxx, 'metric name': xxx}``
.. code:: ipython3 .. code:: ipython3
...@@ -410,19 +643,40 @@ paddle即可使用相关高层API,如:paddle.Model、视觉领域paddle.visi ...@@ -410,19 +643,40 @@ paddle即可使用相关高层API,如:paddle.Model、视觉领域paddle.visi
7. 模型预测 7. 模型预测
----------- -----------
高层API中提供\ ``predict``\ 接口,支持用户使用测试数据来完成模型的预测。 高层API中提供了\ ``predict``\ 接口来方便用户对训练好的模型进行预测验证,只需要基于训练好的模型将需要进行预测测试的数据放到接口中进行计算即可,接口会将经过模型计算得到的预测结果进行返回。
返回格式是一个list,元素数目对应模型的输出数目: \*
模型是单一输出:[(numpy_ndarray_1, numpy_ndarray_2, , numpy_ndarray_n)]
\* 模型是多输出:[(numpy_ndarray_1, numpy_ndarray_2, ,
numpy_ndarray_n), (numpy_ndarray_1, numpy_ndarray_2, ,
numpy_ndarray_n), ]
numpy_ndarray_n是对应原始数据经过模型计算后得到的预测数据,数目对应预测数据集的数目。
.. code:: ipython3 .. code:: ipython3
pred_result = model.predict(val_dataset) pred_result = model.predict(val_dataset)
7.1 使用多卡进行预测
~~~~~~~~~~~~~~~~~~~~
有时我们需要进行预测验证的数据较多,单卡无法满足我们的时间诉求,那么\ ``predict``\ 接口也为用户支持实现了使用多卡模式来运行。
使用起来也是超级简单,无需修改代码程序,只需要使用launch来启动对应的预测脚本即可。
.. code:: bash
$ python3 -m paddle.distributed.launch infer.py
infer.py里面就是包含model.predict的代码程序。
8. 模型部署 8. 模型部署
----------- -----------
8.1 模型存储 8.1 模型存储
~~~~~~~~~~~~ ~~~~~~~~~~~~
模型训练和验证达到我们的预期后,可以使用\ ``save``\ 接口来将我们的模型保存下来,用于后续模型的Fine-tuning或推理部署 模型训练和验证达到我们的预期后,可以使用\ ``save``\ 接口来将我们的模型保存下来,用于后续模型的Fine-tuning(接口参数training=True)或推理部署(接口参数training=False
.. code:: ipython3 .. code:: ipython3
......
线性回归 线性回归
======== ========
NOTE: NOTE: 本示例教程是基于2.0beta版本开发
本示例教程依然在开发中,目前是基于2.0beta版本(由于2.0beta没有正式发版,在用最新developwhl包下载的paddle)。
简要介绍 简要介绍
-------- --------
...@@ -10,20 +9,35 @@ NOTE: ...@@ -10,20 +9,35 @@ NOTE:
经典的线性回归模型主要用来预测一些存在着线性关系的数据集。回归模型可以理解为:存在一个点集,用一条曲线去拟合它分布的过程。如果拟合曲线是一条直线,则称为线性回归。如果是一条二次曲线,则被称为二次回归。线性回归是回归模型中最简单的一种。 经典的线性回归模型主要用来预测一些存在着线性关系的数据集。回归模型可以理解为:存在一个点集,用一条曲线去拟合它分布的过程。如果拟合曲线是一条直线,则称为线性回归。如果是一条二次曲线,则被称为二次回归。线性回归是回归模型中最简单的一种。
本示例简要介绍如何用飞桨开源框架,实现波士顿房价预测。其思路是,假设uci-housing数据集中的房子属性和房价之间的关系可以被属性间的线性组合描述。在模型训练阶段,让假设的预测结果和真实值之间的误差越来越小。在模型预测阶段,预测器会读取训练好的模型,对从未遇见过的房子属性进行房价预测。 本示例简要介绍如何用飞桨开源框架,实现波士顿房价预测。其思路是,假设uci-housing数据集中的房子属性和房价之间的关系可以被属性间的线性组合描述。在模型训练阶段,让假设的预测结果和真实值之间的误差越来越小。在模型预测阶段,预测器会读取训练好的模型,对从未遇见过的房子属性进行房价预测。
环境设置 数据集介绍
-------- ----------
本示例基于飞桨开源框架2.0版本。 本示例采用uci-housing数据集,这是经典线性回归的数据集。数据集共7084条数据,可以拆分成506,每行14列。前13列用来描述房屋的各种信息,最后一列为该类房屋价格中位数。
13列用来描述房屋的各种信息
.. figure:: https://ai-studio-static-online.cdn.bcebos.com/c19602ce74284e3b9a50422f8dc37c0c1c79cf5cd8424994b6a6b073dcb7c057
:alt: avatar
avatar
训练方式一
----------
环境设置
~~~~~~~~
.. code:: ipython3 .. code:: ipython3
import paddle import paddle
import numpy as np import numpy as np
import os import os
import matplotlib
import matplotlib.pyplot as plt import matplotlib.pyplot as plt
import pandas as pd import pandas as pd
import seaborn as sns import seaborn as sns
paddle.disable_static()
paddle.__version__ paddle.__version__
...@@ -31,164 +45,143 @@ NOTE: ...@@ -31,164 +45,143 @@ NOTE:
.. parsed-literal:: .. parsed-literal::
'0.0.0' '2.0.0-beta0'
数据集
------
本示例采用uci-housing数据集,这是经典线性回归的数据集。数据集共506,每行14列。前13列用来描述房屋的各种信息,最后一列为该类房屋价格中位数。飞桨提供了读取uci_housing训练集和测试集的接口,分别为paddle.dataset.uci_housing.train()paddle.dataset.uci_housing.test()
13列用来描述房屋的各种信息
.. figure:: https://ai-studio-static-online.cdn.bcebos.com/c19602ce74284e3b9a50422f8dc37c0c1c79cf5cd8424994b6a6b073dcb7c057 数据处理
:alt: avatar ~~~~~~~~
avatar .. code:: ipython3
下面我们来浏览一下数据是什么样子的: #下载数据
#!wget https://archive.ics.uci.edu/ml/machine-learning-databases/housing/housing.data -O housing.data
.. code:: ipython3 .. code:: ipython3
import matplotlib.pyplot as plt # 从文件导入数据
import matplotlib datafile = './housing.data'
housing_data = np.fromfile(datafile, sep=' ')
train_data=paddle.dataset.uci_housing.train()
sample_data=next(train_data())
print(sample_data[0])
# 画图看特征间的关系,主要是变量两两之间的关系(线性或非线性,有无明显较为相关关系)
feature_names = ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE','DIS', 'RAD', 'TAX', 'PTRATIO', 'B', 'LSTAT', 'MEDV'] feature_names = ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE','DIS', 'RAD', 'TAX', 'PTRATIO', 'B', 'LSTAT', 'MEDV']
feature_num = len(feature_names) feature_num = len(feature_names)
features_np=np.array([x[0] for x in train_data()],np.float32) # 将原始数据进行Reshape,变成[N, 14]这样的形状
labels_np=np.array([x[1] for x in train_data()],np.float32) housing_data = housing_data.reshape([housing_data.shape[0] // feature_num, feature_num])
data_np=np.c_[features_np,labels_np]
df=pd.DataFrame(data_np,columns=feature_names) .. code:: ipython3
# 画图看特征间的关系,主要是变量两两之间的关系(线性或非线性,有无明显较为相关关系)
features_np = np.array([x[:13] for x in housing_data], np.float32)
labels_np = np.array([x[-1] for x in housing_data], np.float32)
data_np = np.c_[features_np, labels_np]
df = pd.DataFrame(data_np, columns=feature_names)
matplotlib.use('TkAgg') matplotlib.use('TkAgg')
%matplotlib inline %matplotlib inline
sns.pairplot(df.dropna()) sns.pairplot(df.dropna(), y_vars=feature_names[-1], x_vars=feature_names[:])
plt.show() plt.show()
.. parsed-literal:: .. image:: https://github.com/PaddlePaddle/FluidDoc/tree/develop/doc/paddle/tutorial/quick_start/linear_regression/linear_regression_files/linear_regression_001.png?raw=true
[-0.0405441 0.06636364 -0.32356227 -0.06916996 -0.03435197 0.05563625
-0.03475696 0.02682186 -0.37171335 -0.21419304 -0.33569506 0.10143217
-0.21172912]
.. image:: linear_regression_files/linear_regression_6_1.png
上图中,对角线上是各属性的直方图,非对角线上的是两个不同属性之间的相关图。
从图中我们可以看出,RM(每栋房平均客房数)、LSTAT(低收入人群占比)、与房价成明显的相关关系、NOX(一氧化碳浓度)和DIS(与波士顿就业中心距离)成明显相关关系等。
.. code:: ipython3 .. code:: ipython3
# 相关性分析 # 相关性分析
fig, ax = plt.subplots(figsize=(15,15)) fig, ax = plt.subplots(figsize=(15, 1))
ax=sns.heatmap(df.corr(), cbar=True, annot=True) corr_data = df.corr().iloc[-1]
ax.set_ylim([14, 0]) corr_data = np.asarray(corr_data).reshape(1, 14)
ax = sns.heatmap(corr_data, cbar=True, annot=True)
plt.show() plt.show()
.. image:: linear_regression_files/linear_regression_8_0.png .. image:: https://github.com/PaddlePaddle/FluidDoc/tree/develop/doc/paddle/tutorial/quick_start/linear_regression/linear_regression_files/linear_regression_002.png?raw=true
**数据归一化处理** **数据归一化处理**\ 下图为大家展示各属性的取值范围分布:
下图为大家展示各属性的取值范围分布:
.. code:: ipython3 .. code:: ipython3
sns.boxplot(data=df.iloc[:,0:13]) sns.boxplot(data=df.iloc[:, 0:13])
.. parsed-literal:: .. parsed-literal::
<matplotlib.axes._subplots.AxesSubplot at 0x1a3adcb410> <matplotlib.axes._subplots.AxesSubplot at 0x1a3e2b4e50>
.. image:: linear_regression_files/linear_regression_11_1.png .. image:: https://github.com/PaddlePaddle/FluidDoc/tree/develop/doc/paddle/tutorial/quick_start/linear_regression/linear_regression_files/linear_regression_003.png?raw=true
做归一化(或 Feature scaling)至少有以下3个理由: 从上图看出,我们各属性的数值范围差异太大,甚至不能够在一个画布上充分的展示各属性具体的最大、最小值以及异常值等。下面我们进行归一化。
- 过大或过小的数值范围会导致计算时的浮点上溢或下溢。 做归一化(或 Feature scaling)至少有以下2个理由:
- 过大或过小的数值范围会导致计算时的浮点上溢或下溢。
- 不同的数值范围会导致不同属性对模型的重要性不同(至少在训练的初始阶段如此),而这个隐含的假设常常是不合理的。这会对优化的过程造成困难,使训练时间大大的加长. - 不同的数值范围会导致不同属性对模型的重要性不同(至少在训练的初始阶段如此),而这个隐含的假设常常是不合理的。这会对优化的过程造成困难,使训练时间大大的加长.
- 很多的机器学习技巧/模型(例如L1L2正则项,向量空间模型-Vector Space
Model)都基于这样的假设:所有的属性取值都差不多是以0为均值且取值范围相近的。
.. code:: ipython3 .. code:: ipython3
features_max=[] features_max = housing_data.max(axis=0)
features_min=[] features_min = housing_data.min(axis=0)
features_avg=[] features_avg = housing_data.sum(axis=0) / housing_data.shape[0]
for i in range(13):
i_feature_max=max([data[1][0][i] for data in enumerate(train_data())])
features_max.append(i_feature_max)
i_feature_min=min([data[1][0][i] for data in enumerate(train_data())])
features_min.append(i_feature_min)
i_feature_avg=sum([data[1][0][i] for data in enumerate(train_data())])/506
features_avg.append(i_feature_avg)
.. code:: ipython3 .. code:: ipython3
BATCH_SIZE=20 BATCH_SIZE = 20
def feature_norm(input): def feature_norm(input):
f_size=input.shape[0] f_size = input.shape
output_features=np.zeros((f_size,13),np.float32) output_features = np.zeros(f_size, np.float32)
for batch_id in range(f_size): for batch_id in range(f_size[0]):
for index in range(13): for index in range(13):
output_features[batch_id][index]=(input[batch_id][index]-features_avg[index])/(features_max[index]-features_min[index]) output_features[batch_id][index] = (input[batch_id][index] - features_avg[index]) / (features_max[index] - features_min[index])
return output_features return output_features
.. code:: ipython3
定义绘制训练过程的损失值变化趋势的方法draw_train_process #只对属性进行归一化
housing_features = feature_norm(housing_data[:, :13])
# print(feature_trian.shape)
housing_data = np.c_[housing_features, housing_data[:, -1]].astype(np.float32)
# print(training_data[0])
.. code:: ipython3 .. code:: ipython3
global iter #归一化后的train_data,我们看下各属性的情况
iter=0 features_np = np.array([x[:13] for x in housing_data],np.float32)
iters=[] labels_np = np.array([x[-1] for x in housing_data],np.float32)
train_costs=[] data_np = np.c_[features_np, labels_np]
df = pd.DataFrame(data_np, columns=feature_names)
def draw_train_process(iters,train_costs): sns.boxplot(data=df.iloc[:, 0:13])
plt.title("training cost" ,fontsize=24)
plt.xlabel("iter", fontsize=14)
plt.ylabel("cost", fontsize=14)
plt.plot(iters, train_costs,color='red',label='training cost')
plt.show()
**数据提供器**
下面我们分别定义了用于训练和测试的数据提供器。提供器每次读入一个大小为BATCH_SIZE的数据批次。如果您希望加一些随机性,它可以同时定义一个批次大小和一个缓存大小。这样的话,每次数据提供器会从缓存中随机读取批次大小那么多的数据。
.. parsed-literal::
<matplotlib.axes._subplots.AxesSubplot at 0x1a3e4cd4d0>
.. image:: https://github.com/PaddlePaddle/FluidDoc/tree/develop/doc/paddle/tutorial/quick_start/linear_regression/linear_regression_files/linear_regression_004.png?raw=true
.. code:: ipython3 .. code:: ipython3
BATCH_SIZE=20 #将训练数据集和测试数据集按照8:2的比例分开
BUF_SIZE=500 ratio = 0.8
offset = int(housing_data.shape[0] * ratio)
train_reader=paddle.batch(paddle.reader.shuffle(paddle.dataset.uci_housing.train(),buf_size=BUF_SIZE),batch_size=BATCH_SIZE) train_data = housing_data[:offset]
test_data = housing_data[offset:]
模型配置 模型配置
-------- ~~~~~~~~
线性回归就是一个从输入到输出的简单的全连接层。 线性回归就是一个从输入到输出的简单的全连接层。
...@@ -197,16 +190,30 @@ NOTE: ...@@ -197,16 +190,30 @@ NOTE:
.. code:: ipython3 .. code:: ipython3
class Regressor(paddle.nn.Layer): class Regressor(paddle.nn.Layer):
def __init__(self): def __init__(self):
super(Regressor,self).__init__() super(Regressor, self).__init__()
self.fc=paddle.nn.Linear(13,1,None) self.fc = paddle.nn.Linear(13, 1,)
def forward(self, inputs):
pred = self.fc(inputs)
return pred
定义绘制训练过程的损失值变化趋势的方法draw_train_process
.. code:: ipython3
train_nums = []
train_costs = []
def forward(self,inputs): def draw_train_process(iters, train_costs):
pred=self.fc(inputs) plt.title("training cost", fontsize=24)
return pred plt.xlabel("iter", fontsize=14)
plt.ylabel("cost", fontsize=14)
plt.plot(iters, train_costs, color='red', label='training cost')
plt.show()
模型训练 模型训练
--------- ~~~~~~~~
下面为大家展示模型训练的代码。 下面为大家展示模型训练的代码。
这里用到的是线性回归模型最常用的损失函数–均方误差(MSE),用来衡量模型预测的房价和真实房价的差异。 这里用到的是线性回归模型最常用的损失函数–均方误差(MSE),用来衡量模型预测的房价和真实房价的差异。
...@@ -214,136 +221,128 @@ NOTE: ...@@ -214,136 +221,128 @@ NOTE:
.. code:: ipython3 .. code:: ipython3
y_preds=[] import paddle.nn.functional as F
labels_list=[] y_preds = []
def train(model): labels_list = []
print('start training ... ')
model.train()
EPOCH_NUM=500
optimizer=paddle.optimizer.SGD(learning_rate=0.001, parameters = model.parameters())
iter=0
for epoch_id in range(EPOCH_NUM): def train(model):
train_cost=0 print('start training ... ')
for batch_id,data in enumerate(train_reader()): # 开启模型训练模式
features_np=np.array([x[0] for x in data],np.float32) model.train()
labels_np=np.array([x[1] for x in data],np.float32) EPOCH_NUM = 500
features=paddle.to_variable(feature_norm(features_np)) train_num = 0
labels=paddle.to_variable(labels_np) optimizer = paddle.optimizer.SGD(learning_rate=0.001, parameters=model.parameters())
#前向计算 for epoch_id in range(EPOCH_NUM):
y_pred=model(features) # 在每轮迭代开始之前,将训练数据的顺序随机的打乱
np.random.shuffle(train_data)
cost=paddle.nn.functional.square_error_cost(y_pred,label=labels) # 将训练数据进行拆分,每个batch包含20条数据
avg_cost=paddle.mean(cost) mini_batches = [train_data[k:k+BATCH_SIZE] for k in range(0, len(train_data), BATCH_SIZE)]
train_cost = [avg_cost.numpy()] for batch_id, data in enumerate(mini_batches):
#反向传播 features_np = np.array(data[:, :13], np.float32)
avg_cost.backward() labels_np = np.array(data[:, -1:], np.float32)
#最小化loss,更新参数 features = paddle.to_tensor(features_np)
opts=optimizer.minimize(avg_cost) labels = paddle.to_tensor(labels_np)
# 清除梯度 #前向计算
model.clear_gradients() y_pred = model(features)
if batch_id%30==0 and epoch_id%30==0: cost = F.mse_loss(y_pred, label=labels)
print("Pass:%d,Cost:%0.5f"%(epoch_id,train_cost[0][0])) train_cost = cost.numpy()[0]
#反向传播
cost.backward()
#最小化loss,更新参数
optimizer.step()
# 清除梯度
optimizer.clear_grad()
if batch_id%30 == 0 and epoch_id%50 == 0:
print("Pass:%d,Cost:%0.5f"%(epoch_id, train_cost))
iter=iter+BATCH_SIZE train_num = train_num + BATCH_SIZE
iters.append(iter) train_nums.append(train_num)
train_costs.append(train_cost[0][0]) train_costs.append(train_cost)
paddle.disable_static()
model = Regressor() model = Regressor()
train(model) train(model)
.. parsed-literal:: .. parsed-literal::
start training ... start training ...
Pass:0,Cost:531.75244 Pass:0,Cost:740.21814
Pass:30,Cost:61.10927 Pass:50,Cost:36.40338
Pass:60,Cost:22.68571 Pass:100,Cost:86.01823
Pass:90,Cost:34.80560 Pass:150,Cost:50.86654
Pass:120,Cost:78.28358 Pass:200,Cost:31.14208
Pass:150,Cost:124.95644 Pass:250,Cost:20.54596
Pass:180,Cost:91.88014 Pass:300,Cost:22.30817
Pass:210,Cost:15.23689 Pass:350,Cost:24.18756
Pass:240,Cost:34.86035 Pass:400,Cost:22.22965
Pass:270,Cost:54.76824 Pass:450,Cost:39.25978
Pass:300,Cost:65.88247
Pass:330,Cost:41.25426
Pass:360,Cost:64.10200
Pass:390,Cost:77.11707
Pass:420,Cost:20.80456
Pass:450,Cost:29.80167
Pass:480,Cost:41.59278
.. code:: ipython3 .. code:: ipython3
matplotlib.use('TkAgg') matplotlib.use('TkAgg')
%matplotlib inline %matplotlib inline
draw_train_process(iters,train_costs) draw_train_process(train_nums, train_costs)
.. image:: linear_regression_files/linear_regression_23_0.png .. image:: https://github.com/PaddlePaddle/FluidDoc/tree/develop/doc/paddle/tutorial/quick_start/linear_regression/linear_regression_files/linear_regression_005.png?raw=true
可以从上图看出,随着训练轮次的增加,损失在呈降低趋势。但由于每次仅基于少量样本更新参数和计算损失,所以损失下降曲线会出现震荡。 可以从上图看出,随着训练轮次的增加,损失在呈降低趋势。但由于每次仅基于少量样本更新参数和计算损失,所以损失下降曲线会出现震荡。
模型预测 模型预测
---------- ~~~~~~~~
.. code:: ipython3 .. code:: ipython3
#获取预测数据 #获取预测数据
INFER_BATCH_SIZE=100 INFER_BATCH_SIZE = 100
infer_reader=paddle.batch(paddle.dataset.uci_housing.test(),batch_size=INFER_BATCH_SIZE)
infer_data = next(infer_reader())
infer_features_np = np.array([data[0] for data in infer_data]).astype("float32")
infer_labels_np= np.array([data[1] for data in infer_data]).astype("float32")
infer_features=paddle.to_variable(feature_norm(infer_features_np)) infer_features_np = np.array([data[:13] for data in test_data]).astype("float32")
infer_labels=paddle.to_variable(infer_labels_np) infer_labels_np = np.array([data[-1] for data in test_data]).astype("float32")
fetch_list=model(infer_features).numpy()
sum_cost=0 infer_features = paddle.to_tensor(infer_features_np)
infer_labels = paddle.to_tensor(infer_labels_np)
fetch_list = model(infer_features)
sum_cost = 0
for i in range(INFER_BATCH_SIZE): for i in range(INFER_BATCH_SIZE):
infer_result=fetch_list[i][0] infer_result = fetch_list[i][0]
ground_truth=infer_labels.numpy()[i] ground_truth = infer_labels[i]
if i%10==0: if i % 10 == 0:
print("No.%d: infer result is %.2f,ground truth is %.2f" % (i, infer_result,ground_truth)) print("No.%d: infer result is %.2f,ground truth is %.2f" % (i, infer_result, ground_truth))
cost=np.power(infer_result-ground_truth,2) cost = paddle.pow(infer_result - ground_truth, 2)
sum_cost+=cost sum_cost += cost
print("平均误差为:",sum_cost/INFER_BATCH_SIZE) mean_loss = sum_cost / INFER_BATCH_SIZE
print("Mean loss is:", mean_loss.numpy())
.. parsed-literal:: .. parsed-literal::
No.0: infer result is 12.20,ground truth is 8.50 No.0: infer result is 12.15,ground truth is 8.50
No.10: infer result is 5.65,ground truth is 7.00 No.10: infer result is 5.21,ground truth is 7.00
No.20: infer result is 14.87,ground truth is 11.70 No.20: infer result is 14.32,ground truth is 11.70
No.30: infer result is 16.60,ground truth is 11.70 No.30: infer result is 16.11,ground truth is 11.70
No.40: infer result is 13.71,ground truth is 10.80 No.40: infer result is 13.42,ground truth is 10.80
No.50: infer result is 16.11,ground truth is 14.90 No.50: infer result is 15.50,ground truth is 14.90
No.60: infer result is 18.78,ground truth is 21.40 No.60: infer result is 18.81,ground truth is 21.40
No.70: infer result is 15.53,ground truth is 13.80 No.70: infer result is 15.42,ground truth is 13.80
No.80: infer result is 18.10,ground truth is 20.60 No.80: infer result is 18.16,ground truth is 20.60
No.90: infer result is 21.39,ground truth is 24.50 No.90: infer result is 21.48,ground truth is 24.50
平均误差为: [12.917107] Mean loss is: [12.195988]
.. code:: ipython3 .. code:: ipython3
def plot_pred_ground(pred, groud): def plot_pred_ground(pred, ground):
plt.figure() plt.figure()
plt.title("Predication v.s. Ground", fontsize=24) plt.title("Predication v.s. Ground truth", fontsize=24)
plt.xlabel("groud price(unit:$1000)", fontsize=14) plt.xlabel("ground truth price(unit:$1000)", fontsize=14)
plt.ylabel("predict price", fontsize=14) plt.ylabel("predict price", fontsize=14)
plt.scatter(pred, groud, alpha=0.5) # scatter:散点图,alpha:"透明度" plt.scatter(ground, pred, alpha=0.5) # scatter:散点图,alpha:"透明度"
plt.plot(groud, groud, c='red') plt.plot(ground, ground, c='red')
plt.show() plt.show()
.. code:: ipython3 .. code:: ipython3
...@@ -352,7 +351,78 @@ NOTE: ...@@ -352,7 +351,78 @@ NOTE:
.. image:: linear_regression_files/linear_regression_28_0.png .. image:: https://github.com/PaddlePaddle/FluidDoc/tree/develop/doc/paddle/tutorial/quick_start/linear_regression/linear_regression_files/linear_regression_001.png?raw=true
上图可以看出,我们训练出来的模型的预测结果与真实结果是较为接近的。 上图可以看出,我们训练出来的模型的预测结果与真实结果是较为接近的。
训练方式二
----------
我们也可以用我们的高层API来做线性回归训练,高层API相较于底层API更加的简洁方便。
.. code:: ipython3
import paddle
paddle.disable_static()
paddle.set_default_dtype("float64")
#step1:用高层API定义数据集,无需进行数据处理等,高层API为您一条龙搞定
train_dataset = paddle.text.datasets.UCIHousing(mode='train')
eval_dataset = paddle.text.datasets.UCIHousing(mode='test')
#step2:定义模型
class UCIHousing(paddle.nn.Layer):
def __init__(self):
super(UCIHousing, self).__init__()
self.fc = paddle.nn.Linear(13, 1, None)
def forward(self, input):
pred = self.fc(input)
return pred
#step3:训练模型
model = paddle.Model(UCIHousing())
model.prepare(paddle.optimizer.Adam(parameters=model.parameters()),
paddle.nn.loss.MSELoss())
model.fit(train_dataset, eval_dataset, epochs=5, batch_size=8, log_freq=20)
.. parsed-literal::
Epoch 1/5
step 20/51 - loss: 520.8663 - 1ms/step
step 40/51 - loss: 611.7135 - 1ms/step
step 51/51 - loss: 620.0662 - 1ms/step
Eval begin...
step 13/13 - loss: 389.7871 - 1ms/step
Eval samples: 102
Epoch 2/5
step 20/51 - loss: 867.4678 - 3ms/step
step 40/51 - loss: 1081.1701 - 2ms/step
step 51/51 - loss: 420.8705 - 2ms/step
Eval begin...
step 13/13 - loss: 387.2432 - 1ms/step
Eval samples: 102
Epoch 3/5
step 20/51 - loss: 810.1555 - 2ms/step
step 40/51 - loss: 840.3570 - 2ms/step
step 51/51 - loss: 421.0806 - 2ms/step
Eval begin...
step 13/13 - loss: 384.7417 - 693us/step
Eval samples: 102
Epoch 4/5
step 20/51 - loss: 647.1215 - 1ms/step
step 40/51 - loss: 682.9673 - 1ms/step
step 51/51 - loss: 422.0570 - 1ms/step
Eval begin...
step 13/13 - loss: 382.2546 - 591us/step
Eval samples: 102
Epoch 5/5
step 20/51 - loss: 713.3719 - 1ms/step
step 40/51 - loss: 567.0962 - 1ms/step
step 51/51 - loss: 456.8702 - 1ms/step
Eval begin...
step 13/13 - loss: 379.7527 - 985us/step
Eval samples: 102
...@@ -14,19 +14,19 @@ ...@@ -14,19 +14,19 @@
"metadata": {}, "metadata": {},
"source": [ "source": [
"## 环境\n", "## 环境\n",
"本教程基于paddle-develop编写,如果您的环境不是本版本,请先安装paddle-develop版本。" "本教程基于paddle-2.0Beta编写,如果您的环境不是此版本,请先安装paddle-2.0Beta版本,使用命令:pip3 install paddlepaddle==2.0Beta。"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 9, "execution_count": 1,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"0.0.0\n" "2.0.0-beta0\n"
] ]
} }
], ],
......
...@@ -7,7 +7,8 @@ ...@@ -7,7 +7,8 @@
环境 环境
---- ----
本教程基于paddle-develop编写,如果您的环境不是本版本,请先安装paddle-develop版本。 本教程基于paddle-2.0Beta编写,如果您的环境不是此版本,请先安装paddle-2.0Beta版本,使用命令:pip3
install paddlepaddle==2.0Beta
.. code:: ipython3 .. code:: ipython3
...@@ -25,7 +26,7 @@ ...@@ -25,7 +26,7 @@
.. parsed-literal:: .. parsed-literal::
0.0.0 2.0.0-beta0
数据集 数据集
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册