Skip to content

D
docs

MindSpore / docs

通知 5
Star 3
Fork 2
D
docs
“aec835c643af5249c7292acac3a600eeb6aa998d”上不存在“...registry/git@gitcode.net:openanolis/dragonwell8_jdk.git”
提交 516b61b6 编写于 作者: J JunYuLiu
浏览文件
操作

Add a new markdown and a notebook

上级 55027189
显示空白变更内容
内联 并排
Showing with 1230 addition and 67 deletion
tutorials/notebook/README.md
浏览文件 @ 516b61b6
......@@ -50,18 +50,19 @@
| 教  程  名  称 | 文  件  名  称 | 教  程  类  别 | 内  容  描  述
| :----------- | :----------- | :------- |:------
| 手写数字分类识别入门体验教程 | [quick_start.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/quick_start.ipynb) | 快速入门 | - CPU平台下从数据集到模型验证的全过程解读 <br/> - 体验教程中各功能模块的使用说明 <br/> - 数据集图形化展示 <br/> - 了解LeNet5具体结构和参数作用 <br/> - 学习使用自定义回调函数 <br/> - loss值与训练步数的变化图 <br/> - 模型精度与训练步数的变化图 <br/> - 使用模型应用到手写图片的预测与分类上
| 线性拟合 | [linear_regression.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/linear_regression.ipynb) | 快速入门 | - 了解线性拟合的算法原理<br/> - 了解在MindSpore中如何实现线性拟合的算法原理 <br/> - 学习使用MindSpore实现AI训练中的正向传播和方向传播<br/> - 可视化线性函数拟合数据的全过程。
| 加载数据集 | [loading_dataset.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/loading_dataset.ipynb) | 使用指南 | - 学习MindSpore中加载数据集的方法 <br/> - 展示加载常用数据集的方法<br/> - 展示加载MindRecord格式数据集的方法<br/> - 展示加载自定义格式数据集的方法
| 将数据集转换为MindSpore数据格式 | [convert_dataset_to_mindspore_data_format.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/convert_dataset_to_mindspore_data_format/convert_dataset_to_mindspore_data_format.ipynb) | 使用指南 | - 展示将MNIST数据集转换为MindSpore数据格式 <br/> - 展示将CSV数据集转换为MindSpore数据格式 <br/> - 展示将CIFAR-10数据集转换为MindSpore数据格式 <br/> - 展示将CIFAR-100数据集转换为MindSpore数据格式 <br/> - 展示将ImageNet数据集转换为MindSpore数据格式 <br/> - 展示用户自定义生成MindSpore数据格式
| 数据处理与数据增强 | [data_loading_enhancement.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/data_loading_enhance/data_loading_enhancement.ipynb) | 使用指南 | - 学习MindSpore中数据处理和增强的方法 <br/> - 展示数据处理、增强方法的实际操作 <br/> - 对比展示数据处理前和处理后的效果<br/> - 表述在数据处理、增强后的意义
| 自然语言处理应用 | [nlp_application.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/nlp_application.ipynb) | 应用实践 | - 展示MindSpore在自然语言处理的应用<br/> - 展示自然语言处理中数据集特定的预处理方法<br/> - 展示如何定义基于LSTM的SentimentNet网络
| 计算机视觉应用 | [computer_vision_application.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/computer_vision_application.ipynb) | 应用实践 | - 学习MindSpore卷积神经网络在计算机视觉应用的过程 <br/> - 学习下载CIFAR-10数据集，搭建运行环境<br/>- 学习使用ResNet-50构建卷积神经网络<br/> - 学习使用Momentum和SoftmaxCrossEntropyWithLogits构建优化器和损失函数<br/> - 学习调试参数训练模型，判断模型精度
| 模型的训练及验证同步方法 | [synchronization_training_and_evaluation.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/synchronization_training_and_evaluation.ipynb) | 应用实践 | - 了解模型训练和验证同步进行的方法<br/> - 学习同步训练和验证中参数设置方法<br/> - 利用绘图函数从保存的模型中挑选出最优模型
| 使用PyNative进行神经网络的训练调试体验 | [debugging_in_pynative_mode.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/debugging_in_pynative_mode.ipynb) | 模型调优 | - GPU平台下从数据集获取单个数据进行单个step训练的数据变化全过程解读 <br/> - 了解PyNative模式下的调试方法 <br/> - 图片数据在训练过程中的变化情况的图形展示 <br/> - 了解构建权重梯度计算函数的方法 <br/> - 展示1个step过程中权重的变化及数据展示
| 自定义调试信息体验文档 | [customized_debugging_information.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/customized_debugging_information.ipynb) | 模型调优 | - 了解MindSpore的自定义调试算子 <br/> - 学习使用自定义调试算子Callback设置定时训练<br/>- 学习设置metrics算子输出相对应的模型精度信息<br/> - 学习设置日志环境变量来控制glog输出日志
| MindInsight的模型溯源和数据溯源体验 | [mindinsight_model_lineage_and_data_lineage.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/mindinsight/mindinsight_model_lineage_and_data_lineage.ipynb) | 模型调优 | - 了解MindSpore中训练数据的采集及展示 <br/> - 学习使用SummaryRecord记录数据 <br/> - 学习使用回调函数SummaryCollector进行数据采集 <br/> - 使用MindInsight进行数据可视化 <br/> - 了解数据溯源和模型溯源的使用方法
| 计算图和数据图可视化 | [calculate_and_datagraphic.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/mindinsight/calculate_and_datagraphic.ipynb) | 模型调优 | - 了解MindSpore中新增可视化功能 <br/> - 学习使用MindInsight可视化看板<br/> - 学习使用查看计算图可视化图的信息的方法<br/> - 学习使用查看数据图中展示的信息的方法
| 标量、直方图、图像和张量可视化 | [mindinsight_image_histogram_scalar_tensor.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/mindinsight/mindinsight_image_histogram_scalar_tensor.ipynb) | 模型调优 | - 了解完整的MindSpore深度学习及MindInsight可视化展示的过程 <br/> - 学习使用MindInsight对训练过程中标量、直方图、图像和张量信息进行可视化展示<br/> - 学习使用Summary算子记录标量、直方图、图像和张量信息<br/> - 学习单独对标量、直方图、图像和张量信息进行记录并可视化展示的方法
| 混合精度 | [mixed_precision.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/mixed_precision.ipynb) | 性能优化 | - 了解混合精度训练的原理 <br/> - 学习在MindSpore中使用混合精度训练 <br/> - 对比单精度训练和混合精度训练的对模型训练的影响
| 模型安全 | [model_security.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/model_security.ipynb) | AI安全和隐私 | - 了解AI算法的安全威胁的概念和影响<br/> - 介绍MindArmour提供的模型安全防护手段<br/> - 学习如何模拟攻击训练模型<br/> - 学习针对被攻击模型进行对抗性防御
| 手写数字分类识别入门体验教程 | [quick_start.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/quick_start.ipynb) | 快速入门 | - CPU平台下从数据集到模型验证的全过程解读 <br/> - 体验教程中各功能模块的使用说明 <br/> - 数据集图形化展示 <br/> - 了解LeNet5具体结构和参数作用 <br/> - 学习使用自定义回调函数 <br/> - loss值与训练步数的变化图 <br/> - 模型精度与训练步数的变化图 <br/> - 使用模型应用到手写图片的预测与分类上
| 线性拟合 | [linear_regression.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/linear_regression.ipynb) | 快速入门 | - 了解线性拟合的算法原理<br/> - 了解在MindSpore中如何实现线性拟合的算法原理 <br/> - 学习使用MindSpore实现AI训练中的正向传播和方向传播<br/> - 可视化线性函数拟合数据的全过程。
| 加载数据集 | [loading_dataset.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/loading_dataset.ipynb) | 使用指南 | - 学习MindSpore中加载数据集的方法 <br/> - 展示加载常用数据集的方法<br/> - 展示加载MindRecord格式数据集的方法<br/> - 展示加载自定义格式数据集的方法
| 将数据集转换为MindSpore数据格式 | [convert_dataset_to_mindspore_data_format.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/convert_dataset_to_mindspore_data_format/convert_dataset_to_mindspore_data_format.ipynb) | 使用指南 | - 展示将MNIST数据集转换为MindSpore数据格式 <br/> - 展示将CSV数据集转换为MindSpore数据格式 <br/> - 展示将CIFAR-10数据集转换为MindSpore数据格式 <br/> - 展示将CIFAR-100数据集转换为MindSpore数据格式 <br/> - 展示将ImageNet数据集转换为MindSpore数据格式 <br/> - 展示用户自定义生成MindSpore数据格式
| 数据处理与数据增强 | [data_loading_enhancement.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/data_loading_enhance/data_loading_enhancement.ipynb) | 使用指南 | - 学习MindSpore中数据处理和增强的方法 <br/> - 展示数据处理、增强方法的实际操作 <br/> - 对比展示数据处理前和处理后的效果<br/> - 表述在数据处理、增强后的意义
| 自然语言处理应用 | [nlp_application.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/nlp_application.ipynb) | 应用实践 | - 展示MindSpore在自然语言处理的应用<br/> - 展示自然语言处理中数据集特定的预处理方法<br/> - 展示如何定义基于LSTM的SentimentNet网络
| 计算机视觉应用 | [computer_vision_application.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/computer_vision_application.ipynb) | 应用实践 | - 学习MindSpore卷积神经网络在计算机视觉应用的过程 <br/> - 学习下载CIFAR-10数据集，搭建运行环境<br/>- 学习使用ResNet-50构建卷积神经网络<br/> - 学习使用Momentum和SoftmaxCrossEntropyWithLogits构建优化器和损失函数<br/> - 学习调试参数训练模型，判断模型精度
| 模型的训练及验证同步方法 | [synchronization_training_and_evaluation.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/synchronization_training_and_evaluation.ipynb) | 应用实践 | - 了解模型训练和验证同步进行的方法<br/> - 学习同步训练和验证中参数设置方法<br/> - 利用绘图函数从保存的模型中挑选出最优模型
| 优化数据准备的性能 | [optimize_the_performance_of_data_preparation.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/optimize_the_performance_of_data_preparation/optimize_the_performance_of_data_preparation.ipynb) | 应用实践 | - 数据加载性能优化<br/> - shuffle性能优化<br/> - 数据增强性能优化<br/> - 性能优化方案总结
| 使用PyNative进行神经网络的训练调试体验 | [debugging_in_pynative_mode.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/debugging_in_pynative_mode.ipynb) | 模型调优 | - GPU平台下从数据集获取单个数据进行单个step训练的数据变化全过程解读 <br/> - 了解PyNative模式下的调试方法 <br/> - 图片数据在训练过程中的变化情况的图形展示 <br/> - 了解构建权重梯度计算函数的方法 <br/> - 展示1个step过程中权重的变化及数据展示
| 自定义调试信息体验文档 | [customized_debugging_information.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/customized_debugging_information.ipynb) | 模型调优 | - 了解MindSpore的自定义调试算子 <br/> - 学习使用自定义调试算子Callback设置定时训练<br/>- 学习设置metrics算子输出相对应的模型精度信息<br/> - 学习设置日志环境变量来控制glog输出日志
| MindInsight的模型溯源和数据溯源体验 | [mindinsight_model_lineage_and_data_lineage.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/mindinsight/mindinsight_model_lineage_and_data_lineage.ipynb) | 模型调优 | - 了解MindSpore中训练数据的采集及展示 <br/> - 学习使用SummaryRecord记录数据 <br/> - 学习使用回调函数SummaryCollector进行数据采集 <br/> - 使用MindInsight进行数据可视化 <br/> - 了解数据溯源和模型溯源的使用方法
| 计算图和数据图可视化 | [calculate_and_datagraphic.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/mindinsight/calculate_and_datagraphic.ipynb) | 模型调优 | - 了解MindSpore中新增可视化功能 <br/> - 学习使用MindInsight可视化看板<br/> - 学习使用查看计算图可视化图的信息的方法<br/> - 学习使用查看数据图中展示的信息的方法
| 标量、直方图、图像和张量可视化 | [mindinsight_image_histogram_scalar_tensor.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/mindinsight/mindinsight_image_histogram_scalar_tensor.ipynb) | 模型调优 | - 了解完整的MindSpore深度学习及MindInsight可视化展示的过程 <br/> - 学习使用MindInsight对训练过程中标量、直方图、图像和张量信息进行可视化展示<br/> - 学习使用Summary算子记录标量、直方图、图像和张量信息<br/> - 学习单独对标量、直方图、图像和张量信息进行记录并可视化展示的方法
| 混合精度 | [mixed_precision.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/mixed_precision.ipynb) | 性能优化 | - 了解混合精度训练的原理 <br/> - 学习在MindSpore中使用混合精度训练 <br/> - 对比单精度训练和混合精度训练的对模型训练的影响
| 模型安全 | [model_security.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/model_security.ipynb) | AI安全和隐私 | - 了解AI算法的安全威胁的概念和影响<br/> - 介绍MindArmour提供的模型安全防护手段<br/> - 学习如何模拟攻击训练模型<br/> - 学习针对被攻击模型进行对抗性防御
tutorials/notebook/computer_vision_application.ipynb
浏览文件 @ 516b61b6
......@@ -71,7 +71,7 @@
"metadata": {},
"source": [
"本次面向Ascend 910 AI处理器硬件平台,将卷积神经网络ResNet加入到案例中，你可以在这里下载完整的样例代码案例作为基础用例：\n",
"https://gitee.com/mindspore/docs/blob/master/tutorials/tutorial_code/resnet"
"https://gitee.com/mindspore/docs/blob/r0.7/tutorials/tutorial_code/resnet"
]
},
{
......
tutorials/notebook/convert_dataset_to_mindspore_data_format/convert_dataset_to_mindspore_data_format.ipynb
浏览文件 @ 516b61b6
......@@ -281,7 +281,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"- 本例中需要的数据位置在https://gitee.com/mindspore/docs/tree/master/tutorials/notebook/convert_dataset_to_mindspore_data_format/csv_data/data.csv\n",
"- 本例中需要的数据位置在https://gitee.com/mindspore/docs/tree/r0.7/tutorials/notebook/convert_dataset_to_mindspore_data_format/csv_data/data.csv\n",
"中，使用过程中可以在此路径下找到文件并下载，并且保存在`jupyter工作目录/dataset/`下，如图所示："
]
},
......@@ -838,7 +838,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"3. 准备需要写入的数据，按照用户定义的Schema形式，准备需要写入的样本列表，本例中需要的数据位置在https://gitee.com/mindspore/docs/tree/master/tutorials/notebook/convert_dataset_to_mindspore_data_format/images/transform.jpg\n",
"3. 准备需要写入的数据，按照用户定义的Schema形式，准备需要写入的样本列表，本例中需要的数据位置在https://gitee.com/mindspore/docs/tree/r0.7/tutorials/notebook/convert_dataset_to_mindspore_data_format/images/transform.jpg\n",
"中，使用过程中可以在此路径下找到图片并下载，并且保存在`jupyter工作目录/dataset/`下。"
]
},
......
tutorials/notebook/customized_debugging_information.ipynb
浏览文件 @ 516b61b6
......@@ -18,7 +18,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"本文将使用[快速入门](https://gitee.com/mindspore/docs/blob/master/tutorials/tutorial_code/lenet.py)作为样例，并通过构建自定义调试函数：`Callback`、`metrics`、`Print算子`、日志打印等，同时将构建的自定义调试函数添加进代码中，通过运行效果来展示具体如何使用MindSpore提供给我们的自定义调试能力，帮助快速调试训练网络。\n",
"本文将使用[快速入门](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/tutorial_code/lenet.py)作为样例，并通过构建自定义调试函数：`Callback`、`metrics`、`Print算子`、日志打印等，同时将构建的自定义调试函数添加进代码中，通过运行效果来展示具体如何使用MindSpore提供给我们的自定义调试能力，帮助快速调试训练网络。\n",
"体验过程如下：\n",
"1. 数据集准备。\n",
"2. 定义深度学习网络LeNet5。\n",
......@@ -46,7 +46,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"这里我们需要将MNIST数据集中随机取出一张图片，并增强成适合LeNet网络的数据格式（如何处理请参考[quick_start.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/quick_start.ipynb)），训练数据集下载地址：{\"<http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz>\", \"<http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz>\"}。\n",
"这里我们需要将MNIST数据集中随机取出一张图片，并增强成适合LeNet网络的数据格式（如何处理请参考[quick_start.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/quick_start.ipynb)），训练数据集下载地址：{\"<http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz>\", \"<http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz>\"}。\n",
"<br/>数据集放在----`Jupyter工作目录+\\MNIST_Data\\train\\`，如下图结构："
]
},
......@@ -303,7 +303,7 @@
"\n",
"`GLOG_logtostderr`：控制日志输出方式，设置为`1`时，日志输出到屏幕；值设置为`0`时，日志输出到文件。设置输出屏幕时，日志部分的信息会显示成红色，设置成输出到文件时，会在`GLOG_log_dir`路径下生成`mindspore.log`文件。\n",
"\n",
"> 更多设置请参考官网：<https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/customized_debugging_information.html>"
"> 更多设置请参考官网：<https://www.mindspore.cn/tutorial/zh-CN/r0.7/advanced_use/customized_debugging_information.html>"
]
},
{
......
tutorials/notebook/data_loading_enhance/data_loading_enhancement.ipynb
浏览文件 @ 516b61b6
......@@ -439,7 +439,7 @@
"source": [
"2. 使用一类图片当作数据，体验操作。在一个数据量比较大的图片数据集中，例如数据集名称叫`images`，它的存储方式是在`images`文件夹下，有不同子类别的文件夹，一个子类别文件夹中的图片属于同一类。所以我们本次体验所使用的图片放置方法，就需要创建`enhance_images`文件夹，接着在`enhance_images`下建一个名为`sample`的子类别文件夹，将图片放在`sample`文件夹中即可。如果有更多类别图片，可以在`enhance_images`下创建对应的子类别文件夹，将图片放入即可。\n",
"\n",
" 增强体验使用的数据位置在<https://gitee.com/mindspore/docs/tree/master/tutorials/notebook/data_loading_enhance/enhance_images/sample>中，使用过程中可以在此路径下找到图片数据，并参照本次体验中图片放置的位置来新建文件夹。"
" 增强体验使用的数据位置在<https://gitee.com/mindspore/docs/tree/r0.7/tutorials/notebook/data_loading_enhance/enhance_images/sample>中，使用过程中可以在此路径下找到图片数据，并参照本次体验中图片放置的位置来新建文件夹。"
]
},
{
......
tutorials/notebook/debugging_in_pynative_mode.ipynb
浏览文件 @ 516b61b6
......@@ -34,7 +34,7 @@
"\n",
"4. 执行神经网络训练，查看网络各参数梯度。\n",
"\n",
"> 你可以在这里找到完整可运行的样例代码：<https://gitee.com/mindspore/docs/blob/master/tutorials/tutorial_code/lenet.py>。"
"> 你可以在这里找到完整可运行的样例代码：<https://gitee.com/mindspore/docs/blob/r0.7/tutorials/tutorial_code/lenet.py>。"
]
},
{
......@@ -55,7 +55,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"这里我们需要将MNIST数据集中随机取出一张图片，并增强成适合LeNet网络的数据格式（如何处理请参考[quick_start.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/quick_start.ipynb)），训练数据集下载地址：{\"<http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz>\", \"<http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz>\"} 。\n",
"这里我们需要将MNIST数据集中随机取出一张图片，并增强成适合LeNet网络的数据格式（如何处理请参考[quick_start.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/quick_start.ipynb)），训练数据集下载地址：{\"<http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz>\", \"<http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz>\"} 。\n",
"<br/>数据集放在----Jupyter工作目录+\\MNIST_Data\\train\\，如下图结构："
]
},
......
tutorials/notebook/linear_regression.ipynb
浏览文件 @ 516b61b6
......@@ -548,7 +548,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"`nn.RMSProp`为完成权重更新的函数，更新方式大致为公式11，但是考虑的因素更多，具体信息请参考[官网说明](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html?highlight=rmsprop#mindspore.nn.RMSProp)。"
"`nn.RMSProp`为完成权重更新的函数，更新方式大致为公式11，但是考虑的因素更多，具体信息请参考[官网说明](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.nn.html?highlight=rmsprop#mindspore.nn.RMSProp)。"
]
},
{
......
tutorials/notebook/loading_dataset.ipynb
浏览文件 @ 516b61b6
......@@ -308,7 +308,7 @@
"\n",
"MindSpore天然支持读取MindSpore数据格式——`MindRecord`存储的数据集，在性能和特性上有更好的支持。 \n",
"\n",
"> 阅读[将数据集转换为MindSpore数据格式](https://www.mindspore.cn/tutorial/zh-CN/master/use/data_preparation/converting_datasets.html)，了解如何将数据集转换为MindSpore数据格式。\n",
"> 阅读[将数据集转换为MindSpore数据格式](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/converting_datasets.html)，了解如何将数据集转换为MindSpore数据格式。\n",
"\n",
"可以通过`MindDataset`对象对数据集进行读取。详细方法如下所示："
]
......@@ -407,7 +407,7 @@
"## 加载自定义数据集\n",
"\n",
"现实场景中，数据集的种类多种多样，对于自定义数据集或者目前不支持直接加载的数据集，有两种方法可以处理。\n",
"一种方法是将数据集转成MindRecord格式（请参考[将数据集转换为MindSpore数据格式](https://www.mindspore.cn/tutorial/zh-CN/master/use/data_preparation/converting_datasets.html)章节），另一种方法是通过`GeneratorDataset`对象加载，以下将展示如何使用`GeneratorDataset`。\n",
"一种方法是将数据集转成MindRecord格式（请参考[将数据集转换为MindSpore数据格式](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/converting_datasets.html)章节），另一种方法是通过`GeneratorDataset`对象加载，以下将展示如何使用`GeneratorDataset`。\n",
"\n",
"1. 定义一个可迭代的对象，用于生成数据集。以下展示了两种示例，一种是含有`yield`返回值的自定义函数，另一种是含有`__getitem__`的自定义类。两种示例都将产生一个含有从0到9数字的数据集。\n",
" \n",
......
tutorials/notebook/mindinsight/calculate_and_datagraphic.ipynb
浏览文件 @ 516b61b6
......@@ -287,7 +287,7 @@
"1. 导入所需的代码包，并示例化训练网络。\n",
"2. 通过MindSpore提供的 `SummaryCollector` 接口，实现收集计算图和数据图。在实例化 `SummaryCollector` 时，在 `collect_specified_data` 参数中，通过设置 `collect_graph` 指定收集计算图，设置 `collect_dataset_graph` 指定收集数据图。\n",
"\n",
"更多 `SummaryCollector` 的用法，请点击[API文档](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.train.html?highlight=summarycollector#mindspore.train.callback.SummaryCollector)查看。\n",
"更多 `SummaryCollector` 的用法，请点击[API文档](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.train.html?highlight=summarycollector#mindspore.train.callback.SummaryCollector)查看。\n",
"\n"
]
},
......@@ -338,7 +338,7 @@
"\n",
"> 其中 /path/ 为 `SummaryCollector` 中参数 `summary_dir` 所指定的目录。\n",
"\n",
"![title](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/mindinsight_map.png)"
"![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/mindinsight_map.png)"
]
},
{
......@@ -352,7 +352,7 @@
"- 节点信息：显示当前所查看节点的信息，包括名称、类型、属性、输入和输出。便于在训练结束后，核对计算正确性时查看。\n",
"- 图例：图例中包括命名空间、聚合节点、虚拟节点、算子节点、常量节点，通过不同图形来区分。\n",
"\n",
"![title](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/cast_map.png)"
"![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/cast_map.png)"
]
},
{
......@@ -384,7 +384,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"![title](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/data_map.png)"
"![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/data_map.png)"
]
},
{
......
tutorials/notebook/mindinsight/mindinsight_image_histogram_scalar_tensor.ipynb
浏览文件 @ 516b61b6
......@@ -305,10 +305,10 @@
"\n",
"当前支持的Summary算子:\n",
"\n",
"- [ScalarSummary](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html?highlight=scalarsummary#mindspore.ops.operations.ScalarSummary): 记录标量数据\n",
"- [TensorSummary](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html?highlight=tensorsummary#mindspore.ops.operations.TensorSummary): 记录张量数据\n",
"- [ImageSummary](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html?highlight=imagesummary#mindspore.ops.operations.ImageSummary): 记录图片数据\n",
"- [HistogramSummary](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html?highlight=histogramsummar#mindspore.ops.operations.HistogramSummary): 将张量数据转为直方图数据记录"
"- [ScalarSummary](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.ops.operations.html?highlight=scalarsummary#mindspore.ops.operations.ScalarSummary): 记录标量数据\n",
"- [TensorSummary](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.ops.operations.html?highlight=tensorsummary#mindspore.ops.operations.TensorSummary): 记录张量数据\n",
"- [ImageSummary](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.ops.operations.html?highlight=imagesummary#mindspore.ops.operations.ImageSummary): 记录图片数据\n",
"- [HistogramSummary](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.ops.operations.html?highlight=histogramsummar#mindspore.ops.operations.HistogramSummary): 将张量数据转为直方图数据记录"
]
},
{
......@@ -393,7 +393,7 @@
"\n",
"下面展示使用`SummaryCollector`来记录标量、直方图信息。\n",
"\n",
"在MindSpore中通过`Callback`机制，提供支持快速简易地收集损失值、参数权重、梯度等信息的`Callback`, 叫做`SummaryCollector`（详细的用法可以参考API文档中[mindspore.train.callback.SummaryCollector](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.train.html?highlight=summarycollector#mindspore.train.callback.SummaryCollector)）。`SummaryCollector`使用方法如下： \n",
"在MindSpore中通过`Callback`机制，提供支持快速简易地收集损失值、参数权重、梯度等信息的`Callback`, 叫做`SummaryCollector`（详细的用法可以参考API文档中[mindspore.train.callback.SummaryCollector](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.train.html?highlight=summarycollector#mindspore.train.callback.SummaryCollector)）。`SummaryCollector`使用方法如下： \n",
"\n",
"`SummaryCollector` 提供 `collect_specified_data` 参数，允许自定义想要收集的数据。\n",
"\n",
......@@ -577,17 +577,17 @@
"\n",
"在本地浏览器中打开地址：`127.0.0.1:8080`，进入到可视化面板。\n",
"\n",
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/mindinsight_panel.png)\n",
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/mindinsight_panel.png)\n",
"\n",
"在上图所示面板中可以看到`summary_01`日志文件目录，点击**训练看板**进入到下图所示的训练数据展示面板，该面板展示了标量数据、直方图、图像和张量信息，并随着训练、测试的进行实时刷新数据，实时显示训练过程参数的变化情况。\n",
"\n",
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/mindinsight_panel2.png)\n",
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/mindinsight_panel2.png)\n",
"\n",
"### 标量可视化\n",
"\n",
"标量可视化用于展示训练过程中标量的变化趋势，点击打开标量信息展示面板，该面板记录了迭代计算过程中的损失值标量信息，如下图展示了损失值标量趋势图。\n",
"\n",
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/scalar_panel.png)\n",
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/scalar_panel.png)\n",
"\n",
"上图展示了神经网络在训练过程中损失值的变化过程。横坐标是训练步骤，纵坐标是损失值。\n",
"\n",
......@@ -599,7 +599,7 @@
"- 分步回退是指对同一个区域连续框选并放大查看时，可以逐步撤销操作。\n",
"- 还原图形是指进行了多次框选后，点击此按钮可以将图还原回原始状态。\n",
"\n",
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/scalar_select.png)\n",
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/scalar_select.png)\n",
"\n",
"上图展示的标量可视化的功能区，提供了根据选择不同标签，水平轴的不同维度和平滑度来查看标量信息的功能。\n",
"\n",
......@@ -614,15 +614,15 @@
"\n",
"直方图用于将用户所指定的张量以直方图的形式展示。点击打开直方图展示面板，以直方图的形式记录了在迭代过程中所有层参数分布信息。\n",
"\n",
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/histogram_panel.png)\n",
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/histogram_panel.png)\n",
"\n",
"如下图为`conv1`层参数分布信息，点击图中右上角，可以将图放大。\n",
"\n",
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/histogram.png)\n",
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/histogram.png)\n",
"\n",
"下图为直方图功能区。\n",
"\n",
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/histogram_func.png)\n",
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/histogram_func.png)\n",
"\n",
"上图展示直方图的功能区，包含以下内容：\n",
"\n",
......@@ -636,11 +636,11 @@
"\n",
"下图为展示`summary_01`记录的图像信息。\n",
"\n",
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/image_panel.png)\n",
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/image_panel.png)\n",
"\n",
"通过滑动上图中的\"步骤\"滑条，查看不同步骤的图片。\n",
"\n",
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/image_function.png)\n",
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/image_function.png)\n",
"\n",
"上图展示图像可视化的功能区，提供了选择查看不同标签，不同亮度和不同对比度来查看图片信息。\n",
"\n",
......@@ -652,7 +652,7 @@
"\n",
"张量可视化用于将张量以表格以及直方图的形式进行展示。\n",
"\n",
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/tensor_func.png)\n",
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/tensor_func.png)\n",
"\n",
"上图展示了张量可视化的功能区，包含以下内容：\n",
"\n",
......@@ -661,7 +661,7 @@
"- 纵轴：可以选择步骤、相对时间、绝对时间中的任意一项，来作为直方图纵轴显示的数据。\n",
"- 视角：可以选择正视和俯视中的一种。正视是指从正面的角度查看直方图，此时不同步骤之间的数据会覆盖在一起。俯视是指 偏移以45度角俯视直方图区域，这时可以呈现不同步骤之间数据的差异。\n",
"\n",
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/tensor.png)\n",
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/tensor.png)\n",
"\n",
"上图中将用户所记录的张量以表格的形式展示，包含以下功能：\n",
"\n",
......@@ -804,7 +804,7 @@
"source": [
"此时点击打开MindInsight**训练列表**看板中的`./summary_loss_only`目录，如下图所示，可以看到只记录有损失值标量信息。\n",
"\n",
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/loss_scalar_only.png)"
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/loss_scalar_only.png)"
]
},
{
......@@ -900,11 +900,11 @@
"source": [
"此时点击打开MindInsight**训练列表**看板中的`./summary_histogram_only`目录。\n",
"\n",
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/histogram_only.png)\n",
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/histogram_only.png)\n",
"\n",
"在MindInsight面板中，如上图所示，只展示了直方图信息。\n",
"\n",
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/histogram_only_all.png)\n",
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/histogram_only_all.png)\n",
"\n",
"点击进入直方图面板，如上图所示，只展示了`conv1`层的直方图信息。"
]
......@@ -1043,7 +1043,7 @@
"source": [
"此时点击打开MindInsight**训练列表**看板中的`./summary_tensor_only`目录，如下图所示，可以看到只记录有张量信息。\n",
"\n",
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/tensor_only.png)"
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/tensor_only.png)"
]
},
{
......@@ -1177,7 +1177,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/image_only.png)\n",
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/image_only.png)\n",
"\n",
"在MindInsight面板中，如上图所示，只展示了输入图像信息。"
]
......@@ -1192,11 +1192,11 @@
"\n",
"点击MindInsight看板中的**对比看板**，打开对比看板，可以得到多次（不同）训练搜集到的标量数据对比信息。\n",
"\n",
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/multi_scalars.png)\n",
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/multi_scalars.png)\n",
"\n",
"上图展示了`summary_01`（上图中红色曲线）和`summary_loss_only`（上图中蓝色曲线）的标量曲线对比效果，横坐标是训练步骤，纵坐标是标量值。\n",
"\n",
"![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/multi_scalars_select.png)\n",
"![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/multi_scalars_select.png)\n",
"\n",
"上图展示的对比看板可视的功能区，提供了根据选择不同训练或标签，水平轴的不同维度和平滑度来进行标量对比的功能。\n",
"\n",
......
tutorials/notebook/mindinsight/mindinsight_model_lineage_and_data_lineage.ipynb
浏览文件 @ 516b61b6
......@@ -29,7 +29,7 @@
"6. 数据溯源的使用。调整数据参数多次训练并存储数据，并使用MindInsight的数据溯源功能对不同数据集下训练产生的模型进行对比分析，了解如何调优。\n",
"\n",
"\n",
"本次体验将使用快速入门案例作为基础用例，将MindInsight的模型溯源和数据溯源的数据记录功能加入到案例中，快速入门案例的源码请参考：<https://gitee.com/mindspore/docs/blob/master/tutorials/tutorial_code/lenet.py>。"
"本次体验将使用快速入门案例作为基础用例，将MindInsight的模型溯源和数据溯源的数据记录功能加入到案例中，快速入门案例的源码请参考：<https://gitee.com/mindspore/docs/blob/r0.7/tutorials/tutorial_code/lenet.py>。"
]
},
{
......@@ -292,7 +292,7 @@
"source": [
"MindSpore 提供 `SummaryCollector` 进行记录训练过程中的信息。通过 `SummaryCollector` 的 `collect_specified_data` 参数，可以自定义记录指定数据。\n",
"\n",
"在本次体验中，我们将记录训练数据与数据集预处理的操作，我们将 `collect_specified_data` 中的 `collect_train_lineage`, `collect_eval_lineage`, `collect_dataset_graph` 设置成 `True`。SummaryCollector的更多用法，请参考[API文档](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.train.html?highlight=collector#mindspore.train.callback.SummaryCollector)。\n"
"在本次体验中，我们将记录训练数据与数据集预处理的操作，我们将 `collect_specified_data` 中的 `collect_train_lineage`, `collect_eval_lineage`, `collect_dataset_graph` 设置成 `True`。SummaryCollector的更多用法，请参考[API文档](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.train.html?highlight=collector#mindspore.train.callback.SummaryCollector)。\n"
]
},
{
......@@ -353,7 +353,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"这里主要展示如何启用及关闭MindInsight，更多的命令集信息，请参考MindSpore官方网站：<https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/visualization_tutorials.html>。\n",
"这里主要展示如何启用及关闭MindInsight，更多的命令集信息，请参考MindSpore官方网站：<https://www.mindspore.cn/tutorial/zh-CN/r0.7/advanced_use/visualization_tutorials.html>。\n",
"\n",
"启动MindInsight服务命令：\n",
"\n",
......@@ -389,7 +389,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"![image](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/model_lineage_all.png)"
"![image](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/model_lineage_all.png)"
]
},
{
......@@ -412,7 +412,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"![image](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/model_lineage_cp.png)"
"![image](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/model_lineage_cp.png)"
]
},
{
......@@ -442,7 +442,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"![image](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/data_lineage.png)"
"![image](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/data_lineage.png)"
]
},
{
......
tutorials/notebook/mixed_precision.ipynb
浏览文件 @ 516b61b6
......@@ -42,7 +42,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"> 你可以在这里找到完整可运行的样例代码：<https://gitee.com/mindspore/mindspore/tree/master/model_zoo/official/cv/resnet>。"
"> 你可以在这里找到完整可运行的样例代码：<https://gitee.com/mindspore/mindspore/tree/r0.7/model_zoo/official/cv/resnet>。"
]
},
{
......@@ -56,7 +56,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"![image](https://www.mindspore.cn/tutorial/zh-CN/master/_images/mix_precision.jpg)"
"![image](https://www.mindspore.cn/tutorial/zh-CN/r0.7/_images/mix_precision.jpg)"
]
},
{
......@@ -952,7 +952,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"当然，如果你想参考单步训练或者手动设置混合精度训练，可以参考官网教程<https://www.mindspore.com/tutorial/zh-CN/master/advanced_use/mixed_precision.html>。"
"当然，如果你想参考单步训练或者手动设置混合精度训练，可以参考官网教程<https://www.mindspore.com/tutorial/zh-CN/r0.7/advanced_use/mixed_precision.html>。"
]
},
{
......
tutorials/notebook/model_security.ipynb
浏览文件 @ 516b61b6
......@@ -579,7 +579,7 @@
"source": [
"### 攻击模型\n",
"\n",
"调用MindArmour提供的FGSM接口（`FastGradientSignMethod`），使用被攻击前抽取的96张数据图像`test_images`作为被攻击数据集，保存被攻击后数据集图像到当前notebook目录下的`ada_data`文件中。其中，参数`eps`为攻击对数据范围产生的单步对抗性摄动的比例，该值越大，则攻击程度越大。关于`FastGradientSignMethod`的详细使用说明，可参考[官方API文档](https://www.mindspore.cn/api/zh-CN/master/api/python/mindarmour/mindarmour.attacks.html?highlight=fastgradientsignmethod#mindarmour.attacks.FastGradientSignMethod)。"
"调用MindArmour提供的FGSM接口（`FastGradientSignMethod`），使用被攻击前抽取的96张数据图像`test_images`作为被攻击数据集，保存被攻击后数据集图像到当前notebook目录下的`ada_data`文件中。其中，参数`eps`为攻击对数据范围产生的单步对抗性摄动的比例，该值越大，则攻击程度越大。关于`FastGradientSignMethod`的详细使用说明，可参考[官方API文档](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindarmour/mindarmour.attacks.html?highlight=fastgradientsignmethod#mindarmour.attacks.FastGradientSignMethod)。"
]
},
{
......
tutorials/notebook/optimize_the_performance_of_data_preparation/optimize_the_performance_of_data_preparation.ipynb 0 → 100644
浏览文件 @ 516b61b6
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# <center/>优化数据准备的性能"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 概述\n",
"\n",
"数据是整个深度学习中最重要的一环，因为数据的好坏决定了最终结果的上限，模型的好坏只是去无限逼近这个上限，所以高质量的数据输入，会在整个深度神经网络中起到积极作用，数据在整个数据处理和数据增强的过程像经过pipeline管道的水一样，源源不断地流向训练系统，如图所示："
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/optimize_the_performance_of_data_preparation/images/pipeline.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"MindSpore为用户提供了数据处理以及数据增强的功能，在数据的整个pipeline过程中，其中的每一步骤，如果都能够进行合理的运用，那么数据的性能会得到很大的优化和提升。本次体验将基于CIFAR-10数据集来为大家展示如何在数据加载、数据处理和数据增强的过程中进行性能的优化。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 整体流程\n",
"- 准备环节。\n",
"- 数据加载性能优化。\n",
"- shuffle性能优化。\n",
"- 数据增强性能优化。\n",
"- 性能优化方案总结。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 准备环节"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 导入模块"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`dataset`模块提供API用来加载和处理数据集。"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import mindspore.dataset as ds"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`numpy`模块用于生成ndarray数组。"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 下载所需数据集"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. 在jupyter工作目录下创建`./dataset/Cifar10Data`目录，本次体验所用的数据集存放在该目录下。\n",
"2. 在jupyter工作目录下创建`./transform`目录，本次体验转换生成的数据集存放在该目录下。\n",
"3. 下载[CIFAR-10二进制格式数据集](https://www.cs.toronto.edu/~kriz/cifar-10-binary.tar.gz)，并将数据集文件解压到`./dataset/Cifar10Data/cifar-10-batches-bin`目录下，数据加载的时候使用该数据集。\n",
"4. 下载[CIFAR-10 Python文件格式数据集](https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz)，并将数据集文件解压到`./dataset/Cifar10Data/cifar-10-batches-py`目录下，数据转换的时候使用该数据集。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"目录结构如下所示：\n",
"\n",
"\n",
" dataset/Cifar10Data\n",
" ├── cifar-10-batches-bin\n",
" │   ├── batches.meta.txt\n",
" │   ├── data_batch_1.bin\n",
" │   ├── data_batch_2.bin\n",
" │   ├── data_batch_3.bin\n",
" │   ├── data_batch_4.bin\n",
" │   ├── data_batch_5.bin\n",
" │   ├── readme.html\n",
" │   └── test_batch.bin\n",
" └── cifar-10-batches-py\n",
" ├── batches.meta\n",
" ├── data_batch_1\n",
" ├── data_batch_2\n",
" ├── data_batch_3\n",
" ├── data_batch_4\n",
" ├── data_batch_5\n",
" ├── readme.html\n",
" └── test_batch\n",
"\n",
"其中：\n",
"- `cifar-10-batches-bin`目录为CIFAR-10二进制格式数据集目录。\n",
"- `cifar-10-batches-py`目录为CIFAR-10 Python文件格式数据集目录。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 数据加载性能优化"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"MindSpore为用户提供了多种数据加载方式，其中包括常用数据集加载、用户自定义数据集加载、MindSpore数据格式加载，详情内容请参考[加载数据集](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/loading_the_datasets.html)。对于数据集加载，底层实现方式的不同，会导致数据集加载的性能存在差异，如下所示："
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"| | 常用数据集 | 用户自定义 | MindRecord |\n",
"| :----: | :----: | :----: | :----: |\n",
"| 底层实现 | C++ | Python | C++ |\n",
"| 性能 | 高 | 中 | 高|"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 性能优化方案"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/optimize_the_performance_of_data_preparation/images/data_loading_performance_scheme.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"数据加载性能优化建议如下：\n",
"- 已经支持的数据集格式优选内置加载算子，具体内容请参考[内置加载算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html)，如果性能仍无法满足需求，则可采取多线程并发方案，请参考本文[多线程优化方案](#多线程优化方案)。\n",
"- 不支持的数据集格式，优选转换为MindSpore数据格式后再使用`MindDataset`类进行加载，具体内容请参考[将数据集转换为MindSpore数据格式](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/converting_datasets.html)，如果性能仍无法满足需求，则可采取多线程并发方案，请参考本文[多线程优化方案](#多线程优化方案)。\n",
"- 不支持的数据集格式，算法快速验证场景，优选用户自定义`GeneratorDataset`类实现，如果性能仍无法满足需求，则可采取多进程并发方案，请参考本文[多进程优化方案](#多进程优化方案)。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 代码示例\n",
"\n",
"基于以上的数据加载性能优化建议，本次体验分别使用内置加载算子`Cifar10Dataset`类、数据转换后使用`MindDataset`类、使用`GeneratorDataset`类进行数据加载，代码演示如下："
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. 使用内置算子`Cifar10Dataset`类加载CIFAR-10数据集，这里使用的是CIFAR-10二进制格式的数据集，加载数据时采取多线程优化方案，开启了4个线程并发完成任务，最后对数据创建了字典迭代器，并通过迭代器读取了一条数据记录。"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'image': array([[[235, 235, 235],\n",
" [230, 230, 230],\n",
" [234, 234, 234],\n",
" ...,\n",
" [248, 248, 248],\n",
" [248, 248, 248],\n",
" [249, 249, 249]],\n",
"\n",
" [[216, 216, 216],\n",
" [213, 213, 213],\n",
" [215, 215, 215],\n",
" ...,\n",
" [254, 254, 254],\n",
" [253, 253, 253],\n",
" [253, 253, 253]],\n",
"\n",
" [[213, 213, 213],\n",
" [217, 217, 217],\n",
" [215, 215, 215],\n",
" ...,\n",
" [255, 255, 255],\n",
" [254, 254, 254],\n",
" [254, 254, 254]],\n",
"\n",
" ...,\n",
"\n",
" [[195, 195, 195],\n",
" [200, 200, 200],\n",
" [202, 202, 202],\n",
" ...,\n",
" [138, 138, 138],\n",
" [143, 143, 143],\n",
" [172, 171, 176]],\n",
"\n",
" [[205, 205, 205],\n",
" [205, 205, 205],\n",
" [211, 211, 211],\n",
" ...,\n",
" [112, 112, 112],\n",
" [130, 130, 132],\n",
" [167, 163, 184]],\n",
"\n",
" [[210, 210, 210],\n",
" [209, 209, 209],\n",
" [213, 213, 213],\n",
" ...,\n",
" [120, 120, 119],\n",
" [146, 146, 146],\n",
" [177, 174, 190]]], dtype=uint8), 'label': array(9, dtype=uint32)}\n"
]
}
],
"source": [
"cifar10_path = \"./dataset/Cifar10Data/cifar-10-batches-bin/\"\n",
"\n",
"# create Cifar10Dataset for reading data\n",
"cifar10_dataset = ds.Cifar10Dataset(cifar10_path,num_parallel_workers=4)\n",
"# create a dictionary iterator and read a data record through the iterator\n",
"print(next(cifar10_dataset.create_dict_iterator()))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"2. 使用`Cifar10ToMR`这个类将CIFAR-10数据集转换为MindSpore数据格式，这里使用的是CIFAR-10 python文件格式的数据集，然后使用`MindDataset`类加载MindSpore数据格式数据集，加载数据采取多线程优化方案，开启了4个线程并发完成任务，最后对数据创建了字典迭代器，并通过迭代器读取了一条数据记录。"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'data': array([255, 216, 255, ..., 63, 255, 217], dtype=uint8), 'id': array(30474, dtype=int64), 'label': array(2, dtype=int64)}\n"
]
}
],
"source": [
"from mindspore.mindrecord import Cifar10ToMR\n",
"\n",
"cifar10_path = './dataset/Cifar10Data/cifar-10-batches-py/'\n",
"cifar10_mindrecord_path = './transform/cifar10.record'\n",
"\n",
"cifar10_transformer = Cifar10ToMR(cifar10_path,cifar10_mindrecord_path)\n",
"# executes transformation from Cifar10 to MindRecord\n",
"cifar10_transformer.transform(['label'])\n",
"\n",
"# create MindDataset for reading data\n",
"cifar10_mind_dataset = ds.MindDataset(dataset_file=cifar10_mindrecord_path,num_parallel_workers=4)\n",
"# create a dictionary iterator and read a data record through the iterator\n",
"print(next(cifar10_mind_dataset.create_dict_iterator()))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"3. 使用`GeneratorDataset`类加载自定义数据集，并且采取多进程优化方案，开启了4个进程并发完成任务，最后对数据创建了字典迭代器，并通过迭代器读取了一条数据记录。"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'data': array([0], dtype=int64)}\n"
]
}
],
"source": [
"def generator_func(num):\n",
" for i in range(num):\n",
" yield (np.array([i]),)\n",
"\n",
"# create GeneratorDataset for reading data\n",
"dataset = ds.GeneratorDataset(source=generator_func(5),column_names=[\"data\"],num_parallel_workers=4)\n",
"# create a dictionary iterator and read a data record through the iterator\n",
"print(next(dataset.create_dict_iterator()))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## shuffle性能优化"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"shuffle操作主要是对有序的数据集或者进行过repeat的数据集进行混洗，MindSpore专门为用户提供了`shuffle`函数，其中设定的`buffer_size`参数越大，混洗程度越大，但时间、计算资源消耗也会大。该接口支持用户在整个pipeline的任何时候都可以对数据进行混洗，具体内容请参考[shuffle处理](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/data_processing_and_augmentation.html#shuffle)。但是因为底层的实现方式不同，该方式的性能不如直接在[内置加载算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html)中设置`shuffle`参数直接对数据进行混洗。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 性能优化方案"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/optimize_the_performance_of_data_preparation/images/shuffle_performance_scheme.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"shuffle性能优化建议如下：\n",
"- 直接使用内置加载算子的`shuffle`参数进行数据的混洗。\n",
"- 如果使用的是`shuffle`函数，当性能仍无法满足需求，可通过调大`buffer_size`参数的值来优化提升性能。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 代码示例\n",
"\n",
"基于以上的shuffle性能优化建议，本次体验分别使用内置加载算子`Cifar10Dataset`类的`shuffle`参数和`Shuffle`函数进行数据的混洗，代码演示如下："
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. 使用内置算子`Cifar10Dataset`类加载CIFAR-10数据集，这里使用的是CIFAR-10二进制格式的数据集，并且设置`shuffle`参数为True来进行数据混洗，最后对数据创建了字典迭代器，并通过迭代器读取了一条数据记录。"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'image': array([[[254, 254, 254],\n",
" [255, 255, 254],\n",
" [255, 255, 254],\n",
" ...,\n",
" [232, 234, 244],\n",
" [226, 230, 242],\n",
" [228, 232, 243]],\n",
"\n",
" [[251, 251, 251],\n",
" [253, 253, 254],\n",
" [255, 255, 255],\n",
" ...,\n",
" [225, 227, 235],\n",
" [227, 231, 241],\n",
" [229, 233, 243]],\n",
"\n",
" [[250, 250, 250],\n",
" [251, 251, 251],\n",
" [253, 253, 253],\n",
" ...,\n",
" [233, 235, 241],\n",
" [233, 236, 245],\n",
" [238, 242, 250]],\n",
"\n",
" ...,\n",
"\n",
" [[ 67, 64, 71],\n",
" [ 65, 62, 69],\n",
" [ 64, 61, 68],\n",
" ...,\n",
" [ 71, 67, 70],\n",
" [ 71, 68, 70],\n",
" [ 69, 65, 68]],\n",
"\n",
" [[ 62, 58, 64],\n",
" [ 59, 55, 61],\n",
" [ 61, 58, 64],\n",
" ...,\n",
" [ 64, 62, 64],\n",
" [ 61, 58, 59],\n",
" [ 62, 60, 61]],\n",
"\n",
" [[ 66, 60, 65],\n",
" [ 64, 59, 64],\n",
" [ 66, 60, 65],\n",
" ...,\n",
" [ 64, 61, 63],\n",
" [ 63, 58, 60],\n",
" [ 61, 56, 58]]], dtype=uint8), 'label': array(9, dtype=uint32)}\n"
]
}
],
"source": [
"cifar10_path = \"./dataset/Cifar10Data/cifar-10-batches-bin/\"\n",
"\n",
"# create Cifar10Dataset for reading data\n",
"cifar10_dataset = ds.Cifar10Dataset(cifar10_path,shuffle=True)\n",
"# create a dictionary iterator and read a data record through the iterator\n",
"print(next(cifar10_dataset.create_dict_iterator()))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"2. 使用`shuffle`函数进行数据混洗，参数`buffer_size`设置为3，数据采用`GeneratorDataset`类自定义生成。"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"before shuffle:\n",
"[0 1 2 3 4]\n",
"[1 2 3 4 5]\n",
"[2 3 4 5 6]\n",
"[3 4 5 6 7]\n",
"[4 5 6 7 8]\n",
"after shuffle:\n",
"[2 3 4 5 6]\n",
"[0 1 2 3 4]\n",
"[4 5 6 7 8]\n",
"[1 2 3 4 5]\n",
"[3 4 5 6 7]\n"
]
}
],
"source": [
"def generator_func():\n",
" for i in range(5):\n",
" yield (np.array([i,i+1,i+2,i+3,i+4]),)\n",
"\n",
"ds1 = ds.GeneratorDataset(source=generator_func,column_names=[\"data\"])\n",
"print(\"before shuffle:\")\n",
"for data in ds1.create_dict_iterator():\n",
" print(data[\"data\"])\n",
"\n",
"ds2 = ds1.shuffle(buffer_size=3)\n",
"print(\"after shuffle:\")\n",
"for data in ds2.create_dict_iterator():\n",
" print(data[\"data\"])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 数据增强性能优化"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"在图片分类的训练中，尤其是当数据集比较小的时候，用户可以使用数据增强的方式来预处理图片，从而丰富数据集。MindSpore为用户提供了多种数据增强的方式，其中包括：\n",
"- 使用内置C算子（`c_transforms`模块）进行数据增强。\n",
"- 使用内置Python算子（`py_transforms`模块）进行数据增强。\n",
"- 用户可根据自己的需求，自定义Python函数进行数据增强。\n",
"\n",
"具体的内容请参考[数据增强](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/data_processing_and_augmentation.html#id3)。因为底层的实现方式不同，所以性能还是有一定的差异，如下所示："
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"| 模块 | 底层接口 | 说明 |\n",
"| :----: | :----: | :----: |\n",
"| c_transforms | C++（基于OpenCV）| 性能高 |\n",
"| py_transforms | Python（基于PIL） | 该模块提供了多种图像增强功能，并提供了PIL Image和Numpy数组之间的传输方法 |\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 性能优化方案"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/optimize_the_performance_of_data_preparation/images/data_enhancement_performance_scheme.png)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"数据增强性能优化建议如下：\n",
"- 优先使用`c_transforms`模块进行数据增强，因为性能最高，如果性能仍无法满足需求，可采取[多线程优化方案](#多线程优化方案)、[Compose优化方案](#Compose优化方案)或者[算子融合优化方案](#算子融合优化方案)。\n",
"- 如果使用了`py_transforms`模块进行数据增强，当性能仍无法满足需求，可采取[多线程优化方案](#多线程优化方案)、[多进程优化方案](#多进程优化方案)、[Compose优化方案](#Compose优化方案)或者[算子融合优化方案](#算子融合优化方案)。\n",
"- `c_transforms`模块是在C++内维护buffer管理，`py_transforms`模块是在Python内维护buffer管理。因为Python和C++切换的性能成本，建议不要混用算子。\n",
"- 如果用户使用了自定义Python函数进行数据增强，当性能仍无法满足需求，可采取[多线程优化方案](#多线程优化方案)或者[多进程优化方案](#多进程优化方案)，如果还是无法提升性能，就需要对自定义的Python代码进行优化。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 代码示例\n",
"\n",
"基于以上的数据增强性能优化建议，本次体验分别使用`c_transforms`模块和自定义Python函数进行了数据增强，演示代码如下所示："
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. 使用`c_transforms`模块进行数据增强，数据增强时采用多线程优化方案，开启了4个线程并发完成任务，并且采用了算子融合优化方案，使用`RandomResizedCrop`融合类替代`RandomResize`类和`RandomCrop`类。"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD8CAYAAAB3lxGOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOy9Xch2W3cedI251v28+6v52pgaTWwrCbQHggeWYlvoSagKWoPxQE2qtKkEemJBqWJTT+qBQnuiLVSUgMVUxFirYpGIiBpKQaUmFcUGJQ1Vo8FQbatFst/nXnN4MP6uMde6n/fZX95Nn833zr2fd/3ca801/8Y1rjHmn6gqPoVP4VP45g3jr3cCPoVP4VP46xs+gcCn8Cl8k4dPIPApfArf5OETCHwKn8I3efgEAp/Cp/BNHj6BwKfwKXyThy8FBETk7xOR/0lEfkZEfvjL+Man8Cl8Ch8nyMceJyAiG4D/GcDfC+DnAPxZAL9dVf/8R/3Qp/ApfAofJXwZTOA3AvgZVf1ZVX0P4McAfN+X8J1P4VP4FD5C2L+EOH8VgP+Nrn8OwG9aHxKR3w3gdwPAEPkNf8O7p8cxyukEkLiS008fCu29c8KWX4Vuy/Ko5B9EIBDYqV/LoOs4H5AhGHEcAyIDYzm/OnIa5Avkd805ADzifx+OVl+4Wu7mgZ5SxVqP5+++JnMvP/OY366/fMEvxwPfAIFWKHSqnSmgque/OTHzXKE67Xz5pl7lg+TkorniZ3/2L/wlVf32NV1fBghcleOpyFT1RwD8CAD88q99pn/Xd/2aHoMLElKI4ILlAjZCAEeeP/5aRBsFJZlIceEFCzQo7qvjGBgQbPuOfd+xbXbc9xu2bcN+u9nf05M9w+dPT/jss8/w9O4dvva1r+Gzr30NT++e8LXP7Pzdu3f47LPP7JmnJ3z22dfwta/Z+bZtBDpw0Dnn/Sr7ulSNXrWSS4HU5fe6LlOyGrYlQL0hewPWmfegmiBsSbAj1Ujmp+oiPs7gUc9nCrREQ6kcdAUj+i2Bux0rNVUmvXAi70ofaKb1hZmtqnh+fsZxv+P5+U7nz3h+/x7Pz8/4/PP3eP/553h+/4zPP//czp+fCxiAPJ8ep8gAQh7GIPnoRP/7v/8f+l9OicKXYw78HACSaPxqAP/Hl/AdDxet/hGcf3HS8MVDi/xFPVm/6sWrV9G9IrxKQV0BgL98vvshcNXTT3VewqQAROv+y3FefqI/fHXvUVgF+YPfe/Sdi2+9RKkefk5OL+oXrOmP6cn7MkDgzwL4dSLy3SLyBOAHAPypLxbFywWyUvUWHtRV/sY/y3L8Aml4KLH6oYceh8ca/OqH67gfmQl8+6UG9I02ruvPlmYWcJJXdvHayM9mxKuCRlpemTt5cH4JkQ/ifKkdIkyiL1gOj4J8AXC6CB/dHFDVu4j8HgD/KYANwB9T1f/xC8YCQNJ+VC60oIpZsWIXa6aXchYI1J+LBikqDxrGKs1dd8tUTAFEFHMqRCbmVMxjQiCYx4G5bZjHARkDc07MOSFzQi//yiacp/uz7o1hZRKmkmUCSn6MlfZflWzP35LXVpRMDVYEJaFKaly0WOe0tNO5zpnVEno5tHNQcbB55j8ERS+7t9qGvaxX7PsU1lzzMc5fy8ZOeM+m0asEsD90JccvhfZ9lQb+UW6vDV+GTwCq+uMAfvz1L1yaUJ5Tq2zx5qNTq7aCx0xc15QuF2qRqqjjRtmeIgLoMLkSj1CHf2sCMvI4ZUJEoHJgTmush1hFpGCEoIJsNQBjDBz3O45tw7zfcdzvBhj3O+b9gG6HgUj83etcPK3shCyhqSLrF9Tc0551DMmmdLZv7fpkuFycsz1c53MeCWDzmJgBAu4TiDQLgwH5ZZq/g3w1iHwjyrfn90yr5VL489d8leBJDeA9QnpxEfB0NizHFugdfdTQkSl8SEL6Y5lOji3KMgH2lUDwpYDAxwnqms6vXPM3BAzhf7VRI+c46HOQWZHldXzARGL4R1UHRBTi2l+OiUMmAOCACb0chzkTj4FxHBARHMdhQuFHPWYyBZ3+G13rPKDHgTknhgjUhUBJ+NO5CaTDT5h/X1pNVQoh/HnF562A+rE9QyAwtfIwjwNzHn59QKem4y3SP0i4ywkry3kBqhIriJiI3hEQSKrYkGkQ+FdJZCwdNwkYUK8vBfkBwb8K7rUMNluxPw6t5S5AkAmjuheChA+FNwEC1nwqZ0bT4xenfi/l5xETwHo/NFXgpV9HQ9NhLMC79BShFWZ29U3SwlMYbV0IdXPyUto/ugIBYwLTmcBx3HEcd8z73jT+vB/Q+4GZrGBi3g8cWDQo9RLUMRjJBYVu18WvQourFgCwZn8o+Je/W9fWcdwdBO4GfHPiOO6NCSQYNM0/MLzsRKwrFXSsfI5iM8SOHP3y2WpWpdWb1o+/LJYChZ63KDG/q4syeaTlWdBVk/Zzq2kAtJxnHFpfE/RPre9zHb8mvAkQANDKOvNLudWoRIFLJpa/pVTiNMofgLeavJkYzLQfw0nItC6WTMsssMCAyOGfOqhGqn/XGnUJ/moOzN1NgOOo43FgHnZ+NFPgjnlsKTCI4wIAK3VmOh3Zr1bM7WrpqyZAKLbwWPBXkFCdmYfjMJPnmAeOu5kIJ8GHYEQexqDxEgPqXcDqrKDyp/1aYM9oAAI1BRFKY5eOLBJvW9aUNMssQwgzR6XBMkiFNe1+igClyZXO6/dVdmU5b5DUSEGAWrGsr7Y5oO7s4nqjflnTYGtBnzN8om/BKOb6oLZnygpZvpEgNJ1NuG8gvyapEMYwE+AYgnEcGPcDUwYJ+6w/p8rzCIegmQFpT7v5MPxbwkdiAMpCTza0OY5Ga2/RPkLr199s1/QUnT8CBTcHIj8zgMDA4H6/OwiQ8KMY1hCBzAF1IDAfzYahYqZXaH8xU0ydOSQTCKCGIM06EeBCqMULIcFC7F6BarEFZtynZsf+gJUJrC8szsMrNnAO0tp704+Xz9b5a8ObAYEzk+ra2gJl3e2/aueEpNTAT2Co9UAV5oRo2KCzGpHOEi51e1SrAQeWDLfxVBVzDGxDuzNQho0QFDMLjuMwNnB3FrBtpfXv3RnY/tKLJZnPdAUNSU1aDMAGkUAiL8ZsyimITHcAQI1Wm6HXm+aqWnnMDKZOHPdgAc94fr47CDxjHubbkBB6GAgMVPls22agMDaMMRMUQuBFFDJc30VZx28jwM6ZXDQC1WQQLCxXFLyaTLQpafeNTLL2Z0HnsrgID8cgPIaB9vwStaj/heJKVtgh4aXwZkBgDVlWXm/MA5qPQA0AgvoENvQaW+OukgyNWlzQhcZ7AliJqJimCG0jISPxNxUyJnRoNkjAWcGoRh6Cf4S9fHTan2Cw9BQcmdY6BiDIdBDzEWQ2YgwYiLxkJoBhjs6gk2HCRHcej/Yr4V9obJYeiEnZb2H/H3fT/vd7jIx7b12lMABIMECBwtg26JxmCgwFtgHVAdXNfTMO1OpCr+rAp8YeJhwgRrUT9fJSpXIjsfN6tdsdJLoqKjDQbKQvMIFG95G/s0MwTREvOwHpNhWsQh/vP4KZysdX0Bx4iJ2R4QuhTgG9AIx8/Jo3nYGFyyv7oDXZJMQqJUBGnTKmVlD1cQiSQhX9/027Tuo2c3/AsW04np9xPN/wjIEt6XA0CsU8jhw2DKBVMHvQDQDIqz7Clh6ADIRTrUwG8+bnGAUar3BWPd5wJUZuXIEAMPXA/fkZdx8Se39+j/th59N7SQbOICAi2MaGsVn+t7E5G/DyyHxVnmTUtRDYijOHdW5Hsahy/2WZUk65fB/5+9KRys7UFwYsqKr5R44D9/uBu5/HvXlMHAexMWrYrT1nkSu4b+M03PqrxgQelh0br4DLZhG3Zh+l1l4cNRcmga43mrW1JCHiZwAIsOFjVAzb2AEG0wYU6Zyu5e84nu+4788YMrBvG7axQY8JqHUdHrebNZDnZ9xuN4yxef6kzJyk1NH4iRoLe9X9mCAwwi4wEIBmP34AVxoNUuVq8hTqFe0ZS44ByPPzM47j2cbFxxj5O4MAyi9ATsJtDIxtMxNqGw0EOF+QDnSQmoCVIMBgiDCFRglJlCXVc2UEayPJlgIQALTzR4OWNA/VU2KO0jw/DtzdbJr+TDKHYLkovC1x0CyT9LVED8tXjQm0AudW1eSSsC28d03aL8wsXNSnrk8ThRjVuE/97MkIOHmuE1MLyOJoKwAYzgDUPeXH/Y778zMGBM/D/AZ6HCaMx4H78477/Rn398/Y9x1jC00OOnq5sHCMurYeCikgcHMmzgHkxJ6J7hgEijCYzwHuZpD8zah0fcLiKyZwvzMIvPdRlSAGUDqLmYCI+Qa4p6CDwDUYCLOFhQ1cjz3o7SLKlRvMmeSrm4FVTtWzQgxpaYfQcJoqjQWZDgQEDIcrDG9PjtVksLgpRwymWM5oYPCa8GZAoAtpCZ9ATgW5soOg/E2X08XZb7MABpsEDYwKdoUirDHwUrRM6PzCJEAM/VUHgm1mN+AhBgbbcAeeA8exH2kybNvuguDfNalpIIDW2ON6YQABArD75tPsXYNxnsI6XGsLfFqzlDVBwh/XOg8830v7MyAEE/BHGwCkY3CY2TO2YgHFaLrwV57ZNLgwEy7YUTOpWttr/PJagTxiAS+YA4CmcNdUYWNd3AuUw8S1l1GkRhBmaZhj2QqI+XzFmMDJlAoNl7Y3EOOjmRLFs8S22jEvpO6enbOrKaDri35bqbDr5+QSWqzA7MTQDrMxAT3Mvj/udxxjw30MiCI95fN+4Njv2J+fsW0bnn26ctjJZybgx6S4IfjRKFzYU/gNAOJcg8WAQMAbdAh9/MkQbDlltYCAmYCImRfmBwhGUH/JBIBspNnQHQRC6DPPzgRAeeznJPQ0yGgFgjgfxBZODUbXW3phPnbNn36BhwBwphXRPvITqjjCV+QjK9kciDKLd7mlnoQfZRK8JrwJEADSsgQAcryF1IfmD6eOnmU24+E4zpUSdjs/ewaC9T7djgog9pFxBVhM6Wg/wyk4fBjwhjkOHPdn3IeBhjFrxRx3jG3D3e3hbXNavEUjtg+dj11A7Hbd11XwgwmgGmN0Cca9EvyBbQvvvZ0zG8hztzBUpwn/cScQuDcQiOLixl0g4Ewg/QHEaB4AwckZuJgFuYjLYiasDYeaB0IL6dJScCH8fL02mR6oblpJwEZasv8oG7Nkj4FfJkhrxunxMRt8kII1vBkQaGApKE/8CgTxALpAn6JblDtXY7z2mAm4hhIThlaYwk/0eCPNgAIh/KP7BjTswePAHD6JSG2+wR3A3A6MY+AIDejaMFcX8oSVkFuiVs9wNjIWfAYEv1d50NPRKLkYEA3vtXAQSLMgZbPO55zWLeggEKbB/f7sXYRoAMpAEIIfAjvGKrQSNKLy6YAwiAmsZgEzinQgjqJy2ZTa9aIsVJfzYnx8fRI9bj5E1VvvhTNI8wdqmmleu96eQ/KLClRRnAHgK2UOAF2UywMvLnRub2em2At7pgOM3J22s0mw4vsCBGK2vaFtZwgNKhopqUprS0SFjSdiKH8cmAAOPOMOJPVTnyTUHXtnex8oisiAwGBQ10iBZ+EvEFhGX0rlI5Y823djI9smtnLSHrQaZBYECIgxARL85wSEO+Y8so6T/5HgpJCm9q5uvxR8yp9d9p6RM/3fKs51WHLS89IU7R63K6bvKHCP3z60aK+ZVwRGxFJ6Wlh4vXzVAV59cFQzW1YfC5mFrwhvBgQmFaBlskSv6+cIJyutDtcvVBCkaaF5A/0F7V9wnywARYwpnClGNYdJFDnVeM6JKWL9v8OmAoftz84Jje6gqOCLv9D6rXsrEd/Tzr9FyiRSP0jou3mAiDfjR8aRXU7ZeLnLDvWXnRDx/oCMDTImtjGtPLdoyEmA7Vmt0meh711+zBckPeOVbzh19zDVB5HBakph60AMxRi2BoTIICEuIEgtrFoK43TOf8hela5/yMj1ky3Kz7uEw/zZti3zUtR+UN2ijVMJU8CaAZtHXzy8CRAgFuY3fBBEOQfy/uMYcCn0vVyYl72M2i1e5GoGAA0S0IjyTDwwomtQpKbWiuCYB+QgbR40z9/uA1yqgru9Zx9mKrkCQcRvsdqowdL+A+rXCAeSGMVPfpnxlH5Jp93Y8lkGgsKccEwphgzMsWEoMMbstRCOLy84i2d43KttHwAWNcIwwnUUQuzArYANIbZ0DbiJF/MPmhOXgeBK2K/+5ulebz7Fs0SQvTzbtmP33o9t281MEu4OjRzWeRuoplTWWQJre39NG38jIBDoWlfiFSQQQlNZNPxJ5v03vk9O/SW4oKBmo12nzCPJjyrCWcRcQhEIbb9PZwIitobAEMFBzMGcga5ttlh5Zz/R/xJ2FAg0QBBqCJyPMg1C2NvRmUA2umFrJNhYBNM6JmphPzMT2NwnQMViL6VAD7Fhv2NsObdi20wDRzGuPMzioenDsYBsjHRc64XuhcnIrag0tw01hoivBCWAzxHpIGBdeFAa5RlgHj4d7SMr816cgxREnhSr7QvT7hjbhn077Hxw+W7WE7OVnpHIJ0g3egOXVhmtkD4Y3ggIAJPNUgIACMjZErQvDlw08TPxJBRtqsixNBbBekpiTQODyqHYbY5ueQToiHf5iADzsJWHshcAsHHx7jOYc6TfoIR6BQKkhrVsrkNi0VgAgwBgU3FPQOCaZ9ttpp56HGNEaUt9UzYMica5Q7aqnxRJYlf2vOVzc8qkQILAahJE8a/+Dwa95fG8Jh5VQsfjxWNSmFSeoq20od1tmbcQ9hjUo8nodL0OYIglwilhSkcR4LbfsG07brEi9bZh7jdbNGZs2HfFtnn5iNQ09yxXbm/+bzPhvjAGfBgEROSPAfheAL+gqn+H3/s2AP8ugO8C8BcB/KOq+pfFUvNHAPw2AP8fgN+lqj/1moQ0N10CgDgAqA/GWfCfGqBHUs+ec4KUaHmwutA5UTUwqHio2+7BZh+AwZzQIZgTNoV4mk9gHoLD1yA4Iue6IbqXWlcfcGIEXUBG/92fr4QSEspAdAuaAW8mwRihajYD3xGsrICguuAkWYFpazPkQphtgpVC4ZN80kOvZh6NLZ9OAFgqoQS/AwI3bSIRXPztftPIkhAEQnUAaPM65iwwmKq2opNqDuDhv1w6bVkHMkZfsvAnCAA4bhP7fuS7c9/TfNm2mAng5pfH5fjlIMclWG09gYAb7yvDa5jAvwngjwL443TvhwH856r6B8X2GvxhAL8PwN8P4Nf5328C8K/hYuORNaguTOCSARBIJBXi36TafN0JfV4sAXXMWBcNSinLQ7kChF5k4HIGAwMrAWj9wQNyaJkAUBszoBNzbhjjwJwb5jEtG2lgd+FOaryaAuEFDq3J+ci4ggnQvAEZGMPHBWz+vjv8EuCkTIJkAtuOMaJXof5qZYUJEZvnMMbm9ywdKrWYgyxlmMklEGSmo2FzLYLOlZC+OXcsK33jgkSQJtea26HUjesDeI4EggPHwSBwFGvw83QqZpoUYT4+Pd2x7zuent7hOCb2fc807HswF8vzmJst1dtr1JRjtewE/ijjnsMPhw+CgKr+aRH5ruX29wH4Hj//UQA/AQOB7wPwx9VUyX8tIt8qIt+pqj//iu/UucBtUi3mFoDgv6UiaDPYSltIYX8vlyWQNQ0Gkowx2ECiCmn9BlRxXqzlmOUIO/wB8eMcAlWbLz9iReJxdEBKEPB0JTWvY/MNtEZwDQLr37ZpPjOGj2zMnqUahz5kc899+QRMDEnz2XYYEJhTsK/96CCgtKLLhaO3KG0Bd/qCsuRds7aBPGYCVNcd6kjaODDBAELBdL7G9NtCqesEn7pHC74kCBw4crhvsClyFrqd+Hy/4+l2s9mC88Dt9pR5CCY4hk2gMl9RaX4zKYQYY7WpcCS/1NYfhW/UJ/C3hGCr6s+LyN/s96+2IPtVAD4MAutFKNyotWYTEkXi3oMLIHgYZPltuQzbvrp5uve5tCVxDingmDoxpkAxMWFre0xfphSqGOrLmIVDaQwozRJstJ6YQZtT34a/MtshLRrXNV8oMcGiHGQL9wJpg274fES/tk83FgXUlmazRjtt9R8YSBsjCBBY62TR5g2HpRWDlmpH6cIQtqRqXn8BBCXweT9/V/BknnVyz3GYYNdU3wP3WN/BmcJxAoRzb0EAQuZ4apavgW/MlxjYYg2JbTu9V0qn2lvWE77x8LEdg1dpueQkQnsR3sIT0qLxSpbqnuPr+lrZ90Jn9Wm611KnFUnT/rHEVBEu++7CAPIrQqsaScYRKVYYk/CmWv9RA5lBdXyFo0oyAZ+EdowVdLYaELPMsAtQiNWG4M+EIxDu6IP4Vmq3HXs4q57MWRXn275h7Df72zb7rmw0DyE0eyRaLX4c5pWP9RjV++Z1Xdvtgg2A8h6nGgBMws0aH4qR99RNTCUTgVlAAcLhDr2D7f2pOOZhE5gO85vI/Y5jHIDcbXn5wxdqOSx/giNBek5bYh1ToYgt2OzjcwGHSX8vdkUuBSSoRpg6gluxBjB/OHyjIPB/Bs0Xke8E8At+/9VbkCntRfjLnt71lhC2NUoThwNEmQWgBBIuuFYRWktvp1ivIJMpQcVmv9XX6ftYG2c4Zfp7yNg08xF0uRqjTdsNoYnGO93h2KZJcwWLa3dfYANjs4U2nKJHv3oNshkLUGwJIOFb2DYDgdhD8bbv2PbNurBuBgLbvmNsO2TbINuGMidm2UqpplxCRzhyfRUgnRgSqxWtNaHtRtNrVBRJOqIs/eYZEOyYy75VjXnbyzaIzQV2WxjBdkzcxx3juBfAjqBTJuw4BgR3YlwHYnqUTgBioD7jm57muQj/DEYS56EkqM1Y+qU11fIdx0m12Boj8eHwjYLAnwLwgwD+oB//I7r/e0Tkx2AOwb/6Gn8AcOYBrGtrFZ+ihx0AghYWbITQKAnQKaz3UiPT4CDSSCsjYWMBfJa/uf4XpYqE23RnJjA08qiMOtZ4xCz0WlxzJABIaOhms4+8PoPBcCCIYcA3bNtofdfbbcfYd5+8tNc3xIDHymVEjjznM8siWcIQXybLgEDWlklsru5J/xnU9gkEQsBTU6bWR9rhdl5Rcz0ghW+eTYJx+LoLknGacLuWZXNSjkxbth11RUQbA6RwJ5Mh5oJqD2d/hrezYJzCEpJIQJlcmMMHwmu6CP8dmBPwbxKRnwPwB2DC/ydE5IcA/K8A/hF//Mdh3YM/A+si/Ce+QFqWD7uIcYalMmcAIHSOzgSKyDdqj3guPhKxp/epCjsRNW2vAiL1dzoTqDOVtFarkuGDU8TmFgyxhjGCuvobyTiY7sHARcWW4cZwoXQBHdtuwr9tvhhHnO9kxwcTME8/hvicABP2bd+wbzZteXcA2LZhcftMRowNcEZRAKBVdlRykIHe4mPJsqrDCpRvbga9bS/Hs2ZnoGAWAPRzeN3MK3NAFcdxQMYz7s6w4OAbYDDngSNZgWubTRIQ1IctR7uIIpgrE5gLC1BFW9glBZrLN8qm2wEBbBDByrheCq/pHfjtD376uy+eVQD/5Ku/zuG0SIDHiRj/LWAfYMzWY7Og3nFt2ol8iyueBIhO9dw4EHXDINfXe4kJIHiA1rkWI4hqna79J+Br3WtpEPMA+RyKQoIc3TgG4PR8bLvZ7MOo++5Cu/notJEsoVgAT6jZXPC3BBAbQBSr/MSSXzIGZBuIYWwFycV4DBsVqrGYqTVk8TJookj1mfjMVdDuUU8ACXfT8nSh9eg5kNBM7+5r5oBO3zFqpAlg7eCOGFYsB5sGkpGaVWSrToc5FHMachwC+QNq5OHFb9FWgnloKLkovGIBDBSr2fOh8GZGDHJgm9yu7V9BiRoLXDEB+G9eGLkKcWhtjrHHwywgWAMXacbTIQHczE4DYUL7U74CEOCCHtoC0QC8T3LxCVL8rmElhHFAxg4hwR8+Km3fd2z7rebo8ww9NxNC+HkiyxjiE1x8rsDmZsSQ9CsUE6i89jUcYuBQmD5wYNaopGamXZls6z3H/ZNO7HS/VUmeSv8nihxzo8VfEwQUMo40A6LHAAocc2JMBXAgzRI4CxlbgqANU54poGWmBBvAC45BijNYDYjJtvKTlsdiQMvs0BfCmwSB1DJCWpoofzY/QTtPyi5dhFcBzs/kZbGGHP8maO+Qu4WYSVQOQCkJkWiMIGYc5rnCzAG1LjUJ6hzVJwFFlDqBmQTDmIA56naIO+7GvmO7PZlDbzPH3u32RGPSB/kN/DzWK8i1CxwEcn5ArcSTi3sMbozRaIPBeAlE2aSEMjhUVchyBP+2tgsqZ11/fam9UxtaB47NedTgoGYO3K38vadFXdvs7hNI7a80QMu7PwdNBgq2YADQNf28OK/r2XwFke7ItUS2pLe7KodgXh8ObwMEBD6V0i9bTutGuw9qJFKPRZ9prLYa5/aYtJYlLaIQdjlFLOEEo/NuVpzDJrFsNi2OITEiL5x70v5yB6GYRRddf6m1TdDHfsOWfzu22w377clMgf0J++0pvfv77QnbKBPgNK9+mbrLC3rE6EEenRh7LqjT03BY5Vh98TKMBph+nKVBStXnWq9r3V6FRz3jl/clBIilyE/mgMzQ2vYX4//HdmDM6f6Qrc4P0/i6KVS3zF1kebqPYaj6YLADQwembljHXSDaafqQlrCAaDm9JcswRqkyGzZG8rh9cngTICAieLrt602QqdUazSMKGZNNRjZc3/E2TIFgC/UG0e6z8PORh/LyuP4rDWSAVACwhYCJLdXFM+P61OGy2XMcgHv7IaO67bbd+vNvBgS32xP23UBg22/e5bf79dJbIF3w2wq9Iwq5FzbNffSyApoHWnsxvETPjTE4w+mmbiu/tVg/PBwmQJxvCf0S3LAzgfWdoPgzqfi1r8FiskFbY9iAKwzbL3LOiW3u/u7E1D2/uUcXrDtjLxc8ydmTnEJp/64mLc+XEfhmuR8sMwtvAgSGCD57eqobq/YnRtCytWgSe0SIEUhdxxPUAKrhnSPS9gxp7ASOlwt4+Dx9+xt1HNZ4LCnLBKAAgXTmuQYaO2QM0/C3J+z7DfvtHW5PTw4C7wocHAjCHNj33eJhFhJg0JiJ529QXgjzEuoAACAASURBVMWXtVqyykNQlDSVEiqw8K+2aTg7GQAUyN60snFf04jXZ6Sq2a95u/YUFgL/NN0UHQDyWH+ZMilTSQFgq/Y3VXHTMBst9ybc9xyLEb0v2QvDwJws1oHNC6eVx1pWmSl7ZryijUZ4EyAgIni3gACPhS5BRGcBfCJVCTwLraai1gul5BqNoPiW80tW0FLRgkIxUBtANECImXwIrdRQLgV/c+E1z72BwnZ7wu3pHXYX/AQBv5faf2MmcGsAAImdfzoV7ZOS0rythtgzmPmM60ZaPwAGIgsQgHSbLvXSGnsv79PVqiE4DwQANfCsIlEgB/UkAICO7YtWjjKGz44s5WBDxf0Nykf0rNwcmLdbAUGaG8EGGtuMAhbwmAPIOpalmxUJ6q8IbwIEhgiebre6wekn26fxHSx5lBIq8XNu1LK8JBzBC6BQu/4uCYvrbOWcmJhEE+aJg0GuEiP5VEM2CGQjB50DwLbvGCM0u89Dd3t/d3PAtP+Wwj+o/3/V9k3gUWVESJrnZxs1ygSLgPuxOabotSWiNiiyVP9SjhzOvOAaGKre4jrrEB0IDLyKBXiqzXnL2t//ViAQiHXV+qQre0ywDcXcFBvFBxHIUeZAF/4yCXifiGsZJlRsiq18SqlcvkogIGNcmANojECycdIzdF4y6g09CrExgSsWIFyv/TdulCwk/M5q//qJiG0GGmAUvolBQh+LYPAxhf+2Yxvm9Nu2G8a+43Z7wu3JBf/pHZ6ezAy4Pb3zrsBoXHtO+d02H1rs+ZLWwKSVa/7rv4VSZo0d2icAgJ1+0UtQ5679Nc+qetXurCxtNQtY9BtWXICFIBZGiatF+5/qsGLMgVzau+8IAygNAomF/mDrJRjSCDSmYEZb9O9FT0wsJLLvNxqQtS3+gGJtWZY5nTXyV+ZBlMSIunSW8pUCgbGaA7GYSlScLOcRTsIbWpu03gkAQtvROdb7ON/39LRvL1qQr91thEDpEZojIlNAtfchi/quO3un87sDwe3JzYHbDU/t/DNnCzVSMI/bVpoBXA5LfpZQxFPOiqflWx+AAQNBPJYiFAQv4xVIX+9jNQuy0K/CuVFcAYCGxkSBWx/HUQDQ/9DYgkU6gKG18AcA0Vq4NEAgHK5jHNjmUfM09jDdiBWkc3BlAWemaZXiQMTKj3Zhal25L4Q3AQKn3oEHws/K+xoMSujy3XhR2CF0odGvrq+AgL7Hwt/YbraD/l/cAVHN6Y0/NFD23btGDyCw7r9uCtyebth3O27bzd/daGBQ9CxcNQbOU2WA89E1b5gv/Mzy3gOhb8N7GTy7Mj6nbzlnfLj6rc0tOdn/5H8QOeWTwbjYQN3rH3UmELpYCN69b3/z+za2w8YVHFOcnfnozBHDsnkp9KW9trCCwVJibOoNZwOvCG8HBJ7YJ8CCH0iHaswXDCCO3JV3AoUrrf6aeOl5Xe/7TcX5Hgt+LdwprbENn2c81fZCNa++0f5tC2F/h23fcbu9w9O7Jxd87ymIsQDbnt19PGnoMQhkMi8ycJWhLoj2lEuxluYFNJlrLPohsiyEQeAuyzHqqapLqNrO+WBgl+VYWC+XR4WBrqgCc7Y6ZodgOQkdyFRJHAsA4rO21oJmckUEx5gYM+ZkDOomHJ0NxI7MabpRRUk/N3DSVlY5qGuMrxoIADdiAk2TEwA0c4BLHXy/hL+f1/NBBZkQANWI0Y4UN+K5l/MT1L40PzEBDfvTn83dlOze2Pbu+b89Yc8egRJ8u2+Owm132n8xffhSq1CmBegN1nOwOvKu9JKVUqn/fBcgmx+o1XGcNqfGAjDCcepfyeqSOgIl2JyFNS8tDoKNzP9iCqnmSkDlkKPA/YLuIexdhRqJrbxpfMeMwZh9PMbA1IHdx20YCGw+WctAYI+uwtiDcZAjOdoomwKcDM8sL1s+aADeS+GNgIB5TesaJ21RbADoQs3HVfjXe/Ru/nYVz4XGz0oAisOuD/VIAgBC+EHXgAs+fUxVaCBQafkbnZtj6VaTg8ie5FGGvctvSeMJFCJzwQtWm/Q6p1YU7Iwrimy4EoAQ8SrSYcvCL+J7HyC1Xx29LOnjmXqhq+W5sy/onHdVtfKa6s2D62YxExoL4EHLnTWFwk5nsDtlVa3bOGZmlvbnORvcRRheJOkF7whbQCuZnvS1SI3+fE14MyDQmQASACDwFXJQjo4FDOJUWeA9oloBR07vZWQpt+XUOYOAVW/MaaAfWlxdVw1jASsA+LF3Edr9sRcTiF6A2+1d+gNyqPB+a2MDZNgONn04aq4h1gSlkhz/lvCzRgc981Ko7mtd3lKSkZKUtFnZFIhRcvG7nVyAAOel/9YEhoR9FSQGgVg/oFhTPOVp1zAHurOwYjp7TiJNwQY0112Aj+AcbaPZAITNp3bnvI3BeasQxM1KewK6FSkhANi+akxgNQdyAd3c6abTPAJtqjxr8DUEmEDA3832+ogFAIm0/R4AaDcFuPFFOjz97hFCAEEBQE0qSaCg6y1B4F2BwNOTzwnw7sKcJnzL3oRBAs/H1k11qR2r8XoWF6H9MAhwaACi0TYXVx4JvU2TCIcY1XMI0lXaueyL33egOOXvnBNV9eXDYsJPgHMwgVT/JyfhCmxpDvg/EVfunub5y6na20inII8YPA8dXnwD/i12snKvS4xKjQ1kXxPeDAjs+9aurZvQwWAxC/yh8/kDANCqGfoo6FmkkNc5utaXvFPPkq3ZACA/EAAwSvvrQNJOifMAgmEg8MTjAZ4aCPAYgOgJ2MYo47OjY4EBqhxLSCj/PgGo1mx4hfA/dFZnAYLWfLazFHQGAmICKfBeSu0eg8FyTnV7eubs5bT0TE0venUp0+PNHRBgELP6lrz6aRhEnE+MAj62+Y0JSK7bMFKLSw3rvgCA2AHLCkTb94tZ2XdeE94MCNxoxKAMqvwUfmICKXMh/Egh1AQCPtazXfBXexanQtXlvrZnSZiwNGIXeAYB9fNcCNT3AJAEBAOBp6d3OQbg9vSZDQpyMyDmEdRIs1j2yyo8HY2o4xUAMEMoOe0LozysL7wg/w+FrpdZ7DlYYLDMWMT5vMq21/9DYIgHyZvf/Gm++YqM2YAgUmuy382CJAdrvij3YZKJ2NbuOTOzAUCNCYiFW0TEJ50RODLLza85v8r0FBOIYerGBL5S5gDw9MTjBKIggaBDIrD16vx31v4NBAgAEhCw0n8hWl9o2u39En52BnYQKGFqqG2JJeHvR8T+fohhonW977cGAk9P73B79y5nCYrEPH9ebXhDNpKg4Fo2viWJQCrKNNMatvulfruus6tr4V8ewIh/v2YxxrTlKsMCqwcgkHJBv0X0suSBQGDh8phiAjiODgBRkOUcZJPgap4+gSkq7cMp+SA/QGr8BQRil+eBcphKqhOh9Efpng0SOLgG4HylfAIAaVx4pkNRCWyBDbGi0BT6ejMoH49uU1qPMLv+IFR0cvoup8bee5DYiE8okQREZVu6wIfGD1YQ3TjC88tj3flw/O2N8tfSYMYakI22Eqn0B4SG4N+9rILmLqrRGtpLHICfpMIJ4V9bKi4e81djsJuxW6XCJpp7Skp5wtsakvRgZDnvtmGLaAJsjsGLWYP5mlKBlkAGgK5/mcu13XiE6VgU6d8LYFHJdnvqFYgC1B7t+vM3El6z0OivgW1B9h2wiVY/oqp/RD7ifoQK+Iq78c1e0ZKVX6XAmocFWekJ0u+n72VjpPNerOR8uSzdYhbSrhO5kH3Panv/CQZszz9fJCSmCzsIhK2/x4Sg3RcRWbsBF9pqbZU0g/I4BKWkxgo5kU8vUeV3NY8vBy3NHN8PdkFFVNYVrfLkaagNQ4iNRXJVsh1k6Tpy0JaSsHUZqfwbm6tbeULZ6kuK0YaiM3oDKCNA1q14GlZz/ereSdh9LwJTVANz+ouYGMxgNZzQI1Og7eScv9fA91V4DRO4A/hnVPWnROTrAH5SRP4zAL8LH3M/Qs6C+PqAgI/71pCrpIplq5cQatJ69CMrrgulU0BAjSRAga/XSo+bSbVJL0iQufABbAA2CIbZ9WJH3op6jBhXHvMF+kKhue49f8fLLoTf8kssKBIdmZR6Lie9kuB/ESCw4bcEBswQuNVKpYTlse3xdvqUw34TeEuTCaELia/H2JjkvBKI8zeUlxjPFX9LaPsrUiwgdlcihURE0K60RoYaC5DU+DOoxlQjdVNTEcqAAcWQbKuPnJvXd7XS/UpUeM1qwz8P30ZMVf9fEflp2NZiH3U/QqbeofVV2FFVfMAgtwqnv1sn0StAbqH+TFyEnLCWZODw0Ca4LKDgw2BKW6TwDwh8vX5sEGwp/CHgsd33iKW/NxsDENOBBwEBqPmdFVxor7IhFWoDemKsu6IWA6VDLm+9AsG5ZKk8xAU06ooZG6oOI02yNNwAGcIB/r0ovbHDFPjGCHTZk7Anlwf2ECkC1LaK4z0HYsvx0tiFAtaUwhQIm51+uziPj5pDuIR/TEvXxMSA+CItw6YjT0A3F2Q5xyzR3hmgrhQWcC6TB+EL+QRE5LsA/HoA/w1+ifsRCm1D9iu+/i1Lggn6aeHQxzv+lBZKMZfebDsbKNZgD1br6Ga0Zuz+GlVyaXymwdJ+LjCoY5/ltw0X9lE+gG3b3ZlUZkA4AVP3NDn2vFPDt/+LKcjleeSZAECLHbzIBE6t3eqgBJ8FU0hYvZYSoDP5SEeBxlO9t2LhGnSPwH1BxkUuuk8gdiImDc2r/XYZkvpvre9sPuvXKBr1/LfyDUKggE7IHNBNHIB7S6skSdbLiQlwwbyWBuALgICIfAuAfx/AP62q/8+5//KUFA6nFCltQ/a3fse36+oTsKGryQlKi3jj6Q1JT19dP3hOgB+uNL+fq0ZaHAoU5tfDldBL+yuHXzkBgQ1D9tTu27Zjj8E/mw0Htm6kMANo4w/3BzSNGUn3Si/tXwK/rgASk3ki7SsL4FVvXyg9187SWEAJbpkE9jsLsDdwFlYyHQrYPS+0uEDzx2rkZUll+DdWALsANG1bj7k/oJVBvCbJ9VvPijcS8cTK6kmO9x3c5rSxg3MgGYFv4woFMMeETOsd0NU0JUaiCZRL9bxe7lt4FQiIyA0GAP+2qv4HfvuXvB8hh9LMIABwxZAV4OIo1FSEMRN1v3G/BRz5Ps9nvQIC5hmyvBxsoEECdewwGIwNIjuG+HoBw6cG7zYAaHcTIBYWzdlk2+imgGsQ28snvNlOOTPpJThmDhR4hvAEPiC13/XfRU15TiWnyGaPw1oUfk+8nMknSF2toflKyLmqehyawBIomMroC/o1xDXwkf6A8g08ynvVp69CQGnj86aQ4zmTdkvOhO1VPQZkIn80Aaeegvoyl8hFZuj8GwCEDw4pcm//vwHgp1X1X6afYj9C4Lwf4e8UC78Zr9yPUPnPhdxm2fp/cQ++pxv9NlG/0Rv0X33BBIZ/KcQN0G5pYSdbAxGhczsRFgABwpNfU3xpr8BYASjXA7z5fIBabILXBmirzSxe+GACzAagPefxIJcQqHQQeU1GwFtsT/q7+p3KegGQ0qaahasI5oE8PgQi2PfXdfozHe7Zz70DLo8Xf6o+g3D6OaWb0lQauMS7jQ2J+61OuvbJkl7S3vwPrTyJQbwgzHo6qXb7RcJrmMBvAfA7APwPIvLf+b1/Hh9zP0KvqAyCXLIpxt2kQ0ukaYXVWRSQbO1uVU31gTZYCMtjAQo+aEMPH14KmhcvNv1UKI22n5gA0wTffA/UUJJWjvxgrm7jFW/rCoQQWaMZKdlijiOUcKVtWYVpz4VmjnJRtbRolA9pYWr89YeHWrTUPl8HWyqqmp9eX3EmEBvFdEcvxyjtM4a90s4rSgKVOPejUIxNYKbifj9wHHccx4F5mGlwHLUJyYkVBdJHPr3NtS3ls+gcYqdmeoe3iaFiKxRP8XUKrTzG8s0OhpUF7tbtadQX6u06vKZ34M+g1ziHj7IfocI8tRXEaFLYmpPKHk59U961Vqitlo/qkFpAIFsN3W+v0nMhOALogeyZy7QM9e5LMUoVIxrT5hXUsGG36XO14QHuvsuKJTnM36b6DMpVa9a7V0AAioPvZ+k0DR0acAGEqyAtUoQp0p22DWoaGwnB0ai3jHNlV5cfbmZZWQNkBiznLVBZqNrGo8f9wHFMO+ediFJD06ter+aTsoZjvUIBBPXteEmRSAT1BWitHUzECEnV4V2DV8JPecIFCBDnq89e5P1BeDMjBicluNOqIDilOePfCD2r3OzWI7IS27pWCsTqs8sql6HeENtttS76qW5QeeG73Qcp7S7tBaEI4ts8aw0IKmpRa2rNlfs10M9Gv5YK6UnvUtMsOy+DeD/jAN3rsvmgwKsseRJSaHs9P96St9AFFdTGBicwYBZg/1TREIVPwfEPtQTUtapingR/Nr9AgS6lOZVF8xgtx6hTzbQhwMD7F2UO8xP4DsZnBkCAz3lEFtfCBCKHjwr9OrwJEFCgmQMyqnBJtzZap1ctMwQ2AYWfWTROPhIQXTQv9VfeUqeeLkwehQw1QRWxHYaHxSNt6vAq/KMdrRKFBB+piSekmQYAWqPsHuwoG6EzrRJkOhtgoKQxIgFsDoSGUZxKu6t3yYdUeRC3ZIPsRM3zkk4UfTxE+5SzXm+ZjCgLR8YGAvlJVgT2zP0eTMDAIEyBAATM2hOwciuNiUKZGUTcVbaKiMNDTI+HrUg9RW3w08kEmMhBRpFHMAhYuWX9ZV2+XJZreBMgsNI2nSDqrU6zadx7ahDQdfJCipiEnSuOACFXxsnf48PqDTrs1oWxCty+ti6iEa/OUBIOAOFE8iq3s9jk0hlDFYMBosL9ACWTKdarhiMN4G9kIlN7gD3zDyh6xql0/EAoCayCoRao62N0M+Yt6OmtuH7p+1R/4KQXYDJQ5hcoEfaYmwD+N4/DNih1NhDlkkWy5Ea8nrPrMoEwqqYcqHG0b8dQYGswIlJmAJsEfB4pb+esFKpdxH+vBYO3AQI4mwPVrEqzn7RBC8uvjPwZT5yW9tdA8LUhq5zYtfZXc4QXRmlkSYpPLAACW6JaKgKWGVbAQaFJW7E8NgbAotKyupZUNQgGgBgz0OOksiMp7jGWxitmpS1OpBPyui0WVa6xAa8DggYpl0ygefnjYxEf5S18AiX4PF5gopQTIXJ+lco2E8Ltz9Mzkb0XqtOdytMcyjoBtYVOrx2CDGyJ+Zfl5fBSSdRH5XcObwIEFFc+gRiIq5DpA07GheLh61V7tUshwXN4WUyB01JkWhZ0IwoZvxV9LB5l24bnBUxz+/RhKTaQDkIUE8gKn4oZ69mjZpvVJwtwSmmzhPtw2qWQrgC0eZPpr1PpAs+KQ7MwQoybGRa9EWtd8BJtoS1Rgsq+j7UB65KDkxg2jRjaMz5CYhPCoYA6E5j3YAQOBIczgVldMcJtixhlYjq3Qa160xx/YF2ZlnBaAxIKHRNzCsbqF0DPR8IRmQVRj90kuAbeR+FNgAAA84DHebFoH6HnZTdRtc9toiuH+r21XsmKi0ZcA2yEfo802AeEXmkLcEabV18sBDGo1CYLMf3PJcZChFjmZswpmwYI0R2qRhySYWSBVCVzb0KENjAINHqPxKjpkUsNxI0qysu/nQUQpEZwmuvfBkwAvSuPG2+BANX+CcsrtecfXmICKxuIY8Y2pzsGq3uQ/yQLun/vusFxyWrWT4BBjquYYcdHWrwMOa8MAAFiASoRPwk6OzTnVJ8DEebHh8PbAAHtIGCL5HihTjEDWc1uiva4vn/F0LomsquV+Odzy80ECJq4wi+LhsAP32Nwy78hNV04Vo01rV4aYtocUqi6196YobEAmZhjw/Btr6fvItzSqnVeSZeeRqDmO/jDLGx5z73hbQDQnCSM7a0Wf1uWS6QDQ9zzI/GITEv+cWW1+l1rC+3BBigkPMVq/FvJlkizprZeNTDq3WABrE84dRpFrtlJwmXGYq7Q7BK0OQi++lAMKONVohm8pzmeO0MoUDiOA2NsZNpsCWSvCW8DBKDmDYsrRTrRw/miw8wCgBr2onCqrT62htIOpdZc1TWc/UZVS05lLrQOSm/H4RODbBdimhYsO4ZPHbZIxcHOdxqSsBEBzGlCLtMXihyQMWvNuVhKnBp9sZoIQg0ILpDdHAAJQpZGCrzWAJlJjizXrgkYBCbnpcAqHSH8de6VGaxKuvAHGEirWG8DJEoXh8pbo8QlKCz4DAg5a5C7BGeVBxTpSEUI+cIMshVRORHZuIAva1djxJ8vOR4rDDMAhMKQ6XUSYM2ADdtzYgxsx4b7fYeMA5s7O18T3gYIrExA6ETUKHKYAtSISivHS4umi7gZv+k9jdrLiGdS+4gh13gLmjuK9gcAiE8MGuIThHyqMIINZG8AqCECmDMB5pAJ5BbmtflEnqfABRSxKROaRfwdB5wRGgjVMrOxliBMAoEZQ2ndQRYNsTQsSquSVovyqfPrnZCjclmW6nwxK/wbBQ7+WQdAhWa1nh1qkU4S+gUQLM9aQLBMIgqhFyq3KxYQpoZw2UTbo8uoo1pElIeT05qLCQRq7DHWOvC6OWYB9lTNFYzv+47tODDcxPlqgQBw4oOaA3EEYqoT6YG74vQNnlnTLfdKIfTrR7GI2hDm+GS2SPvLKcKxOlAuG7YBBAAWgbTGWeBj+YRMW/fuAgQGzx1ANah415gEAEyoL9rZPW2Lo4w1Zq6o47al8oi5B/6CLAYHxXYukBgNRwDWQaB6XxgEkgl4fPCqV63bWUcte1oRsTlAAl8MIQBQsyxWEAmju9rKSvcBbl+i1NZY+D2NAlofIwDTFxUNRiChaLJZlMaL+sk5EbQi0jF3jNbFeXwVzYHOBFLrquYIPN4YpIBgsdWbbum8oP9aNcQYcQkGMZsxplr5YKBgA4KRgr9JLBiyOwhsyJ6A8KRPcoSF1qLk5xZSLjy1K00XpqDXIXQ61NIRwjVGLqltn2KhKK1ujeWAzpkj5+JcSdvotGlaoZVMjFdzoM5rIdHaTmvk2ogOAlWkSIazmBIiYZJJUfMgdQHIpOkf2/cLCDwAt/DopzmwNAhuL8lptFpcMU9+z9qvRC+Rs4AE+FxDsquFAJmZ5T5pQFONdNzud8jYsB1HMQEf/PSa8DZAQAEcVXAaS/ORksSQ7B3INrMO5c3IuCI0z5nMlW273utgECBgQBCJEsSagWEOhFNwkx2b7MDYEPMDNHsHfAowaZx1kcsQllym2lcXjoVGBzWiEJIhI1fZATbbMpsGmohnam38COGe3DXmWsQbEXdz5XFOp+jd/8DnmV4CNN1GVpb6s1VLqxnhgJikKyoaDv4h4FF35TQDlS8u7zsgTDYHaGUh/6vGV01rNQfab5Ggq3YUsWWdeT0TEAjUgUAXgFGqH6ubu9fTcRzY7gfGuLeBTzEX4jXhTYBA2psexD3lAhf8oTUu39iR/RaM4CJGivl0P9hePAEd/S1+RZBrHajmjdTGsjgGa+2ADTxgSAPjFeSN52mtbocSCIwxTJDnwBzmNNSk11omwvCiGcOGukKgI4YuUqaSJVujRwp4MADvKtPOCspXQH4DFAj0qbUGljo2iEzoNqC6IdYjzP0RyCwIZgARDPHhouLm0Qj7uOpboi6EXKVhYk0ChxXwQOcnRrAyid5mqiSXBkfPMQa0NhRvpqYP/031CowRvyx8lRhMMwVymPOR3ZwxE/KYBeavCW8CBMLNltcpYNydUk4nejFIpDcQpRhDyztEd/XejcugEr5FmKS2Cs/tbo4/XgZsbDb/f7v5MuG1N+C+3SCyuda3ap0xRFgBDBf4abMQB8wDXKvmUqMPoPOjaXbKFxCSTS2vjw9oHJaeC3qPRTASIJbptG0tvoiRfAJ57uAzZORcoCGjplN4mUf98G7QKuK7KXnVTAniZTWqZFtTFVYfOk5CzubAmk9+pmvw+jfnWcFXAQopdxB1SkegMqlLz9dhWHgBs5hsk8FYop5QQ41Tw88D98P+pt+7H3fIfWC/33G/P2NsA/f7gW274zXhbYCACG4b7UU4yHbi7al4Q1I+tnOi/7n6sFbjC22cpyR0sZ8fTfKJBUG3/VbntBDIvj9h92XCY6/A/XYDZEBVOhCgJgrVSDLq+nEKWjvTxsYc1J9MpgBf1/ZV5Ddg00mjrNEaX/O6hwJOOm4Px3loqwmhkclLDwHHGfFV1aBmaVKCtCidPUIUj2d1el2Gvox/q0JJ4FVb915o0xMwJCiW8AmCNdg3ZuLtYmLwoJw2QIcYqNK5LMucM9s6Dv++A4ZO/7P47scd9/vdTIB7p/7b/W69A/d7nm++aO1rwpsAAQDYBoMAykHUNmbESfCVzoFoJCH0So0lBD7eo0aXgxLqmAOBfOFPXgtw8+XBUvv7kZkAXAva+CbxGYLBCoLi13JWYRqkwLkTbd2SSrA4CLNspAkvQZvnWRv4MV8N4U+BpvOk+TloKpiSprQXAFT5LqQZJYgCSPli1AWeVyVOgZYazyGeUIGnXVb2VwLdFlMODZ9/i1+A/AXcjrI1kdY/sYi2SvGEHj7ngOMhIBbUCsexqlH2xmgsNRxxaJ3TRKfwAzAzqOt7O7/fv2pMYOdtyEj7OxCAGh8LPjOqSwbQjtw6eQ236uZLAIgFQsfmwl3bgO/7rYPAtmO78VbhN2/AwQBQrABwwYcLfw0EmVoed7a30zwKgY/7qHLpYEHvsVXAwr8wAi920voAPM0R7/QBW7YQpkUi9HIjaFnUIX2nhNDTpNd99iawMIKsSc+Taou1bAEH/1XTM+UPgeap05m24ItFzTMu3pxkTszj3px187BeFmavvJ/g8B6b8PR3RnCkKaHzaACgOnE/DmMAczorCACYpf3vd2z3Z9uV+G7L070mvA0QgGAfZ3NgBYPglinUjPiN8mf1eSuJN6JZMk9GCX/7i12Cduy3sP333BhkxHluFW7rA8aiobYPogu98jkIDO5ZcwAAIABJREFUAMjeThDwEiHN2gbh0L0oOyufcQEEXdaDIbH8xcYujUXEAqJDIBMYQzDVOkQhiild+Kse65axti78pf/jaQLn0P5TcqEWdmyywZDnxBTYHDj5SAgQeC2/NAPYhJAAgHqvxlAQAwjNvFB0nZM2IJXWOwJUlywP7bU/cV+MgYCl80jGcD/utu4BgcB0EDjud9wDBJ7vsO3s7hjjGa8Jr9mG7DMAfxrAO3/+T6rqHxCR7wbwYwC+DcBPAfgdqvpeRN7Bti37DQD+LwDfr6p/8QPfwMY+AcFjEACagBuwBwOoCsyKlNQfHjmZBSFQY/PGv+wN6LZVrAS8bTtut6em8c002Orc/QW6MgFYm6suQpxAIAfhpKDz1XKeIOD3r5iAP5CTilL4SWgu66NKpzP/8NNcvJqgGsDLcWn9Rgk5u/d8tmiYCErABUQFU3SdB64OPrtdWl4DLAgAqsy1Py/1Po8fiF4U08L3ouB309RzTtpxeGDb7N0YB2DCPWrCTwLMSDNjOhhYj4yP4Thmav6YI1BdgXeMY2R6xjFwP+7Yjo/HBD4H8FtV9a+JLT3+Z0TkPwHwewH8K6r6YyLyrwP4IdiWYz8E4C+r6q8VkR8A8IcAfP9LHxAIbsQEkPQfxFGtYUcdwu01heY4dL/rR6D6+LVa9sICAO6m2VL4h9g+gSb4t9wV+LY/mRPQTYPmMNyj12AHxPwAZQ7weZkDCQJAHzCFJksVruW2TIYcelpZjTUKmAXEuvZlAhQjCGHHEFsbP0yyaSxtTF1Mq7U2iQ1wwhd/QGlwejBMgBByocqlzGt+ya4S01ZGEILOZsCFOUB9C5Z+Bw0zLWpAldnk9xT6+/Ozeejvz7g/3zHngT13jrLu0W3fAKgxqmW0n87Ru2jnYUc92vXhPQLHsZzPie2+Q4axgG27m/J6fsZGvSwvhdcsNKoA/ppf3vxPAfxWAP+Y3/9RAP8CDAS+z88B4E8C+KMiIsoDAZYgAuxXPoFoDMEKRglTChKDQaSZTIACAxZ+Ohe4wNdy4EITgULwbzcHgdsTbrd35hvYategdcMQCPsD+j4B64AhoDOB1uhTsZGGS0W+gMaFwxAiplml5IvNgTiv98wmHwOYPoNTIL4qLjCmGCgIE/OHVQsm8oVAJW4ps3EnugB5TDHVKw/dVWgBDTn7qpHUb03YF39BgkTkpjkbyYE3J455T8fb8/3ZhN/BIADh2Hfc9oltbg5kFm/te0hDtMknUGM1HASOA3PeU+vfwxF4py7COXFs5gc4NusiNGbtG9a8Irx285ENwE8C+LUA/lUAfwHAX1HVcD/GVmMAbUOmqncR+asAfiWAv7TEmduQ/Y1f/zq2QdRlYQCgRh12s1FoL1wtrZbNUmq94RD4JhgJMj66jTcGlbre9926Ad0vsMfeANRtaOMHtjYEFJAc3zRFq5dAfElx44YJaEKC0J0dpOX8t64YlZQnOwPZk2/dk8MHE+kYqRlnRqtRLxCxiVR2HBhjYhJdVQYsSkoydTLfsksTVOaowVPhBIxrniw1fCt2zkeROO1MRENrB3AWyxKJsfcC0YnY9XdKthCrLRXMIRhT2roJUHLkHeSpj7UIwjygMfuHOwiD7k8RcgYKjjkxjsNynUO7lx6HXJKMcHsZehzrX8YzyTA/9twBVT0A/J0i8q0A/kMAf/vVY5HOF37jOHMbsr/tO75Dt41QawEABgE5CQ4czelDpOXjOoej0ph2eEGOJsC+XfgYfZfgvRyDW+4YXJuD1Lu1e3A091iqjBVT0NS6R4YMl1Y7ZwCgPGvtxFNHy7hhjRgIOfioTl8M1UcojgHuqVjPz38X2/4utd5GEbKJIYJcfi3AgJZhS4ekmzajgfZSII0WRHkWCMS9WNorhSPPBYco5hTMw6iD+DahqhOxNV4Aigm72d33497oOXfhjSkeJzDHMEAQsZ6EceA4AmyRadZt8zQfqKXNag4DokVJ9M6MHM4SQ7SFyiEA6zXhC/UOqOpfEZGfAPCbAXyriOzOBnirsdiG7OdEZAfwKwD83y9GLOJ2U10nCIBBALRKtqannQUi3l3BIAQ1B9Zs5QcY29bOB59nF6HvGEznY9vJn1Cbhg6xOQPhPxuitSiSAud+cjQwyKS3wu9mQYJfvmy6mBVYCk4y39CWkoIsQ1K4h45LoY/ybVS6knWJ+sy0Ws/GAgIrEJzGO8SMyJPZUZQ94TN9AH4/8stThEnDzjkxxBblONJ4s1rTGX6VAoCi5e4QnOUYDDpvC3wIxhw4RDDmgTlHMoEYzx8O05q6bQajMZKwZTi/nQGMmFg3a2pyMgLP20ebOyAi3w7g2QHgawD+Hpiz778E8A/Degh+EH0bsh8E8F/57//FS/4Az163XxL96zz/ojGz8GusUVjvxVj0KBgT6i7cqf1z37/txApisFANFa7zAJKYFy5JX1dbTHLTyeGa2Zp0afAzVUK2+2r+CwCwOQCmykvpisI2QPHRcGpAAFXaKfdK+EPw1+OL1ZllnuMY/J6kGeDp0tiAJf5c8FuDr/eb4CudLyDA5gCgmBLAZ845nWUOHNF34xvE6BzAmG4qxCeKHeXgnFmOuTADjnkU5Xe2lUKukmaBCIwNZEUbSGXJ+F6HARSVEjIDBjB8un2w22wiqojNVl8TXsMEvhPAj7pfYAD4E6r6H4vInwfwYyLyLwL4c7D9CuHHf0tEfgbGAH7gg18Qwbaz4HTBX51cqsi93jUAPChxCn6cW7yhubexYexbOvGii48pfQOHsSXtZ1CIxSBAKCwLIkezVWgOtVXgtMb+ozX3r9nAKpRIoXSSUVsoqOby5zxCLo9NWFDCH+eraeLfWvTToxOEScSg0IR+nVwFGh8h0csQmZHUkMlIMteU7uW3BLWYAJWj9Uy727oHANT3B7Z52LTKT5TRTO2avQPcRdhWLR44jsMUtXv/RQwQkg4G+5reY3CMbLq23Fh0khUjtqLwVad0uAM1ppnHylNlDsj8SExAVf97AL/+4v7PAviNF/d/EbUv4auCCPo45xCqBQwCBKYDvkw/F7iWc6EnoQxACG++DfKpTT8LBDbv3+1AcAKHjbsQCWwyrdHYWX401kfp91cpJ/Bot/NO1/4ssFT+9Gb8Hu8sGhPnOFbG8QCf1iRfZKjpLyoLOf+5eVDjI4otpG8DLuBE87Ee45kFEHjzUdVB5gBxEx+dN8fhx/i2A4B3E4Y/4DR0NxyDhw0WGnOYX8A9/w0E0vxSG4l5TBxbLTCyDdvjEsN6abg8ILaonSkTawex3kRW4LQJaTg+VHsW3syIweYYvBD+/KNO99SC2aZL8BMAXDvb7L+bD/t1B5+fmzlA5oIL+zZianAAgmR3YjgWM73A+dobZoGC9scgYLkRkqiuQSk2/QAQNC1oohNKNIZd534L7XnkV06afk0XpVvQGxqn+Gw0BGUNBhATtZwFXLKI+KeEndnMCgIMlPFM9MYohtNzd44KbIguBuYcGJvZ8jprTYOKk7rzjuoRmJPPZzMH5pScKzD8aJ0Q4YfxlaTmwJjig4ssrxsGZqwTkf4Sp3kiiEVKbCn+8pskV3Lge014EyCAB0zgCgws/9YzML1t5KicpOQ1DDhAwIb7PiUA3G428ed2u2GLhRrTBIjZgzycmHoWiG0AqXioGca/ocHsumnEpL15kSBwBQDiDacofAeAVcufWQI5Ty/Ec62Pqgo5HU9gVYiQgsyE5OxCEG/YHQSueEfEz07JmJpb+S4wyLxmPmOSVvT3S7KBGJkHqA3Vnbays26CETa7lECdmcC9TenN3oExsLljcHMv/ZRak0HEgMCYiLXpMQa2MWC7E9X4giFbY1HhJ2twmf4Ty7JhzVdsyXFVxf14T3dY8IvSY/goPOUpudQgm7DW4h4yJOm/2fQu7BtT/pqyG9+mmfuRUMQEFvF0s9msIMHMv2a55vmqYVPoS57qX3qsOUXj2tNWABFA4F/LNF41ivO9uNMEv6WxpY6IwQX91PULq8AHA+ggkGAiod8ir5FfTq2PCUnnofgAKR96LASAVKHZlTZtgs79fsfz83s835/x/vP3eP/e/p6fn+3vbjPzYnhwlHMIMfYNYw7s2c7ct0CjONPnQcBZlV5KTHwFYnNKj/ydyz+UooEDdaMHUMhFfVyENwECgOJ+0GQHEn7LSK03Zgtz8NEEMxyAiYok2OzUa0cHiGAMjYFYbGcgABqFZhaQND2vOwB0q9tj7ZBep+120GgScmIEaEdiAMrknlXz6c75o5HKaGSevjNLYRCoZy91UMwboDIuq9wHvjTjp+qha/sozRVeqIxLTkKpJng0D0ICwZFAcE+h99GA9xgTcM8eAR7IA7flBRtU1E3JcC6GYqECzgPBQYJEdAG6EvP2m7KQDJJZQWeSdf4VAoETEyDBXwW07MhB50A0WBliGl4GJDz5Ul172x4swLv4YoAPmQ/1raY4vDGRzayl8btQEmVvDY6FEg/qiCpSl2fi/SsA0P6dayYArILzwZCCj2xsBQp0no8LTgXXn/DIeteh1Z9kfLZKUY+o2//X+QhoWT0dUZ45fymZlDOBu43/f//8jOf37/H5558TEzA2YGBgPQKeoMyz+YgsvfvG/qXYO4KEPGl9lYfJM6/LGD1TO8YeE9z4b5SwrxpfTicvhrcDAvcFBJpA1hz//IMtzWXnG7UtR99tYNuGd+/FZJ8yCcrj78N8yebPGpIgokJp9SRqNcXmqENRdO62audXZXDSalfUuqgwg8H5yEC1xPz4Ij6cv2QaFsHntHX/gFy0O1muAgBMaEwg+rJuxuq9sbtTL7hAhzNFmxEp/B36olS9taDmL4gZgc/PZg68f/8en78vEHj//Iy7A0FMGZbMewj3SLNpc5Nzo2nENeahgwFShi2uwQyW56SIJGvlodQyeH3Ms4J6TXgbIADFfZY5kBoZAlu22zbmsPsbYsZbuNrEh48GdvDOLt0E6N19MqpAWfBFeOx4kwo/d2/zhWZO4U8WoGC7ncFhLYNXFFTxiuaPeKT9H7eE1zUQRcm19HOwAnrMDE6o0ATBurqsr36k/c7sjpnRygiSJLxO4TV2UgBOk4N8NZ7n+3P6AcIHcG/Ov8M0NrxPH5LUf4h0AFjmPqykKAyCaoIRj09FzvbqJm4oKzpfFcw6CvVD4W2AwIkJMDU/Uvsb2k6Ys4/Wo5SB4RMpYux5DA3OQUI5ToAdgzFrkPwPzS9gNcboakXteolBQIn+q14ARABCxMHx1Ve0fXBh1ZcafjUH6PzU/cf5uAjSH2Ft2pV8v2bGwCBxBRwdBGJvvvAH1AYc4j6gEn4uiZBkpd+XrEg9f3aQGQTEvIJjzrT9gwm8d3Pg+fkZ79McsL/jfmAMhWzuAxAhgaUdhUTIN1BNS1CsoJpZCX/5sby97nsxBBluaoxs5zaBTpfZqY845zm8CRCAzsUnEEwgJuNseW8btiPQ8EYywnaMAl4AoHwBNlAoWUFO/tmaXcVj3nlTjN78XMi1a/2YzdiH4ZI5cAkEqy3P5cJfxglAGiughxrAfLAdkKnz6IkmQ49ZwCPGkBQfUb4m9DHbkgfEMAgkfW7MAOCJQ7Kk+pzWYonaKxFALfx5HAee73e8f/+c5sDze+speH5vwm9A4It7wjZ22aQ2idl2H1sSOwvRak88kapjZIBioUT2WC1D1gctVmLt156NCVG8V6ENcPoKgYACOCYtihi0HxPQsPeNAUAGhgpkThsxlTHAZbfbSwEIjNC1etBK1aySrjX/orOJ+qewk5AzGq9j8U9Dc4F271Q4cUq0L347AUEeHpsCWVg9tlcGaYImTejX3+O8g0HQf95rPljeI25ftRAxGfCTg/6FQA7bVicxecepfrKBbg7ESr/WM3D3iUdSgOvtriamSZoJaxtz8n9pHpg14O+NYAWLObst41m2YbKg6qNpbUv1OX3h0leENwECCC3qIURRo9uP7qYjiWbtSVCnQVt/kUCrAnMCkAmfmgHFYcM2zSitLwuqufl9JXlyvXICgMzDCQDqOigosAh0Mx0eiQHqeU/I4/N2soRFq15erc/yHW1zHR6ZAl34mZKHUNgQXkAwp2KMSb+tf3AzosCeNXx9g36m+zmunzfpmBPPz9YL8Pnnn+Pz9378/Bfz7/58d4eh+wZ4YdFhswNjfYYClZH8w/xWVkYGYgJIge7qPYm2GiAV605O359CR2+Ljhqo9d4mYqqajAmZH3FRkS87hEC1O17R4U2uP/YP9Ik9vEYgYny1xqIjsajE4feAMSdk6NJqFiBAVY6LQAFBCjvAbADtnr/R/AQXABAfoqSsXvh0+qG/cwaAeKzxJEg2Tw7d3DnXzPr0orpaSVn8zAQq/XFdTkFr3AYIcy7+ABQAsKnQ0nUBBudncVqKO0Dh+f1zgUACQF1HT4A5B2uegM4+JJinJ9toxtEAoKD9GuK1lXQ3NXNnKv/LHEcZsakVAABPw/gKmQOhPTNkt090FyEzXY4lnsZbY/x50pAJKprQqwwops0aj7HZwLk3IO5pr6RuCkS6Y20Dovmp/VcmYBHYgQwMDcob9Doacx2VEtDfyx/WQ5Pj1zWJB0EiDoKU9VuxmpPGCyWgqiUEhems7fvCKJfDlBvorEBAHvbFZAlBnrEWwP2OYx7Z/29/7+no3YGu+QM8amlw5IjB9S9aS/osU6FRIWZQggVNBdP8SU3RsDLy+CijknENCF5nCgBvBQSQytEv4IIQNld4Ur1BpBkQg33KHMiJPcLLfCvGnDjGgPhiGnMqkLYpo2sk4YoJLOdJ3UAaH2B2wKwAKN+AxUURgUEgnGKV9xCEzhquwGAtWL6Qi/PXQMPZfu0l9Pid1VFXQqo42cunc34HpEwbJOd3VudihFgH0BYEib7+u48D+BzvFwbwi84KkvrTHo22Wk8tGmrmQKxW5LMUUV3YWrWKLvSnAs3GVYqrTIIR7YjfdWZsQ6TjS25WLeXzUngzINB8Ap5ppjxWYN5jAAaAcTYHaGJKgPOEAkdsdpko4/YaF5mbA84+UnOTkCUqA1VpoeUXUwDoYNC0v1/X1bL5SHiLkx4zGFVqVl/iKSyauZ/H9UVY6fcJUGodx3MM13GuAl+Cvgr9BQi0RAQTKEZQXY/skNRaCPS4Zzff/X7g/fvFH/De/QG/aKBQ6w7E5COj5xAkAJTwEyOI3bKDBaj68nLSRoGey43aDsgvMAMIqN2FaRXtNXPrA+kwwL0oL4U3AQKlKS1cDgILdCffQPUx1z4BPMMvkVNLZFy6W/df0StuYgQMCoorCZ9r/c4GgAsAOB2vtLmljYeFxrmSwOjVe0v5XQug5Vvo/IPhxTbUoOzi0auXjQEAH9b6KxD0OBcgBZrwc7yA7+Pnw33N+18Ov+fn93h+X8f3ZB4keDPdVwPq3E6Mfwc9JzSvJZJKRV78wNsjql11IED/A1Ce2TIHcpZx3telTTwObwIEkiZ7iD38JMuv2MBpl6DmGORhxQKoUMGl6J4FO48LGFwKP5175FlZAQz526L9da0YXU5r8MwYk84lzRZkLur1DgcRViEv7d8dhCsrePT+SyEA7uWn2GF4dvitANDv1ZeWWnPhak5FPlfF/bk8/GHz34/7mQlQz8Dnn39OWeN6svENxQ4WxyAKOCRZAKmU1NoMAJotr/wAIJOA4yYm4KZibQnHUEK9KR8IbwQE0NwYIyk2YOvxRbde/dU+1zyj0Jv0jHkFvt03dcuk5gaoQFcB7xaVohp4DgHOuJBCzkN5zyAQzGHRndq/lDsO5wKmamUgeiGv1+J/Jdjl1a89mlo68j1c3P84oUBCL0Cg2/F8f00T+1IUQAw1ZiZgffX2RjCA+53NgXtzDN5j1uBzPQsQV4y0gup2/UsfgbeNOaCDHXtacSz5PDsZJ2KzWvXdq3UqxUftaw0iudfEa8KrQUBsjcH/FsD/rqrfKyLfjY+0DZlp7EpxTRUmAeR7at7+w/eMlmNCJOz9aBQzWYGhpZTWju/4P7kOHxm4quQbwBUQ2MMrEDAzCOHn8/AhYGnIEbmILUphmiQahIHBhVRgaU4X9+z+Sj/reBXX49aT7OyXgBGh+UNbOnvOONdv9OskzK2s7ZkObtPzWZo6FgQNj39s+nGcqX1ocmYoINMlWWrUca0anKXn3fdm3tpCJtBBayT43An1BVBP/gW/5v8Ws6OZIqjyyHb3ivBFmMA/BeCnAfxyv/5D+EjbkLGAAQ4CaptgSgAE35sOAAfTHx9tJt5PSj0KwGgDXBTrOWn+9MIzKJGuPoGCnn8L+39WhbHmuGQIDhAxldRWOxIM3bCprThzHgOPRV7Pvwv9qwijURZA4LLvJcTUnPN4+s4XsB6qu3BNqTYwOL8XdY0quxNwiTOn8glMEvq+T8A9Bw7NRutjBWJrT8mjXPjDNGXqH2aByNFKVOAsN+dGWLw5D0ZhCmpK+37tjaBpduhUzEG7F41aqpwBwEvr1fXx2h2IfjWAfwDAvwTg94qV7kfbhgywnXszeN0OF3rAdsQNJhBAEN0iignFgWyoDNVVcxl11ehyHlaULNdYhH+5l/E2sAgkR6I71GhdMogEhAIHEZuFNtWXqRoK+J5261BToXzlD/0RAjkXstzxhxnBoxAr82B5rj56ZgYfRoNY6WdlAVcAEPdWFrA6WptTUKwPKcCmVgE+byQ6Z80M1KXvP7r5oolIO4/ydequmqsKV6DMsBPUlVUoKhGBDmDqyG3KBjOB3ASm1kecOm3cizOF+lqHxteE1zKBPwzgnwPwdb/+lfiI25B9/Vt+2SJQpfmZBSRDcOZvNTFBO/1RHAALsX+UDixAxhhWAAh3fBN8/+fMJvr9biueAeHR3xhGETcFdAs+bPHGbjzxEfFz7g6L37jxlsaX5fxDwZ4tIKDvgIWWQeHDza8EXggAzvR1BYViAlrnyjUTZWDbvVlaNAV0/bMVguOamNojEZIwB6Sajz8egim+buCUCczwZyGHFifBAFp34ZxqG73S3odrr8TaVnJnJSqBLyb+Fl6z+cj3AvgFVf1JEfmeKo5TWHnZ1W91g7Yh+45v/zZtPoEJ2xMOSBNgJACIOf+JapsH9SAZIDQM6i1M6ap/dRX6QH8EQnMG1H5bM3O6Vvsntx7PNe/1VLlKtG8qeZ63iaFbNYBtM4aT+Tj3p2c2Ml0BGNE1F2MiAiIeA0EIaJZou17fo01UFlC4CqvgX/kDKq76Rml/135KRFhLQDOvXmm8e1AsChpdhrFycPkFqH6S+hdsBg4LHOgRdej7C+QbaoPasjQCON3EjbQqEDsKzSb8s/1V2pZBSrGiMDNdv34tHLyGCfwWAP+giPw2AJ/BfAJ/GB9xG7Kie5GRcABerCmoyMlRIVjD7aNA8ZmNpYCi0NsFyTcCiPt1ntWdLbIj7Wny6mVhq4ZDKgR8tmOtfR82n6YJoLpDVbGFbQpgg0LHSPBa+9mzubbKL4AI4Sx7NnL0SGBLM4dtHYL2Mt7XWIBH4UPU/3F6Fh8LMQIDbDN3sofA3kpnYM4WvNg3oLYLJzbAe7oTCmSbMSTIuo5vcqlUzURdDcSgJojtwCRim4QMWrKclUcDhUmTlWb4L+prX5wHvG7zkd8P4PdbBuR7APyzqvqPi8i/h4+0DZl9h5hAAED28xcQTN/Cy0wC9eWnpu02ywUWjUbLTIiNLjHMp8CTVQLimSGIe2+BMwDENR+4+hkEanRZOZAaACRdVesFyMBry/kX0gQo0AqxjjRRKlD9xQVoeRaC8o20nGYWiAuOXNL605tSQAA8BoUeWDgZEGaPpCaD5OrDfRtw8g/QrkFN87ZaLhOKBTxIUXil0jnoT4o4oxX45q8+vBcTMe5FbF8qGEjzWIPzQCQsYGAjCNdlxblQwc3xxfBLGSfw+/CxtiFDpy65LZUIcsYgbSgKni3oMwaZBDsbd6ZQBWWg7kIENGodDCBMgAQHoG1TrU2YikVgic/6dQnZZ+1Nz2yAgWJOxbaZBzmF3rfKbqPrPE08zbamP3PdxzvwMgJizAUznSv1UW2I412/wemKa3rnIlxR/8cA0G+G/Zu79jIwQFp8ca2qtSaA7xNg4wCod4B6CJqpJtHfHpwioLazku608zn9vrepTIE5rn31hABv30Qke7EmcpcjPYLqH5jzjjkFxyG+45X6kmVw4IuCpfKPOvgyQEBVfwLAT/j5R9uGDFir2zIsI1YH2nL7MLMwfLSgbA4IGwQbYuNHmeFBhe34EhtMKIBw0KWtS+iZwhHnDAAh/PQ7MQdmFSJiforjcAG37qk4X5kAHy1vPlhoHE7/ZgpAepW00hMe/1Bave7jyrVOmg4rcfU6eEkbR0PjW2RmscN1BaIWjWABDxSARs0sidFgAKs54L4Bi+M8qclAwAYD3Z9roFDMH+AtxXKwT2OUw3cOIh8MQAJun69dhVhra2uHUV75F/WgAh2S3z/mATl8kZL7ILUzoXMzUFEHCN0bey1FVGX8ofBmRgy2tnfBAGIF11hdKAQ/QUA2zCmpwjS2K4PkLrTpLtRYACQQvb7L9nSn/AQGpHLWFYrE057OnDlpUMpcNE55pYMdQGyL8Km2NRVTxNo+zGkvj26iQjzLcYETaATmFRNoQvVCXa1MikGgt70H19LjMIIm9P3yWjjvblqYtXEHjM4pp2oK+72NEeDpwc7YmH5D3QT1AT4ATAhjkq5g5CI12toXmwixFoAJbtVFMLNMaXMAHvYdZwKH935bGQXKl8+gfBRLHXzVQKBpjBAoWk9t2zbs+w0YGySZwG6A4GAwfLGHQw5TmNM0w0RtPhmUvFF1/6ySEDEYnAEgtH8AleQCkabF1aeduuAftH31caR2WHsNpio22Eq1c8Se9jZGAL51VnJndZMgsABnTs1XMbOyTIPmx6aXCBSzPBpKcDWdgIA1UP5L5oM9ujZW5hedCShVzAnBGeOlAAAgAElEQVQEMr3qzxQoJHNQpeHAz23YcBsvENOGGxBMW6tn+galvklojEszv9RAjexc/lzw50S+Jx7PEL8XZoFG78CBedgWZnLc7blskRNjbs4CNhzbhn3eiwGQeVhO7g+HNwECXegsBCUOANj2HfttN82P8A3siIlEwIapB+QAcKhPLSh7UcNRgNDAsYus7VAfjd/ly9udO9bIZ8AsIJaFljHMc6+2+COwoTaunLSkVQxVLcpZ/oHo1VAcm+1lN3X06arA/9/e24Zc123nQdeYa9/vSZN+pElNOJhiDIZoEdKGQBMqUg0VE6T+KdIgNNWA/qglgmATBX/4K/6xjSClUikWqq1GoyVIaon929gPS8WmxyY1mmPSpK0xBnve5957zeGP8XWNuda+7/t9z3vy3M85z3y4n7X22muvNb/GNa4x5pxjphlQdSeL8iMNmv8JIOWEinj/B3OAQSQFCSls9cgy6oVBIISZKSkxKQGbAWVGlAaj92abdBbQznE0ExpLcCZ2pXUB1+stASAjBs3VQaiUB3+TT06bU7IbTBm20egUjAACMAjA5/vbPRjAsAkMtkhuRn9yj386LgdkB6YIdtf+li9bKKe6Yc4NYx+YW+1O1OYwjHcMBA4pzQFnAxk63LR/OgfBjsLNnDEwu3qKIuZrC8xPAMC1Q9Hz3e12RW15HufhL0hbms8h0JjJ540+AECN5hnj2AsMlvh2Zo0wGFi+ROBj1pttkqkXIMfFvYOn5o/yVlIlwa0TxBoK075WN/YUafcezhMctT0/mukIBMwM/A1yPEqrzwKHpsXz/QwKUeYzEHBABQ/N1gzBG2n+nEZ8AgDWR8Jc8WFoN6FE1fwBKu6dt0hVsY7fLFACgYylaO09MdwYsHPvLd4fJPumCLBPyZmxUeYwG0YOJ+8nICC1n8YL0isBAfJu+2c2Byzs8obLw4P7AkrwwxwIn4BAsaOWccaSrik7wo2TiHvbsd+uS6hm6+gzaWdUZmz7NLLzjn1k+HIdGwC1zRBkSxCw+eq3HI8uENBGPUMTAMDcB+bcbBqpOiDo7Fo/eqkr+lVIWYCtRucBCOKbFQDSLKJn1h+SCWSHE1QH9Osp3E342RwoECizouzryEcbFvT8IY4SZfT2Y01O5l5FDr62lYQMCN03EI5kZ4Lwgb1YZDaDCQjmsFmCOkLYTYEENGAOYEx33roZIGJbko8IDRoT4Wr9AQSQHdgBUwLDfAUiA2Nu2HZjoPu2RM0eEatx+GjC8+mVgMCSAsnYKXiJ0QHS/ignIWQzUyC0hKihtM83yPoIB0xsM327knNujd0eKm9Qp63zsW3YHJW3zTuub7FePoE+OeW2306YgGsJF+x9bkkL+8wxL8Qsqo4oMtCEVFHCWxSdmQ2PURebaGCCFQBYMNGEm4GAtXsHAwaC2qjTro/2vqL0PT8p/FT4ZFOk0csJu5Pw35ZzYgXxW170RZVrS9rVhV/NnI/FPCKIsGJW30AsaFN3JkIkTQIJwbcxQ59SHP6D3evVlJmVb2CMHWO6o3wO7GOQOVraf8iweTBU18+lVwICRPnysx3ruxBupE0WBY3tmHYM1/w+f0Bjv3fF3AeGAHtoj8lgUAtAwjm0zxiy4w4r7XyM2NxkYG475nbB2AbmdsHUmTvY8m62oXUALDZkCEB13n0f/tvNl5pq1Y1SXS3naUfnjcVmcmgqAq9g7SgMCKD8sVAUcwjGVUuryzeguZ8g3IlJNivcH6DRjvFOpvcdEHr+KJ84iwVYgr0ygeO2YssKwshD5B1I5sQLsnu6L3D1SxRISC3vVnB5DTQUPtFIol3dXNRhPogxMGNfwtjvcNmj8B0DAYB6sjdEOMP8zz9LECgBxhbOudhOLEKOTMiuHlR0wxTNyRZWLS5sseGEd4TbXo67fd9xmxwL/wgEvKVZmCxxTaEOLrVYJTpqmdUkwK69p0qyBxHB7RYCTEFRSSBWAKhP/F/luZhMAcJ9+x3OHiKLCxj74zVNL7hp8pRpEHUY6w2ifu8xj+mAs/gBGARyQRAtDSYTLMKF7bHPILGBdcJQ+BKMlCkBnfgsYgVW4ZI0shLkqFU8qzEJzZ/pzzHvjFSwUPV1MOmgmpg56rQK+3Fz0nfYJxD0Kc67JmAwCKo/hmCLzUdz92FF7lUoEXlVMXfFPgauqXHKL5Be4n337amt49j5LYUewhVvtnUHgdozbtsuUNB2UL4uIEcCsgP3DgMgx4f33Rr0egvKXsFGQsMfqfJZKvo9shy8A9MyzwEFDNEacewz0zsQualcproLRsxsTI2vUnWJOg9BymWz0UaNCSgBQuRBc2JWhBM3QK/Q4hE1aN9vdL7nvoLxm7Z+wN8VoJhol+AY1esAylW2IqRUvm1zEr/WeEUwKZ+LIOrKT2xl4nBzAovG5/p715lAH54qwc9lwkQN4dNqjQoJtk1sui1iTH2mWWB2tG9FFg2VtGtPNhCx6a/Xmlp6vd668GflmjDFBCYzCzaMfXdQ2LNMBgZ9FACgztU0b60VNyAIOw8AapEMwnzIcxbO6JTVASL/8xCcdWDIzB1uEc4kGam9vEFSK1o+CIiW9gsdnSvl0kSISELSVtEBZSZo1Nk6bJrDdvQGYgcGArfcRmyftaVYhRa7ZhvvPmWYTQaexBXmABy8gCb/La3Xggi06wpoTOjwc6Xz6AdmKiiCz45pzkd4fSZo+roXSPhixhEEpKHSk+mVgMBC7zCpg5NZ4F5bgZVvJABsuDwY7S9/a+xlaOGebsO3j/Y3rM7B2223SLOPV1z3W0aeLeStiUEpQJtp/WQC7hPYfOy2kVetT80hRtrQphvDOqcQ7dVapVZxCQoUy3cBlP0POobAV/6Hr7sYY1jkouE7O2P4BqHDlRO1jZtbTJXje/ZFAOrh3KTnKf0C8O/YHLC88qKqHlCDfENsHihsGPC2O803YQ8fwE4MoZ1PP96OIwTNB2KZdQYTGvuYRPK/AwIYKNpXCqW4lzQm5nQq6lHURgZyFGd2cC8lEv0zgGHdyen59CpAwLsyfXb0R6wGZP+AM4ERIDBwuQw8XDbsCQIR581BQDQ3ikxzgEBgj4kkj4948/iI6/XmO9NeASC1f+4NH7MEBwt+9wnkcEQZ13lcKVvYeQPDV57t2HeUk0wn9n0zxXSwmd1uRhe4VcB4I1Y7D2fjBuhm+zxs1vnUWYBEhwQfHQASIDoolfAYJwmNr25clw9iBSuH53TaxkSq2UGgmUDuR5l77i1gkYT3totQLh9OJ23sLEQAsNPoQDgHo8BJ350ZBJuLJo16p+bmvp1VFYxWDt8666gyJutJ9ktv5Hek8C+mwbsGAsCJOYDq4HwekGFMIHwCgu3i+w/OAejmwjNsnFaH7xMfjaS1vDSG7mL4yNnA4+Ojh50mAQotGo6abcO27b4t9Z6mwbjEtmiAMQmklhDXsib0EVNQzHSBTSeWCJ9GXGJMA7ZJgr8KXqOCEJ+9WHmOckTgUhtCJZtf4Nu0UcdT+i7oqsDnw5NtrosGBXm/nUunGZBHOEAsTCBGbggQciJP9pV6V8y/uO019BcjAn0uAM0SnXu73gEni41YmiwBAHfkajXBel+uarKH19Lrtc8XsHIZqZ4PZ/D5AOQMDGX3boHAiTmQJgA5BtV9BG7uSADANnC5bOa40jABfKhwDgC+l3s4cda5AtcrbrdHYwJv3uDx8REfvnmDDz+02PO1TTRtf5Y+AWMCY2ECMX6bGn+QWbH54iDfMl2NAwBjYCgwd6QJMMa08WEJTcmhsGI14mwAsDr7CgA2Op8Z0NTSliaWafYYPVC3Z3EAAEFpzWJs3nFd26t3xhB8uOCDbdxgMYocqo0pv908KMHgCUU5FyA2Drldc4ux2632EDRGxecUVWjvw4QxlJcMBUTNSXiTlvv/4hrdG4u6eEArmTWrpmfzl/4SIDwjxRPKZBESfGYDL0mvBAS6K8UaVrHfJq7jBpFHzAlbHHRTXB5uuD7ueHhzw8PDAx4eHvHwwWM6h3bffHLf6/xzH34On3vzId48vjHb/3bDNWbxqYehptVk8QfAg0EoZE7T5NOAwVaQ7GDrLjR1zHa0BSOKkV5etb1j3VGmQyFDbX45mx3OJAYJtNUNAUBMbPGtsWrexHEoaTUH4vO2bXi4PNgkp4cHbwnNJdQiFttwxLPAeXHvvaIHwYg2jU5J5ykwfJ7tr40JcCCWU/aTppL7M+aOuduq0ZuDlCmFWCYcOwrXcO30BV01PyByJJmvI83u29yFA3PfKcAJypwLwTU5N4ViU85dCcTRggqksqsh0lrkVqkYRsh6xpVs0vR8eiUgANPckdSCcuz7xLjtuMoNU21W1tyBy23idlPcbhPXxxsuDzseHm9tfFjT1rthzh1v3jziMbaZ4skiGeijzxhkjZaaDgYIAwr4TDFbhSjwCZ65fXSsJxCPKKMaJoD1s9K4qO+nDXHO1Oago2Rn6mPopHkTBIgRMBDwegw/n5eLC8vFRlrEgnRimF9DMLAFmxDBNpzluHmQi6+i7qprOl07OffvWdPeBQEKv3ZwGMKON7G1+Pt2xW0M7GIuGdFihtAQ9r3AIAAgZwoCwUjLeRviLssRrsgNhCvmX9VHmgLuzDXmsnn0KKurTescTejp8wkEHKT8CVPlufRKQEAA3mZLgbl7A8Iiq4zbxO063fa+4nK5Yrs8YtsuuFw+wOXygFwiHMt1ZwWK+NyHn8OHbz7Eh073r75BZUScDRYQAT95rBiIoFAA5sTuUz91Fxdi6wwDwMiAoT5vYUyMbSQgqNoWa+qsYM5JS5GZIlNHRDjXFjpMvpN13PjgeJR+HpGLL7svULrY4qVNbEhybgOX6eaWCC4+6nHZLnhwn0eO5yNsaDo/AwBhge9sGRA3gUygoBGRSUtQo33oXHXiOgSiE/O2GQgMGw42L+vMPqDTlIPuNHFrssZ1s0dDpuLfGjOCIlkFE5iKubuwL4ylWItaANlhQ9kxKmOrA6dbCDEkHnsfEAgEO6P/028jWAYv5MWY8CpAIBA2kiqw7+qLZwS3m9qeg7JjbDe3vy/YxiXPx3Ypm6o1rJ2/efMhPvfhh+bwc7vxGlN5c1af2nnY2qHVYRTZR28hc5oDTabXf2hC2B6Cc8PYbERiuPNNt2IHKbQ+CUR2X4jiw3Ky1E1eTM1CmqPINxAsYJwJ/kJpHVjmfjFmo4pNBPsY2ATAtFEDwcA2gIdtw+VywQcPD/jg4cEorb84dBlwBABlABCjrGnekr8ry9UEf2UF/RjnmwigO27XCx62gZtExAQXoknaf3Y/QJuRSKgk9C/9FqC6SyDzNp0Tu4jvF9BNmDbXQS1ehAGomQLGCIIj2m+iT8WkYvOZmN2fcSPduZoOldZzTtjDnfTSzUd+BsCvwDjvTVW/VUS+CsCfAfD1AH4GwL+kqr8kZuz9EIDvAvAPAPx+Vf2rz76jmQOoCtx32OSgCWDHGDtkGBCMsWHIluehJdnhEoj6+PiIx6uZA7fblcyBYzCJYgJoPTUq36jgRLoFVKDDd51Sp8rxTzUfIcrRZCQDTIjAjxw5tuqCPwStLAppn3OeQbKAc8EHAQCc0m9O9fdtw9wvmNvwFY1W1k0E2xA8bAMfXDZ88PCAbYwS6lCKqeilljtLOQcZBFLoGRCYWp+BwD7BNn2cCxT7bgCwuUkjKHNA40i/SWZAgq/pE2BtS2YA1WHl2ZmrakQZobkG5NcIpsF9qsZjW5+NvwAA+460g9etBdl1QMAZELwsfRQm8M+oKm8g8v0AflxVf1BEvt8//yEA3wngG/3vt8N2Jfrtzz+emYBt/QwVWt7r67ljkosMyNj8PFYWOvonU6rz6/URj2/e4Hq9OhNwv8Asp+CuEzuZAjUkBUjEkdXIg0Jj+0OxBpkzpi0bAxhu/22bNU8EEjZHnjOBaNCk/khBTxDS/NQ7TR67T6BPHy3HXFrh4ZQTm5i0uea8jIF926CbQOcOmRMD6gCw4YPLBZ96eMCv+dQHOSGqUf0TwT8cwcITbav1OWbtzQUEOESbj+uH6bcBmPsVj5fNzJZhQT7CHGAmoH4+PZrQUWzKVyEEnnUcNtefTRr3J0w3GmP0oWJHFjudOjHH5uLt5gCMIUjYIgQCAcTFRkJhapqNqpFf7heyFuxu+nzMgX8RwO/08/8MFoD0D/n1P6kmQX9RRL5SRD6tqj9//1GLT2DafP851Sj6HlNDFbYWU+g3ktdE4HTYBTP66JCcSHK92aSgxxv5BLRMgqBvOQwHwtgAZJd+09yhhQ0YTOAmNp+As40NCmBTzWXGLPBp53l+NYXDtX2sNQitGUBnNVU1GGC0AkAwgbW+4wkPOy4+NfnhYkxgvw3ovhvbgZkJwQK+7IMH/JpPfQqXywVe4fY0ZwY5N9sFX9fzRfi5vOxkC1s+vOW619Jsm+pdC4UEiv12wYeXDZcxrDwgcyBZAAGAzxkIbpcTl7wy2z8G2Lw3GI21wZyaAV9rSJICywSYYbMlyJhQ3WxTGQC6ubnpgpwRLmONAflxMGZrQ3FzwPpoCf9LOcFLQUAB/A9iOfpjarsHfW0Itqr+vIh8jd+b25B5ii3KGggIbUP2a7/iy9G6qSI9w3Of2G9mr99usUbbGiEQMCimxNRg7/xjWOAGGdLWkvfRAUJspdGBpPNIFKC1IInAIbiQmZ3jMD67hzau8XyqBz+h4jebkpxLXjn98ZoAYK+IEQE40xhYs9PbAbjdHswU8OHUEJZwSgnURwYGLtuGh8uGh8slBT7mP2AIgYAJvS5sIJhACrxT6ZgenMKvCt0dDFShY2IfAp0Wf28TyenV83bDZTMA2Hw+yHF0gMCATAMLtOJtGkCgic2rQVAKKGu/TIhoo+OWZ85adFZb7AQ8aWF4WwYIeBnsO/NDhXwISXoYqj293DR4KQj8DlX9ORf0Py8if/OJe8/efPBSKG1D9jW/6au1CwZsGEsVU2I4beawmv0+n+N/Jo1TURuLOACIwBeUXJe48xVias+VfuVsI3XdG19wOGbuV2F2bcCf74NAoE10rqNfIj7a7fUZ2RHi95Fm/fwkVeyCWSspt5iGe8Ntu+H24BOqtg23xytul6uDbk2Cwhg+HFpAECDAgBD1wIJvQVcXEAhTIM/3Tqvj/uwIx7IlO4KNvsyhvlZiyzpKAQ8ApzYPFmUa1tvCh4aR4F8CGYJ8tu9hRTxSTJkYYvMZbJg5zEgqSGobdeZfwV9EajlXmiWoyEdPLHE4TS8CAVX9OT/+ooj8CGy/gV8Imi8inwbwi357bEMWibcou5sYyYxlDmhEcBGFDsGmNq12zhIOjc86U4ZSrgQRFaxWk0UMgVxQEkAQjCDNcKLRnSoGIed6XleZhWWXuxCjRhzCD5CPixo4NBwbI3EXAInyIzWGMSMSiPjpWB5HJwo4zQ4giAlWW0XfuXqk3gefinsxcACQ8R8NDBSy2TAob6yBQaZAzI1ADAWWOZDm15zAXtoaO5kDNCJQDj+gHGqhuavFjIQ4MxSf70D3Z60KDr9c2zMAh2Z048jMUI5LVwDNPzDMFzRHhCOfvs+mPXJotW91DQe8aGfS/I0FSFwBwl34kvSSDUm/AsBQ1V/x838OwL+P2m7sB3HchuzfEJE/DXMI/vLT/gDPPWvHfDeMzo4J24M0JtQYMreJKh6zPx8QG1H4Crt93nKjiVtq/76UtNPupWXXll7YgJzcw/TQRFmc3nXAewoQDkSB1XwGqPDOEBoiNAc0w2PDf2oH4gm6aC5nBO0vVtttlwRTEYFsPnylVtdDij7r8MCb04Uf8LBo3QcQjMC86+UIRIBEo+9af47WB+CrGjwAgPrS6Vg2XYM1aYH3BmZgd9uPRxOYmTXuRYBWAMfm3URus+eAMMIJzTkgpqfCTE/oM+Vm6VtPUkBKL2ECXwvgR/zhFwD/uar+mIj8JQD/pYh8L4D/E7Xr0H8PGx78KdgQ4b/y3AtC2OuCWghnrxx1jrPZLBtfgx2z5VATSLInqN9T1N6Wjrrg33YKLzVpmHCZ+iol6ckFGu93p9Kqe8IBlp1EfSahVNTj9igGAbdnw6EXPgRBBbr0N6dQi1NBVLgq+wJEJEj06SSEP8yB/XbDPjbs1xtulytuY3Nz4Ibb8F18Llcr8dx8BGRaO8GcoeIhtI0FeN2I2C47KOo/6dhAIJhA+ATINEhfCZsD2uukKREYAExiAluwsaD0oVWzXhamp3FvtedSkdXvFMgRAaXRAfevzGn7FAyt0aQxxfc18JBiMFD1ncy8O9WiowCAau/qkUuWX5ResiHp3wbwzSfX/z6A7zi5rgD+wAvfn4lRzRfV2SIb0TbNck5gyMyFxznssrvHNf/Fwgu7tk8S+ra0lGYMzlysbHmS8kEwIHT1b8djpSsihPRK6eN5DU+8DkSAOUbO1R+0GjHtROgBDDJoxQoAVE+Us7rNtX943PfLTqaA+wRc8G9jayAwYpGLe7iHC4pkI5rge4xuWKBtpJZsIBBDg/si9Hs5CZkVgMCafQO9lYoNDBFgDNvdWeF1RUOWfjwq+hLunuKaK54E/VgCTdo/AAGK4VOLLU4gMYE5T6U2FnDznojVnxaKtwDCJ2YO/GqlRmMkaFx4vqMhFZJeD9e2SSujol2jLysQOchnmySU5gCPDHC+PEPgI9/A9yE1fNiQYUWuDs0DqUCAgDg19IUmQ52JnminACzHB3jEH3FtwQVJLUaZULg5oLMxgrmtpgD7C8x5KDLgQdYzDxCf8hV+k0BzQ24TDMEBANhpVkITIwMFCvDzBACW4Cxcr1+bw+FHN5m2IZBps7uCasdjJDR+Vrac1h1cqKuNwzQJ7e/KqJmZ4Qfo5tAIXxHchSNAzQ6sdzP/KCiIgCf42OmVgABpV/QGq6267ThGRdJJBM5527EKcPd9A/aM7RfCb1GAS/j30IQ5swvlhGk2/goEgjXfjSG4ZjCWG5qBNPLabs4MZIh5r4HcbFl86jFT/VrWCuuICQTSOjHbq4fZcYCzgIl9lKDfbgO3ccPtcnNzwM2AseF6KSZgBkDVVwwHjuTazqR8vXsIHW/A2jZl9VGAsv/DJNCaO2D2XwbiXE2CahmpOg0mAPIDiIZFmWxiZvtIPXuprzaHg47redn+5L+AOruVnIsiUv4BwE2A1nZo760RhO4g5Kv546On+TS9DhBQo4ORrHOgRgIqCnSjfo2O5ZmWFl7/AOos3eaOyD6sAZJtxP2g34I7Gj2r3evj7LmDcOqMtOHrWZU/vdO588n5X9eCEe0nWZWzA6uO+x06nG02J2PHLruBwPWG69hwfbzhcbtiyMBlbLgM22Zt23dsD7utkYjzmwVVMX8Azx0YOVJQLKAcZ3MVfDIBEgSqE1g5XRMns4sRA8SwoPkAYrRCVCFTsItvHzZG5UUtqtKc5k+aCgcJbTXNAn0U/ggl37U/9UpqrLrOysGYCMG9sKkcJqZmOcP4NeagDibqE7OWznMnvQoQUFVcr9f8bA0TaOnaIxSAa3NeYinwKDpq9NNBPmQwabaI1BDNtM0+55yYI+iwjeFO9WMAk7dGr1ICAvDED6fCqpi+GzJEoTJzajGVHGzfmfkzqvOOkwVA629zGuNzjJC7stuZCHBCCkKMFuy7hWDffHLV4/VqJsDYsI0Nu4PA2G+2C9Plhu1mAVYkwquNkUODAQRmDsSoQMQLqFGCmCdQ53YsQKzRiOjk4eSdewgUTOiHBXmxziO+uYu4ghgQtdBzUxUSzkkQRfe1HClubBZUzSF8AdGXs8bdpxTEKK6tMxCRJu+xZ0QxTfCtvWM+oQQoIPJtOx9Bej6eS68CBKaqh/KyVGAvTtEKCOauhfpeYBmCoTbzS/y+AIYZ+7uNkZSfRwJqeMw75ebXtmInWZ+smBOmGQz8zOnemNXRpy+9rUVC540kUvHlK5jHOAGBeEbYATiiAH0ODWPn0aHspqzbaTEcbrJjGzuu1xuGDLwZj+afCN+VApfrBeOy5TZscS6bbc2Wwu+jAnkEUe/wAfBfUsA6r9GVMsDSVwqbCBZRomMOidn+FvlJRozHB9DVnI19Toyp2GXGajBgqu0UqILUA+x0Tc1P/SLZZm+L6h0OvKMDupyCNxF7/y+WE4kGiHk51b7jsOaYwBx68tzz9CpAQHXi8ZFBICZPSIKB+vUagy1KJWIbkWgCgdMktbjDU53+uW3NqwYTANJGJUDIYSimgziYCAwAcRw6LXb8sKXHsX11NH52lm6juGnSo/+sHQdkDR6hJFTPU5qAfyvZiXOegBgQXMcNMdtOYNQZLrwRZXlcfGusrUDAdml2wU8wgC+WQXViFp6YADaL9mlSQHvvcGo83O8gAgyIbTByu7lyCBotuQeEAcAgJ5wpkLDJ55iQXbCLGBB4zcwJ1PqQAAJe6cmUH+2YAkgTPcJEsVmWyO3DnpLWxt0U5sR0mz98C4M66PR3D32q/Xt6JSDQzYFYMdjOEwjoL7qxe6XVPehps4HsK5WK9jOGaelmFxYIKIOBVh6zrgN/Mh39BTqN3s9Zi42s00l2nJL/oJnq896PlDEWGIXmFjcDJH8bbw+QWXwN6GyAc5126bQFW8OBYL/tuGHgOowRhGkFwAKsbgNy23LmYGzTjm00AMhptkOqA4cmJTCweivvOdNAAbJetqwjWICOWTM/o53EV5mOzTYDDa+8HW3lnoi3jW/qEf0LMjNKsrVX1VT2BTYKtIwCa5xewYkDAQDRpiAMIJOB35X9SaPvIE0M9RPNjlnQ/u6ZA1Px5s1ju6Y53zXWg1Ut8XmtyPMlmRgZE7BWZJsNWN7/GE4kDUjaP+3VyZ0VeX9nA1iUrjfUUFuKG2aIiMUSjGcSohRwhSlN8QlpQVQ0fK03UBL2npHqtl3NZDex8afKr5Y/wPT/zXfL9eco2hZt2yitn/vhbcVcigmAVhJKTmXooMrUWvzrStgAACAASURBVHOWpxTaQ4BaHDQierRvwKnTmYCtAUkmsG22elNjMlNfIDanQnzdBMDbw1Vbt9Woi81OqiZ+QBzN6jZ/i8UfEIuvuLHifHlPtZs7KZf6iklF3P5TPdjuC9KrAAFdfAIiYfVFzdO5+Gru3FILuZEGN8vpOQl+0lGce6njHKhKL/oK6riBxN5MtJxUhgXniKXG4WwqEGAAcI0XPgEP9NGZAdVZ1JWpLi8paaxmkxJzEUkXQmwVBmPftmejz+21vigFeBO+ZdeO29VZQPgrHARAqwlr5WCcE1OLd1JBUoFqfSFaNw5YVOkxBJex5fk2bB2+rQuZ2T4RUNX2hRzZ5oP6wdQJ3AQie6tTqPmphuc/lw5z7Uc+k3USs/NliAUe1X4Z9Trak+5rj8/2877noG1drvpZNKxNSZs2wpUXToLUnKRXAwK325WuCCJWgKSAh4PM6TwAkQ0BDhFjX6FegQwAfmQQsBdbZ+AhKwYCBgvXMErsYO3EQCG0iECcARjizwxXtoKADUOVz0BGNwmwOJI0BUSyrN0L0OlrXElW4B2qDIZgRDZjErBty0X3Vj6eUFQh1Wsr7DwSAygmsIBAz15TwZLHYARWnxYwxAKfXDYL0zU3A9ncQUgDDsWjTVUE6GifKKVoMZ2xxVReA2/xCU6iLJ29lo/nKxPoAFBb2TE4cHstzRePJQYYGKOJ5JwvKZ/Akrun0qsBgSuBgFm6sXFjZwJjuA06NltY5Ig/M36+3aqkDQGGgnxpq6Sga+p0fMK9+zGbLNjzWASeKSQJjIEI+RlGjUyEKRBAEPkLW280IKgpxFkOrQ6XLIJKV+WiEiqBQMxbUGNRW7IO0u5e367s0pMu7t8wOi+ATLOpmTO7674BAQoMDr2T/RTUbMLXBAA2AlkvlNcXDycj3xXM0WaaxnAtsOVog23CohjTNmSJTV/t2H0DTVtHBrV/aPa/E4hiAcFynWedPW+tGrqc/UaizqJi4q4Chnn+uNP0KkAAsIkq7YrO0AOI0QFAMlz2GBtk2zLM2IjINtlzpH9ub+vnRe99kkjajfEoe1YgfMwNKLvsCAQcL5+nxvJsRwakTIKjYzBChLm2KVbzFAhoK6gq3dFAwLd3F8HmYdvifBs2LyA3VIl6H7VLMxm8dZ4M5SQdFOkdLRjCDgF8G/MYUyfpqr/4Dgx29Rw9vDefjnK8Rn3baJJOm7Y91zBwkYWcptkzX5q/HJiH+QGMdEsVsEgzzvBOaEQR1hp+ufR7ehUgEFQzP/u19W8C1gllYIwdYw9WYJ2zOiOQSJudBnk9U3TYcFIp0EwFredUJxRECOrIO5cjjkNhYaTUAY1mprXUsqMpXNmR0DtNE3wf6lJGo8zHwge0jglgWh11oMAgj4ehSt5ViQQvQFKAXjoSRu7VZ4U/Vka/3oR9Oef7Sb0SBizvUPpJAbwVpVYdngruoZySY/n5mU25FWCW7B/TiVCn8HveFc6WeXao9R2984in0qsAAaCi7wA+bdgF0qaMa3r2c3PQ2ARjMBMogU1NFTHwVnDgDhS56LKU9DwckUXPfXtyb8W14hMMYhEJ+xb8y8pDYZNwh8yOUp0T9KxiAzM/1/tXAAhgkwQD/kkKvXdYAwQpU4HBwJmASMt4PweA8x6+0GdO0q6Ehq76oTYl8MnvvD2XJmznMTy5pgR3GWUejGHzB9SZ2JRSBC6CvRhSbyFBDxPgHLe4zE2/I3ClHIT2bc0P8bUkZB/FFOd3kglAbTpwpN2987E70E7e+tBIQUvjfBBFbZNrltDbjMzVqSLJoZPGfbEjse3A4wAEdnSdMQNNrcwAEM8tIWfwiv5drZ8aC2ZWFBiEfyGYwcEgQFwMNhV5bCCA6uDugnWNGLsOOQgEAI7R8lesftXK3MhP9cylDU5+F3k6A4IAoRUAsojdMjo8d9XW4Q+I0YFgBkxC2P23EpwW7DUA4CQEfOYfWiM1lPi5KevgsvgduXr046VXAQIKc6JFyiAXPo3VxqcVt/RKD8jYiabyBqDLRqAECp1qj9ahWKNkH5YSvwACG3ZyECJO1xWMpPDXdzxygNZJEmQOgr+SFWYVBABazsaoz/yFVh5Y+AuUYjTZyj1I19WeiNJNAs8ra6kCAsptajKS0Ds99RQ3zlhbCmPUPf9yUbVAl/475yyP3D+E2yRriArsR5FijaHf8zngI+U4H1uifrCaZP32zNyySk6roDrBi9OrAAEoajNHoMX+u91mxQG87an9kw0kMyibtU23jXn4QeEdFAaBQO73F7QQDhJwoVkAYPMdiFs0JABrVz7z2QAdAHKjTx8F6Pf1B5wK/xMg0AFAFwAg7UJaRiD5OYBpZP0dhUMPva3T5M6W0DtnCk2vvxK2YmMJ4nGd/mIeAgQ1GamJDtBkg6iCiPheEmHyuW0dk7SU26jn6ejjsIcWmICGAxfgp9KeAWMOAyd+1m+qTNquKAzE15Gx59KrAAFdfAK7RwTe94nr9YarB7W4Xm/ND9C2DCcg4ONg0KAhsOzUYyA2DOmAEBMvvAkcUMbYcLn4luQjF6cvx/vn1hkKmHKxkIMX10o/ogv9yTHqMn7GYNCcrLM+87CB0OtiqK6EowChOrJ1wrBF9UQqnlRIqT2pjljwUWysCT5rfGHxcMEPrbi8vJtu2tRulE1DScAjFMu67XsAgB6Cy8az+b4j2BOTYSJDeeSp4LJ8B6U5IdQG7fpHTC/dhuwrAfxxAP+kv+dfBfAZfELbkAm6ANj4v3tofWOMyRUHZI+emNZhZzyr1F1QZ1tHbusEIhZ/+BBKwwUAVIefrunH2DCnxziMDkMCHA3fjihtdFKfuTowdvsVAoEelDI0Og81LmGsCQS8cjrzvcsEgq/HdFQ02aDKXiDJRV6Rk1b6QOVTXVFOTznvASwImg2fBtvC/PqioGmLxjIgyvWKmUrkZkPP1G2CZifriWNqbsn9I2UIxjwT6shlp/FZLAaN7BMFePdUhd2zsD80rDp9XzPFzonFk+mlTOCHAPyYqv4eEfkAwJcD+HfwSW1DJmggsPnkn6Hqm4kMP59Uk+YZj/BVtrpKa7GITMx5FHaEsLclurV0l6/ZPgdqXvJtw/R49QEC2xbbdA80dsG7KRH/LJ+AoICk/BdDIg4B7bfnQs7x6+247Gxz4hkK6tyGQD07BgILAPhv4pytkw4C5d8oBhLfH23Xs25dYeIov6HBqb5yQrQFJ7TRojFxm7XUGqoZQHb3DVQsFJptV8/sZcScCwGZf74aZQyMCSgmdHgMwDQxbVl6CXflU1BrIgB084FmB1q54X2wlFonQ/SgrI/kOOgOydVQ+HjpJSHHfz2AfxrA7wcAVX0E8Cgin9g2ZAJg20pwpm4YvvjG/qZN2Dj0SoXtAGfQLI7e0Fj4YUAA17Tiu81AVu3fBRHis+fEthLftgsus7RtB4Et5ykEKMR01bLJj8ODZRIUIIhILtAxQavfFzCU8O++BXvcL6TmAgC4lnkItM7LZi+NY/VZC1NY/8Ls5uX6yhaepqUdAGTpv32IU/P2Cdugo/ZwDGoeodJvHlwkgowYCPDoBpx1CiyAawhQsAIDfluMFsOE0Rdig5BkpFRI/tiYAwg4GptlRiRUB+c1V5OhO/hUnSHbPZnBC3HhJUzgGwD8XQB/QkS+GcBfAfB9+Dy3IWtJbMVXpDGtAcYW0zklF15UB4t54K7lIICH3o4loobyfXZbor/UsbOBDhCA4OFyQ2x8YdktEAgn4XoelDb/fM5ALCdmk4DBoBYtGeiYoFe0n5wj79GTAxh4xVtqGi+rVzK82qyusq8dHXOsb6YzoQYIxDqOGv+o95fGXj6fklusIGhmXeTueJ86E5h77VE4o+4226hUx4CIj+pICFWxggHYUKACAyOZxvD1/2P6FGW2KRgIpI7NCejMIW7Id56YB2uKsgYw834D4SPIZ4nf2anFs+klIHAB8C0A/qCq/oSI/BCM+t9LZ28+9AWhvQg/9fDQPOPhJBtM5dxbG5y22aDqWkKl5oyHLSazzpP+9Xs6E+ggsG1bblDCTKADwYbL5YLL5SHPAdqEQrWd907SQSAEe98HbCd40DM6AOyu6fb9BrdTiK6Ke6ZZJYTwx8wyo+UcoCTuLowg+xzx26cFXU+us26X6qVNcMpyqmnROokNtZGRmjY9c4eivW1aGsFnTTR8MZGU7yXfCRLMDNUmbZJQzRWIvhQMhWuuyte0v79MwEygmuRUYuSkXhfckfZermmh8+fTS0DgswA+q6o/4Z9/GAYCn9c2ZEp7Ef66r/hy3UYxgbl52CcdGFMw9gCEDBvYOkssHkmaxR1fyIMfiNxg2O3/g1DW0TRMrAKs5b4m8BsulwdcLg94eHjA5XLBw8ODleNOxCIWVvYaB9jNudu0aM9jrGysPRPc5nX6yyBQIyDu1Brcyxz46HMFtSw7vfdthXoA1jAFJpCrGEG/ae27NvhJWs0AQbUvz7Sc0NzQU2d3jFrd7rk9me6xZ+FeE9DIvAg/T9JritEIGOtRVWBUXIfwS8WirmACjAUsfCv9b6ZAK7ucyn8BQDe7Otwo9IgKC6v7hJiAqv4dEflZEfkmVf0MbMORv+F/34NPZBsyYJBPYMw+VbUomdhmo0EVU0POWpgjghY/DztVBoNDXWOt3MftTdBv+w2xp5z9lEHABP+DDx7wwQcf4HKxI4CMdBPBONTDnfcOMg70cb/t2MYNu2usqZM2T7nlEOrtdiUQQIJIG3bUnAOI6iX0GWU6RFBzkXCICqC2PDWIWpkCJQHPjUt3QFg4dKOv8YPyoXAsyH3uZAY5I3JhV1UCgPiLpdCxAMl8T+EoTc0/4rMFpIlVetnvuL0GahcpAG1ub1w6gACSnS3NQL9hr8qx7sTrPljIOn04nsX3vxADXjw68AcB/CkfGfjbsK3FBj6xbcj6EGH6AGixSk271JzMoihKmKsOw9E1eR45gwB/tmtM72du522OyRuxgIgpkAKWk4fKHHh4eHAmINg2C3xpjs06jzIftYUJ35YBOzz/urKBvl/g7VYgMMaADg9o4uGp0zPNZlACgAl62ZpVN9W3aTSA92bntPS++0zANNhB8PPr0NIx6uFLsLUE3iaTlSkUDlI0ACgfzrYPzG2HRX+++PN9VETKFOA9yYVHAk7+ohK0fpLXjppf6rBcOmrv8wHWleR3AGiVfzh9Lr10V+K/BuBbT776BLcho3Ontaltp0W7mZfQCLaG3UKHqwWI3F3ANegkEAFL+S1E2ux+e2FjBlWYOHQHX22aUc6rDgyb2/eCMQxM9n0kjU3tmx2qOo6qWtDOHHrMem2jBDmr0iPtAposYJ2AtIJAAQDy3a3zxn0Bjuv5GL2PLWrnOVOA38dHALlRTLCAMKlqB6k9WRGzgtqdaEaLdY0r8JBe1dxZt5T7PtlqpeXli4iVg/Z8dTAw0eT+tQICV1XrcgkG59JbjuwCrXo2j3DVQjA2P55Kr2LGIICWYaayYxvYpu11d3HnmEyL2yd7RYo1LYpcfWggkBvVAEiczvcopGbGneSphL+EkMftmVZavkcDAZGBqdNHOyh6cfzEx6wZEFQVlxxq7CsVcyelWZuH3m43C8iiDAJl2sQzGh1FvJfAQPhaZycrELykg90HggCf0d5VINBNgAIDK7fmbtKm6ff9ZutOvMEzLmG9rgS+nRMb0SW/7vsBP4ufGf4ohCCHPyVe2N8JLueTAEC+Ri4AORtL+P0ckvEmYjp8VzDPp1cDAt0+qo6XjEAVug3cBHDHQKP9IfRmGojHqqwZ5C3ok6I+NwS4U2nkf6i95jXBBkp55rULHnVHY1ORESsh7V2lIep8IwBoDRnmSLIBMgluNyjgu9v6IiuZlRdmAKzpT4S+gUJqHjdPgGwXvLCDnTf08i7U81oAFgYEmjCV8yWcIeQIQHYC53yL1meZTAFdeTboel52PqDZ004tovht9TQysCS/Bn19uJZZkfp9gjizAQKCkdf7DNZ3DgRYQ8eCFbOPFYqNhH4mCxAxO1t2q5Q51feO8PDSU4AYOfC3AGxTZRU3lF5T9K3omDWDrzRFAUBNGpqz1qfzUKEJIGuRYkJz7jUJKRuSHGWxN4CbA1ffISgceQfHYA6HdXOgaWEJQOydp2v+qqfzzrXU35M2QXXoYCUhCbw9fNR1jQDt6R/osycr/jQLXQRpicouQFht9vB5FAMIgc/JSnecdk3cxa8yyND37VoVuSuhU7AgwTepT0B+7u8l6VWAQBQoPw/BUJu5tXlDZAP7FmIi4htG+LY4u0CGQmL3mrDdJiIQMQCkUyobUBetRqdHM4B9ArOckSEgo8wB0+ZsStSzSuDqpdHIc3cQiHDeeQ+DD+8YbPPlc7LQi0CgM5AU/mAKrHEWrd3Nihcmlp8AHJIUQbVBzQcg0KWyJyugNRTB6waQw6pHILN3YzAYVP5svgROZL00f/gEFvsyOgC9D72eGzrF7fcFvuqqnt3A+IwVEEC/kyDQ0bA63hgWcHTTYgq6Fs5PFTAAgEJhe8APQVvpFZ5c8+nE8w7tWM9rn6MzlnaKDhHT4GuGWUwd7iBgeQjm4CVdgGAcWID9MZOwgCu1X+DNveMigjFjqqvtuzjCibcIeRDV1IjceROgAhzq93JWWXeaNCvy5JssWwgr2Kyrup7tvC+Yqg1IYwKWTfNmgcy3huBH9qWUQeRTWnaLBRg48DfcZYn8c1vmS1F1mPcwaHDV0O+iqk7ArAn58vk5VnuWXgcIgKvSmcCynULOGNz3ZAKQvSLd7hY3P2YI2oZMCgGvPus14w5eQmY53ANmAxQnsBb3gBqimMC2bQfBXzVNChrqtZfL5eAXiB9Nj2C8zxomu91uuF6vDgI1oSWZQAr4uTA3zUWd6vibyDOps7tteZJauameCejstvK1tHO1FaNIYKj9GwTqDjEFxvA19VE+RMT6PC8mYNCT5qEumWUTIRyFccxG0ybJWT/BBKhS6ruj9hc6jy+Tvfhz2QHYwGBlCwEEL0yvBwRYuWs1Us4f8BbanH+rAjHHcEuk8Kj5arEArKNIOlkY6ssvgKxEv6vdkFoox+l9tZoLoP3VtXDWcbK+1BtFBE1rwd3CB/Ojecpr/Hue/ImomVHBotRWwdn7ukB3n0B1oNA++X10sNTgrZaeTO2uUxBYgSD1r9c7MzDUN6r9XADBtK3Q3TRIoVnX8cebJDJFTt8wO7i+yQGsXAh/Bsn8wqCo7lKDx4+o2PkcflA9I27KsPOs9bnd6J1V1pe106sBAS0YPla2J2aQ0amzsWlyhwkAWmX1l4ERAGfVlb9RuO1ZM/Zutyuu1w3Xx0c8PjxiGwMfXi64XGxoMMJ0swZtIpHXK29Rnuv1EbfrNQFm332SjNYmqjFbslGLpUN6d8hnH+obMT9wCViZNxgSH+7rJXlZOnk02uSkeiq5447CR8LPD01w4791whkBQb2DHLY06jK1lnKzo7KAiNMijCz0UfcNIFB9ruFgB4FUTB1lkP9SyDsAJHPgDvFMejUgsPaUItBaSlypwgcs9BNsnXf4D9T8gumUmwO5D50e3kIXRGjiVVVeOgP3iG50xePjowXg9J15Ac0wZjZkBV9JWFqPtcCK1ux9//DNG7x5fIPr9RFXB4M9FsNwpzwIQrwnugmqM4EFNynOoUOHoEPVt71SZyz2lNgKi5nTWQolexQXOvNZioD4oh1qcWVRJ6ccnWVZQjF4s+W6iZxyvtGWaZQL9rHkPISYHeoh7WjZdgvhJuvfqpnXa6g+INU2DQDOwCCOBDJZTkh7R2iBaptSAs+lVwMCrUOGpvOUuiIro6/UMgAQDIU5wzS2a/IoMWMaHdcg/3cAgTtJ5ANlBtxuN9yuN1y3R3uv2+7m8bff7fvuW3dvB+2UQrpSOTEwwxA8vnmDN2/e4PGxg0CuZJzECCLbYjVUmqZrE2nlq7o+E9ZcG+DCnuPV6zVdwYXbcqlSNrEReXWVDElwQbbMYha0XMeDe//IviDFAGKn5L6lWz2LJ35NB4Co55yD4SyMHcH5XqG/gdbWLPgdGCrTZS7QA9uR6r6BwGjAgCh/AkEHlufS6wCBRSLzlAqRQAAchCsaXwcw1JZ8Wkgyq8AhkrvKB6VtvPbs3clAtWxwcsbJEFy2DZftAhF4KPKRQ3URiNQ6wTjk9/RvCB4fH3G7uTmwV4gsFv4CyKPZVP2HAOBOZ6iyrnw9fqOIXXYOQ6vE1eiNh+ckUC0PT9bRXtjpfz/X03JEf2jaUgQZuo38AgeGl3MRamISL1QqFhCbxujSN58GeADlxEPlMX7fGQLVEoPAQchBz6nv0No7APodYgJHp8vR7gMIAEBr8n2utO1dOCE6fFGYWqioMaE6fOPy8hqvPTSA4bD0xTvHvu+4jRu26xWPEsIe0YB8W2uduHl4q227VFBTEQp/3jVTW7UogsfrYzKB2/UxY+XtKwtgpkQdMpteuL5O6veErlfdU/W45IuUn4UxgwX5wADWp2rkVxxfPL9B87AKfj/v5a0XlUklNXV6iyNt8y4FZNa04QisWASh/U/NAR6OJIGEoJ5PQt80+AoA0TbUWEWIGl3IOmtAwhof/Byh97xDIHAU+PIIV8+x/0TY+6sY0zzgAybsKjOZgA7B1ApPdr/b11h/rkKUEpRgAbsIrj7GFPskCOCLWIwxXK833GLCzygAiKnELa5gLl2uo/kc3uCRfAI3D5mVE2TIUbWmBEf/UN1JyqYFCVcCoSLoUomdsQCrFu+ALcRu8qp6P7Voazqub5+glduqe73HjWUCVF6YBaqSEPoLotwRTyF8AsHQRgpcvWHSSEAwgFqyXcuVuzlApikYgNCA5ui9r2ud5nNZOI+SyqkAo2v+Dv7Hdx1r/jy9EhDAMjrgKeTW+mKhLqjA9HkMmJNQeXSAKv/svahnhz0sSnkIx6DbiWPcIFcBVCsqMY31RQwBHuvn0Od9lV8/Fxk+8nDF7VqjA9ERa9FShd6Iioo4AFQhLKvNzmFN2OpfD3f4/y7sseSYgnBUyUuTLwFzT0yCvJOOJ/lb6HeyNenmRJMbiU1WpRgAmwPREdz5mcOANBO0YjnutbHsgZcUEzgK/VnfW9nAOQi0qqBOexR8Pnahl4Cnd4sJLGlRcdyHD+cEAmgOQzjFtvHjNZRzpDATYkgMMe6s5KTSGIcXnwPAQ2rWkeAaJeh77EsQ6wmiQ+b5CQiMIbjdbnjz5g3ePH6Ix8crHt0kuN1utISWVyT6ttssUKvUsVr2CzWRyT/HjboQ+yarrrmhOeOybtHDT9YUphgDSrEJIROkA0HkMxYxcZw9hqB2FgoizQwXYp0WNVgVEk7ACE6isVS54j/0rd5wEOiDbypGIYiunzkGz0cEqjytCZnBhG9GO4RWD/DnntHEO+l1gkDTXKtuqKurhltBNP6GN0Db9ziGpXJ4qgNB5oOAQKfZ/0a3FfsQ3G4BAsilrTuZA32N/8oEzAQIP8EYA7f9hsfHRzy+eYPr7WamQTgLr1caKdhpMkvksypEgcNkJO7MNeYO+n2xg9Dx4h0qPfju+GwjDPnrEsP7XXD9Jjp25Tvyym2tgth8wmeJetMN0Di+C63vG9DWe8Bmj+YTncnt66QrNrk4F1KlS+p/+KuhyJWidxYQwi3tucfaUejxImL9SQ7ZilUG72rtOb3bCpxeJwhk6trl0LGK1+al0ghI9DWNsITTZAAgIFDq5KJSTaMKhQOBeIjvfeSClav/5uJOJN41uajpIMcgxa4j+rrvew4PXm9XXB+vOVrAIcd1sVOzPmSBAwLJ1LQMq6cAEM8KgCHbX4oFCWo/QvvMoNITd0eNC4S1vX3PYb8VJ5os0K49wNsrPP9QiHr4+QCDYT/bSfgnjRasw4FIoY6P4YRkp2Q3A4Lqr74BZv9M6atNFiDvJae29T4rzs9EvDvroW2eSq8UBM50iQLZMN1/fLidUDcWEUGQm4rN0PgsLmeMIB4aWmH6/HXXjh4LmMaZL7htGy63Swm9sGOQnYDSRwpo34Hb9YqrrxC8eizBG01VnjRngDtrOP7ONXEX9GQE0KeBAAagmj2qFmCzozHfKXVfHJ5mBmeC3hmKxmO9/iw2OL3UpSXXdyzLtxUTU/1H0/Pu2jTMqzIBigUkW/L3hCmAAG0pf0Os02jzBUDC74DQAQB53bqfcrXlfx1UiYPFTeH/ERpBCJPoBeklm498E2y7sUjfAODfA/An8QltQ+YvqvNWaj30oG7HgmpMC30RQzkcK54obtOXd4BgeamtTnQQErMVVRX72LHNDfuwvRKv21Z7JJKAr+cZ3py0iU71+QE8TblCaR291pMqQehM29UzQWcAiK+583nD0JOiprpYNwA4mXOwmg7UaHU1O/+RD2i0zQBMkAXINSVIAID/KQV9mVMdNCzKszqgqUdh5mHAWKLc69Vekqwy+hCzuDvmQAMBMDvoRc86bIon2kcP4tDqyRGygwD5WV6QXhJt+DMAfqsXYgPwfwH4EVjY8U9mGzIq3FNfNu3Pidz5ibJkd4nyDDWOe++dK6gUAw7RamuMUD0jnYTGDiySj87d9jqU0VmAhN0fGoFBoAOAyDC/gg9JptNq37vdSpNYmBmxVRma+zA7MjT9AQiiPug2B8sML760RGidrLJnnFEs3sVV58k9/X/LiwLqgu9tal8HgJPmRoACHCDsNKPSA9BpUNaEnyNGhecN1KcQgkx2/sEceIoJoMwKemYVfoXAszbCAtZVnRn4WHsE4ufSRzUHvgPAT6vq/yGf4DZkwMJcjsof2dn1eG2tEccAFzS1mYPF53x9YUg5MwLUeXR8r1R7hXqoMPtOpm+aSn/GNEbrHEljl3vP/kqD0erBnCQ008mlzS8QWY+RjigXSNjYyXTsXGfsIOpAwc/zsscTm8I59OpDKx7aDsRW6DfFXBbnmAuXRpwEVVssIkThMve9aAAAFYdJREFUQ/Ddxh8zfIq+FgKlnWuOwMxh2IpTUGbIat9zENcahhz3Rwf8PMvOWn+tLer/vHipvtOl20uWC1427Q3zZPqoIPB7AfwXfv7JbUN2lhZmdKwuutIG9v2SCGqNgTNC2gRT0u6354kW7c0w1IzLqUG75srnrdDeQKG0RwMCiO+NwiDC7zoeQ/tHJ+91Ia4Vojza+0Gz98/Kdri1np0783gdndTBfaE/u67tm3OtR+UWwQivuFr4OMM6co4GIVg0+kTNbRASZgA0PXhPIIhZhNl+IABYwuDXCBCDQbGkbGcJEGDhLKoRxCPKleVuIKDpFuPP5QSk/tba5en0YhAQ23PgdwP4geduPbl26AVC25B92aceDk8ICqoR1jl20A3EC22Xmom396rKQaJ/bwRbeq7pKEIIjJaHlTe6r2p9Oqb+vatOKA5XlTbXXLVg9e2V7Zy8KzPIseg7kuq938ZPRZc81rOf6k5P259PsYEz+LD2A4DcNbnR6ejqIWCkYcMv4OeH0ZMlVwEoTLwzF/HsyGQwAgL1uJ5VvzAiafbEc6krNqG/+/0nzqyvJkC/nAQA+GhM4DsB/FVV/QX//IltQ/Ybft1XaKGXf+//99lpUblw7W4OP2soyetueiNX94VgDDVeCEUMFungflPDX5qrDu2NXW+dyOSaJHpIeWwj83I4J03O7/GWb5yE/R8Bbk+0OJm2/pw100edrqA8tnwtxfu8UiK9f7r3jhKDwY7WFl2n+gD87jClRKwtc4hYk9TYveSRt0f41mzRj1KT17WqkXDInVSIRI+t68e6z0ysF7ACAss1g0JV4fJ0PWu18zSevyXTd6NMAcC2G/seP/8e9G3Ifp9Y+ja8cBuyZhtD+uf1L+hc0DIRDyC5dAzqKMNuwRj2V3TOphubnVeboW6bL0XdBGMTbBTTfwz7vPl3vGMS76A8hr2zLzNFy6OV/U6dtPqJa7J8ppsPGrtri9ZN6Ebu3KvWS3t4+bub4Y/yB29LrM9f25yHVe/sgcB5YlOAQpgffCs602m4AsF5gasuuA3upwXUW1pMOV1vk2N10bV6+sIOeZREZ/u7l17EBETkywH8LgD/Ol3+QXxC25D5O7goIPM9gVIEbst793HeIzmfPXwAMdEnqGQNUgV1HWCveThWAlkVtVWWpKe4tbkSOku7vHyxSM2TPecpLR2MIMyWYgLxpijLmtGnCTklU5v1rBVojj94ubrpv7p3ktk4Zk2WkZRFKUCqfujvnjkQKYZ5i2nZUyTeScAU/e6IBNy+999zytiyr5QZU1+AGxZCbUmX2+3xjBe3OV6+Ddk/APDVy7W/j09yG7JWj50AdY1Y1yQbjLUqAcEQyCxHXyw7IUIF5G/dRs/ez/bx+jkKe0dss6XYf3DsKLKeH5/kcskdwsvhAlvAFs97uUHInaouep3e60UpCOfpuc63gma+ku44YiYzPEEOry7sKh6dIwO592PAQymBzC8LnUT1iYMsCToBQWEf9cXIO9sap22hx1PNgU166lHJpGmnpBi5DAuOPIN/mV7RjMGqrNgi24bkqFK8YaJyOiCUQCcQwJyCQ2I02ltZvZG1V1QH5dIr+aU/o6++W3+vDg5F4HrkHOpc/NbWYXrrpbwDCOlcZz33etTl2Auxavr4qt4ueM5ev5c+CjGQ/h8BOX8uRnbYxr2ZA9ZRog0yVsAc4FGiKi75CBYzqUYSqg2boFuHK/BhYHhC2bcJPitcsjng56SmsgMcWrT9hiZ9reV6Ir0aEJDeQtlJcxNcWbq3agFCniuhtuY+gzkRCEjJCQ7BhDpzQCfccIscokYZaVRWJYErxxSeUs7CX7H2X+unNEGjhK6Z+7Aa33XsCJydAyAkGDwjzh9F2u8+owu/nctB+Ot49mcPyEfQ0K/1kTsjHpk6VysAjLzVOxBCf1p4eXmdcPMsebh7qzd2Y2+NAVSnzEGSF1KBVwECPEae15bz1P5Ao2rWJo7UGizArw3/zSRtq8g5AX1O/J2Xp+YnNGbtnxq+g4WBU2n+aqeX9ZRmBrRrJfyA1Hg5A9czjb/moPGPJ7J376sXss56zpPavwt/zbXACQCET2ApBw0NxuzOyqkwTlTu452oL0vLSyqhNd/UK6mA/CH5offdlQFov3dt8353+5qVRwp9Hp8Dv0qvAgQA1P4CiM5fHl7xCR957lFw6rMAPhwkc0JgE4NkqoPCyWQLGI+mUcCnUzpbihKsi21S92o1fEQq4vOXvPDIGInqNcA5AtOpWN7tEOc08zw9bRg8zTnoKU3DPy/4dc9Tf8D9WI7nwBMfaoVk/EeK5F5ZtQO1qi1PGIOoa0n+oR7OuYQAw3WVD1GHcKvC1z34tdlnRUJ9M1eUa/T+eEBPrwYEGhMQIOpyuvobqOm0wyurpoVOwIN8Ri3alF6tKaVEj6wihQTXGqmBO6VkAqBn+HkJPpbrktdjchNfe4nIxbMiU/VZDu85zWs9Jct6YKLxdG139+/y5Ggo6Nl9hyv9ro+u/Zktngm+/XaM9Xsh4T9zOOK0zaMnrAyjyhLtWBGeo5lsnVLVcvUsPdSInOQrHt8uiVCfVeiEr5T0PPhU8jHs/fGcp6a1cXoVIGBj9MQEUIgn6PvRiVg4L5m0+7AzAev/XikOAOYbIES1FzQE73k5z2PN2qMrB8FHAYsLfTtfhL93i3Yx3tje2eaML+dPMYFT8yABjN7fQDGSlILMw3klPd3l/G0v1vhYhP788/EZx/OnWEeVSk/KebeoCGnV1kgB0JqjDGkC8MPz3dqBNfKdaC352BB4BCOYrvEjYIorSAxAY6HluwQCAE5AoMAABAL5OQXfPoujZQCAuFMogKIzgVWglraW9QoDADOB5TnxGUbnzgCg5PGoX5qeuCPsp4JPjISfFmXFen/rwFSm6Ktn1HXxwbwsnfOEe3Z+fHcU8vNz/sxHFv7Ibh9xWJI41CVQaJwYjB9+E74kbdesTmnZsdsMNOmVtLzkudA54jPnC6idth0MjBT46JnMXBUpc9qkWPFFci9IrwIEBB0ELOiHegBcWwloBXb6rzb7K02AsIumuk0WjiFHyEBQ8JFRfIV/6uwLve7DPN4Q8Z0/L/wz6Q9I0+M+CPA1Zh3Pa/8zgab8sjmxnBeVdU24UNuqBxaSj+IXON553873Z5+AwPp9/bZfi3d2QWcWcHKPcBlxp4xnYJDq+sQB1wFC4HUcow6xHmYRdmYDzCRsFbV0EHClplMBGR5a3+rCAunIs0u7I70KEMBiDoQjsNnwIMcgMwGY8AuBhLiDhJnAUWtrHhmGpf9HSQ8f2SmXABCPZCBY2UAvfH/smXY/PSdzpDX2HRawlDkXYeV3BUA8UYgFLD3xL2YC5+notAtmcG8UoLOHfuT84UQg1zLg/Dz/W5pfVkBIBX8ws2ytigtgglSBRXGAGDXqeZFgR1EX9FnDFwA3BQIEBAAmPNYNoLCt6T2s+0vS6wABAEPIHJAmRQUCXgEBBOkfkAgIAYslF6sIZZYpgBo2Yo3Nae3g51Wo7eyo/b2haOShzl8KBD2fPc/3v+M8nlH/zoRK+DvzWCWpOuLKCD5OOpoAT2n/50CgC3jVRa+DJ2VBeknyuWsRpWv3FQjyc/4s+iCPTvA9vZ8FwIrY1urxechwJgEfHUD6BADYUQHxiZHqi+RUpL3jqfQqQEAAyOgZHmrKKY4aaDjKJJg67XvbddRpki0LXM8bHW5aXJ2hrWzgXgV2EIBSR0CwAhI8DcFiMMi30BPpMwlhjymwzlbs39XT5HA9zRaNfEp+kAj/ra6zqL+LlPAfbe+Pns5AILfd9newl3+MeufKSvgYQFhsoAtt/e4O/JLG78XT5e5j/ZcZtTzbgYPzE5u7niFPAYEBxyBAMCbg4dGmmLCrpplgfUMQG7uoJLl7Nr0KEDhPYYcdS1JxJj10qJALJIBDpUBCqwEslZDSzxblJonaxUrWm6qB4+vS/sj3aZoEQJsrkBScn1j/sYmBJvArm1G0/1tf7CMizAbS20y/q7xp1gGImrI2fjodBdDS4vQT8YjNz/gDQrOdsJLQlNGm4gvG5OC8i+e07KTgMcnpRVzLokTyY1TBtZXb/SOeF+WB2Oj1WRnj/XTs7+F+60BGbcXtwvU2FsV6L71SEBDUhhrVFiIWjTeuBRhkw6YasL8hFX+eSNrTENnoYVV+F476vstbF2olf0AyAb+hCb4ez4uiswbXJ489rYwlzCoHEj7Pdyv/ICsjWcDCCFoVnaXTPPXn3PP6N6CgDh73tTYHavUnUEDbGFLP76r1s3wN6I7Cn2RAuH2iXAWc7D/h8hyWQS91G5mL8+pRmp8176lCHXfqHqf+kbP0SkGgkhVanKZWkNBsC6A6hFP7+iWS3iZqaqu94/uEnuspV5+p30Ca92h3kfb3m+K8T1BigUdez3IgbPVlYhCZBTUJqTrkmo94QwRM0WAP7gtgs0CL19bvV21FGvm0/pZccE+sxbTsOLsHBPS+pi1PAMFfrLnCshRzNfyxVg4sIAW3wIDrkB+i0EAehOMuH5vaP+qPAI2OmaGoU8oTv5Uw4Dzx4+hd7zgTAAqN6Yr4LEAfAU0mMATiM7ViKDGHUxxVV2L1gtczN05WUgLq7ddaTdqRhZ/pamMDi62eQn3v3I/NHCCfxNkssdD4NR2bP1Me8gdUWykcUh36DggcOm+e9Oef0foVBECC34X05Bw1QUchrpBpBKTlleZYNqFfgelQkuVzMABnVakcdClLB6yzgCiFN9Q/11Wlggaoa2wBBtPOop5PrxIEGIRVuGLgsQEHxpyYCCAYEJn1Y0VaBiEjo1WItHfcS020uFdrwIJQ4xUcGxgBqxnQ2QELNdI+t6/9zQ0dzoU/f/tECYAu9HH+5EiD90Gm7aWVXwoCytmn/BBVjs+NFdR5E9D4zcoiQHWd5WD2dswlC789dhGgtR6FP7ngMwgQmyrWAsqrnw9pdcrsKnO2gBYYzLIyje44kfBJQ8EAooQvCxz2KkEg04KSXD8MDhlB6NjisI6i8bj6PjTRIRUVj0Y5zBGn3zJFzY7rHCGAIEYJmAkchHil/XFPIoDmswwENDt6d3guZccZExCszKDYS69Dtl3BHTffsJ5EVjp70sNNKwDUeRMM7+WlUUlYCSgStELuBagJOvfy2rVmf0crTDsXf1E6VyMwhcyaDxQARs9iH0AqjdAdzyrtQlMr7rIQXuov3/UyIvBKQYCENRvZESBMgCG1vaQkE3Cd4B1AIha938UVxILLOh95xpqlzy7kmXf3hCQFloGAAUGRnajoOb1fe67uMoA8b9W2pNL88a7QXscAKUk7CuwkvN13QODknVwWZhchoA1oFtBhU4Dz0AW2A0LVTzytg9u9/J5R6HMB6qzKWw8Ru8+q1BevEQsIxXXX+ZmgwLB4591CFevdtoALEHr3eDkReKUgADStnwIrrpVRTECBFlQkQpWXBniig+WRf4vCWG+kYAIr6xaI+SMYDEQgMor+a7Qd+wesg4rTx9DqjZ5T+VWR78985LldOHRa/j0ATAfFBIJ4Rr13ZTtRP0cbllQOv5IurZNo8vxYgydtFIIPet8xL/HO+yzwmNZsn5avPbPyGp+tzqbX37Bz6bcWA1iUTyi1hSXESRGXe2Xxvr2SPvR3PeP/bumjRBv+VU1r56i6Oe+AefezJT+tvZN3vzCdPq5Tt7tPfMGLnhzmeb4odJ8+87C49SOV/iOkE5TCkYOdp5cW9Pn0fBWsgv/EPXJ+3+dVg/njexml6wdA+3ivfLUgYOnEnkuD+l76JDqxnpzdedUTgP3sW17Qt7vfoj/0pbUQGufu9Ai+fvehkt+fBmN6rryfhxy/7KdfCPB6CWi+NL0wfx+lnpZ7X9KfztIrB4GPKWGfd3/4CA+4Q8uevSkA5JmG0zbNdTFxnsgmd4iYb3DuCO156MJ8gg6yAtPxGadPeJIMffIC/PHkYf3VC/O1hma+xw7Pf/zEx/U3z5TqY1ajPBeP7lcjicivAPjM287HFyj9JgB/721n4guQ3pfr3Uv/iKr+Q+vF1+IY/IyqfuvbzsQXIonIX/5iLNv7cn3xpFduDrxP79P79IVO70HgfXqfvsTTawGB/+RtZ+ALmL5Yy/a+XF8k6VU4Bt+n9+l9envptTCB9+l9ep/eUnoPAu/T+/Qlnt46CIjIPy8inxGRnxKR73/b+fkoSUR+s4j8BRH5SRH5X0Xk+/z6V4nInxeRv+XH3+jXRUT+Iy/rXxeRb3m7JXg6icgmIv+ziPyof/5HReQnvFx/RkQ+8Ouf8s8/5d9//dvM93NJRL5SRH5YRP6mt923f7G02cdJbxUERGQD8B8D+E4AvwXAd4vIb3mbefqI6Qbg31LVfwLAtwH4A57/7wfw46r6jQB+3D8DVs5v9L9/DcAf/dXP8kdK3wfgJ+nzfwDgD3u5fgnA9/r17wXwS6r6jwH4w37fa04/BODHVPUfB/DNsDJ+sbTZR08cYOJX+w/AtwP4c/T5BwD8wNvM0+dZnv8OwO+CzX78tF/7NGwyFAD8MQDfTffnfa/tD8DXwYThnwXwo7BJqX8PwGVtOwB/DsC3+/nF75O3XYY75fr1AP73NX9fDG32cf/etjnwDwP4Wfr8Wb/2ziWnwL8NwE8A+FpV/XkA8OPX+G3vUnn/CIB/G7W57VcD+H9U9eafOe9ZLv/+l/3+15i+AcDfBfAn3NT54yLyFfjiaLOPld42CJwteXjnxixF5NcC+K8B/Juq+v8+devJtVdXXhH5FwD8oqr+Fb58cusxaMPxu9eWLgC+BcAfVdXfBuD/Q1H/s/Qule1jpbcNAp8F8Jvp89cB+Lm3lJePlUTkAQYAf0pV/xu//Asi8mn//tMAftGvvyvl/R0AfreI/AyAPw0zCf4IgK8UkVhvwnnPcvn3vwHA//2rmeGPkD4L4LOq+hP++YdhoPCut9nHTm8bBP4SgG90r/MHAH4vgD/7lvP04iQWfuY/BfCTqvof0ld/FsD3+Pn3wHwFcf33ucf52wD8clDQ15RU9QdU9etU9ethbfI/quq/DOAvAPg9fttarijv7/H7X6W2VNW/A+BnReSb/NJ3APgbeMfb7PNKb9spAeC7APxvAH4awL/7tvPzEfP+T8Go4V8H8Nf877tg9vCPA/hbfvwqv19goyE/DeB/AfCtb7sMLyjj7wTwo37+DQD+JwA/BeC/AvApv/5l/vmn/PtveNv5fqZMvxXAX/Z2+28B/MYvpjb7qH/vpw2/T+/Tl3h62+bA+/Q+vU9vOb0HgffpffoST+9B4H16n77E03sQeJ/epy/x9B4E3qf36Us8vQeB9+l9+hJP70HgfXqfvsTT/w+tCba+RPSREwAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"import mindspore.dataset.transforms.c_transforms as c_transforms\n",
"import mindspore.dataset.transforms.vision.c_transforms as C\n",
"import matplotlib.pyplot as plt\n",
"cifar10_path = \"./dataset/Cifar10Data/cifar-10-batches-bin/\"\n",
"\n",
"# create Cifar10Dataset for reading data\n",
"cifar10_dataset = ds.Cifar10Dataset(cifar10_path,num_parallel_workers=4)\n",
"transforms = C.RandomResizedCrop((800,800))\n",
"# apply the transform to the dataset through dataset.map()\n",
"cifar10_dataset = cifar10_dataset.map(input_columns=\"image\",operations=transforms,num_parallel_workers=4)\n",
"\n",
"data = next(cifar10_dataset.create_dict_iterator())\n",
"plt.imshow(data[\"image\"])\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"2. 使用自定义Python函数进行数据增强，数据增强时采用多进程优化方案，开启了4个进程并发完成任务。"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"before map:\n",
"[0 1 2 3 4]\n",
"[1 2 3 4 5]\n",
"[2 3 4 5 6]\n",
"[3 4 5 6 7]\n",
"[4 5 6 7 8]\n",
"after map:\n",
"[ 0 1 4 9 16]\n",
"[ 1 4 9 16 25]\n",
"[ 4 9 16 25 36]\n",
"[ 9 16 25 36 49]\n",
"[16 25 36 49 64]\n"
]
}
],
"source": [
"def generator_func():\n",
" for i in range(5):\n",
" yield (np.array([i,i+1,i+2,i+3,i+4]),)\n",
"\n",
"ds3 = ds.GeneratorDataset(source=generator_func,column_names=[\"data\"])\n",
"print(\"before map:\")\n",
"for data in ds3.create_dict_iterator():\n",
" print(data[\"data\"])\n",
"\n",
"func = lambda x:x**2\n",
"ds4 = ds3.map(input_columns=\"data\",operations=func,python_multiprocessing=True,num_parallel_workers=4)\n",
"print(\"after map:\")\n",
"for data in ds4.create_dict_iterator():\n",
" print(data[\"data\"])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 性能优化方案总结"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 多线程优化方案"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"在数据pipeline过程中，相关算子一般都有线程数设置参数，来提升处理并发度，提升性能，例如：\n",
"- 在数据加载的过程中，内置数据加载类有`num_parallel_workers`参数用来设置线程数。\n",
"- 在数据增强的过程中，`map`函数有`num_parallel_workers`参数用来设置线程数。\n",
"- 在Batch的过程中，`batch`函数有`num_parallel_workers`参数用来设置线程数。\n",
"\n",
"具体内容请参考[内置加载算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html)。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 多进程优化方案"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"数据处理中Python实现的算子均支持多进程的模式，例如：\n",
"- `GeneratorDataset`这个类默认是多进程模式，它的`num_parallel_workers`参数表示的是开启的进程数，默认为1，具体内容请参考[GeneratorDataset](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html#mindspore.dataset.GeneratorDataset)。\n",
"- 如果使用Python自定义函数或者`py_transforms`模块进行数据增强的时候，当`map`函数的参数`python_multiprocessing`设置为True时，此时参数`num_parallel_workers`表示的是进程数，参数`python_multiprocessing`默认为False，此时参数`num_parallel_workers`表示的是线程数，具体的内容请参考[内置加载算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html)。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Compose优化方案"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Map算子可以接收Tensor算子列表，并将按照顺序应用所有的这些算子，与为每个Tensor算子使用的Map算子相比，此类“胖Map算子”可以获得更好的性能，如图所示："
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/optimize_the_performance_of_data_preparation/images/compose.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 算子融合优化方案"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"提供某些融合算子，这些算子将两个或多个算子的功能聚合到一个算子中。具体内容请参考[数据增强算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.transforms.vision.html)，与它们各自组件的流水线相比，这种融合算子提供了更好的性能。如图所示："
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/optimize_the_performance_of_data_preparation/images/operator_fusion.png)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.5"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
tutorials/notebook/quick_start.ipynb
浏览文件 @ 516b61b6
......@@ -34,7 +34,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"说明：<br/>你可以在这里找到完整可运行的样例代码：<https://gitee.com/mindspore/docs/blob/master/tutorials/tutorial_code/lenet.py>。"
"说明：<br/>你可以在这里找到完整可运行的样例代码：<https://gitee.com/mindspore/docs/blob/r0.7/tutorials/tutorial_code/lenet.py>。"
]
},
{
......@@ -470,7 +470,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"<img src=\"https://www.mindspore.cn/tutorial/zh-CN/master/_images/LeNet_5.jpg\" alt=\"LeNet5\">"
"<img src=\"https://www.mindspore.cn/tutorial/zh-CN/r0.7/_images/LeNet_5.jpg\" alt=\"LeNet5\">"
]
},
{
......
tutorials/notebook/synchronization_training_and_evaluation.ipynb
浏览文件 @ 516b61b6
......@@ -208,7 +208,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"实现思想：每隔n个epoch验证一次模型精度，由于在自定义函数中实现，如需了解自定义回调函数的详细用法，请参考[API说明](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.train.html?highlight=callback#mindspore.train.callback.Callback)。\n",
"实现思想：每隔n个epoch验证一次模型精度，由于在自定义函数中实现，如需了解自定义回调函数的详细用法，请参考[API说明](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.train.html?highlight=callback#mindspore.train.callback.Callback)。\n",
"\n",
"核心实现：回调函数的`epoch_end`内设置验证点，如下：\n",
"\n",
......
tutorials/source_zh_cn/advanced_use/optimize_the_performance_of_data_preparation.md 0 → 100644
浏览文件 @ 516b61b6
# 优化数据准备的性能
`Ascend` `GPU` `CPU` `数据准备` `初级` `中级` `高级`
<!-- TOC -->
- [优化数据准备的性能](#优化数据准备的性能)
- [概述](#概述)
- [整体流程](#整体流程)
- [准备环节](#准备环节)
- [导入模块](#导入模块)
- [下载所需数据集](#下载所需数据集)
- [数据加载性能优化](#数据加载性能优化)
- [性能优化方案](#性能优化方案)
- [代码示例](#代码示例)
- [shuffle性能优化](#shuffle性能优化)
- [性能优化方案](#性能优化方案-1)
- [代码示例](#代码示例-1)
- [数据增强性能优化](#数据增强性能优化)
- [性能优化方案](#性能优化方案-2)
- [代码示例](#代码示例-2)
- [性能优化方案总结](#性能优化方案总结)
- [多线程优化方案](#多线程优化方案)
- [多进程优化方案](#多进程优化方案)
- [Compose优化方案](#compose优化方案)
- [算子融合优化方案](#算子融合优化方案)
<!-- /TOC -->
<a href="https://gitee.com/mindspore/docs/blob/r0.7/tutorials/source_zh_cn/advanced_use/optimize_the_performance_of_data_preparation.md" target="_blank"><img src="../_static/logo_source.png"></a>&nbsp;&nbsp;
<a href="https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/optimize_the_performance_of_data_preparation/optimize_the_performance_of_data_preparation.ipynb" target="_blank"><img src="../_static/logo_notebook.png"></a>
## 概述
数据是整个深度学习中最重要的一环，因为数据的好坏决定了最终结果的上限，模型的好坏只是去无限逼近这个上限，所以高质量的数据输入，会在整个深度神经网络中起到积极作用，数据在整个数据处理和数据增强的过程像经过pipeline管道的水一样，源源不断地流向训练系统，如图所示：
![title](./images/pipeline.png)
MindSpore为用户提供了数据处理以及数据增强的功能，在数据的整个pipeline过程中，其中的每一步骤，如果都能够进行合理的运用，那么数据的性能会得到很大的优化和提升。本次体验将基于CIFAR-10数据集来为大家展示如何在数据加载、数据处理和数据增强的过程中进行性能的优化。
## 整体流程
- 准备环节。
- 数据加载性能优化。
- shuffle性能优化。
- 数据增强性能优化。
- 性能优化方案总结。
## 准备环节
### 导入模块
`dataset`模块提供API用来加载和处理数据集。
```python
import mindspore.dataset as ds
```
`numpy`模块用于生成ndarray数组。
```python
import numpy as np
```
### 下载所需数据集
1. 在当前工作目录下创建`./dataset/Cifar10Data`目录，本次体验所用的数据集存放在该目录下。
2. 在当前工作目录下创建`./transform`目录，本次体验转换生成的数据集存放在该目录下。
3. 下载[CIFAR-10二进制格式数据集](https://www.cs.toronto.edu/~kriz/cifar-10-binary.tar.gz)，并将数据集文件解压到`./dataset/Cifar10Data/cifar-10-batches-bin`目录下，数据加载的时候使用该数据集。
4. 下载[CIFAR-10 Python文件格式数据集](https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz)，并将数据集文件解压到`./dataset/Cifar10Data/cifar-10-batches-py`目录下，数据转换的时候使用该数据集。
目录结构如下所示：
dataset/Cifar10Data
├── cifar-10-batches-bin
│   ├── batches.meta.txt
│   ├── data_batch_1.bin
│   ├── data_batch_2.bin
│   ├── data_batch_3.bin
│   ├── data_batch_4.bin
│   ├── data_batch_5.bin
│   ├── readme.html
│   └── test_batch.bin
└── cifar-10-batches-py
├── batches.meta
├── data_batch_1
├── data_batch_2
├── data_batch_3
├── data_batch_4
├── data_batch_5
├── readme.html
└── test_batch
其中：
- `cifar-10-batches-bin`目录为CIFAR-10二进制格式数据集目录。
- `cifar-10-batches-py`目录为CIFAR-10 Python文件格式数据集目录。
## 数据加载性能优化
MindSpore为用户提供了多种数据加载方式，其中包括常用数据集加载、用户自定义数据集加载、MindSpore数据格式加载，详情内容请参考[加载数据集](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/loading_the_datasets.html)。对于数据集加载，底层实现方式的不同，会导致数据集加载的性能存在差异，如下所示：
| | 常用数据集 | 用户自定义 | MindRecord |
| :----: | :----: | :----: | :----: |
| 底层实现 | C++ | Python | C++ |
| 性能 | 高 | 中 | 高|
### 性能优化方案
![title](./images/data_loading_performance_scheme.png)
数据加载性能优化建议如下：
- 已经支持的数据集格式优选内置加载算子，具体内容请参考[内置加载算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html)，如果性能仍无法满足需求，则可采取多线程并发方案，请参考本文[多线程优化方案](#多线程优化方案)。
- 不支持的数据集格式，优选转换为MindSpore数据格式后再使用`MindDataset`类进行加载，具体内容请参考[将数据集转换为MindSpore数据格式](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/converting_datasets.html)，如果性能仍无法满足需求，则可采取多线程并发方案，请参考本文[多线程优化方案](#多线程优化方案)。
- 不支持的数据集格式，算法快速验证场景，优选用户自定义`GeneratorDataset`类实现，如果性能仍无法满足需求，则可采取多进程并发方案，请参考本文[多进程优化方案](#多进程优化方案)。
### 代码示例
基于以上的数据加载性能优化建议，本次体验分别使用内置加载算子`Cifar10Dataset`类、数据转换后使用`MindDataset`类、使用`GeneratorDataset`类进行数据加载，代码演示如下：
1. 使用内置算子`Cifar10Dataset`类加载CIFAR-10数据集，这里使用的是CIFAR-10二进制格式的数据集，加载数据时采取多线程优化方案，开启了4个线程并发完成任务，最后对数据创建了字典迭代器，并通过迭代器读取了一条数据记录。
```python
cifar10_path = "./dataset/Cifar10Data/cifar-10-batches-bin/"
# create Cifar10Dataset for reading data
cifar10_dataset = ds.Cifar10Dataset(cifar10_path,num_parallel_workers=4)
# create a dictionary iterator and read a data record through the iterator
print(next(cifar10_dataset.create_dict_iterator()))
```
输出：
{'image': array([[[235, 235, 235],
[230, 230, 230],
[234, 234, 234],
...,
[248, 248, 248],
[248, 248, 248],
[249, 249, 249]],
...,
[120, 120, 119],
[146, 146, 146],
[177, 174, 190]]], dtype=uint8), 'label': array(9, dtype=uint32)}
2. 使用`Cifar10ToMR`这个类将CIFAR-10数据集转换为MindSpore数据格式，这里使用的是CIFAR-10 python文件格式的数据集，然后使用`MindDataset`类加载MindSpore数据格式数据集，加载数据采取多线程优化方案，开启了4个线程并发完成任务，最后对数据创建了字典迭代器，并通过迭代器读取了一条数据记录。
```python
from mindspore.mindrecord import Cifar10ToMR
cifar10_path = './dataset/Cifar10Data/cifar-10-batches-py/'
cifar10_mindrecord_path = './transform/cifar10.record'
cifar10_transformer = Cifar10ToMR(cifar10_path,cifar10_mindrecord_path)
# executes transformation from Cifar10 to MindRecord
cifar10_transformer.transform(['label'])
# create MindDataset for reading data
cifar10_mind_dataset = ds.MindDataset(dataset_file=cifar10_mindrecord_path,num_parallel_workers=4)
# create a dictionary iterator and read a data record through the iterator
print(next(cifar10_mind_dataset.create_dict_iterator()))
```
输出：
{'data': array([255, 216, 255, ..., 63, 255, 217], dtype=uint8), 'id': array(30474, dtype=int64), 'label': array(2, dtype=int64)}
3. 使用`GeneratorDataset`类加载自定义数据集，并且采取多进程优化方案，开启了4个进程并发完成任务，最后对数据创建了字典迭代器，并通过迭代器读取了一条数据记录。
```python
def generator_func(num):
for i in range(num):
yield (np.array([i]),)
# create GeneratorDataset for reading data
dataset = ds.GeneratorDataset(source=generator_func(5),column_names=["data"],num_parallel_workers=4)
# create a dictionary iterator and read a data record through the iterator
print(next(dataset.create_dict_iterator()))
```
输出：
{'data': array([0], dtype=int64)}
## shuffle性能优化
shuffle操作主要是对有序的数据集或者进行过repeat的数据集进行混洗，MindSpore专门为用户提供了`shuffle`函数，其中设定的`buffer_size`参数越大，混洗程度越大，但时间、计算资源消耗也会大。该接口支持用户在整个pipeline的任何时候都可以对数据进行混洗，具体内容请参考[shuffle处理](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/data_processing_and_augmentation.html#shuffle)。但是因为底层的实现方式不同，该方式的性能不如直接在[内置加载算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html)中设置`shuffle`参数直接对数据进行混洗。
### 性能优化方案
![title](./images/shuffle_performance_scheme.png)
shuffle性能优化建议如下：
- 直接使用内置加载算子的`shuffle`参数进行数据的混洗。
- 如果使用的是`shuffle`函数，当性能仍无法满足需求，可通过调大`buffer_size`参数的值来优化提升性能。
### 代码示例
基于以上的shuffle性能优化建议，本次体验分别使用内置加载算子`Cifar10Dataset`类的`shuffle`参数和`Shuffle`函数进行数据的混洗，代码演示如下：
1. 使用内置算子`Cifar10Dataset`类加载CIFAR-10数据集，这里使用的是CIFAR-10二进制格式的数据集，并且设置`shuffle`参数为True来进行数据混洗，最后对数据创建了字典迭代器，并通过迭代器读取了一条数据记录。
```python
cifar10_path = "./dataset/Cifar10Data/cifar-10-batches-bin/"
# create Cifar10Dataset for reading data
cifar10_dataset = ds.Cifar10Dataset(cifar10_path,shuffle=True)
# create a dictionary iterator and read a data record through the iterator
print(next(cifar10_dataset.create_dict_iterator()))
```
输出：
{'image': array([[[254, 254, 254],
[255, 255, 254],
[255, 255, 254],
...,
[232, 234, 244],
[226, 230, 242],
[228, 232, 243]],
...,
[ 64, 61, 63],
[ 63, 58, 60],
[ 61, 56, 58]]], dtype=uint8), 'label': array(9, dtype=uint32)}
2. 使用`shuffle`函数进行数据混洗，参数`buffer_size`设置为3，数据采用`GeneratorDataset`类自定义生成。
```python
def generator_func():
for i in range(5):
yield (np.array([i,i+1,i+2,i+3,i+4]),)
ds1 = ds.GeneratorDataset(source=generator_func,column_names=["data"])
print("before shuffle:")
for data in ds1.create_dict_iterator():
print(data["data"])
ds2 = ds1.shuffle(buffer_size=3)
print("after shuffle:")
for data in ds2.create_dict_iterator():
print(data["data"])
```
输出：
before shuffle:
[0 1 2 3 4]
[1 2 3 4 5]
[2 3 4 5 6]
[3 4 5 6 7]
[4 5 6 7 8]
after shuffle:
[2 3 4 5 6]
[0 1 2 3 4]
[4 5 6 7 8]
[1 2 3 4 5]
[3 4 5 6 7]
## 数据增强性能优化
在图片分类的训练中，尤其是当数据集比较小的时候，用户可以使用数据增强的方式来预处理图片，从而丰富数据集。MindSpore为用户提供了多种数据增强的方式，其中包括：
- 使用内置C算子（`c_transforms`模块）进行数据增强。
- 使用内置Python算子（`py_transforms`模块）进行数据增强。
- 用户可根据自己的需求，自定义Python函数进行数据增强。
具体的内容请参考[数据增强](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/data_processing_and_augmentation.html#id3)。因为底层的实现方式不同，所以性能还是有一定的差异，如下所示：
| 模块 | 底层接口 | 说明 |
| :----: | :----: | :----: |
| c_transforms | C++（基于OpenCV）| 性能高 |
| py_transforms | Python（基于PIL） | 该模块提供了多种图像增强功能，并提供了PIL Image和Numpy数组之间的传输方法 |
### 性能优化方案
![title](./images/data_enhancement_performance_scheme.png)
数据增强性能优化建议如下：
- 优先使用`c_transforms`模块进行数据增强，因为性能最高，如果性能仍无法满足需求，可采取[多线程优化方案](#多线程优化方案)、[Compose优化方案](#Compose优化方案)或者[算子融合优化方案](#算子融合优化方案)。
- 如果使用了`py_transforms`模块进行数据增强，当性能仍无法满足需求，可采取[多线程优化方案](#多线程优化方案)、[多进程优化方案](#多进程优化方案)、[Compose优化方案](#Compose优化方案)或者[算子融合优化方案](#算子融合优化方案)。
- `c_transforms`模块是在C++内维护buffer管理，`py_transforms`模块是在Python内维护buffer管理。因为Python和C++切换的性能成本，建议不要混用算子。
- 如果用户使用了自定义Python函数进行数据增强，当性能仍无法满足需求，可采取[多线程优化方案](#多线程优化方案)或者[多进程优化方案](#多进程优化方案)，如果还是无法提升性能，就需要对自定义的Python代码进行优化。
### 代码示例
基于以上的数据增强性能优化建议，本次体验分别使用`c_transforms`模块和自定义Python函数进行了数据增强，演示代码如下所示：
1. 使用`c_transforms`模块进行数据增强，数据增强时采用多线程优化方案，开启了4个线程并发完成任务，并且采用了算子融合优化方案，使用`RandomResizedCrop`融合类替代`RandomResize`类和`RandomCrop`类。
```python
import mindspore.dataset.transforms.c_transforms as c_transforms
import mindspore.dataset.transforms.vision.c_transforms as C
import matplotlib.pyplot as plt
cifar10_path = "./dataset/Cifar10Data/cifar-10-batches-bin/"
# create Cifar10Dataset for reading data
cifar10_dataset = ds.Cifar10Dataset(cifar10_path,num_parallel_workers=4)
transforms = C.RandomResizedCrop((800,800))
# apply the transform to the dataset through dataset.map()
cifar10_dataset = cifar10_dataset.map(input_columns="image",operations=transforms,num_parallel_workers=4)
data = next(cifar10_dataset.create_dict_iterator())
plt.imshow(data["image"])
plt.show()
```
输出：
![png](./images/cifar10_c_transforms.png)
2. 使用自定义Python函数进行数据增强，数据增强时采用多进程优化方案，开启了4个进程并发完成任务。
```python
def generator_func():
for i in range(5):
yield (np.array([i,i+1,i+2,i+3,i+4]),)
ds3 = ds.GeneratorDataset(source=generator_func,column_names=["data"])
print("before map:")
for data in ds3.create_dict_iterator():
print(data["data"])
func = lambda x:x**2
ds4 = ds3.map(input_columns="data",operations=func,python_multiprocessing=True,num_parallel_workers=4)
print("after map:")
for data in ds4.create_dict_iterator():
print(data["data"])
```
输出：
before map:
[0 1 2 3 4]
[1 2 3 4 5]
[2 3 4 5 6]
[3 4 5 6 7]
[4 5 6 7 8]
after map:
[ 0 1 4 9 16]
[ 1 4 9 16 25]
[ 4 9 16 25 36]
[ 9 16 25 36 49]
[16 25 36 49 64]
## 性能优化方案总结
### 多线程优化方案
在数据pipeline过程中，相关算子一般都有线程数设置参数，来提升处理并发度，提升性能，例如：
- 在数据加载的过程中，内置数据加载类有`num_parallel_workers`参数用来设置线程数。
- 在数据增强的过程中，`map`函数有`num_parallel_workers`参数用来设置线程数。
- 在Batch的过程中，`batch`函数有`num_parallel_workers`参数用来设置线程数。
具体内容请参考[内置加载算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html)。
### 多进程优化方案
数据处理中Python实现的算子均支持多进程的模式，例如：
- `GeneratorDataset`这个类默认是多进程模式，它的`num_parallel_workers`参数表示的是开启的进程数，默认为1，具体内容请参考[GeneratorDataset](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html#mindspore.dataset.GeneratorDataset)。
- 如果使用Python自定义函数或者`py_transforms`模块进行数据增强的时候，当`map`函数的参数`python_multiprocessing`设置为True时，此时参数`num_parallel_workers`表示的是进程数，参数`python_multiprocessing`默认为False，此时参数`num_parallel_workers`表示的是线程数，具体的内容请参考[内置加载算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html)。
### Compose优化方案
Map算子可以接收Tensor算子列表，并将按照顺序应用所有的这些算子，与为每个Tensor算子使用的Map算子相比，此类“胖Map算子”可以获得更好的性能，如图所示：
![title](./images/compose.png)
### 算子融合优化方案
提供某些融合算子，这些算子将两个或多个算子的功能聚合到一个算子中。具体内容请参考[数据增强算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.transforms.vision.html)，与它们各自组件的流水线相比，这种融合算子提供了更好的性能。如图所示：
![title](./images/operator_fusion.png)
tutorials/source_zh_cn/index.rst
浏览文件 @ 516b61b6
......@@ -55,6 +55,7 @@ MindSpore教程
advanced_use/graph_kernel_fusion
advanced_use/quantization_aware
advanced_use/gradient_accumulation
advanced_use/optimize_the_performance_of_data_preparation
.. toctree::
:glob:
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或