diff --git a/tutorials/notebook/README.md b/tutorials/notebook/README.md index f7eac5dd4b49ceb37ad71ce574f11d0165e53954..1e7d99639feb66523dbc29b4a8ce59fc68173d4b 100644 --- a/tutorials/notebook/README.md +++ b/tutorials/notebook/README.md @@ -50,18 +50,19 @@ | 教  程  名  称 | 文  件  名  称 | 教  程  类  别 | 内  容  描  述 | :----------- | :----------- | :------- |:------ -| 手写数字分类识别入门体验教程 | [quick_start.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/quick_start.ipynb) | 快速入门 | - CPU平台下从数据集到模型验证的全过程解读
- 体验教程中各功能模块的使用说明
- 数据集图形化展示
- 了解LeNet5具体结构和参数作用
- 学习使用自定义回调函数
- loss值与训练步数的变化图
- 模型精度与训练步数的变化图
- 使用模型应用到手写图片的预测与分类上 -| 线性拟合 | [linear_regression.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/linear_regression.ipynb) | 快速入门 | - 了解线性拟合的算法原理
- 了解在MindSpore中如何实现线性拟合的算法原理
- 学习使用MindSpore实现AI训练中的正向传播和方向传播
- 可视化线性函数拟合数据的全过程。 -| 加载数据集 | [loading_dataset.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/loading_dataset.ipynb) | 使用指南 | - 学习MindSpore中加载数据集的方法
- 展示加载常用数据集的方法
- 展示加载MindRecord格式数据集的方法
- 展示加载自定义格式数据集的方法 -| 将数据集转换为MindSpore数据格式 | [convert_dataset_to_mindspore_data_format.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/convert_dataset_to_mindspore_data_format/convert_dataset_to_mindspore_data_format.ipynb) | 使用指南 | - 展示将MNIST数据集转换为MindSpore数据格式
- 展示将CSV数据集转换为MindSpore数据格式
- 展示将CIFAR-10数据集转换为MindSpore数据格式
- 展示将CIFAR-100数据集转换为MindSpore数据格式
- 展示将ImageNet数据集转换为MindSpore数据格式
- 展示用户自定义生成MindSpore数据格式 -| 数据处理与数据增强 | [data_loading_enhancement.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/data_loading_enhance/data_loading_enhancement.ipynb) | 使用指南 | - 学习MindSpore中数据处理和增强的方法
- 展示数据处理、增强方法的实际操作
- 对比展示数据处理前和处理后的效果
- 表述在数据处理、增强后的意义 -| 自然语言处理应用 | [nlp_application.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/nlp_application.ipynb) | 应用实践 | - 展示MindSpore在自然语言处理的应用
- 展示自然语言处理中数据集特定的预处理方法
- 展示如何定义基于LSTM的SentimentNet网络 -| 计算机视觉应用 | [computer_vision_application.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/computer_vision_application.ipynb) | 应用实践 | - 学习MindSpore卷积神经网络在计算机视觉应用的过程
- 学习下载CIFAR-10数据集,搭建运行环境
- 学习使用ResNet-50构建卷积神经网络
- 学习使用Momentum和SoftmaxCrossEntropyWithLogits构建优化器和损失函数
- 学习调试参数训练模型,判断模型精度 -| 模型的训练及验证同步方法 | [synchronization_training_and_evaluation.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/synchronization_training_and_evaluation.ipynb) | 应用实践 | - 了解模型训练和验证同步进行的方法
- 学习同步训练和验证中参数设置方法
- 利用绘图函数从保存的模型中挑选出最优模型 -| 使用PyNative进行神经网络的训练调试体验 | [debugging_in_pynative_mode.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/debugging_in_pynative_mode.ipynb) | 模型调优 | - GPU平台下从数据集获取单个数据进行单个step训练的数据变化全过程解读
- 了解PyNative模式下的调试方法
- 图片数据在训练过程中的变化情况的图形展示
- 了解构建权重梯度计算函数的方法
- 展示1个step过程中权重的变化及数据展示 -| 自定义调试信息体验文档 | [customized_debugging_information.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/customized_debugging_information.ipynb) | 模型调优 | - 了解MindSpore的自定义调试算子
- 学习使用自定义调试算子Callback设置定时训练
- 学习设置metrics算子输出相对应的模型精度信息
- 学习设置日志环境变量来控制glog输出日志 -| MindInsight的模型溯源和数据溯源体验 | [mindinsight_model_lineage_and_data_lineage.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/mindinsight/mindinsight_model_lineage_and_data_lineage.ipynb) | 模型调优 | - 了解MindSpore中训练数据的采集及展示
- 学习使用SummaryRecord记录数据
- 学习使用回调函数SummaryCollector进行数据采集
- 使用MindInsight进行数据可视化
- 了解数据溯源和模型溯源的使用方法 -| 计算图和数据图可视化 | [calculate_and_datagraphic.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/mindinsight/calculate_and_datagraphic.ipynb) | 模型调优 | - 了解MindSpore中新增可视化功能
- 学习使用MindInsight可视化看板
- 学习使用查看计算图可视化图的信息的方法
- 学习使用查看数据图中展示的信息的方法 -| 标量、直方图、图像和张量可视化 | [mindinsight_image_histogram_scalar_tensor.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/mindinsight/mindinsight_image_histogram_scalar_tensor.ipynb) | 模型调优 | - 了解完整的MindSpore深度学习及MindInsight可视化展示的过程
- 学习使用MindInsight对训练过程中标量、直方图、图像和张量信息进行可视化展示
- 学习使用Summary算子记录标量、直方图、图像和张量信息
- 学习单独对标量、直方图、图像和张量信息进行记录并可视化展示的方法 -| 混合精度 | [mixed_precision.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/mixed_precision.ipynb) | 性能优化 | - 了解混合精度训练的原理
- 学习在MindSpore中使用混合精度训练
- 对比单精度训练和混合精度训练的对模型训练的影响 -| 模型安全 | [model_security.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/model_security.ipynb) | AI安全和隐私 | - 了解AI算法的安全威胁的概念和影响
- 介绍MindArmour提供的模型安全防护手段
- 学习如何模拟攻击训练模型
- 学习针对被攻击模型进行对抗性防御 +| 手写数字分类识别入门体验教程 | [quick_start.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/quick_start.ipynb) | 快速入门 | - CPU平台下从数据集到模型验证的全过程解读
- 体验教程中各功能模块的使用说明
- 数据集图形化展示
- 了解LeNet5具体结构和参数作用
- 学习使用自定义回调函数
- loss值与训练步数的变化图
- 模型精度与训练步数的变化图
- 使用模型应用到手写图片的预测与分类上 +| 线性拟合 | [linear_regression.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/linear_regression.ipynb) | 快速入门 | - 了解线性拟合的算法原理
- 了解在MindSpore中如何实现线性拟合的算法原理
- 学习使用MindSpore实现AI训练中的正向传播和方向传播
- 可视化线性函数拟合数据的全过程。 +| 加载数据集 | [loading_dataset.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/loading_dataset.ipynb) | 使用指南 | - 学习MindSpore中加载数据集的方法
- 展示加载常用数据集的方法
- 展示加载MindRecord格式数据集的方法
- 展示加载自定义格式数据集的方法 +| 将数据集转换为MindSpore数据格式 | [convert_dataset_to_mindspore_data_format.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/convert_dataset_to_mindspore_data_format/convert_dataset_to_mindspore_data_format.ipynb) | 使用指南 | - 展示将MNIST数据集转换为MindSpore数据格式
- 展示将CSV数据集转换为MindSpore数据格式
- 展示将CIFAR-10数据集转换为MindSpore数据格式
- 展示将CIFAR-100数据集转换为MindSpore数据格式
- 展示将ImageNet数据集转换为MindSpore数据格式
- 展示用户自定义生成MindSpore数据格式 +| 数据处理与数据增强 | [data_loading_enhancement.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/data_loading_enhance/data_loading_enhancement.ipynb) | 使用指南 | - 学习MindSpore中数据处理和增强的方法
- 展示数据处理、增强方法的实际操作
- 对比展示数据处理前和处理后的效果
- 表述在数据处理、增强后的意义 +| 自然语言处理应用 | [nlp_application.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/nlp_application.ipynb) | 应用实践 | - 展示MindSpore在自然语言处理的应用
- 展示自然语言处理中数据集特定的预处理方法
- 展示如何定义基于LSTM的SentimentNet网络 +| 计算机视觉应用 | [computer_vision_application.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/computer_vision_application.ipynb) | 应用实践 | - 学习MindSpore卷积神经网络在计算机视觉应用的过程
- 学习下载CIFAR-10数据集,搭建运行环境
- 学习使用ResNet-50构建卷积神经网络
- 学习使用Momentum和SoftmaxCrossEntropyWithLogits构建优化器和损失函数
- 学习调试参数训练模型,判断模型精度 +| 模型的训练及验证同步方法 | [synchronization_training_and_evaluation.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/synchronization_training_and_evaluation.ipynb) | 应用实践 | - 了解模型训练和验证同步进行的方法
- 学习同步训练和验证中参数设置方法
- 利用绘图函数从保存的模型中挑选出最优模型 +| 优化数据准备的性能 | [optimize_the_performance_of_data_preparation.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/optimize_the_performance_of_data_preparation/optimize_the_performance_of_data_preparation.ipynb) | 应用实践 | - 数据加载性能优化
- shuffle性能优化
- 数据增强性能优化
- 性能优化方案总结 +| 使用PyNative进行神经网络的训练调试体验 | [debugging_in_pynative_mode.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/debugging_in_pynative_mode.ipynb) | 模型调优 | - GPU平台下从数据集获取单个数据进行单个step训练的数据变化全过程解读
- 了解PyNative模式下的调试方法
- 图片数据在训练过程中的变化情况的图形展示
- 了解构建权重梯度计算函数的方法
- 展示1个step过程中权重的变化及数据展示 +| 自定义调试信息体验文档 | [customized_debugging_information.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/customized_debugging_information.ipynb) | 模型调优 | - 了解MindSpore的自定义调试算子
- 学习使用自定义调试算子Callback设置定时训练
- 学习设置metrics算子输出相对应的模型精度信息
- 学习设置日志环境变量来控制glog输出日志 +| MindInsight的模型溯源和数据溯源体验 | [mindinsight_model_lineage_and_data_lineage.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/mindinsight/mindinsight_model_lineage_and_data_lineage.ipynb) | 模型调优 | - 了解MindSpore中训练数据的采集及展示
- 学习使用SummaryRecord记录数据
- 学习使用回调函数SummaryCollector进行数据采集
- 使用MindInsight进行数据可视化
- 了解数据溯源和模型溯源的使用方法 +| 计算图和数据图可视化 | [calculate_and_datagraphic.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/mindinsight/calculate_and_datagraphic.ipynb) | 模型调优 | - 了解MindSpore中新增可视化功能
- 学习使用MindInsight可视化看板
- 学习使用查看计算图可视化图的信息的方法
- 学习使用查看数据图中展示的信息的方法 +| 标量、直方图、图像和张量可视化 | [mindinsight_image_histogram_scalar_tensor.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/mindinsight/mindinsight_image_histogram_scalar_tensor.ipynb) | 模型调优 | - 了解完整的MindSpore深度学习及MindInsight可视化展示的过程
- 学习使用MindInsight对训练过程中标量、直方图、图像和张量信息进行可视化展示
- 学习使用Summary算子记录标量、直方图、图像和张量信息
- 学习单独对标量、直方图、图像和张量信息进行记录并可视化展示的方法 +| 混合精度 | [mixed_precision.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/mixed_precision.ipynb) | 性能优化 | - 了解混合精度训练的原理
- 学习在MindSpore中使用混合精度训练
- 对比单精度训练和混合精度训练的对模型训练的影响 +| 模型安全 | [model_security.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/model_security.ipynb) | AI安全和隐私 | - 了解AI算法的安全威胁的概念和影响
- 介绍MindArmour提供的模型安全防护手段
- 学习如何模拟攻击训练模型
- 学习针对被攻击模型进行对抗性防御 diff --git a/tutorials/notebook/computer_vision_application.ipynb b/tutorials/notebook/computer_vision_application.ipynb index f6a65a867c7f1be61b2a417ff1f4273718940790..bd966948cb06438e77cf383a1348e3dc56379342 100644 --- a/tutorials/notebook/computer_vision_application.ipynb +++ b/tutorials/notebook/computer_vision_application.ipynb @@ -71,7 +71,7 @@ "metadata": {}, "source": [ "本次面向Ascend 910 AI处理器硬件平台,将卷积神经网络ResNet加入到案例中,你可以在这里下载完整的样例代码案例作为基础用例:\n", - "https://gitee.com/mindspore/docs/blob/master/tutorials/tutorial_code/resnet" + "https://gitee.com/mindspore/docs/blob/r0.7/tutorials/tutorial_code/resnet" ] }, { diff --git a/tutorials/notebook/convert_dataset_to_mindspore_data_format/convert_dataset_to_mindspore_data_format.ipynb b/tutorials/notebook/convert_dataset_to_mindspore_data_format/convert_dataset_to_mindspore_data_format.ipynb index 60f0ffe0c6ad6562918a6295e1e0f7293abc0cfb..6e9332fc81a8dd4d12469c539ac70762368df244 100644 --- a/tutorials/notebook/convert_dataset_to_mindspore_data_format/convert_dataset_to_mindspore_data_format.ipynb +++ b/tutorials/notebook/convert_dataset_to_mindspore_data_format/convert_dataset_to_mindspore_data_format.ipynb @@ -281,7 +281,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "- 本例中需要的数据位置在https://gitee.com/mindspore/docs/tree/master/tutorials/notebook/convert_dataset_to_mindspore_data_format/csv_data/data.csv\n", + "- 本例中需要的数据位置在https://gitee.com/mindspore/docs/tree/r0.7/tutorials/notebook/convert_dataset_to_mindspore_data_format/csv_data/data.csv\n", "中,使用过程中可以在此路径下找到文件并下载,并且保存在`jupyter工作目录/dataset/`下,如图所示:" ] }, @@ -838,7 +838,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "3. 准备需要写入的数据,按照用户定义的Schema形式,准备需要写入的样本列表,本例中需要的数据位置在https://gitee.com/mindspore/docs/tree/master/tutorials/notebook/convert_dataset_to_mindspore_data_format/images/transform.jpg\n", + "3. 准备需要写入的数据,按照用户定义的Schema形式,准备需要写入的样本列表,本例中需要的数据位置在https://gitee.com/mindspore/docs/tree/r0.7/tutorials/notebook/convert_dataset_to_mindspore_data_format/images/transform.jpg\n", "中,使用过程中可以在此路径下找到图片并下载,并且保存在`jupyter工作目录/dataset/`下。" ] }, diff --git a/tutorials/notebook/customized_debugging_information.ipynb b/tutorials/notebook/customized_debugging_information.ipynb index 7ef6762a17ec7e4a8d8ce8389d6bed61fea52422..0f30cdcfb7b9a7ea2ccca66840e57fdc72d7f232 100644 --- a/tutorials/notebook/customized_debugging_information.ipynb +++ b/tutorials/notebook/customized_debugging_information.ipynb @@ -18,7 +18,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "本文将使用[快速入门](https://gitee.com/mindspore/docs/blob/master/tutorials/tutorial_code/lenet.py)作为样例,并通过构建自定义调试函数:`Callback`、`metrics`、`Print算子`、日志打印等,同时将构建的自定义调试函数添加进代码中,通过运行效果来展示具体如何使用MindSpore提供给我们的自定义调试能力,帮助快速调试训练网络。\n", + "本文将使用[快速入门](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/tutorial_code/lenet.py)作为样例,并通过构建自定义调试函数:`Callback`、`metrics`、`Print算子`、日志打印等,同时将构建的自定义调试函数添加进代码中,通过运行效果来展示具体如何使用MindSpore提供给我们的自定义调试能力,帮助快速调试训练网络。\n", "体验过程如下:\n", "1. 数据集准备。\n", "2. 定义深度学习网络LeNet5。\n", @@ -46,7 +46,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "这里我们需要将MNIST数据集中随机取出一张图片,并增强成适合LeNet网络的数据格式(如何处理请参考[quick_start.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/quick_start.ipynb)),训练数据集下载地址:{\"\", \"\"}。\n", + "这里我们需要将MNIST数据集中随机取出一张图片,并增强成适合LeNet网络的数据格式(如何处理请参考[quick_start.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/quick_start.ipynb)),训练数据集下载地址:{\"\", \"\"}。\n", "
数据集放在----`Jupyter工作目录+\\MNIST_Data\\train\\`,如下图结构:" ] }, @@ -303,7 +303,7 @@ "\n", "`GLOG_logtostderr`:控制日志输出方式,设置为`1`时,日志输出到屏幕;值设置为`0`时,日志输出到文件。设置输出屏幕时,日志部分的信息会显示成红色,设置成输出到文件时,会在`GLOG_log_dir`路径下生成`mindspore.log`文件。\n", "\n", - "> 更多设置请参考官网:" + "> 更多设置请参考官网:" ] }, { diff --git a/tutorials/notebook/data_loading_enhance/data_loading_enhancement.ipynb b/tutorials/notebook/data_loading_enhance/data_loading_enhancement.ipynb index fb562620846ff291b4c5438bb568de8baf6af099..893cbb572d6d3b6783684640e27f6a8a9fe44676 100644 --- a/tutorials/notebook/data_loading_enhance/data_loading_enhancement.ipynb +++ b/tutorials/notebook/data_loading_enhance/data_loading_enhancement.ipynb @@ -439,7 +439,7 @@ "source": [ "2. 使用一类图片当作数据,体验操作。在一个数据量比较大的图片数据集中,例如数据集名称叫`images`,它的存储方式是在`images`文件夹下,有不同子类别的文件夹,一个子类别文件夹中的图片属于同一类。所以我们本次体验所使用的图片放置方法,就需要创建`enhance_images`文件夹,接着在`enhance_images`下建一个名为`sample`的子类别文件夹,将图片放在`sample`文件夹中即可。如果有更多类别图片,可以在`enhance_images`下创建对应的子类别文件夹,将图片放入即可。\n", "\n", - " 增强体验使用的数据位置在中,使用过程中可以在此路径下找到图片数据,并参照本次体验中图片放置的位置来新建文件夹。" + " 增强体验使用的数据位置在中,使用过程中可以在此路径下找到图片数据,并参照本次体验中图片放置的位置来新建文件夹。" ] }, { diff --git a/tutorials/notebook/debugging_in_pynative_mode.ipynb b/tutorials/notebook/debugging_in_pynative_mode.ipynb index b068dddd05fc8bde544cf34f234b405b02db40dd..2d07612ea30389b378d88b717f21dccc174f0964 100644 --- a/tutorials/notebook/debugging_in_pynative_mode.ipynb +++ b/tutorials/notebook/debugging_in_pynative_mode.ipynb @@ -34,7 +34,7 @@ "\n", "4. 执行神经网络训练,查看网络各参数梯度。\n", "\n", - "> 你可以在这里找到完整可运行的样例代码:。" + "> 你可以在这里找到完整可运行的样例代码:。" ] }, { @@ -55,7 +55,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "这里我们需要将MNIST数据集中随机取出一张图片,并增强成适合LeNet网络的数据格式(如何处理请参考[quick_start.ipynb](https://gitee.com/mindspore/docs/blob/master/tutorials/notebook/quick_start.ipynb)),训练数据集下载地址:{\"\", \"\"} 。\n", + "这里我们需要将MNIST数据集中随机取出一张图片,并增强成适合LeNet网络的数据格式(如何处理请参考[quick_start.ipynb](https://gitee.com/mindspore/docs/blob/r0.7/tutorials/notebook/quick_start.ipynb)),训练数据集下载地址:{\"\", \"\"} 。\n", "
数据集放在----Jupyter工作目录+\\MNIST_Data\\train\\,如下图结构:" ] }, diff --git a/tutorials/notebook/linear_regression.ipynb b/tutorials/notebook/linear_regression.ipynb index 291d0a4c5b4e26470649618d4d2aa767f806acdb..9237cc74c2d0516ffbe8c98ac9c1bfdecab6b722 100644 --- a/tutorials/notebook/linear_regression.ipynb +++ b/tutorials/notebook/linear_regression.ipynb @@ -548,7 +548,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "`nn.RMSProp`为完成权重更新的函数,更新方式大致为公式11,但是考虑的因素更多,具体信息请参考[官网说明](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html?highlight=rmsprop#mindspore.nn.RMSProp)。" + "`nn.RMSProp`为完成权重更新的函数,更新方式大致为公式11,但是考虑的因素更多,具体信息请参考[官网说明](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.nn.html?highlight=rmsprop#mindspore.nn.RMSProp)。" ] }, { diff --git a/tutorials/notebook/loading_dataset.ipynb b/tutorials/notebook/loading_dataset.ipynb index d1865e5fd7e5ef2a2db65bae5fc165cfc6feec09..62be30b1b676a4c6d5a66d740f37cf577e14f999 100644 --- a/tutorials/notebook/loading_dataset.ipynb +++ b/tutorials/notebook/loading_dataset.ipynb @@ -308,7 +308,7 @@ "\n", "MindSpore天然支持读取MindSpore数据格式——`MindRecord`存储的数据集,在性能和特性上有更好的支持。 \n", "\n", - "> 阅读[将数据集转换为MindSpore数据格式](https://www.mindspore.cn/tutorial/zh-CN/master/use/data_preparation/converting_datasets.html),了解如何将数据集转换为MindSpore数据格式。\n", + "> 阅读[将数据集转换为MindSpore数据格式](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/converting_datasets.html),了解如何将数据集转换为MindSpore数据格式。\n", "\n", "可以通过`MindDataset`对象对数据集进行读取。详细方法如下所示:" ] @@ -407,7 +407,7 @@ "## 加载自定义数据集\n", "\n", "现实场景中,数据集的种类多种多样,对于自定义数据集或者目前不支持直接加载的数据集,有两种方法可以处理。\n", - "一种方法是将数据集转成MindRecord格式(请参考[将数据集转换为MindSpore数据格式](https://www.mindspore.cn/tutorial/zh-CN/master/use/data_preparation/converting_datasets.html)章节),另一种方法是通过`GeneratorDataset`对象加载,以下将展示如何使用`GeneratorDataset`。\n", + "一种方法是将数据集转成MindRecord格式(请参考[将数据集转换为MindSpore数据格式](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/converting_datasets.html)章节),另一种方法是通过`GeneratorDataset`对象加载,以下将展示如何使用`GeneratorDataset`。\n", "\n", "1. 定义一个可迭代的对象,用于生成数据集。以下展示了两种示例,一种是含有`yield`返回值的自定义函数,另一种是含有`__getitem__`的自定义类。两种示例都将产生一个含有从0到9数字的数据集。\n", " \n", diff --git a/tutorials/notebook/mindinsight/calculate_and_datagraphic.ipynb b/tutorials/notebook/mindinsight/calculate_and_datagraphic.ipynb index 2eb475d31541d52e7af5edc5c2b76a55cc3374b2..71d3b6d17e1af0e05cb541b298766f6b26ab3928 100644 --- a/tutorials/notebook/mindinsight/calculate_and_datagraphic.ipynb +++ b/tutorials/notebook/mindinsight/calculate_and_datagraphic.ipynb @@ -287,7 +287,7 @@ "1. 导入所需的代码包,并示例化训练网络。\n", "2. 通过MindSpore提供的 `SummaryCollector` 接口,实现收集计算图和数据图。在实例化 `SummaryCollector` 时,在 `collect_specified_data` 参数中,通过设置 `collect_graph` 指定收集计算图,设置 `collect_dataset_graph` 指定收集数据图。\n", "\n", - "更多 `SummaryCollector` 的用法,请点击[API文档](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.train.html?highlight=summarycollector#mindspore.train.callback.SummaryCollector)查看。\n", + "更多 `SummaryCollector` 的用法,请点击[API文档](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.train.html?highlight=summarycollector#mindspore.train.callback.SummaryCollector)查看。\n", "\n" ] }, @@ -338,7 +338,7 @@ "\n", "> 其中 /path/ 为 `SummaryCollector` 中参数 `summary_dir` 所指定的目录。\n", "\n", - "![title](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/mindinsight_map.png)" + "![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/mindinsight_map.png)" ] }, { @@ -352,7 +352,7 @@ "- 节点信息:显示当前所查看节点的信息,包括名称、类型、属性、输入和输出。便于在训练结束后,核对计算正确性时查看。\n", "- 图例:图例中包括命名空间、聚合节点、虚拟节点、算子节点、常量节点,通过不同图形来区分。\n", "\n", - "![title](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/cast_map.png)" + "![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/cast_map.png)" ] }, { @@ -384,7 +384,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "![title](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/data_map.png)" + "![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/data_map.png)" ] }, { diff --git a/tutorials/notebook/mindinsight/mindinsight_image_histogram_scalar_tensor.ipynb b/tutorials/notebook/mindinsight/mindinsight_image_histogram_scalar_tensor.ipynb index 08a68bdb080a475a114bfbbc57f463113e64f289..7313052db809619cec900a445f02d6eb737dd2c3 100644 --- a/tutorials/notebook/mindinsight/mindinsight_image_histogram_scalar_tensor.ipynb +++ b/tutorials/notebook/mindinsight/mindinsight_image_histogram_scalar_tensor.ipynb @@ -305,10 +305,10 @@ "\n", "当前支持的Summary算子:\n", "\n", - "- [ScalarSummary](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html?highlight=scalarsummary#mindspore.ops.operations.ScalarSummary): 记录标量数据\n", - "- [TensorSummary](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html?highlight=tensorsummary#mindspore.ops.operations.TensorSummary): 记录张量数据\n", - "- [ImageSummary](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html?highlight=imagesummary#mindspore.ops.operations.ImageSummary): 记录图片数据\n", - "- [HistogramSummary](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html?highlight=histogramsummar#mindspore.ops.operations.HistogramSummary): 将张量数据转为直方图数据记录" + "- [ScalarSummary](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.ops.operations.html?highlight=scalarsummary#mindspore.ops.operations.ScalarSummary): 记录标量数据\n", + "- [TensorSummary](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.ops.operations.html?highlight=tensorsummary#mindspore.ops.operations.TensorSummary): 记录张量数据\n", + "- [ImageSummary](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.ops.operations.html?highlight=imagesummary#mindspore.ops.operations.ImageSummary): 记录图片数据\n", + "- [HistogramSummary](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.ops.operations.html?highlight=histogramsummar#mindspore.ops.operations.HistogramSummary): 将张量数据转为直方图数据记录" ] }, { @@ -393,7 +393,7 @@ "\n", "下面展示使用`SummaryCollector`来记录标量、直方图信息。\n", "\n", - "在MindSpore中通过`Callback`机制,提供支持快速简易地收集损失值、参数权重、梯度等信息的`Callback`, 叫做`SummaryCollector`(详细的用法可以参考API文档中[mindspore.train.callback.SummaryCollector](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.train.html?highlight=summarycollector#mindspore.train.callback.SummaryCollector))。`SummaryCollector`使用方法如下: \n", + "在MindSpore中通过`Callback`机制,提供支持快速简易地收集损失值、参数权重、梯度等信息的`Callback`, 叫做`SummaryCollector`(详细的用法可以参考API文档中[mindspore.train.callback.SummaryCollector](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.train.html?highlight=summarycollector#mindspore.train.callback.SummaryCollector))。`SummaryCollector`使用方法如下: \n", "\n", "`SummaryCollector` 提供 `collect_specified_data` 参数,允许自定义想要收集的数据。\n", "\n", @@ -577,17 +577,17 @@ "\n", "在本地浏览器中打开地址:`127.0.0.1:8080`,进入到可视化面板。\n", "\n", - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/mindinsight_panel.png)\n", + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/mindinsight_panel.png)\n", "\n", "在上图所示面板中可以看到`summary_01`日志文件目录,点击**训练看板**进入到下图所示的训练数据展示面板,该面板展示了标量数据、直方图、图像和张量信息,并随着训练、测试的进行实时刷新数据,实时显示训练过程参数的变化情况。\n", "\n", - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/mindinsight_panel2.png)\n", + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/mindinsight_panel2.png)\n", "\n", "### 标量可视化\n", "\n", "标量可视化用于展示训练过程中标量的变化趋势,点击打开标量信息展示面板,该面板记录了迭代计算过程中的损失值标量信息,如下图展示了损失值标量趋势图。\n", "\n", - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/scalar_panel.png)\n", + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/scalar_panel.png)\n", "\n", "上图展示了神经网络在训练过程中损失值的变化过程。横坐标是训练步骤,纵坐标是损失值。\n", "\n", @@ -599,7 +599,7 @@ "- 分步回退是指对同一个区域连续框选并放大查看时,可以逐步撤销操作。\n", "- 还原图形是指进行了多次框选后,点击此按钮可以将图还原回原始状态。\n", "\n", - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/scalar_select.png)\n", + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/scalar_select.png)\n", "\n", "上图展示的标量可视化的功能区,提供了根据选择不同标签,水平轴的不同维度和平滑度来查看标量信息的功能。\n", "\n", @@ -614,15 +614,15 @@ "\n", "直方图用于将用户所指定的张量以直方图的形式展示。点击打开直方图展示面板,以直方图的形式记录了在迭代过程中所有层参数分布信息。\n", "\n", - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/histogram_panel.png)\n", + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/histogram_panel.png)\n", "\n", "如下图为`conv1`层参数分布信息,点击图中右上角,可以将图放大。\n", "\n", - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/histogram.png)\n", + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/histogram.png)\n", "\n", "下图为直方图功能区。\n", "\n", - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/histogram_func.png)\n", + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/histogram_func.png)\n", "\n", "上图展示直方图的功能区,包含以下内容:\n", "\n", @@ -636,11 +636,11 @@ "\n", "下图为展示`summary_01`记录的图像信息。\n", "\n", - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/image_panel.png)\n", + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/image_panel.png)\n", "\n", "通过滑动上图中的\"步骤\"滑条,查看不同步骤的图片。\n", "\n", - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/image_function.png)\n", + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/image_function.png)\n", "\n", "上图展示图像可视化的功能区,提供了选择查看不同标签,不同亮度和不同对比度来查看图片信息。\n", "\n", @@ -652,7 +652,7 @@ "\n", "张量可视化用于将张量以表格以及直方图的形式进行展示。\n", "\n", - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/tensor_func.png)\n", + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/tensor_func.png)\n", "\n", "上图展示了张量可视化的功能区,包含以下内容:\n", "\n", @@ -661,7 +661,7 @@ "- 纵轴:可以选择步骤、相对时间、绝对时间中的任意一项,来作为直方图纵轴显示的数据。\n", "- 视角:可以选择正视和俯视中的一种。正视是指从正面的角度查看直方图,此时不同步骤之间的数据会覆盖在一起。俯视是指 偏移以45度角俯视直方图区域,这时可以呈现不同步骤之间数据的差异。\n", "\n", - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/tensor.png)\n", + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/tensor.png)\n", "\n", "上图中将用户所记录的张量以表格的形式展示,包含以下功能:\n", "\n", @@ -804,7 +804,7 @@ "source": [ "此时点击打开MindInsight**训练列表**看板中的`./summary_loss_only`目录,如下图所示,可以看到只记录有损失值标量信息。\n", "\n", - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/loss_scalar_only.png)" + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/loss_scalar_only.png)" ] }, { @@ -900,11 +900,11 @@ "source": [ "此时点击打开MindInsight**训练列表**看板中的`./summary_histogram_only`目录。\n", "\n", - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/histogram_only.png)\n", + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/histogram_only.png)\n", "\n", "在MindInsight面板中,如上图所示,只展示了直方图信息。\n", "\n", - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/histogram_only_all.png)\n", + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/histogram_only_all.png)\n", "\n", "点击进入直方图面板,如上图所示,只展示了`conv1`层的直方图信息。" ] @@ -1043,7 +1043,7 @@ "source": [ "此时点击打开MindInsight**训练列表**看板中的`./summary_tensor_only`目录,如下图所示,可以看到只记录有张量信息。\n", "\n", - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/tensor_only.png)" + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/tensor_only.png)" ] }, { @@ -1177,7 +1177,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/image_only.png)\n", + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/image_only.png)\n", "\n", "在MindInsight面板中,如上图所示,只展示了输入图像信息。" ] @@ -1192,11 +1192,11 @@ "\n", "点击MindInsight看板中的**对比看板**,打开对比看板,可以得到多次(不同)训练搜集到的标量数据对比信息。\n", "\n", - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/multi_scalars.png)\n", + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/multi_scalars.png)\n", "\n", "上图展示了`summary_01`(上图中红色曲线)和`summary_loss_only`(上图中蓝色曲线)的标量曲线对比效果,横坐标是训练步骤,纵坐标是标量值。\n", "\n", - "![](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/multi_scalars_select.png)\n", + "![](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/multi_scalars_select.png)\n", "\n", "上图展示的对比看板可视的功能区,提供了根据选择不同训练或标签,水平轴的不同维度和平滑度来进行标量对比的功能。\n", "\n", diff --git a/tutorials/notebook/mindinsight/mindinsight_model_lineage_and_data_lineage.ipynb b/tutorials/notebook/mindinsight/mindinsight_model_lineage_and_data_lineage.ipynb index 3a09e78dc423d7634c1f32c9ba7ff91109afb600..e2f8f721fd109727ad83933439ff2d61823b16ad 100644 --- a/tutorials/notebook/mindinsight/mindinsight_model_lineage_and_data_lineage.ipynb +++ b/tutorials/notebook/mindinsight/mindinsight_model_lineage_and_data_lineage.ipynb @@ -29,7 +29,7 @@ "6. 数据溯源的使用。调整数据参数多次训练并存储数据,并使用MindInsight的数据溯源功能对不同数据集下训练产生的模型进行对比分析,了解如何调优。\n", "\n", "\n", - "本次体验将使用快速入门案例作为基础用例,将MindInsight的模型溯源和数据溯源的数据记录功能加入到案例中,快速入门案例的源码请参考:。" + "本次体验将使用快速入门案例作为基础用例,将MindInsight的模型溯源和数据溯源的数据记录功能加入到案例中,快速入门案例的源码请参考:。" ] }, { @@ -292,7 +292,7 @@ "source": [ "MindSpore 提供 `SummaryCollector` 进行记录训练过程中的信息。通过 `SummaryCollector` 的 `collect_specified_data` 参数,可以自定义记录指定数据。\n", "\n", - "在本次体验中,我们将记录训练数据与数据集预处理的操作,我们将 `collect_specified_data` 中的 `collect_train_lineage`, `collect_eval_lineage`, `collect_dataset_graph` 设置成 `True`。SummaryCollector的更多用法,请参考[API文档](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.train.html?highlight=collector#mindspore.train.callback.SummaryCollector)。\n" + "在本次体验中,我们将记录训练数据与数据集预处理的操作,我们将 `collect_specified_data` 中的 `collect_train_lineage`, `collect_eval_lineage`, `collect_dataset_graph` 设置成 `True`。SummaryCollector的更多用法,请参考[API文档](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.train.html?highlight=collector#mindspore.train.callback.SummaryCollector)。\n" ] }, { @@ -353,7 +353,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "这里主要展示如何启用及关闭MindInsight,更多的命令集信息,请参考MindSpore官方网站:。\n", + "这里主要展示如何启用及关闭MindInsight,更多的命令集信息,请参考MindSpore官方网站:。\n", "\n", "启动MindInsight服务命令:\n", "\n", @@ -389,7 +389,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "![image](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/model_lineage_all.png)" + "![image](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/model_lineage_all.png)" ] }, { @@ -412,7 +412,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "![image](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/model_lineage_cp.png)" + "![image](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/model_lineage_cp.png)" ] }, { @@ -442,7 +442,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "![image](https://gitee.com/mindspore/docs/raw/master/tutorials/notebook/mindinsight/images/data_lineage.png)" + "![image](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/mindinsight/images/data_lineage.png)" ] }, { diff --git a/tutorials/notebook/mixed_precision.ipynb b/tutorials/notebook/mixed_precision.ipynb index b57154f1e1a6bfb4cde295dbbe8a36c48cd1c006..59f64ef33b80cba89e4b09e06fe194022f6f1d27 100644 --- a/tutorials/notebook/mixed_precision.ipynb +++ b/tutorials/notebook/mixed_precision.ipynb @@ -42,7 +42,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "> 你可以在这里找到完整可运行的样例代码:。" + "> 你可以在这里找到完整可运行的样例代码:。" ] }, { @@ -56,7 +56,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "![image](https://www.mindspore.cn/tutorial/zh-CN/master/_images/mix_precision.jpg)" + "![image](https://www.mindspore.cn/tutorial/zh-CN/r0.7/_images/mix_precision.jpg)" ] }, { @@ -952,7 +952,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "当然,如果你想参考单步训练或者手动设置混合精度训练,可以参考官网教程。" + "当然,如果你想参考单步训练或者手动设置混合精度训练,可以参考官网教程。" ] }, { diff --git a/tutorials/notebook/model_security.ipynb b/tutorials/notebook/model_security.ipynb index f1c00155f64a3c3bde554b7628e17e1fd94d0bc3..3e39acbed410e7bcfc5d21052d0899e3f18a425b 100644 --- a/tutorials/notebook/model_security.ipynb +++ b/tutorials/notebook/model_security.ipynb @@ -579,7 +579,7 @@ "source": [ "### 攻击模型\n", "\n", - "调用MindArmour提供的FGSM接口(`FastGradientSignMethod`),使用被攻击前抽取的96张数据图像`test_images`作为被攻击数据集,保存被攻击后数据集图像到当前notebook目录下的`ada_data`文件中。其中,参数`eps`为攻击对数据范围产生的单步对抗性摄动的比例,该值越大,则攻击程度越大。关于`FastGradientSignMethod`的详细使用说明,可参考[官方API文档](https://www.mindspore.cn/api/zh-CN/master/api/python/mindarmour/mindarmour.attacks.html?highlight=fastgradientsignmethod#mindarmour.attacks.FastGradientSignMethod)。" + "调用MindArmour提供的FGSM接口(`FastGradientSignMethod`),使用被攻击前抽取的96张数据图像`test_images`作为被攻击数据集,保存被攻击后数据集图像到当前notebook目录下的`ada_data`文件中。其中,参数`eps`为攻击对数据范围产生的单步对抗性摄动的比例,该值越大,则攻击程度越大。关于`FastGradientSignMethod`的详细使用说明,可参考[官方API文档](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindarmour/mindarmour.attacks.html?highlight=fastgradientsignmethod#mindarmour.attacks.FastGradientSignMethod)。" ] }, { diff --git a/tutorials/notebook/optimize_the_performance_of_data_preparation/images/compose.png b/tutorials/notebook/optimize_the_performance_of_data_preparation/images/compose.png new file mode 100644 index 0000000000000000000000000000000000000000..a1dcbf92d4ce37bd9b794b6c04bc38b131cc3a40 Binary files /dev/null and b/tutorials/notebook/optimize_the_performance_of_data_preparation/images/compose.png differ diff --git a/tutorials/notebook/optimize_the_performance_of_data_preparation/images/data_enhancement_performance_scheme.png b/tutorials/notebook/optimize_the_performance_of_data_preparation/images/data_enhancement_performance_scheme.png new file mode 100644 index 0000000000000000000000000000000000000000..a21caea16f4ee0852be47c3e56b32d184f06a7de Binary files /dev/null and b/tutorials/notebook/optimize_the_performance_of_data_preparation/images/data_enhancement_performance_scheme.png differ diff --git a/tutorials/notebook/optimize_the_performance_of_data_preparation/images/data_loading_performance_scheme.png b/tutorials/notebook/optimize_the_performance_of_data_preparation/images/data_loading_performance_scheme.png new file mode 100644 index 0000000000000000000000000000000000000000..fd32feee9d720141fc1bfcf3bb03cd40363316e6 Binary files /dev/null and b/tutorials/notebook/optimize_the_performance_of_data_preparation/images/data_loading_performance_scheme.png differ diff --git a/tutorials/notebook/optimize_the_performance_of_data_preparation/images/operator_fusion.png b/tutorials/notebook/optimize_the_performance_of_data_preparation/images/operator_fusion.png new file mode 100644 index 0000000000000000000000000000000000000000..bd3a88cfb04825f7469e76bcd48988a596ce222d Binary files /dev/null and b/tutorials/notebook/optimize_the_performance_of_data_preparation/images/operator_fusion.png differ diff --git a/tutorials/notebook/optimize_the_performance_of_data_preparation/images/pipeline.png b/tutorials/notebook/optimize_the_performance_of_data_preparation/images/pipeline.png new file mode 100644 index 0000000000000000000000000000000000000000..5fb3f3defd20eb700c0e16d6dff5d57a1d2007c9 Binary files /dev/null and b/tutorials/notebook/optimize_the_performance_of_data_preparation/images/pipeline.png differ diff --git a/tutorials/notebook/optimize_the_performance_of_data_preparation/images/shuffle_performance_scheme.png b/tutorials/notebook/optimize_the_performance_of_data_preparation/images/shuffle_performance_scheme.png new file mode 100644 index 0000000000000000000000000000000000000000..d09ca3dda379502827d58c1269599fa4381cbf76 Binary files /dev/null and b/tutorials/notebook/optimize_the_performance_of_data_preparation/images/shuffle_performance_scheme.png differ diff --git a/tutorials/notebook/optimize_the_performance_of_data_preparation/optimize_the_performance_of_data_preparation.ipynb b/tutorials/notebook/optimize_the_performance_of_data_preparation/optimize_the_performance_of_data_preparation.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..896ca2376d516a21027d59fb4b3cffab12232dfc --- /dev/null +++ b/tutorials/notebook/optimize_the_performance_of_data_preparation/optimize_the_performance_of_data_preparation.ipynb @@ -0,0 +1,772 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#
优化数据准备的性能" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 概述\n", + "\n", + "数据是整个深度学习中最重要的一环,因为数据的好坏决定了最终结果的上限,模型的好坏只是去无限逼近这个上限,所以高质量的数据输入,会在整个深度神经网络中起到积极作用,数据在整个数据处理和数据增强的过程像经过pipeline管道的水一样,源源不断地流向训练系统,如图所示:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/optimize_the_performance_of_data_preparation/images/pipeline.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "MindSpore为用户提供了数据处理以及数据增强的功能,在数据的整个pipeline过程中,其中的每一步骤,如果都能够进行合理的运用,那么数据的性能会得到很大的优化和提升。本次体验将基于CIFAR-10数据集来为大家展示如何在数据加载、数据处理和数据增强的过程中进行性能的优化。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 整体流程\n", + "- 准备环节。\n", + "- 数据加载性能优化。\n", + "- shuffle性能优化。\n", + "- 数据增强性能优化。\n", + "- 性能优化方案总结。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 准备环节" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 导入模块" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`dataset`模块提供API用来加载和处理数据集。" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import mindspore.dataset as ds" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`numpy`模块用于生成ndarray数组。" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 下载所需数据集" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "1. 在jupyter工作目录下创建`./dataset/Cifar10Data`目录,本次体验所用的数据集存放在该目录下。\n", + "2. 在jupyter工作目录下创建`./transform`目录,本次体验转换生成的数据集存放在该目录下。\n", + "3. 下载[CIFAR-10二进制格式数据集](https://www.cs.toronto.edu/~kriz/cifar-10-binary.tar.gz),并将数据集文件解压到`./dataset/Cifar10Data/cifar-10-batches-bin`目录下,数据加载的时候使用该数据集。\n", + "4. 下载[CIFAR-10 Python文件格式数据集](https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz),并将数据集文件解压到`./dataset/Cifar10Data/cifar-10-batches-py`目录下,数据转换的时候使用该数据集。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "目录结构如下所示:\n", + "\n", + "\n", + " dataset/Cifar10Data\n", + " ├── cifar-10-batches-bin\n", + " │   ├── batches.meta.txt\n", + " │   ├── data_batch_1.bin\n", + " │   ├── data_batch_2.bin\n", + " │   ├── data_batch_3.bin\n", + " │   ├── data_batch_4.bin\n", + " │   ├── data_batch_5.bin\n", + " │   ├── readme.html\n", + " │   └── test_batch.bin\n", + " └── cifar-10-batches-py\n", + " ├── batches.meta\n", + " ├── data_batch_1\n", + " ├── data_batch_2\n", + " ├── data_batch_3\n", + " ├── data_batch_4\n", + " ├── data_batch_5\n", + " ├── readme.html\n", + " └── test_batch\n", + "\n", + "其中:\n", + "- `cifar-10-batches-bin`目录为CIFAR-10二进制格式数据集目录。\n", + "- `cifar-10-batches-py`目录为CIFAR-10 Python文件格式数据集目录。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 数据加载性能优化" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "MindSpore为用户提供了多种数据加载方式,其中包括常用数据集加载、用户自定义数据集加载、MindSpore数据格式加载,详情内容请参考[加载数据集](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/loading_the_datasets.html)。对于数据集加载,底层实现方式的不同,会导致数据集加载的性能存在差异,如下所示:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "| | 常用数据集 | 用户自定义 | MindRecord |\n", + "| :----: | :----: | :----: | :----: |\n", + "| 底层实现 | C++ | Python | C++ |\n", + "| 性能 | 高 | 中 | 高|" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 性能优化方案" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/optimize_the_performance_of_data_preparation/images/data_loading_performance_scheme.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "数据加载性能优化建议如下:\n", + "- 已经支持的数据集格式优选内置加载算子,具体内容请参考[内置加载算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html),如果性能仍无法满足需求,则可采取多线程并发方案,请参考本文[多线程优化方案](#多线程优化方案)。\n", + "- 不支持的数据集格式,优选转换为MindSpore数据格式后再使用`MindDataset`类进行加载,具体内容请参考[将数据集转换为MindSpore数据格式](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/converting_datasets.html),如果性能仍无法满足需求,则可采取多线程并发方案,请参考本文[多线程优化方案](#多线程优化方案)。\n", + "- 不支持的数据集格式,算法快速验证场景,优选用户自定义`GeneratorDataset`类实现,如果性能仍无法满足需求,则可采取多进程并发方案,请参考本文[多进程优化方案](#多进程优化方案)。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 代码示例\n", + "\n", + "基于以上的数据加载性能优化建议,本次体验分别使用内置加载算子`Cifar10Dataset`类、数据转换后使用`MindDataset`类、使用`GeneratorDataset`类进行数据加载,代码演示如下:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "1. 使用内置算子`Cifar10Dataset`类加载CIFAR-10数据集,这里使用的是CIFAR-10二进制格式的数据集,加载数据时采取多线程优化方案,开启了4个线程并发完成任务,最后对数据创建了字典迭代器,并通过迭代器读取了一条数据记录。" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'image': array([[[235, 235, 235],\n", + " [230, 230, 230],\n", + " [234, 234, 234],\n", + " ...,\n", + " [248, 248, 248],\n", + " [248, 248, 248],\n", + " [249, 249, 249]],\n", + "\n", + " [[216, 216, 216],\n", + " [213, 213, 213],\n", + " [215, 215, 215],\n", + " ...,\n", + " [254, 254, 254],\n", + " [253, 253, 253],\n", + " [253, 253, 253]],\n", + "\n", + " [[213, 213, 213],\n", + " [217, 217, 217],\n", + " [215, 215, 215],\n", + " ...,\n", + " [255, 255, 255],\n", + " [254, 254, 254],\n", + " [254, 254, 254]],\n", + "\n", + " ...,\n", + "\n", + " [[195, 195, 195],\n", + " [200, 200, 200],\n", + " [202, 202, 202],\n", + " ...,\n", + " [138, 138, 138],\n", + " [143, 143, 143],\n", + " [172, 171, 176]],\n", + "\n", + " [[205, 205, 205],\n", + " [205, 205, 205],\n", + " [211, 211, 211],\n", + " ...,\n", + " [112, 112, 112],\n", + " [130, 130, 132],\n", + " [167, 163, 184]],\n", + "\n", + " [[210, 210, 210],\n", + " [209, 209, 209],\n", + " [213, 213, 213],\n", + " ...,\n", + " [120, 120, 119],\n", + " [146, 146, 146],\n", + " [177, 174, 190]]], dtype=uint8), 'label': array(9, dtype=uint32)}\n" + ] + } + ], + "source": [ + "cifar10_path = \"./dataset/Cifar10Data/cifar-10-batches-bin/\"\n", + "\n", + "# create Cifar10Dataset for reading data\n", + "cifar10_dataset = ds.Cifar10Dataset(cifar10_path,num_parallel_workers=4)\n", + "# create a dictionary iterator and read a data record through the iterator\n", + "print(next(cifar10_dataset.create_dict_iterator()))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "2. 使用`Cifar10ToMR`这个类将CIFAR-10数据集转换为MindSpore数据格式,这里使用的是CIFAR-10 python文件格式的数据集,然后使用`MindDataset`类加载MindSpore数据格式数据集,加载数据采取多线程优化方案,开启了4个线程并发完成任务,最后对数据创建了字典迭代器,并通过迭代器读取了一条数据记录。" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'data': array([255, 216, 255, ..., 63, 255, 217], dtype=uint8), 'id': array(30474, dtype=int64), 'label': array(2, dtype=int64)}\n" + ] + } + ], + "source": [ + "from mindspore.mindrecord import Cifar10ToMR\n", + "\n", + "cifar10_path = './dataset/Cifar10Data/cifar-10-batches-py/'\n", + "cifar10_mindrecord_path = './transform/cifar10.record'\n", + "\n", + "cifar10_transformer = Cifar10ToMR(cifar10_path,cifar10_mindrecord_path)\n", + "# executes transformation from Cifar10 to MindRecord\n", + "cifar10_transformer.transform(['label'])\n", + "\n", + "# create MindDataset for reading data\n", + "cifar10_mind_dataset = ds.MindDataset(dataset_file=cifar10_mindrecord_path,num_parallel_workers=4)\n", + "# create a dictionary iterator and read a data record through the iterator\n", + "print(next(cifar10_mind_dataset.create_dict_iterator()))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "3. 使用`GeneratorDataset`类加载自定义数据集,并且采取多进程优化方案,开启了4个进程并发完成任务,最后对数据创建了字典迭代器,并通过迭代器读取了一条数据记录。" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'data': array([0], dtype=int64)}\n" + ] + } + ], + "source": [ + "def generator_func(num):\n", + " for i in range(num):\n", + " yield (np.array([i]),)\n", + "\n", + "# create GeneratorDataset for reading data\n", + "dataset = ds.GeneratorDataset(source=generator_func(5),column_names=[\"data\"],num_parallel_workers=4)\n", + "# create a dictionary iterator and read a data record through the iterator\n", + "print(next(dataset.create_dict_iterator()))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## shuffle性能优化" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "shuffle操作主要是对有序的数据集或者进行过repeat的数据集进行混洗,MindSpore专门为用户提供了`shuffle`函数,其中设定的`buffer_size`参数越大,混洗程度越大,但时间、计算资源消耗也会大。该接口支持用户在整个pipeline的任何时候都可以对数据进行混洗,具体内容请参考[shuffle处理](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/data_processing_and_augmentation.html#shuffle)。但是因为底层的实现方式不同,该方式的性能不如直接在[内置加载算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html)中设置`shuffle`参数直接对数据进行混洗。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 性能优化方案" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/optimize_the_performance_of_data_preparation/images/shuffle_performance_scheme.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "shuffle性能优化建议如下:\n", + "- 直接使用内置加载算子的`shuffle`参数进行数据的混洗。\n", + "- 如果使用的是`shuffle`函数,当性能仍无法满足需求,可通过调大`buffer_size`参数的值来优化提升性能。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 代码示例\n", + "\n", + "基于以上的shuffle性能优化建议,本次体验分别使用内置加载算子`Cifar10Dataset`类的`shuffle`参数和`Shuffle`函数进行数据的混洗,代码演示如下:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "1. 使用内置算子`Cifar10Dataset`类加载CIFAR-10数据集,这里使用的是CIFAR-10二进制格式的数据集,并且设置`shuffle`参数为True来进行数据混洗,最后对数据创建了字典迭代器,并通过迭代器读取了一条数据记录。" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'image': array([[[254, 254, 254],\n", + " [255, 255, 254],\n", + " [255, 255, 254],\n", + " ...,\n", + " [232, 234, 244],\n", + " [226, 230, 242],\n", + " [228, 232, 243]],\n", + "\n", + " [[251, 251, 251],\n", + " [253, 253, 254],\n", + " [255, 255, 255],\n", + " ...,\n", + " [225, 227, 235],\n", + " [227, 231, 241],\n", + " [229, 233, 243]],\n", + "\n", + " [[250, 250, 250],\n", + " [251, 251, 251],\n", + " [253, 253, 253],\n", + " ...,\n", + " [233, 235, 241],\n", + " [233, 236, 245],\n", + " [238, 242, 250]],\n", + "\n", + " ...,\n", + "\n", + " [[ 67, 64, 71],\n", + " [ 65, 62, 69],\n", + " [ 64, 61, 68],\n", + " ...,\n", + " [ 71, 67, 70],\n", + " [ 71, 68, 70],\n", + " [ 69, 65, 68]],\n", + "\n", + " [[ 62, 58, 64],\n", + " [ 59, 55, 61],\n", + " [ 61, 58, 64],\n", + " ...,\n", + " [ 64, 62, 64],\n", + " [ 61, 58, 59],\n", + " [ 62, 60, 61]],\n", + "\n", + " [[ 66, 60, 65],\n", + " [ 64, 59, 64],\n", + " [ 66, 60, 65],\n", + " ...,\n", + " [ 64, 61, 63],\n", + " [ 63, 58, 60],\n", + " [ 61, 56, 58]]], dtype=uint8), 'label': array(9, dtype=uint32)}\n" + ] + } + ], + "source": [ + "cifar10_path = \"./dataset/Cifar10Data/cifar-10-batches-bin/\"\n", + "\n", + "# create Cifar10Dataset for reading data\n", + "cifar10_dataset = ds.Cifar10Dataset(cifar10_path,shuffle=True)\n", + "# create a dictionary iterator and read a data record through the iterator\n", + "print(next(cifar10_dataset.create_dict_iterator()))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "2. 使用`shuffle`函数进行数据混洗,参数`buffer_size`设置为3,数据采用`GeneratorDataset`类自定义生成。" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "before shuffle:\n", + "[0 1 2 3 4]\n", + "[1 2 3 4 5]\n", + "[2 3 4 5 6]\n", + "[3 4 5 6 7]\n", + "[4 5 6 7 8]\n", + "after shuffle:\n", + "[2 3 4 5 6]\n", + "[0 1 2 3 4]\n", + "[4 5 6 7 8]\n", + "[1 2 3 4 5]\n", + "[3 4 5 6 7]\n" + ] + } + ], + "source": [ + "def generator_func():\n", + " for i in range(5):\n", + " yield (np.array([i,i+1,i+2,i+3,i+4]),)\n", + "\n", + "ds1 = ds.GeneratorDataset(source=generator_func,column_names=[\"data\"])\n", + "print(\"before shuffle:\")\n", + "for data in ds1.create_dict_iterator():\n", + " print(data[\"data\"])\n", + "\n", + "ds2 = ds1.shuffle(buffer_size=3)\n", + "print(\"after shuffle:\")\n", + "for data in ds2.create_dict_iterator():\n", + " print(data[\"data\"])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 数据增强性能优化" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "在图片分类的训练中,尤其是当数据集比较小的时候,用户可以使用数据增强的方式来预处理图片,从而丰富数据集。MindSpore为用户提供了多种数据增强的方式,其中包括:\n", + "- 使用内置C算子(`c_transforms`模块)进行数据增强。\n", + "- 使用内置Python算子(`py_transforms`模块)进行数据增强。\n", + "- 用户可根据自己的需求,自定义Python函数进行数据增强。\n", + "\n", + "具体的内容请参考[数据增强](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/data_processing_and_augmentation.html#id3)。因为底层的实现方式不同,所以性能还是有一定的差异,如下所示:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "| 模块 | 底层接口 | 说明 |\n", + "| :----: | :----: | :----: |\n", + "| c_transforms | C++(基于OpenCV)| 性能高 |\n", + "| py_transforms | Python(基于PIL) | 该模块提供了多种图像增强功能,并提供了PIL Image和Numpy数组之间的传输方法 |\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 性能优化方案" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/optimize_the_performance_of_data_preparation/images/data_enhancement_performance_scheme.png)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "数据增强性能优化建议如下:\n", + "- 优先使用`c_transforms`模块进行数据增强,因为性能最高,如果性能仍无法满足需求,可采取[多线程优化方案](#多线程优化方案)、[Compose优化方案](#Compose优化方案)或者[算子融合优化方案](#算子融合优化方案)。\n", + "- 如果使用了`py_transforms`模块进行数据增强,当性能仍无法满足需求,可采取[多线程优化方案](#多线程优化方案)、[多进程优化方案](#多进程优化方案)、[Compose优化方案](#Compose优化方案)或者[算子融合优化方案](#算子融合优化方案)。\n", + "- `c_transforms`模块是在C++内维护buffer管理,`py_transforms`模块是在Python内维护buffer管理。因为Python和C++切换的性能成本,建议不要混用算子。\n", + "- 如果用户使用了自定义Python函数进行数据增强,当性能仍无法满足需求,可采取[多线程优化方案](#多线程优化方案)或者[多进程优化方案](#多进程优化方案),如果还是无法提升性能,就需要对自定义的Python代码进行优化。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 代码示例\n", + "\n", + "基于以上的数据增强性能优化建议,本次体验分别使用`c_transforms`模块和自定义Python函数进行了数据增强,演示代码如下所示:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "1. 使用`c_transforms`模块进行数据增强,数据增强时采用多线程优化方案,开启了4个线程并发完成任务,并且采用了算子融合优化方案,使用`RandomResizedCrop`融合类替代`RandomResize`类和`RandomCrop`类。" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD8CAYAAAB3lxGOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOy9Xch2W3cedI251v28+6v52pgaTWwrCbQHggeWYlvoSagKWoPxQE2qtKkEemJBqWJTT+qBQnuiLVSUgMVUxFirYpGIiBpKQaUmFcUGJQ1Vo8FQbatFst/nXnN4MP6uMde6n/fZX95Nn833zr2fd/3ca801/8Y1rjHmn6gqPoVP4VP45g3jr3cCPoVP4VP46xs+gcCn8Cl8k4dPIPApfArf5OETCHwKn8I3efgEAp/Cp/BNHj6BwKfwKXyThy8FBETk7xOR/0lEfkZEfvjL+Man8Cl8Ch8nyMceJyAiG4D/GcDfC+DnAPxZAL9dVf/8R/3Qp/ApfAofJXwZTOA3AvgZVf1ZVX0P4McAfN+X8J1P4VP4FD5C2L+EOH8VgP+Nrn8OwG9aHxKR3w3gdwPAEPkNf8O7p8cxyukEkLiS008fCu29c8KWX4Vuy/Ko5B9EIBDYqV/LoOs4H5AhGHEcAyIDYzm/OnIa5Avkd805ADzifx+OVl+4Wu7mgZ5SxVqP5+++JnMvP/OY366/fMEvxwPfAIFWKHSqnSmgque/OTHzXKE67Xz5pl7lg+TkorniZ3/2L/wlVf32NV1fBghcleOpyFT1RwD8CAD88q99pn/Xd/2aHoMLElKI4ILlAjZCAEeeP/5aRBsFJZlIceEFCzQo7qvjGBgQbPuOfd+xbXbc9xu2bcN+u9nf05M9w+dPT/jss8/w9O4dvva1r+Gzr30NT++e8LXP7Pzdu3f47LPP7JmnJ3z22dfwta/Z+bZtBDpw0Dnn/Sr7ulSNXrWSS4HU5fe6LlOyGrYlQL0hewPWmfegmiBsSbAj1Ujmp+oiPs7gUc9nCrREQ6kcdAUj+i2Bux0rNVUmvXAi70ofaKb1hZmtqnh+fsZxv+P5+U7nz3h+/x7Pz8/4/PP3eP/553h+/4zPP//czp+fCxiAPJ8ep8gAQh7GIPnoRP/7v/8f+l9OicKXYw78HACSaPxqAP/Hl/AdDxet/hGcf3HS8MVDi/xFPVm/6sWrV9G9IrxKQV0BgL98vvshcNXTT3VewqQAROv+y3FefqI/fHXvUVgF+YPfe/Sdi2+9RKkefk5OL+oXrOmP6cn7MkDgzwL4dSLy3SLyBOAHAPypLxbFywWyUvUWHtRV/sY/y3L8Aml4KLH6oYceh8ca/OqH67gfmQl8+6UG9I02ruvPlmYWcJJXdvHayM9mxKuCRlpemTt5cH4JkQ/ifKkdIkyiL1gOj4J8AXC6CB/dHFDVu4j8HgD/KYANwB9T1f/xC8YCQNJ+VC60oIpZsWIXa6aXchYI1J+LBikqDxrGKs1dd8tUTAFEFHMqRCbmVMxjQiCYx4G5bZjHARkDc07MOSFzQi//yiacp/uz7o1hZRKmkmUCSn6MlfZflWzP35LXVpRMDVYEJaFKaly0WOe0tNO5zpnVEno5tHNQcbB55j8ERS+7t9qGvaxX7PsU1lzzMc5fy8ZOeM+m0asEsD90JccvhfZ9lQb+UW6vDV+GTwCq+uMAfvz1L1yaUJ5Tq2zx5qNTq7aCx0xc15QuF2qRqqjjRtmeIgLoMLkSj1CHf2sCMvI4ZUJEoHJgTmush1hFpGCEoIJsNQBjDBz3O45tw7zfcdzvBhj3O+b9gG6HgUj83etcPK3shCyhqSLrF9Tc0551DMmmdLZv7fpkuFycsz1c53MeCWDzmJgBAu4TiDQLgwH5ZZq/g3w1iHwjyrfn90yr5VL489d8leBJDeA9QnpxEfB0NizHFugdfdTQkSl8SEL6Y5lOji3KMgH2lUDwpYDAxwnqms6vXPM3BAzhf7VRI+c46HOQWZHldXzARGL4R1UHRBTi2l+OiUMmAOCACb0chzkTj4FxHBARHMdhQuFHPWYyBZ3+G13rPKDHgTknhgjUhUBJ+NO5CaTDT5h/X1pNVQoh/HnF562A+rE9QyAwtfIwjwNzHn59QKem4y3SP0i4ywkry3kBqhIriJiI3hEQSKrYkGkQ+FdJZCwdNwkYUK8vBfkBwb8K7rUMNluxPw6t5S5AkAmjuheChA+FNwEC1nwqZ0bT4xenfi/l5xETwHo/NFXgpV9HQ9NhLMC79BShFWZ29U3SwlMYbV0IdXPyUto/ugIBYwLTmcBx3HEcd8z73jT+vB/Q+4GZrGBi3g8cWDQo9RLUMRjJBYVu18WvQourFgCwZn8o+Je/W9fWcdwdBO4GfHPiOO6NCSQYNM0/MLzsRKwrFXSsfI5iM8SOHP3y2WpWpdWb1o+/LJYChZ63KDG/q4syeaTlWdBVk/Zzq2kAtJxnHFpfE/RPre9zHb8mvAkQANDKOvNLudWoRIFLJpa/pVTiNMofgLeavJkYzLQfw0nItC6WTMsssMCAyOGfOqhGqn/XGnUJ/moOzN1NgOOo43FgHnZ+NFPgjnlsKTCI4wIAK3VmOh3Zr1bM7WrpqyZAKLbwWPBXkFCdmYfjMJPnmAeOu5kIJ8GHYEQexqDxEgPqXcDqrKDyp/1aYM9oAAI1BRFKY5eOLBJvW9aUNMssQwgzR6XBMkiFNe1+igClyZXO6/dVdmU5b5DUSEGAWrGsr7Y5oO7s4nqjflnTYGtBnzN8om/BKOb6oLZnygpZvpEgNJ1NuG8gvyapEMYwE+AYgnEcGPcDUwYJ+6w/p8rzCIegmQFpT7v5MPxbwkdiAMpCTza0OY5Ga2/RPkLr199s1/QUnT8CBTcHIj8zgMDA4H6/OwiQ8KMY1hCBzAF1IDAfzYahYqZXaH8xU0ydOSQTCKCGIM06EeBCqMULIcFC7F6BarEFZtynZsf+gJUJrC8szsMrNnAO0tp704+Xz9b5a8ObAYEzk+ra2gJl3e2/aueEpNTAT2Co9UAV5oRo2KCzGpHOEi51e1SrAQeWDLfxVBVzDGxDuzNQho0QFDMLjuMwNnB3FrBtpfXv3RnY/tKLJZnPdAUNSU1aDMAGkUAiL8ZsyimITHcAQI1Wm6HXm+aqWnnMDKZOHPdgAc94fr47CDxjHubbkBB6GAgMVPls22agMDaMMRMUQuBFFDJc30VZx28jwM6ZXDQC1WQQLCxXFLyaTLQpafeNTLL2Z0HnsrgID8cgPIaB9vwStaj/heJKVtgh4aXwZkBgDVlWXm/MA5qPQA0AgvoENvQaW+OukgyNWlzQhcZ7AliJqJimCG0jISPxNxUyJnRoNkjAWcGoRh6Cf4S9fHTan2Cw9BQcmdY6BiDIdBDzEWQ2YgwYiLxkJoBhjs6gk2HCRHcej/Yr4V9obJYeiEnZb2H/H3fT/vd7jIx7b12lMABIMECBwtg26JxmCgwFtgHVAdXNfTMO1OpCr+rAp8YeJhwgRrUT9fJSpXIjsfN6tdsdJLoqKjDQbKQvMIFG95G/s0MwTREvOwHpNhWsQh/vP4KZysdX0Bx4iJ2R4QuhTgG9AIx8/Jo3nYGFyyv7oDXZJMQqJUBGnTKmVlD1cQiSQhX9/027Tuo2c3/AsW04np9xPN/wjIEt6XA0CsU8jhw2DKBVMHvQDQDIqz7Clh6ADIRTrUwG8+bnGAUar3BWPd5wJUZuXIEAMPXA/fkZdx8Se39+j/th59N7SQbOICAi2MaGsVn+t7E5G/DyyHxVnmTUtRDYijOHdW5Hsahy/2WZUk65fB/5+9KRys7UFwYsqKr5R44D9/uBu5/HvXlMHAexMWrYrT1nkSu4b+M03PqrxgQelh0br4DLZhG3Zh+l1l4cNRcmga43mrW1JCHiZwAIsOFjVAzb2AEG0wYU6Zyu5e84nu+4788YMrBvG7axQY8JqHUdHrebNZDnZ9xuN4yxef6kzJyk1NH4iRoLe9X9mCAwwi4wEIBmP34AVxoNUuVq8hTqFe0ZS44ByPPzM47j2cbFxxj5O4MAyi9ATsJtDIxtMxNqGw0EOF+QDnSQmoCVIMBgiDCFRglJlCXVc2UEayPJlgIQALTzR4OWNA/VU2KO0jw/DtzdbJr+TDKHYLkovC1x0CyT9LVED8tXjQm0AudW1eSSsC28d03aL8wsXNSnrk8ThRjVuE/97MkIOHmuE1MLyOJoKwAYzgDUPeXH/Y778zMGBM/D/AZ6HCaMx4H78477/Rn398/Y9x1jC00OOnq5sHCMurYeCikgcHMmzgHkxJ6J7hgEijCYzwHuZpD8zah0fcLiKyZwvzMIvPdRlSAGUDqLmYCI+Qa4p6CDwDUYCLOFhQ1cjz3o7SLKlRvMmeSrm4FVTtWzQgxpaYfQcJoqjQWZDgQEDIcrDG9PjtVksLgpRwymWM5oYPCa8GZAoAtpCZ9ATgW5soOg/E2X08XZb7MABpsEDYwKdoUirDHwUrRM6PzCJEAM/VUHgm1mN+AhBgbbcAeeA8exH2kybNvuguDfNalpIIDW2ON6YQABArD75tPsXYNxnsI6XGsLfFqzlDVBwh/XOg8830v7MyAEE/BHGwCkY3CY2TO2YgHFaLrwV57ZNLgwEy7YUTOpWttr/PJagTxiAS+YA4CmcNdUYWNd3AuUw8S1l1GkRhBmaZhj2QqI+XzFmMDJlAoNl7Y3EOOjmRLFs8S22jEvpO6enbOrKaDri35bqbDr5+QSWqzA7MTQDrMxAT3Mvj/udxxjw30MiCI95fN+4Njv2J+fsW0bnn26ctjJZybgx6S4IfjRKFzYU/gNAOJcg8WAQMAbdAh9/MkQbDlltYCAmYCImRfmBwhGUH/JBIBspNnQHQRC6DPPzgRAeeznJPQ0yGgFgjgfxBZODUbXW3phPnbNn36BhwBwphXRPvITqjjCV+QjK9kciDKLd7mlnoQfZRK8JrwJEADSsgQAcryF1IfmD6eOnmU24+E4zpUSdjs/ewaC9T7djgog9pFxBVhM6Wg/wyk4fBjwhjkOHPdn3IeBhjFrxRx3jG3D3e3hbXNavEUjtg+dj11A7Hbd11XwgwmgGmN0Cca9EvyBbQvvvZ0zG8hztzBUpwn/cScQuDcQiOLixl0g4Ewg/QHEaB4AwckZuJgFuYjLYiasDYeaB0IL6dJScCH8fL02mR6oblpJwEZasv8oG7Nkj4FfJkhrxunxMRt8kII1vBkQaGApKE/8CgTxALpAn6JblDtXY7z2mAm4hhIThlaYwk/0eCPNgAIh/KP7BjTswePAHD6JSG2+wR3A3A6MY+AIDejaMFcX8oSVkFuiVs9wNjIWfAYEv1d50NPRKLkYEA3vtXAQSLMgZbPO55zWLeggEKbB/f7sXYRoAMpAEIIfAjvGKrQSNKLy6YAwiAmsZgEzinQgjqJy2ZTa9aIsVJfzYnx8fRI9bj5E1VvvhTNI8wdqmmleu96eQ/KLClRRnAHgK2UOAF2UywMvLnRub2em2At7pgOM3J22s0mw4vsCBGK2vaFtZwgNKhopqUprS0SFjSdiKH8cmAAOPOMOJPVTnyTUHXtnex8oisiAwGBQ10iBZ+EvEFhGX0rlI5Y823djI9smtnLSHrQaZBYECIgxARL85wSEO+Y8so6T/5HgpJCm9q5uvxR8yp9d9p6RM/3fKs51WHLS89IU7R63K6bvKHCP3z60aK+ZVwRGxFJ6Wlh4vXzVAV59cFQzW1YfC5mFrwhvBgQmFaBlskSv6+cIJyutDtcvVBCkaaF5A/0F7V9wnywARYwpnClGNYdJFDnVeM6JKWL9v8OmAoftz84Jje6gqOCLv9D6rXsrEd/Tzr9FyiRSP0jou3mAiDfjR8aRXU7ZeLnLDvWXnRDx/oCMDTImtjGtPLdoyEmA7Vmt0meh711+zBckPeOVbzh19zDVB5HBakph60AMxRi2BoTIICEuIEgtrFoK43TOf8hela5/yMj1ky3Kz7uEw/zZti3zUtR+UN2ijVMJU8CaAZtHXzy8CRAgFuY3fBBEOQfy/uMYcCn0vVyYl72M2i1e5GoGAA0S0IjyTDwwomtQpKbWiuCYB+QgbR40z9/uA1yqgru9Zx9mKrkCQcRvsdqowdL+A+rXCAeSGMVPfpnxlH5Jp93Y8lkGgsKccEwphgzMsWEoMMbstRCOLy84i2d43KttHwAWNcIwwnUUQuzArYANIbZ0DbiJF/MPmhOXgeBK2K/+5ulebz7Fs0SQvTzbtmP33o9t281MEu4OjRzWeRuoplTWWQJre39NG38jIBDoWlfiFSQQQlNZNPxJ5v03vk9O/SW4oKBmo12nzCPJjyrCWcRcQhEIbb9PZwIitobAEMFBzMGcga5ttlh5Zz/R/xJ2FAg0QBBqCJyPMg1C2NvRmUA2umFrJNhYBNM6JmphPzMT2NwnQMViL6VAD7Fhv2NsObdi20wDRzGuPMzioenDsYBsjHRc64XuhcnIrag0tw01hoivBCWAzxHpIGBdeFAa5RlgHj4d7SMr816cgxREnhSr7QvT7hjbhn077Hxw+W7WE7OVnpHIJ0g3egOXVhmtkD4Y3ggIAJPNUgIACMjZErQvDlw08TPxJBRtqsixNBbBekpiTQODyqHYbY5ueQToiHf5iADzsJWHshcAsHHx7jOYc6TfoIR6BQKkhrVsrkNi0VgAgwBgU3FPQOCaZ9ttpp56HGNEaUt9UzYMica5Q7aqnxRJYlf2vOVzc8qkQILAahJE8a/+Dwa95fG8Jh5VQsfjxWNSmFSeoq20od1tmbcQ9hjUo8nodL0OYIglwilhSkcR4LbfsG07brEi9bZh7jdbNGZs2HfFtnn5iNQ09yxXbm/+bzPhvjAGfBgEROSPAfheAL+gqn+H3/s2AP8ugO8C8BcB/KOq+pfFUvNHAPw2AP8fgN+lqj/1moQ0N10CgDgAqA/GWfCfGqBHUs+ec4KUaHmwutA5UTUwqHio2+7BZh+AwZzQIZgTNoV4mk9gHoLD1yA4Iue6IbqXWlcfcGIEXUBG/92fr4QSEspAdAuaAW8mwRihajYD3xGsrICguuAkWYFpazPkQphtgpVC4ZN80kOvZh6NLZ9OAFgqoQS/AwI3bSIRXPztftPIkhAEQnUAaPM65iwwmKq2opNqDuDhv1w6bVkHMkZfsvAnCAA4bhP7fuS7c9/TfNm2mAng5pfH5fjlIMclWG09gYAb7yvDa5jAvwngjwL443TvhwH856r6B8X2GvxhAL8PwN8P4Nf5328C8K/hYuORNaguTOCSARBIJBXi36TafN0JfV4sAXXMWBcNSinLQ7kChF5k4HIGAwMrAWj9wQNyaJkAUBszoBNzbhjjwJwb5jEtG2lgd+FOaryaAuEFDq3J+ci4ggnQvAEZGMPHBWz+vjv8EuCkTIJkAtuOMaJXof5qZYUJEZvnMMbm9ywdKrWYgyxlmMklEGSmo2FzLYLOlZC+OXcsK33jgkSQJtea26HUjesDeI4EggPHwSBwFGvw83QqZpoUYT4+Pd2x7zuent7hOCb2fc807HswF8vzmJst1dtr1JRjtewE/ijjnsMPhw+CgKr+aRH5ruX29wH4Hj//UQA/AQOB7wPwx9VUyX8tIt8qIt+pqj//iu/UucBtUi3mFoDgv6UiaDPYSltIYX8vlyWQNQ0Gkowx2ECiCmn9BlRxXqzlmOUIO/wB8eMcAlWbLz9iReJxdEBKEPB0JTWvY/MNtEZwDQLr37ZpPjOGj2zMnqUahz5kc899+QRMDEnz2XYYEJhTsK/96CCgtKLLhaO3KG0Bd/qCsuRds7aBPGYCVNcd6kjaODDBAELBdL7G9NtCqesEn7pHC74kCBw4crhvsClyFrqd+Hy/4+l2s9mC88Dt9pR5CCY4hk2gMl9RaX4zKYQYY7WpcCS/1NYfhW/UJ/C3hGCr6s+LyN/s96+2IPtVAD4MAutFKNyotWYTEkXi3oMLIHgYZPltuQzbvrp5uve5tCVxDingmDoxpkAxMWFre0xfphSqGOrLmIVDaQwozRJstJ6YQZtT34a/MtshLRrXNV8oMcGiHGQL9wJpg274fES/tk83FgXUlmazRjtt9R8YSBsjCBBY62TR5g2HpRWDlmpH6cIQtqRqXn8BBCXweT9/V/BknnVyz3GYYNdU3wP3WN/BmcJxAoRzb0EAQuZ4apavgW/MlxjYYg2JbTu9V0qn2lvWE77x8LEdg1dpueQkQnsR3sIT0qLxSpbqnuPr+lrZ90Jn9Wm611KnFUnT/rHEVBEu++7CAPIrQqsaScYRKVYYk/CmWv9RA5lBdXyFo0oyAZ+EdowVdLYaELPMsAtQiNWG4M+EIxDu6IP4Vmq3HXs4q57MWRXn275h7Df72zb7rmw0DyE0eyRaLX4c5pWP9RjV++Z1Xdvtgg2A8h6nGgBMws0aH4qR99RNTCUTgVlAAcLhDr2D7f2pOOZhE5gO85vI/Y5jHIDcbXn5wxdqOSx/giNBek5bYh1ToYgt2OzjcwGHSX8vdkUuBSSoRpg6gluxBjB/OHyjIPB/Bs0Xke8E8At+/9VbkCntRfjLnt71lhC2NUoThwNEmQWgBBIuuFYRWktvp1ivIJMpQcVmv9XX6ftYG2c4Zfp7yNg08xF0uRqjTdsNoYnGO93h2KZJcwWLa3dfYANjs4U2nKJHv3oNshkLUGwJIOFb2DYDgdhD8bbv2PbNurBuBgLbvmNsO2TbINuGMidm2UqpplxCRzhyfRUgnRgSqxWtNaHtRtNrVBRJOqIs/eYZEOyYy75VjXnbyzaIzQV2WxjBdkzcxx3juBfAjqBTJuw4BgR3YlwHYnqUTgBioD7jm57muQj/DEYS56EkqM1Y+qU11fIdx0m12Boj8eHwjYLAnwLwgwD+oB//I7r/e0Tkx2AOwb/6Gn8AcOYBrGtrFZ+ihx0AghYWbITQKAnQKaz3UiPT4CDSSCsjYWMBfJa/uf4XpYqE23RnJjA08qiMOtZ4xCz0WlxzJABIaOhms4+8PoPBcCCIYcA3bNtofdfbbcfYd5+8tNc3xIDHymVEjjznM8siWcIQXybLgEDWlklsru5J/xnU9gkEQsBTU6bWR9rhdl5Rcz0ghW+eTYJx+LoLknGacLuWZXNSjkxbth11RUQbA6RwJ5Mh5oJqD2d/hrezYJzCEpJIQJlcmMMHwmu6CP8dmBPwbxKRnwPwB2DC/ydE5IcA/K8A/hF//Mdh3YM/A+si/Ce+QFqWD7uIcYalMmcAIHSOzgSKyDdqj3guPhKxp/epCjsRNW2vAiL1dzoTqDOVtFarkuGDU8TmFgyxhjGCuvobyTiY7sHARcWW4cZwoXQBHdtuwr9tvhhHnO9kxwcTME8/hvicABP2bd+wbzZteXcA2LZhcftMRowNcEZRAKBVdlRykIHe4mPJsqrDCpRvbga9bS/Hs2ZnoGAWAPRzeN3MK3NAFcdxQMYz7s6w4OAbYDDngSNZgWubTRIQ1IctR7uIIpgrE5gLC1BFW9glBZrLN8qm2wEBbBDByrheCq/pHfjtD376uy+eVQD/5Ku/zuG0SIDHiRj/LWAfYMzWY7Og3nFt2ol8iyueBIhO9dw4EHXDINfXe4kJIHiA1rkWI4hqna79J+Br3WtpEPMA+RyKQoIc3TgG4PR8bLvZ7MOo++5Cu/notJEsoVgAT6jZXPC3BBAbQBSr/MSSXzIGZBuIYWwFycV4DBsVqrGYqTVk8TJookj1mfjMVdDuUU8ACXfT8nSh9eg5kNBM7+5r5oBO3zFqpAlg7eCOGFYsB5sGkpGaVWSrToc5FHMachwC+QNq5OHFb9FWgnloKLkovGIBDBSr2fOh8GZGDHJgm9yu7V9BiRoLXDEB+G9eGLkKcWhtjrHHwywgWAMXacbTIQHczE4DYUL7U74CEOCCHtoC0QC8T3LxCVL8rmElhHFAxg4hwR8+Km3fd2z7rebo8ww9NxNC+HkiyxjiE1x8rsDmZsSQ9CsUE6i89jUcYuBQmD5wYNaopGamXZls6z3H/ZNO7HS/VUmeSv8nihxzo8VfEwQUMo40A6LHAAocc2JMBXAgzRI4CxlbgqANU54poGWmBBvAC45BijNYDYjJtvKTlsdiQMvs0BfCmwSB1DJCWpoofzY/QTtPyi5dhFcBzs/kZbGGHP8maO+Qu4WYSVQOQCkJkWiMIGYc5rnCzAG1LjUJ6hzVJwFFlDqBmQTDmIA56naIO+7GvmO7PZlDbzPH3u32RGPSB/kN/DzWK8i1CxwEcn5ArcSTi3sMbozRaIPBeAlE2aSEMjhUVchyBP+2tgsqZ11/fam9UxtaB47NedTgoGYO3K38vadFXdvs7hNI7a80QMu7PwdNBgq2YADQNf28OK/r2XwFke7ItUS2pLe7KodgXh8ObwMEBD6V0i9bTutGuw9qJFKPRZ9prLYa5/aYtJYlLaIQdjlFLOEEo/NuVpzDJrFsNi2OITEiL5x70v5yB6GYRRddf6m1TdDHfsOWfzu22w377clMgf0J++0pvfv77QnbKBPgNK9+mbrLC3rE6EEenRh7LqjT03BY5Vh98TKMBph+nKVBStXnWq9r3V6FRz3jl/clBIilyE/mgMzQ2vYX4//HdmDM6f6Qrc4P0/i6KVS3zF1kebqPYaj6YLADQwembljHXSDaafqQlrCAaDm9JcswRqkyGzZG8rh9cngTICAieLrt602QqdUazSMKGZNNRjZc3/E2TIFgC/UG0e6z8PORh/LyuP4rDWSAVACwhYCJLdXFM+P61OGy2XMcgHv7IaO67bbd+vNvBgS32xP23UBg22/e5bf79dJbIF3w2wq9Iwq5FzbNffSyApoHWnsxvETPjTE4w+mmbiu/tVg/PBwmQJxvCf0S3LAzgfWdoPgzqfi1r8FiskFbY9iAKwzbL3LOiW3u/u7E1D2/uUcXrDtjLxc8ydmTnEJp/64mLc+XEfhmuR8sMwtvAgSGCD57eqobq/YnRtCytWgSe0SIEUhdxxPUAKrhnSPS9gxp7ASOlwt4+Dx9+xt1HNZ4LCnLBKAAgXTmuQYaO2QM0/C3J+z7DfvtHW5PTw4C7wocHAjCHNj33eJhFhJg0JiJ529QXgjzEuoAACAASURBVMWXtVqyykNQlDSVEiqw8K+2aTg7GQAUyN60snFf04jXZ6Sq2a95u/YUFgL/NN0UHQDyWH+ZMilTSQFgq/Y3VXHTMBst9ybc9xyLEb0v2QvDwJws1oHNC6eVx1pWmSl7ZryijUZ4EyAgIni3gACPhS5BRGcBfCJVCTwLraai1gul5BqNoPiW80tW0FLRgkIxUBtANECImXwIrdRQLgV/c+E1z72BwnZ7wu3pHXYX/AQBv5faf2MmcGsAAImdfzoV7ZOS0rythtgzmPmM60ZaPwAGIgsQgHSbLvXSGnsv79PVqiE4DwQANfCsIlEgB/UkAICO7YtWjjKGz44s5WBDxf0Nykf0rNwcmLdbAUGaG8EGGtuMAhbwmAPIOpalmxUJ6q8IbwIEhgiebre6wekn26fxHSx5lBIq8XNu1LK8JBzBC6BQu/4uCYvrbOWcmJhEE+aJg0GuEiP5VEM2CGQjB50DwLbvGCM0u89Dd3t/d3PAtP+Wwj+o/3/V9k3gUWVESJrnZxs1ygSLgPuxOabotSWiNiiyVP9SjhzOvOAaGKre4jrrEB0IDLyKBXiqzXnL2t//ViAQiHXV+qQre0ywDcXcFBvFBxHIUeZAF/4yCXifiGsZJlRsiq18SqlcvkogIGNcmANojECycdIzdF4y6g09CrExgSsWIFyv/TdulCwk/M5q//qJiG0GGmAUvolBQh+LYPAxhf+2Yxvm9Nu2G8a+43Z7wu3JBf/pHZ6ezAy4Pb3zrsBoXHtO+d02H1rs+ZLWwKSVa/7rv4VSZo0d2icAgJ1+0UtQ5679Nc+qetXurCxtNQtY9BtWXICFIBZGiatF+5/qsGLMgVzau+8IAygNAomF/mDrJRjSCDSmYEZb9O9FT0wsJLLvNxqQtS3+gGJtWZY5nTXyV+ZBlMSIunSW8pUCgbGaA7GYSlScLOcRTsIbWpu03gkAQtvROdb7ON/39LRvL1qQr91thEDpEZojIlNAtfchi/quO3un87sDwe3JzYHbDU/t/DNnCzVSMI/bVpoBXA5LfpZQxFPOiqflWx+AAQNBPJYiFAQv4xVIX+9jNQuy0K/CuVFcAYCGxkSBWx/HUQDQ/9DYgkU6gKG18AcA0Vq4NEAgHK5jHNjmUfM09jDdiBWkc3BlAWemaZXiQMTKj3Zhal25L4Q3AQKn3oEHws/K+xoMSujy3XhR2CF0odGvrq+AgL7Hwt/YbraD/l/cAVHN6Y0/NFD23btGDyCw7r9uCtyebth3O27bzd/daGBQ9CxcNQbOU2WA89E1b5gv/Mzy3gOhb8N7GTy7Mj6nbzlnfLj6rc0tOdn/5H8QOeWTwbjYQN3rH3UmELpYCN69b3/z+za2w8YVHFOcnfnozBHDsnkp9KW9trCCwVJibOoNZwOvCG8HBJ7YJ8CCH0iHaswXDCCO3JV3AoUrrf6aeOl5Xe/7TcX5Hgt+LdwprbENn2c81fZCNa++0f5tC2F/h23fcbu9w9O7Jxd87ymIsQDbnt19PGnoMQhkMi8ycJWhLoj2lEuxluYFNJlrLPohsiyEQeAuyzHqqapLqNrO+WBgl+VYWC+XR4WBrqgCc7Y6ZodgOQkdyFRJHAsA4rO21oJmckUEx5gYM+ZkDOomHJ0NxI7MabpRRUk/N3DSVlY5qGuMrxoIADdiAk2TEwA0c4BLHXy/hL+f1/NBBZkQANWI0Y4UN+K5l/MT1L40PzEBDfvTn83dlOze2Pbu+b89Yc8egRJ8u2+Owm132n8xffhSq1CmBegN1nOwOvKu9JKVUqn/fBcgmx+o1XGcNqfGAjDCcepfyeqSOgIl2JyFNS8tDoKNzP9iCqnmSkDlkKPA/YLuIexdhRqJrbxpfMeMwZh9PMbA1IHdx20YCGw+WctAYI+uwtiDcZAjOdoomwKcDM8sL1s+aADeS+GNgIB5TesaJ21RbADoQs3HVfjXe/Ru/nYVz4XGz0oAisOuD/VIAgBC+EHXgAs+fUxVaCBQafkbnZtj6VaTg8ie5FGGvctvSeMJFCJzwQtWm/Q6p1YU7Iwrimy4EoAQ8SrSYcvCL+J7HyC1Xx29LOnjmXqhq+W5sy/onHdVtfKa6s2D62YxExoL4EHLnTWFwk5nsDtlVa3bOGZmlvbnORvcRRheJOkF7whbQCuZnvS1SI3+fE14MyDQmQASACDwFXJQjo4FDOJUWeA9oloBR07vZWQpt+XUOYOAVW/MaaAfWlxdVw1jASsA+LF3Edr9sRcTiF6A2+1d+gNyqPB+a2MDZNgONn04aq4h1gSlkhz/lvCzRgc981Ko7mtd3lKSkZKUtFnZFIhRcvG7nVyAAOel/9YEhoR9FSQGgVg/oFhTPOVp1zAHurOwYjp7TiJNwQY0112Aj+AcbaPZAITNp3bnvI3BeasQxM1KewK6FSkhANi+akxgNQdyAd3c6abTPAJtqjxr8DUEmEDA3832+ogFAIm0/R4AaDcFuPFFOjz97hFCAEEBQE0qSaCg6y1B4F2BwNOTzwnw7sKcJnzL3oRBAs/H1k11qR2r8XoWF6H9MAhwaACi0TYXVx4JvU2TCIcY1XMI0lXaueyL33egOOXvnBNV9eXDYsJPgHMwgVT/JyfhCmxpDvg/EVfunub5y6na20inII8YPA8dXnwD/i12snKvS4xKjQ1kXxPeDAjs+9aurZvQwWAxC/yh8/kDANCqGfoo6FmkkNc5utaXvFPPkq3ZACA/EAAwSvvrQNJOifMAgmEg8MTjAZ4aCPAYgOgJ2MYo47OjY4EBqhxLSCj/PgGo1mx4hfA/dFZnAYLWfLazFHQGAmICKfBeSu0eg8FyTnV7eubs5bT0TE0venUp0+PNHRBgELP6lrz6aRhEnE+MAj62+Y0JSK7bMFKLSw3rvgCA2AHLCkTb94tZ2XdeE94MCNxoxKAMqvwUfmICKXMh/Egh1AQCPtazXfBXexanQtXlvrZnSZiwNGIXeAYB9fNcCNT3AJAEBAOBp6d3OQbg9vSZDQpyMyDmEdRIs1j2yyo8HY2o4xUAMEMoOe0LozysL7wg/w+FrpdZ7DlYYLDMWMT5vMq21/9DYIgHyZvf/Gm++YqM2YAgUmuy382CJAdrvij3YZKJ2NbuOTOzAUCNCYiFW0TEJ50RODLLza85v8r0FBOIYerGBL5S5gDw9MTjBKIggaBDIrD16vx31v4NBAgAEhCw0n8hWl9o2u39En52BnYQKGFqqG2JJeHvR8T+fohhonW977cGAk9P73B79y5nCYrEPH9ebXhDNpKg4Fo2viWJQCrKNNMatvulfruus6tr4V8ewIh/v2YxxrTlKsMCqwcgkHJBv0X0suSBQGDh8phiAjiODgBRkOUcZJPgap4+gSkq7cMp+SA/QGr8BQRil+eBcphKqhOh9Efpng0SOLgG4HylfAIAaVx4pkNRCWyBDbGi0BT6ejMoH49uU1qPMLv+IFR0cvoup8bee5DYiE8okQREZVu6wIfGD1YQ3TjC88tj3flw/O2N8tfSYMYakI22Eqn0B4SG4N+9rILmLqrRGtpLHICfpMIJ4V9bKi4e81djsJuxW6XCJpp7Skp5wtsakvRgZDnvtmGLaAJsjsGLWYP5mlKBlkAGgK5/mcu13XiE6VgU6d8LYFHJdnvqFYgC1B7t+vM3El6z0OivgW1B9h2wiVY/oqp/RD7ifoQK+Iq78c1e0ZKVX6XAmocFWekJ0u+n72VjpPNerOR8uSzdYhbSrhO5kH3Panv/CQZszz9fJCSmCzsIhK2/x4Sg3RcRWbsBF9pqbZU0g/I4BKWkxgo5kU8vUeV3NY8vBy3NHN8PdkFFVNYVrfLkaagNQ4iNRXJVsh1k6Tpy0JaSsHUZqfwbm6tbeULZ6kuK0YaiM3oDKCNA1q14GlZz/ereSdh9LwJTVANz+ouYGMxgNZzQI1Og7eScv9fA91V4DRO4A/hnVPWnROTrAH5SRP4zAL8LH3M/Qs6C+PqAgI/71pCrpIplq5cQatJ69CMrrgulU0BAjSRAga/XSo+bSbVJL0iQufABbAA2CIbZ9WJH3op6jBhXHvMF+kKhue49f8fLLoTf8kssKBIdmZR6Lie9kuB/ESCw4bcEBswQuNVKpYTlse3xdvqUw34TeEuTCaELia/H2JjkvBKI8zeUlxjPFX9LaPsrUiwgdlcihURE0K60RoYaC5DU+DOoxlQjdVNTEcqAAcWQbKuPnJvXd7XS/UpUeM1qwz8P30ZMVf9fEflp2NZiH3U/QqbeofVV2FFVfMAgtwqnv1sn0StAbqH+TFyEnLCWZODw0Ca4LKDgw2BKW6TwDwh8vX5sEGwp/CHgsd33iKW/NxsDENOBBwEBqPmdFVxor7IhFWoDemKsu6IWA6VDLm+9AsG5ZKk8xAU06ooZG6oOI02yNNwAGcIB/r0ovbHDFPjGCHTZk7Anlwf2ECkC1LaK4z0HYsvx0tiFAtaUwhQIm51+uziPj5pDuIR/TEvXxMSA+CItw6YjT0A3F2Q5xyzR3hmgrhQWcC6TB+EL+QRE5LsA/HoA/w1+ifsRCm1D9iu+/i1Lggn6aeHQxzv+lBZKMZfebDsbKNZgD1br6Ga0Zuz+GlVyaXymwdJ+LjCoY5/ltw0X9lE+gG3b3ZlUZkA4AVP3NDn2vFPDt/+LKcjleeSZAECLHbzIBE6t3eqgBJ8FU0hYvZYSoDP5SEeBxlO9t2LhGnSPwH1BxkUuuk8gdiImDc2r/XYZkvpvre9sPuvXKBr1/LfyDUKggE7IHNBNHIB7S6skSdbLiQlwwbyWBuALgICIfAuAfx/AP62q/8+5//KUFA6nFCltQ/a3fse36+oTsKGryQlKi3jj6Q1JT19dP3hOgB+uNL+fq0ZaHAoU5tfDldBL+yuHXzkBgQ1D9tTu27Zjj8E/mw0Htm6kMANo4w/3BzSNGUn3Si/tXwK/rgASk3ki7SsL4FVvXyg9187SWEAJbpkE9jsLsDdwFlYyHQrYPS+0uEDzx2rkZUll+DdWALsANG1bj7k/oJVBvCbJ9VvPijcS8cTK6kmO9x3c5rSxg3MgGYFv4woFMMeETOsd0NU0JUaiCZRL9bxe7lt4FQiIyA0GAP+2qv4HfvuXvB8hh9LMIABwxZAV4OIo1FSEMRN1v3G/BRz5Ps9nvQIC5hmyvBxsoEECdewwGIwNIjuG+HoBw6cG7zYAaHcTIBYWzdlk2+imgGsQ28snvNlOOTPpJThmDhR4hvAEPiC13/XfRU15TiWnyGaPw1oUfk+8nMknSF2toflKyLmqehyawBIomMroC/o1xDXwkf6A8g08ynvVp69CQGnj86aQ4zmTdkvOhO1VPQZkIn80Aaeegvoyl8hFZuj8GwCEDw4pcm//vwHgp1X1X6afYj9C4Lwf4e8UC78Zr9yPUPnPhdxm2fp/cQ++pxv9NlG/0Rv0X33BBIZ/KcQN0G5pYSdbAxGhczsRFgABwpNfU3xpr8BYASjXA7z5fIBabILXBmirzSxe+GACzAagPefxIJcQqHQQeU1GwFtsT/q7+p3KegGQ0qaahasI5oE8PgQi2PfXdfozHe7Zz70DLo8Xf6o+g3D6OaWb0lQauMS7jQ2J+61OuvbJkl7S3vwPrTyJQbwgzHo6qXb7RcJrmMBvAfA7APwPIvLf+b1/Hh9zP0KvqAyCXLIpxt2kQ0ukaYXVWRSQbO1uVU31gTZYCMtjAQo+aEMPH14KmhcvNv1UKI22n5gA0wTffA/UUJJWjvxgrm7jFW/rCoQQWaMZKdlijiOUcKVtWYVpz4VmjnJRtbRolA9pYWr89YeHWrTUPl8HWyqqmp9eX3EmEBvFdEcvxyjtM4a90s4rSgKVOPejUIxNYKbifj9wHHccx4F5mGlwHLUJyYkVBdJHPr3NtS3ls+gcYqdmeoe3iaFiKxRP8XUKrTzG8s0OhpUF7tbtadQX6u06vKZ34M+g1ziHj7IfocI8tRXEaFLYmpPKHk59U961Vqitlo/qkFpAIFsN3W+v0nMhOALogeyZy7QM9e5LMUoVIxrT5hXUsGG36XO14QHuvsuKJTnM36b6DMpVa9a7V0AAioPvZ+k0DR0acAGEqyAtUoQp0p22DWoaGwnB0ai3jHNlV5cfbmZZWQNkBiznLVBZqNrGo8f9wHFMO+ediFJD06ter+aTsoZjvUIBBPXteEmRSAT1BWitHUzECEnV4V2DV8JPecIFCBDnq89e5P1BeDMjBicluNOqIDilOePfCD2r3OzWI7IS27pWCsTqs8sql6HeENtttS76qW5QeeG73Qcp7S7tBaEI4ts8aw0IKmpRa2rNlfs10M9Gv5YK6UnvUtMsOy+DeD/jAN3rsvmgwKsseRJSaHs9P96St9AFFdTGBicwYBZg/1TREIVPwfEPtQTUtapingR/Nr9AgS6lOZVF8xgtx6hTzbQhwMD7F2UO8xP4DsZnBkCAz3lEFtfCBCKHjwr9OrwJEFCgmQMyqnBJtzZap1ctMwQ2AYWfWTROPhIQXTQv9VfeUqeeLkwehQw1QRWxHYaHxSNt6vAq/KMdrRKFBB+piSekmQYAWqPsHuwoG6EzrRJkOhtgoKQxIgFsDoSGUZxKu6t3yYdUeRC3ZIPsRM3zkk4UfTxE+5SzXm+ZjCgLR8YGAvlJVgT2zP0eTMDAIEyBAATM2hOwciuNiUKZGUTcVbaKiMNDTI+HrUg9RW3w08kEmMhBRpFHMAhYuWX9ZV2+XJZreBMgsNI2nSDqrU6zadx7ahDQdfJCipiEnSuOACFXxsnf48PqDTrs1oWxCty+ti6iEa/OUBIOAOFE8iq3s9jk0hlDFYMBosL9ACWTKdarhiMN4G9kIlN7gD3zDyh6xql0/EAoCayCoRao62N0M+Yt6OmtuH7p+1R/4KQXYDJQ5hcoEfaYmwD+N4/DNih1NhDlkkWy5Ea8nrPrMoEwqqYcqHG0b8dQYGswIlJmAJsEfB4pb+esFKpdxH+vBYO3AQI4mwPVrEqzn7RBC8uvjPwZT5yW9tdA8LUhq5zYtfZXc4QXRmlkSYpPLAACW6JaKgKWGVbAQaFJW7E8NgbAotKyupZUNQgGgBgz0OOksiMp7jGWxitmpS1OpBPyui0WVa6xAa8DggYpl0ygefnjYxEf5S18AiX4PF5gopQTIXJ+lco2E8Ltz9Mzkb0XqtOdytMcyjoBtYVOrx2CDGyJ+Zfl5fBSSdRH5XcObwIEFFc+gRiIq5DpA07GheLh61V7tUshwXN4WUyB01JkWhZ0IwoZvxV9LB5l24bnBUxz+/RhKTaQDkIUE8gKn4oZ69mjZpvVJwtwSmmzhPtw2qWQrgC0eZPpr1PpAs+KQ7MwQoybGRa9EWtd8BJtoS1Rgsq+j7UB65KDkxg2jRjaMz5CYhPCoYA6E5j3YAQOBIczgVldMcJtixhlYjq3Qa160xx/YF2ZlnBaAxIKHRNzCsbqF0DPR8IRmQVRj90kuAbeR+FNgAAA84DHebFoH6HnZTdRtc9toiuH+r21XsmKi0ZcA2yEfo802AeEXmkLcEabV18sBDGo1CYLMf3PJcZChFjmZswpmwYI0R2qRhySYWSBVCVzb0KENjAINHqPxKjpkUsNxI0qysu/nQUQpEZwmuvfBkwAvSuPG2+BANX+CcsrtecfXmICKxuIY8Y2pzsGq3uQ/yQLun/vusFxyWrWT4BBjquYYcdHWrwMOa8MAAFiASoRPwk6OzTnVJ8DEebHh8PbAAHtIGCL5HihTjEDWc1uiva4vn/F0LomsquV+Odzy80ECJq4wi+LhsAP32Nwy78hNV04Vo01rV4aYtocUqi6196YobEAmZhjw/Btr6fvItzSqnVeSZeeRqDmO/jDLGx5z73hbQDQnCSM7a0Wf1uWS6QDQ9zzI/GITEv+cWW1+l1rC+3BBigkPMVq/FvJlkizprZeNTDq3WABrE84dRpFrtlJwmXGYq7Q7BK0OQi++lAMKONVohm8pzmeO0MoUDiOA2NsZNpsCWSvCW8DBKDmDYsrRTrRw/miw8wCgBr2onCqrT62htIOpdZc1TWc/UZVS05lLrQOSm/H4RODbBdimhYsO4ZPHbZIxcHOdxqSsBEBzGlCLtMXihyQMWvNuVhKnBp9sZoIQg0ILpDdHAAJQpZGCrzWAJlJjizXrgkYBCbnpcAqHSH8de6VGaxKuvAHGEirWG8DJEoXh8pbo8QlKCz4DAg5a5C7BGeVBxTpSEUI+cIMshVRORHZuIAva1djxJ8vOR4rDDMAhMKQ6XUSYM2ADdtzYgxsx4b7fYeMA5s7O18T3gYIrExA6ETUKHKYAtSISivHS4umi7gZv+k9jdrLiGdS+4gh13gLmjuK9gcAiE8MGuIThHyqMIINZG8AqCECmDMB5pAJ5BbmtflEnqfABRSxKROaRfwdB5wRGgjVMrOxliBMAoEZQ2ndQRYNsTQsSquSVovyqfPrnZCjclmW6nwxK/wbBQ7+WQdAhWa1nh1qkU4S+gUQLM9aQLBMIgqhFyq3KxYQpoZw2UTbo8uoo1pElIeT05qLCQRq7DHWOvC6OWYB9lTNFYzv+47tODDcxPlqgQBw4oOaA3EEYqoT6YG74vQNnlnTLfdKIfTrR7GI2hDm+GS2SPvLKcKxOlAuG7YBBAAWgbTGWeBj+YRMW/fuAgQGzx1ANah415gEAEyoL9rZPW2Lo4w1Zq6o47al8oi5B/6CLAYHxXYukBgNRwDWQaB6XxgEkgl4fPCqV63bWUcte1oRsTlAAl8MIQBQsyxWEAmju9rKSvcBbl+i1NZY+D2NAlofIwDTFxUNRiChaLJZlMaL+sk5EbQi0jF3jNbFeXwVzYHOBFLrquYIPN4YpIBgsdWbbum8oP9aNcQYcQkGMZsxplr5YKBgA4KRgr9JLBiyOwhsyJ6A8KRPcoSF1qLk5xZSLjy1K00XpqDXIXQ61NIRwjVGLqltn2KhKK1ujeWAzpkj5+JcSdvotGlaoZVMjFdzoM5rIdHaTmvk2ogOAlWkSIazmBIiYZJJUfMgdQHIpOkf2/cLCDwAt/DopzmwNAhuL8lptFpcMU9+z9qvRC+Rs4AE+FxDsquFAJmZ5T5pQFONdNzud8jYsB1HMQEf/PSa8DZAQAEcVXAaS/ORksSQ7B3INrMO5c3IuCI0z5nMlW273utgECBgQBCJEsSagWEOhFNwkx2b7MDYEPMDNHsHfAowaZx1kcsQllym2lcXjoVGBzWiEJIhI1fZATbbMpsGmohnam38COGe3DXmWsQbEXdz5XFOp+jd/8DnmV4CNN1GVpb6s1VLqxnhgJikKyoaDv4h4FF35TQDlS8u7zsgTDYHaGUh/6vGV01rNQfab5Ggq3YUsWWdeT0TEAjUgUAXgFGqH6ubu9fTcRzY7gfGuLeBTzEX4jXhTYBA2psexD3lAhf8oTUu39iR/RaM4CJGivl0P9hePAEd/S1+RZBrHajmjdTGsjgGa+2ADTxgSAPjFeSN52mtbocSCIwxTJDnwBzmNNSk11omwvCiGcOGukKgI4YuUqaSJVujRwp4MADvKtPOCspXQH4DFAj0qbUGljo2iEzoNqC6IdYjzP0RyCwIZgARDPHhouLm0Qj7uOpboi6EXKVhYk0ChxXwQOcnRrAyid5mqiSXBkfPMQa0NhRvpqYP/031CowRvyx8lRhMMwVymPOR3ZwxE/KYBeavCW8CBMLNltcpYNydUk4nejFIpDcQpRhDyztEd/XejcugEr5FmKS2Cs/tbo4/XgZsbDb/f7v5MuG1N+C+3SCyuda3ap0xRFgBDBf4abMQB8wDXKvmUqMPoPOjaXbKFxCSTS2vjw9oHJaeC3qPRTASIJbptG0tvoiRfAJ57uAzZORcoCGjplN4mUf98G7QKuK7KXnVTAniZTWqZFtTFVYfOk5CzubAmk9+pmvw+jfnWcFXAQopdxB1SkegMqlLz9dhWHgBs5hsk8FYop5QQ41Tw88D98P+pt+7H3fIfWC/33G/P2NsA/f7gW274zXhbYCACG4b7UU4yHbi7al4Q1I+tnOi/7n6sFbjC22cpyR0sZ8fTfKJBUG3/VbntBDIvj9h92XCY6/A/XYDZEBVOhCgJgrVSDLq+nEKWjvTxsYc1J9MpgBf1/ZV5Ddg00mjrNEaX/O6hwJOOm4Px3loqwmhkclLDwHHGfFV1aBmaVKCtCidPUIUj2d1el2Gvox/q0JJ4FVb915o0xMwJCiW8AmCNdg3ZuLtYmLwoJw2QIcYqNK5LMucM9s6Dv++A4ZO/7P47scd9/vdTIB7p/7b/W69A/d7nm++aO1rwpsAAQDYBoMAykHUNmbESfCVzoFoJCH0So0lBD7eo0aXgxLqmAOBfOFPXgtw8+XBUvv7kZkAXAva+CbxGYLBCoLi13JWYRqkwLkTbd2SSrA4CLNspAkvQZvnWRv4MV8N4U+BpvOk+TloKpiSprQXAFT5LqQZJYgCSPli1AWeVyVOgZYazyGeUIGnXVb2VwLdFlMODZ9/i1+A/AXcjrI1kdY/sYi2SvGEHj7ngOMhIBbUCsexqlH2xmgsNRxxaJ3TRKfwAzAzqOt7O7/fv2pMYOdtyEj7OxCAGh8LPjOqSwbQjtw6eQ236uZLAIgFQsfmwl3bgO/7rYPAtmO78VbhN2/AwQBQrABwwYcLfw0EmVoed7a30zwKgY/7qHLpYEHvsVXAwr8wAi920voAPM0R7/QBW7YQpkUi9HIjaFnUIX2nhNDTpNd99iawMIKsSc+Taou1bAEH/1XTM+UPgeap05m24ItFzTMu3pxkTszj3px187BeFmavvJ/g8B6b8PR3RnCkKaHzaACgOnE/DmMAczorCACYpf3vd2z3Z9uV+G7L070mvA0QgGAfZ3NgBYPglinUjPiN8mf1eSuJN6JZMk9GCX/7i12Cduy3sP333BhkxHluFW7rA8aiobYPogu98jkIDO5ZcwAAIABJREFUAMjeThDwEiHN2gbh0L0oOyufcQEEXdaDIbH8xcYujUXEAqJDIBMYQzDVOkQhiild+Kse65axti78pf/jaQLn0P5TcqEWdmyywZDnxBTYHDj5SAgQeC2/NAPYhJAAgHqvxlAQAwjNvFB0nZM2IJXWOwJUlywP7bU/cV+MgYCl80jGcD/utu4BgcB0EDjud9wDBJ7vsO3s7hjjGa8Jr9mG7DMAfxrAO3/+T6rqHxCR7wbwYwC+DcBPAfgdqvpeRN7Bti37DQD+LwDfr6p/8QPfwMY+AcFjEACagBuwBwOoCsyKlNQfHjmZBSFQY/PGv+wN6LZVrAS8bTtut6em8c002Orc/QW6MgFYm6suQpxAIAfhpKDz1XKeIOD3r5iAP5CTilL4SWgu66NKpzP/8NNcvJqgGsDLcWn9Rgk5u/d8tmiYCErABUQFU3SdB64OPrtdWl4DLAgAqsy1Py/1Po8fiF4U08L3ouB309RzTtpxeGDb7N0YB2DCPWrCTwLMSDNjOhhYj4yP4Thmav6YI1BdgXeMY2R6xjFwP+7Yjo/HBD4H8FtV9a+JLT3+Z0TkPwHwewH8K6r6YyLyrwP4IdiWYz8E4C+r6q8VkR8A8IcAfP9LHxAIbsQEkPQfxFGtYUcdwu01heY4dL/rR6D6+LVa9sICAO6m2VL4h9g+gSb4t9wV+LY/mRPQTYPmMNyj12AHxPwAZQ7weZkDCQJAHzCFJksVruW2TIYcelpZjTUKmAXEuvZlAhQjCGHHEFsbP0yyaSxtTF1Mq7U2iQ1wwhd/QGlwejBMgBByocqlzGt+ya4S01ZGEILOZsCFOUB9C5Z+Bw0zLWpAldnk9xT6+/Ozeejvz7g/3zHngT13jrLu0W3fAKgxqmW0n87Ru2jnYUc92vXhPQLHsZzPie2+Q4axgG27m/J6fsZGvSwvhdcsNKoA/ppf3vxPAfxWAP+Y3/9RAP8CDAS+z88B4E8C+KMiIsoDAZYgAuxXPoFoDMEKRglTChKDQaSZTIACAxZ+Ohe4wNdy4EITgULwbzcHgdsTbrd35hvYategdcMQCPsD+j4B64AhoDOB1uhTsZGGS0W+gMaFwxAiplml5IvNgTiv98wmHwOYPoNTIL4qLjCmGCgIE/OHVQsm8oVAJW4ps3EnugB5TDHVKw/dVWgBDTn7qpHUb03YF39BgkTkpjkbyYE3J455T8fb8/3ZhN/BIADh2Hfc9oltbg5kFm/te0hDtMknUGM1HASOA3PeU+vfwxF4py7COXFs5gc4NusiNGbtG9a8Irx285ENwE8C+LUA/lUAfwHAX1HVcD/GVmMAbUOmqncR+asAfiWAv7TEmduQ/Y1f/zq2QdRlYQCgRh12s1FoL1wtrZbNUmq94RD4JhgJMj66jTcGlbre9926Ad0vsMfeANRtaOMHtjYEFJAc3zRFq5dAfElx44YJaEKC0J0dpOX8t64YlZQnOwPZk2/dk8MHE+kYqRlnRqtRLxCxiVR2HBhjYhJdVQYsSkoydTLfsksTVOaowVPhBIxrniw1fCt2zkeROO1MRENrB3AWyxKJsfcC0YnY9XdKthCrLRXMIRhT2roJUHLkHeSpj7UIwjygMfuHOwiD7k8RcgYKjjkxjsNynUO7lx6HXJKMcHsZehzrX8YzyTA/9twBVT0A/J0i8q0A/kMAf/vVY5HOF37jOHMbsr/tO75Dt41QawEABgE5CQ4czelDpOXjOoej0ph2eEGOJsC+XfgYfZfgvRyDW+4YXJuD1Lu1e3A091iqjBVT0NS6R4YMl1Y7ZwCgPGvtxFNHy7hhjRgIOfioTl8M1UcojgHuqVjPz38X2/4utd5GEbKJIYJcfi3AgJZhS4ekmzajgfZSII0WRHkWCMS9WNorhSPPBYco5hTMw6iD+DahqhOxNV4Aigm72d33497oOXfhjSkeJzDHMEAQsZ6EceA4AmyRadZt8zQfqKXNag4DokVJ9M6MHM4SQ7SFyiEA6zXhC/UOqOpfEZGfAPCbAXyriOzOBnirsdiG7OdEZAfwKwD83y9GLOJ2U10nCIBBALRKtqannQUi3l3BIAQ1B9Zs5QcY29bOB59nF6HvGEznY9vJn1Cbhg6xOQPhPxuitSiSAud+cjQwyKS3wu9mQYJfvmy6mBVYCk4y39CWkoIsQ1K4h45LoY/ybVS6knWJ+sy0Ws/GAgIrEJzGO8SMyJPZUZQ94TN9AH4/8stThEnDzjkxxBblONJ4s1rTGX6VAoCi5e4QnOUYDDpvC3wIxhw4RDDmgTlHMoEYzx8O05q6bQajMZKwZTi/nQGMmFg3a2pyMgLP20ebOyAi3w7g2QHgawD+Hpiz778E8A/Degh+EH0bsh8E8F/57//FS/4Az163XxL96zz/ojGz8GusUVjvxVj0KBgT6i7cqf1z37/txApisFANFa7zAJKYFy5JX1dbTHLTyeGa2Zp0afAzVUK2+2r+CwCwOQCmykvpisI2QPHRcGpAAFXaKfdK+EPw1+OL1ZllnuMY/J6kGeDp0tiAJf5c8FuDr/eb4CudLyDA5gCgmBLAZ845nWUOHNF34xvE6BzAmG4qxCeKHeXgnFmOuTADjnkU5Xe2lUKukmaBCIwNZEUbSGXJ+F6HARSVEjIDBjB8un2w22wiqojNVl8TXsMEvhPAj7pfYAD4E6r6H4vInwfwYyLyLwL4c7D9CuHHf0tEfgbGAH7gg18Qwbaz4HTBX51cqsi93jUAPChxCn6cW7yhubexYexbOvGii48pfQOHsSXtZ1CIxSBAKCwLIkezVWgOtVXgtMb+ozX3r9nAKpRIoXSSUVsoqOby5zxCLo9NWFDCH+eraeLfWvTToxOEScSg0IR+nVwFGh8h0csQmZHUkMlIMteU7uW3BLWYAJWj9Uy727oHANT3B7Z52LTKT5TRTO2avQPcRdhWLR44jsMUtXv/RQwQkg4G+5reY3CMbLq23Fh0khUjtqLwVad0uAM1ppnHylNlDsj8SExAVf97AL/+4v7PAviNF/d/EbUv4auCCPo45xCqBQwCBKYDvkw/F7iWc6EnoQxACG++DfKpTT8LBDbv3+1AcAKHjbsQCWwyrdHYWX401kfp91cpJ/Bot/NO1/4ssFT+9Gb8Hu8sGhPnOFbG8QCf1iRfZKjpLyoLOf+5eVDjI4otpG8DLuBE87Ee45kFEHjzUdVB5gBxEx+dN8fhx/i2A4B3E4Y/4DR0NxyDhw0WGnOYX8A9/w0E0vxSG4l5TBxbLTCyDdvjEsN6abg8ILaonSkTawex3kRW4LQJaTg+VHsW3syIweYYvBD+/KNO99SC2aZL8BMAXDvb7L+bD/t1B5+fmzlA5oIL+zZianAAgmR3YjgWM73A+dobZoGC9scgYLkRkqiuQSk2/QAQNC1oohNKNIZd534L7XnkV06afk0XpVvQGxqn+Gw0BGUNBhATtZwFXLKI+KeEndnMCgIMlPFM9MYohtNzd44KbIguBuYcGJvZ8jprTYOKk7rzjuoRmJPPZzMH5pScKzD8aJ0Q4YfxlaTmwJjig4ssrxsGZqwTkf4Sp3kiiEVKbCn+8pskV3Lge014EyCAB0zgCgws/9YzML1t5KicpOQ1DDhAwIb7PiUA3G428ed2u2GLhRrTBIjZgzycmHoWiG0AqXioGca/ocHsumnEpL15kSBwBQDiDacofAeAVcufWQI5Ty/Ec62Pqgo5HU9gVYiQgsyE5OxCEG/YHQSueEfEz07JmJpb+S4wyLxmPmOSVvT3S7KBGJkHqA3Vnbays26CETa7lECdmcC9TenN3oExsLljcHMv/ZRak0HEgMCYiLXpMQa2MWC7E9X4giFbY1HhJ2twmf4Ty7JhzVdsyXFVxf14T3dY8IvSY/goPOUpudQgm7DW4h4yJOm/2fQu7BtT/pqyG9+mmfuRUMQEFvF0s9msIMHMv2a55vmqYVPoS57qX3qsOUXj2tNWABFA4F/LNF41ivO9uNMEv6WxpY6IwQX91PULq8AHA+ggkGAiod8ir5FfTq2PCUnnofgAKR96LASAVKHZlTZtgs79fsfz83s835/x/vP3eP/e/p6fn+3vbjPzYnhwlHMIMfYNYw7s2c7ct0CjONPnQcBZlV5KTHwFYnNKj/ydyz+UooEDdaMHUMhFfVyENwECgOJ+0GQHEn7LSK03Zgtz8NEEMxyAiYok2OzUa0cHiGAMjYFYbGcgABqFZhaQND2vOwB0q9tj7ZBep+120GgScmIEaEdiAMrknlXz6c75o5HKaGSevjNLYRCoZy91UMwboDIuq9wHvjTjp+qha/sozRVeqIxLTkKpJng0D0ICwZFAcE+h99GA9xgTcM8eAR7IA7flBRtU1E3JcC6GYqECzgPBQYJEdAG6EvP2m7KQDJJZQWeSdf4VAoETEyDBXwW07MhB50A0WBliGl4GJDz5Ul172x4swLv4YoAPmQ/1raY4vDGRzayl8btQEmVvDY6FEg/qiCpSl2fi/SsA0P6dayYArILzwZCCj2xsBQp0no8LTgXXn/DIeteh1Z9kfLZKUY+o2//X+QhoWT0dUZ45fymZlDOBu43/f//8jOf37/H5558TEzA2YGBgPQKeoMyz+YgsvfvG/qXYO4KEPGl9lYfJM6/LGD1TO8YeE9z4b5SwrxpfTicvhrcDAvcFBJpA1hz//IMtzWXnG7UtR99tYNuGd+/FZJ8yCcrj78N8yebPGpIgokJp9SRqNcXmqENRdO62audXZXDSalfUuqgwg8H5yEC1xPz4Ij6cv2QaFsHntHX/gFy0O1muAgBMaEwg+rJuxuq9sbtTL7hAhzNFmxEp/B36olS9taDmL4gZgc/PZg68f/8en78vEHj//Iy7A0FMGZbMewj3SLNpc5Nzo2nENeahgwFShi2uwQyW56SIJGvlodQyeH3Ms4J6TXgbIADFfZY5kBoZAlu22zbmsPsbYsZbuNrEh48GdvDOLt0E6N19MqpAWfBFeOx4kwo/d2/zhWZO4U8WoGC7ncFhLYNXFFTxiuaPeKT9H7eE1zUQRcm19HOwAnrMDE6o0ATBurqsr36k/c7sjpnRygiSJLxO4TV2UgBOk4N8NZ7n+3P6AcIHcG/Ov8M0NrxPH5LUf4h0AFjmPqykKAyCaoIRj09FzvbqJm4oKzpfFcw6CvVD4W2AwIkJMDU/Uvsb2k6Ys4/Wo5SB4RMpYux5DA3OQUI5ToAdgzFrkPwPzS9gNcboakXteolBQIn+q14ARABCxMHx1Ve0fXBh1ZcafjUH6PzU/cf5uAjSH2Ft2pV8v2bGwCBxBRwdBGJvvvAH1AYc4j6gEn4uiZBkpd+XrEg9f3aQGQTEvIJjzrT9gwm8d3Pg+fkZ79McsL/jfmAMhWzuAxAhgaUdhUTIN1BNS1CsoJpZCX/5sby97nsxBBluaoxs5zaBTpfZqY845zm8CRCAzsUnEEwgJuNseW8btiPQ8EYywnaMAl4AoHwBNlAoWUFO/tmaXcVj3nlTjN78XMi1a/2YzdiH4ZI5cAkEqy3P5cJfxglAGiughxrAfLAdkKnz6IkmQ49ZwCPGkBQfUb4m9DHbkgfEMAgkfW7MAOCJQ7Kk+pzWYonaKxFALfx5HAee73e8f/+c5sDze+speH5vwm9A4It7wjZ22aQ2idl2H1sSOwvRak88kapjZIBioUT2WC1D1gctVmLt156NCVG8V6ENcPoKgYACOCYtihi0HxPQsPeNAUAGhgpkThsxlTHAZbfbSwEIjNC1etBK1aySrjX/orOJ+qewk5AzGq9j8U9Dc4F271Q4cUq0L347AUEeHpsCWVg9tlcGaYImTejX3+O8g0HQf95rPljeI25ftRAxGfCTg/6FQA7bVicxecepfrKBbg7ESr/WM3D3iUdSgOvtriamSZoJaxtz8n9pHpg14O+NYAWLObst41m2YbKg6qNpbUv1OX3h0leENwECCC3qIURRo9uP7qYjiWbtSVCnQVt/kUCrAnMCkAmfmgHFYcM2zSitLwuqufl9JXlyvXICgMzDCQDqOigosAh0Mx0eiQHqeU/I4/N2soRFq15erc/yHW1zHR6ZAl34mZKHUNgQXkAwp2KMSb+tf3AzosCeNXx9g36m+zmunzfpmBPPz9YL8Pnnn+Pz9378/Bfz7/58d4eh+wZ4YdFhswNjfYYClZH8w/xWVkYGYgJIge7qPYm2GiAV605O359CR2+Ljhqo9d4mYqqajAmZH3FRkS87hEC1O17R4U2uP/YP9Ik9vEYgYny1xqIjsajE4feAMSdk6NJqFiBAVY6LQAFBCjvAbADtnr/R/AQXABAfoqSsXvh0+qG/cwaAeKzxJEg2Tw7d3DnXzPr0orpaSVn8zAQq/XFdTkFr3AYIcy7+ABQAsKnQ0nUBBudncVqKO0Dh+f1zgUACQF1HT4A5B2uegM4+JJinJ9toxtEAoKD9GuK1lXQ3NXNnKv/LHEcZsakVAABPw/gKmQOhPTNkt090FyEzXY4lnsZbY/x50pAJKprQqwwops0aj7HZwLk3IO5pr6RuCkS6Y20Dovmp/VcmYBHYgQwMDcob9Doacx2VEtDfyx/WQ5Pj1zWJB0EiDoKU9VuxmpPGCyWgqiUEhems7fvCKJfDlBvorEBAHvbFZAlBnrEWwP2OYx7Z/29/7+no3YGu+QM8amlw5IjB9S9aS/osU6FRIWZQggVNBdP8SU3RsDLy+CijknENCF5nCgBvBQSQytEv4IIQNld4Ur1BpBkQg33KHMiJPcLLfCvGnDjGgPhiGnMqkLYpo2sk4YoJLOdJ3UAaH2B2wKwAKN+AxUURgUEgnGKV9xCEzhquwGAtWL6Qi/PXQMPZfu0l9Pid1VFXQqo42cunc34HpEwbJOd3VudihFgH0BYEib7+u48D+BzvFwbwi84KkvrTHo22Wk8tGmrmQKxW5LMUUV3YWrWKLvSnAs3GVYqrTIIR7YjfdWZsQ6TjS25WLeXzUngzINB8Ap5ppjxWYN5jAAaAcTYHaGJKgPOEAkdsdpko4/YaF5mbA84+UnOTkCUqA1VpoeUXUwDoYNC0v1/X1bL5SHiLkx4zGFVqVl/iKSyauZ/H9UVY6fcJUGodx3MM13GuAl+Cvgr9BQi0RAQTKEZQXY/skNRaCPS4Zzff/X7g/fvFH/De/QG/aKBQ6w7E5COj5xAkAJTwEyOI3bKDBaj68nLSRoGey43aDsgvMAMIqN2FaRXtNXPrA+kwwL0oL4U3AQKlKS1cDgILdCffQPUx1z4BPMMvkVNLZFy6W/df0StuYgQMCoorCZ9r/c4GgAsAOB2vtLmljYeFxrmSwOjVe0v5XQug5Vvo/IPhxTbUoOzi0auXjQEAH9b6KxD0OBcgBZrwc7yA7+Pnw33N+18Ov+fn93h+X8f3ZB4keDPdVwPq3E6Mfwc9JzSvJZJKRV78wNsjql11IED/A1Ce2TIHcpZx3telTTwObwIEkiZ7iD38JMuv2MBpl6DmGORhxQKoUMGl6J4FO48LGFwKP5175FlZAQz526L9da0YXU5r8MwYk84lzRZkLur1DgcRViEv7d8dhCsrePT+SyEA7uWn2GF4dvitANDv1ZeWWnPhak5FPlfF/bk8/GHz34/7mQlQz8Dnn39OWeN6svENxQ4WxyAKOCRZAKmU1NoMAJotr/wAIJOA4yYm4KZibQnHUEK9KR8IbwQE0NwYIyk2YOvxRbde/dU+1zyj0Jv0jHkFvt03dcuk5gaoQFcB7xaVohp4DgHOuJBCzkN5zyAQzGHRndq/lDsO5wKmamUgeiGv1+J/Jdjl1a89mlo68j1c3P84oUBCL0Cg2/F8f00T+1IUQAw1ZiZgffX2RjCA+53NgXtzDN5j1uBzPQsQV4y0gup2/UsfgbeNOaCDHXtacSz5PDsZJ2KzWvXdq3UqxUftaw0iudfEa8KrQUBsjcH/FsD/rqrfKyLfjY+0DZlp7EpxTRUmAeR7at7+w/eMlmNCJOz9aBQzWYGhpZTWju/4P7kOHxm4quQbwBUQ2MMrEDAzCOHn8/AhYGnIEbmILUphmiQahIHBhVRgaU4X9+z+Sj/reBXX49aT7OyXgBGh+UNbOnvOONdv9OskzK2s7ZkObtPzWZo6FgQNj39s+nGcqX1ocmYoINMlWWrUca0anKXn3fdm3tpCJtBBayT43An1BVBP/gW/5v8Ws6OZIqjyyHb3ivBFmMA/BeCnAfxyv/5D+EjbkLGAAQ4CaptgSgAE35sOAAfTHx9tJt5PSj0KwGgDXBTrOWn+9MIzKJGuPoGCnn8L+39WhbHmuGQIDhAxldRWOxIM3bCprThzHgOPRV7Pvwv9qwijURZA4LLvJcTUnPN4+s4XsB6qu3BNqTYwOL8XdY0quxNwiTOn8glMEvq+T8A9Bw7NRutjBWJrT8mjXPjDNGXqH2aByNFKVOAsN+dGWLw5D0ZhCmpK+37tjaBpduhUzEG7F41aqpwBwEvr1fXx2h2IfjWAfwDAvwTg94qV7kfbhgywnXszeN0OF3rAdsQNJhBAEN0iignFgWyoDNVVcxl11ehyHlaULNdYhH+5l/E2sAgkR6I71GhdMogEhAIHEZuFNtWXqRoK+J5261BToXzlD/0RAjkXstzxhxnBoxAr82B5rj56ZgYfRoNY6WdlAVcAEPdWFrA6WptTUKwPKcCmVgE+byQ6Z80M1KXvP7r5oolIO4/ydequmqsKV6DMsBPUlVUoKhGBDmDqyG3KBjOB3ASm1kecOm3cizOF+lqHxteE1zKBPwzgnwPwdb/+lfiI25B9/Vt+2SJQpfmZBSRDcOZvNTFBO/1RHAALsX+UDixAxhhWAAh3fBN8/+fMJvr9biueAeHR3xhGETcFdAs+bPHGbjzxEfFz7g6L37jxlsaX5fxDwZ4tIKDvgIWWQeHDza8EXggAzvR1BYViAlrnyjUTZWDbvVlaNAV0/bMVguOamNojEZIwB6Sajz8egim+buCUCczwZyGHFifBAFp34ZxqG73S3odrr8TaVnJnJSqBLyb+Fl6z+cj3AvgFVf1JEfmeKo5TWHnZ1W91g7Yh+45v/zZtPoEJ2xMOSBNgJACIOf+JapsH9SAZIDQM6i1M6ap/dRX6QH8EQnMG1H5bM3O6Vvsntx7PNe/1VLlKtG8qeZ63iaFbNYBtM4aT+Tj3p2c2Ml0BGNE1F2MiAiIeA0EIaJZou17fo01UFlC4CqvgX/kDKq76Rml/135KRFhLQDOvXmm8e1AsChpdhrFycPkFqH6S+hdsBg4LHOgRdej7C+QbaoPasjQCON3EjbQqEDsKzSb8s/1V2pZBSrGiMDNdv34tHLyGCfwWAP+giPw2AJ/BfAJ/GB9xG7Kie5GRcABerCmoyMlRIVjD7aNA8ZmNpYCi0NsFyTcCiPt1ntWdLbIj7Wny6mVhq4ZDKgR8tmOtfR82n6YJoLpDVbGFbQpgg0LHSPBa+9mzubbKL4AI4Sx7NnL0SGBLM4dtHYL2Mt7XWIBH4UPU/3F6Fh8LMQIDbDN3sofA3kpnYM4WvNg3oLYLJzbAe7oTCmSbMSTIuo5vcqlUzURdDcSgJojtwCRim4QMWrKclUcDhUmTlWb4L+prX5wHvG7zkd8P4PdbBuR7APyzqvqPi8i/h4+0DZl9h5hAAED28xcQTN/Cy0wC9eWnpu02ywUWjUbLTIiNLjHMp8CTVQLimSGIe2+BMwDENR+4+hkEanRZOZAaACRdVesFyMBry/kX0gQo0AqxjjRRKlD9xQVoeRaC8o20nGYWiAuOXNL605tSQAA8BoUeWDgZEGaPpCaD5OrDfRtw8g/QrkFN87ZaLhOKBTxIUXil0jnoT4o4oxX45q8+vBcTMe5FbF8qGEjzWIPzQCQsYGAjCNdlxblQwc3xxfBLGSfw+/CxtiFDpy65LZUIcsYgbSgKni3oMwaZBDsbd6ZQBWWg7kIENGodDCBMgAQHoG1TrU2YikVgic/6dQnZZ+1Nz2yAgWJOxbaZBzmF3rfKbqPrPE08zbamP3PdxzvwMgJizAUznSv1UW2I412/wemKa3rnIlxR/8cA0G+G/Zu79jIwQFp8ca2qtSaA7xNg4wCod4B6CJqpJtHfHpwioLazku608zn9vrepTIE5rn31hABv30Qke7EmcpcjPYLqH5jzjjkFxyG+45X6kmVw4IuCpfKPOvgyQEBVfwLAT/j5R9uGDFir2zIsI1YH2nL7MLMwfLSgbA4IGwQbYuNHmeFBhe34EhtMKIBw0KWtS+iZwhHnDAAh/PQ7MQdmFSJiforjcAG37qk4X5kAHy1vPlhoHE7/ZgpAepW00hMe/1Bave7jyrVOmg4rcfU6eEkbR0PjW2RmscN1BaIWjWABDxSARs0sidFgAKs54L4Bi+M8qclAwAYD3Z9roFDMH+AtxXKwT2OUw3cOIh8MQAJun69dhVhra2uHUV75F/WgAh2S3z/mATl8kZL7ILUzoXMzUFEHCN0bey1FVGX8ofBmRgy2tnfBAGIF11hdKAQ/QUA2zCmpwjS2K4PkLrTpLtRYACQQvb7L9nSn/AQGpHLWFYrE057OnDlpUMpcNE55pYMdQGyL8Km2NRVTxNo+zGkvj26iQjzLcYETaATmFRNoQvVCXa1MikGgt70H19LjMIIm9P3yWjjvblqYtXEHjM4pp2oK+72NEeDpwc7YmH5D3QT1AT4ATAhjkq5g5CI12toXmwixFoAJbtVFMLNMaXMAHvYdZwKH935bGQXKl8+gfBRLHXzVQKBpjBAoWk9t2zbs+w0YGySZwG6A4GAwfLGHQw5TmNM0w0RtPhmUvFF1/6ySEDEYnAEgtH8AleQCkabF1aeduuAftH31caR2WHsNpio22Eq1c8Se9jZGAL51VnJndZMgsABnTs1XMbOyTIPmx6aXCBSzPBpKcDWdgIA1UP5L5oM9ujZW5hedCShVzAnBGeOlAAAgAElEQVQEMr3qzxQoJHNQpeHAz23YcBsvENOGGxBMW6tn+galvklojEszv9RAjexc/lzw50S+Jx7PEL8XZoFG78CBedgWZnLc7blskRNjbs4CNhzbhn3eiwGQeVhO7g+HNwECXegsBCUOANj2HfttN82P8A3siIlEwIapB+QAcKhPLSh7UcNRgNDAsYus7VAfjd/ly9udO9bIZ8AsIJaFljHMc6+2+COwoTaunLSkVQxVLcpZ/oHo1VAcm+1lN3X06arA/9/e24Zc123nQdeYa9/vSZN+pElNOJhiDIZoEdKGQBMqUg0VE6T+KdIgNNWA/qglgmATBX/4K/6xjSClUikWqq1GoyVIaon929gPS8WmxyY1mmPSpK0xBnve5957zeGP8XWNuda+7/t9z3vy3M85z3y4n7X22muvNb/GNa4x5pxjphlQdSeL8iMNmv8JIOWEinj/B3OAQSQFCSls9cgy6oVBIISZKSkxKQGbAWVGlAaj92abdBbQznE0ExpLcCZ2pXUB1+stASAjBs3VQaiUB3+TT06bU7IbTBm20egUjAACMAjA5/vbPRjAsAkMtkhuRn9yj386LgdkB6YIdtf+li9bKKe6Yc4NYx+YW+1O1OYwjHcMBA4pzQFnAxk63LR/OgfBjsLNnDEwu3qKIuZrC8xPAMC1Q9Hz3e12RW15HufhL0hbms8h0JjJ540+AECN5hnj2AsMlvh2Zo0wGFi+ROBj1pttkqkXIMfFvYOn5o/yVlIlwa0TxBoK075WN/YUafcezhMctT0/mukIBMwM/A1yPEqrzwKHpsXz/QwKUeYzEHBABQ/N1gzBG2n+nEZ8AgDWR8Jc8WFoN6FE1fwBKu6dt0hVsY7fLFACgYylaO09MdwYsHPvLd4fJPumCLBPyZmxUeYwG0YOJ+8nICC1n8YL0isBAfJu+2c2Byzs8obLw4P7AkrwwxwIn4BAsaOWccaSrik7wo2TiHvbsd+uS6hm6+gzaWdUZmz7NLLzjn1k+HIdGwC1zRBkSxCw+eq3HI8uENBGPUMTAMDcB+bcbBqpOiDo7Fo/eqkr+lVIWYCtRucBCOKbFQDSLKJn1h+SCWSHE1QH9Osp3E342RwoECizouzryEcbFvT8IY4SZfT2Y01O5l5FDr62lYQMCN03EI5kZ4Lwgb1YZDaDCQjmsFmCOkLYTYEENGAOYEx33roZIGJbko8IDRoT4Wr9AQSQHdgBUwLDfAUiA2Nu2HZjoPu2RM0eEatx+GjC8+mVgMCSAsnYKXiJ0QHS/ignIWQzUyC0hKihtM83yPoIB0xsM327knNujd0eKm9Qp63zsW3YHJW3zTuub7FePoE+OeW2306YgGsJF+x9bkkL+8wxL8Qsqo4oMtCEVFHCWxSdmQ2PURebaGCCFQBYMNGEm4GAtXsHAwaC2qjTro/2vqL0PT8p/FT4ZFOk0csJu5Pw35ZzYgXxW170RZVrS9rVhV/NnI/FPCKIsGJW30AsaFN3JkIkTQIJwbcxQ59SHP6D3evVlJmVb2CMHWO6o3wO7GOQOVraf8iweTBU18+lVwICRPnysx3ruxBupE0WBY3tmHYM1/w+f0Bjv3fF3AeGAHtoj8lgUAtAwjm0zxiy4w4r7XyM2NxkYG475nbB2AbmdsHUmTvY8m62oXUALDZkCEB13n0f/tvNl5pq1Y1SXS3naUfnjcVmcmgqAq9g7SgMCKD8sVAUcwjGVUuryzeguZ8g3IlJNivcH6DRjvFOpvcdEHr+KJ84iwVYgr0ygeO2YssKwshD5B1I5sQLsnu6L3D1SxRISC3vVnB5DTQUPtFIol3dXNRhPogxMGNfwtjvcNmj8B0DAYB6sjdEOMP8zz9LECgBxhbOudhOLEKOTMiuHlR0wxTNyRZWLS5sseGEd4TbXo67fd9xmxwL/wgEvKVZmCxxTaEOLrVYJTpqmdUkwK69p0qyBxHB7RYCTEFRSSBWAKhP/F/luZhMAcJ9+x3OHiKLCxj74zVNL7hp8pRpEHUY6w2ifu8xj+mAs/gBGARyQRAtDSYTLMKF7bHPILGBdcJQ+BKMlCkBnfgsYgVW4ZI0shLkqFU8qzEJzZ/pzzHvjFSwUPV1MOmgmpg56rQK+3Fz0nfYJxD0Kc67JmAwCKo/hmCLzUdz92FF7lUoEXlVMXfFPgauqXHKL5Be4n337amt49j5LYUewhVvtnUHgdozbtsuUNB2UL4uIEcCsgP3DgMgx4f33Rr0egvKXsFGQsMfqfJZKvo9shy8A9MyzwEFDNEacewz0zsQualcproLRsxsTI2vUnWJOg9BymWz0UaNCSgBQuRBc2JWhBM3QK/Q4hE1aN9vdL7nvoLxm7Z+wN8VoJhol+AY1esAylW2IqRUvm1zEr/WeEUwKZ+LIOrKT2xl4nBzAovG5/p715lAH54qwc9lwkQN4dNqjQoJtk1sui1iTH2mWWB2tG9FFg2VtGtPNhCx6a/Xmlp6vd668GflmjDFBCYzCzaMfXdQ2LNMBgZ9FACgztU0b60VNyAIOw8AapEMwnzIcxbO6JTVASL/8xCcdWDIzB1uEc4kGam9vEFSK1o+CIiW9gsdnSvl0kSISELSVtEBZSZo1Nk6bJrDdvQGYgcGArfcRmyftaVYhRa7ZhvvPmWYTQaexBXmABy8gCb/La3Xggi06wpoTOjwc6Xz6AdmKiiCz45pzkd4fSZo+roXSPhixhEEpKHSk+mVgMBC7zCpg5NZ4F5bgZVvJABsuDwY7S9/a+xlaOGebsO3j/Y3rM7B2223SLOPV1z3W0aeLeStiUEpQJtp/WQC7hPYfOy2kVetT80hRtrQphvDOqcQ7dVapVZxCQoUy3cBlP0POobAV/6Hr7sYY1jkouE7O2P4BqHDlRO1jZtbTJXje/ZFAOrh3KTnKf0C8O/YHLC88qKqHlCDfENsHihsGPC2O803YQ8fwE4MoZ1PP96OIwTNB2KZdQYTGvuYRPK/AwIYKNpXCqW4lzQm5nQq6lHURgZyFGd2cC8lEv0zgGHdyen59CpAwLsyfXb0R6wGZP+AM4ERIDBwuQw8XDbsCQIR581BQDQ3ikxzgEBgj4kkj4948/iI6/XmO9NeASC1f+4NH7MEBwt+9wnkcEQZ13lcKVvYeQPDV57t2HeUk0wn9n0zxXSwmd1uRhe4VcB4I1Y7D2fjBuhm+zxs1vnUWYBEhwQfHQASIDoolfAYJwmNr25clw9iBSuH53TaxkSq2UGgmUDuR5l77i1gkYT3totQLh9OJ23sLEQAsNPoQDgHo8BJ350ZBJuLJo16p+bmvp1VFYxWDt8666gyJutJ9ktv5Hek8C+mwbsGAsCJOYDq4HwekGFMIHwCgu3i+w/OAejmwjNsnFaH7xMfjaS1vDSG7mL4yNnA4+Ojh50mAQotGo6abcO27b4t9Z6mwbjEtmiAMQmklhDXsib0EVNQzHSBTSeWCJ9GXGJMA7ZJgr8KXqOCEJ+9WHmOckTgUhtCJZtf4Nu0UcdT+i7oqsDnw5NtrosGBXm/nUunGZBHOEAsTCBGbggQciJP9pV6V8y/uO019BcjAn0uAM0SnXu73gEni41YmiwBAHfkajXBel+uarKH19Lrtc8XsHIZqZ4PZ/D5AOQMDGX3boHAiTmQJgA5BtV9BG7uSADANnC5bOa40jABfKhwDgC+l3s4cda5AtcrbrdHYwJv3uDx8REfvnmDDz+02PO1TTRtf5Y+AWMCY2ECMX6bGn+QWbH54iDfMl2NAwBjYCgwd6QJMMa08WEJTcmhsGI14mwAsDr7CgA2Op8Z0NTSliaWafYYPVC3Z3EAAEFpzWJs3nFd26t3xhB8uOCDbdxgMYocqo0pv908KMHgCUU5FyA2Drldc4ux2632EDRGxecUVWjvw4QxlJcMBUTNSXiTlvv/4hrdG4u6eEArmTWrpmfzl/4SIDwjxRPKZBESfGYDL0mvBAS6K8UaVrHfJq7jBpFHzAlbHHRTXB5uuD7ueHhzw8PDAx4eHvHwwWM6h3bffHLf6/xzH34On3vzId48vjHb/3bDNWbxqYehptVk8QfAg0EoZE7T5NOAwVaQ7GDrLjR1zHa0BSOKkV5etb1j3VGmQyFDbX45mx3OJAYJtNUNAUBMbPGtsWrexHEoaTUH4vO2bXi4PNgkp4cHbwnNJdQiFttwxLPAeXHvvaIHwYg2jU5J5ykwfJ7tr40JcCCWU/aTppL7M+aOuduq0ZuDlCmFWCYcOwrXcO30BV01PyByJJmvI83u29yFA3PfKcAJypwLwTU5N4ViU85dCcTRggqksqsh0lrkVqkYRsh6xpVs0vR8eiUgANPckdSCcuz7xLjtuMoNU21W1tyBy23idlPcbhPXxxsuDzseHm9tfFjT1rthzh1v3jziMbaZ4skiGeijzxhkjZaaDgYIAwr4TDFbhSjwCZ65fXSsJxCPKKMaJoD1s9K4qO+nDXHO1Oago2Rn6mPopHkTBIgRMBDwegw/n5eLC8vFRlrEgnRimF9DMLAFmxDBNpzluHmQi6+i7qprOl07OffvWdPeBQEKv3ZwGMKON7G1+Pt2xW0M7GIuGdFihtAQ9r3AIAAgZwoCwUjLeRviLssRrsgNhCvmX9VHmgLuzDXmsnn0KKurTescTejp8wkEHKT8CVPlufRKQEAA3mZLgbl7A8Iiq4zbxO063fa+4nK5Yrs8YtsuuFw+wOXygFwiHMt1ZwWK+NyHn8OHbz7Eh073r75BZUScDRYQAT95rBiIoFAA5sTuUz91Fxdi6wwDwMiAoT5vYUyMbSQgqNoWa+qsYM5JS5GZIlNHRDjXFjpMvpN13PjgeJR+HpGLL7svULrY4qVNbEhybgOX6eaWCC4+6nHZLnhwn0eO5yNsaDo/AwBhge9sGRA3gUygoBGRSUtQo33oXHXiOgSiE/O2GQgMGw42L+vMPqDTlIPuNHFrssZ1s0dDpuLfGjOCIlkFE5iKubuwL4ylWItaANlhQ9kxKmOrA6dbCDEkHnsfEAgEO6P/028jWAYv5MWY8CpAIBA2kiqw7+qLZwS3m9qeg7JjbDe3vy/YxiXPx3Ypm6o1rJ2/efMhPvfhh+bwc7vxGlN5c1af2nnY2qHVYRTZR28hc5oDTabXf2hC2B6Cc8PYbERiuPNNt2IHKbQ+CUR2X4jiw3Ky1E1eTM1CmqPINxAsYJwJ/kJpHVjmfjFmo4pNBPsY2ATAtFEDwcA2gIdtw+VywQcPD/jg4cEorb84dBlwBABlABCjrGnekr8ry9UEf2UF/RjnmwigO27XCx62gZtExAQXoknaf3Y/QJuRSKgk9C/9FqC6SyDzNp0Tu4jvF9BNmDbXQS1ehAGomQLGCIIj2m+iT8WkYvOZmN2fcSPduZoOldZzTtjDnfTSzUd+BsCvwDjvTVW/VUS+CsCfAfD1AH4GwL+kqr8kZuz9EIDvAvAPAPx+Vf2rz76jmQOoCtx32OSgCWDHGDtkGBCMsWHIluehJdnhEoj6+PiIx6uZA7fblcyBYzCJYgJoPTUq36jgRLoFVKDDd51Sp8rxTzUfIcrRZCQDTIjAjxw5tuqCPwStLAppn3OeQbKAc8EHAQCc0m9O9fdtw9wvmNvwFY1W1k0E2xA8bAMfXDZ88PCAbYwS6lCKqeilljtLOQcZBFLoGRCYWp+BwD7BNn2cCxT7bgCwuUkjKHNA40i/SWZAgq/pE2BtS2YA1WHl2ZmrakQZobkG5NcIpsF9qsZjW5+NvwAA+460g9etBdl1QMAZELwsfRQm8M+oKm8g8v0AflxVf1BEvt8//yEA3wngG/3vt8N2Jfrtzz+emYBt/QwVWt7r67ljkosMyNj8PFYWOvonU6rz6/URj2/e4Hq9OhNwv8Asp+CuEzuZAjUkBUjEkdXIg0Jj+0OxBpkzpi0bAxhu/22bNU8EEjZHnjOBaNCk/khBTxDS/NQ7TR67T6BPHy3HXFrh4ZQTm5i0uea8jIF926CbQOcOmRMD6gCw4YPLBZ96eMCv+dQHOSGqUf0TwT8cwcITbav1OWbtzQUEOESbj+uH6bcBmPsVj5fNzJZhQT7CHGAmoH4+PZrQUWzKVyEEnnUcNtefTRr3J0w3GmP0oWJHFjudOjHH5uLt5gCMIUjYIgQCAcTFRkJhapqNqpFf7heyFuxu+nzMgX8RwO/08/8MFoD0D/n1P6kmQX9RRL5SRD6tqj9//1GLT2DafP851Sj6HlNDFbYWU+g3ktdE4HTYBTP66JCcSHK92aSgxxv5BLRMgqBvOQwHwtgAZJd+09yhhQ0YTOAmNp+As40NCmBTzWXGLPBp53l+NYXDtX2sNQitGUBnNVU1GGC0AkAwgbW+4wkPOy4+NfnhYkxgvw3ovhvbgZkJwQK+7IMH/JpPfQqXywVe4fY0ZwY5N9sFX9fzRfi5vOxkC1s+vOW619Jsm+pdC4UEiv12wYeXDZcxrDwgcyBZAAGAzxkIbpcTl7wy2z8G2Lw3GI21wZyaAV9rSJICywSYYbMlyJhQ3WxTGQC6ubnpgpwRLmONAflxMGZrQ3FzwPpoCf9LOcFLQUAB/A9iOfpjarsHfW0Itqr+vIh8jd+b25B5ii3KGggIbUP2a7/iy9G6qSI9w3Of2G9mr99usUbbGiEQMCimxNRg7/xjWOAGGdLWkvfRAUJspdGBpPNIFKC1IInAIbiQmZ3jMD67hzau8XyqBz+h4jebkpxLXjn98ZoAYK+IEQE40xhYs9PbAbjdHswU8OHUEJZwSgnURwYGLtuGh8uGh8slBT7mP2AIgYAJvS5sIJhACrxT6ZgenMKvCt0dDFShY2IfAp0Wf28TyenV83bDZTMA2Hw+yHF0gMCATAMLtOJtGkCgic2rQVAKKGu/TIhoo+OWZ85adFZb7AQ8aWF4WwYIeBnsO/NDhXwISXoYqj293DR4KQj8DlX9ORf0Py8if/OJe8/efPBSKG1D9jW/6au1CwZsGEsVU2I4beawmv0+n+N/Jo1TURuLOACIwBeUXJe48xVias+VfuVsI3XdG19wOGbuV2F2bcCf74NAoE10rqNfIj7a7fUZ2RHi95Fm/fwkVeyCWSspt5iGe8Ntu+H24BOqtg23xytul6uDbk2Cwhg+HFpAECDAgBD1wIJvQVcXEAhTIM/3Tqvj/uwIx7IlO4KNvsyhvlZiyzpKAQ8ApzYPFmUa1tvCh4aR4F8CGYJ8tu9hRTxSTJkYYvMZbJg5zEgqSGobdeZfwV9EajlXmiWoyEdPLHE4TS8CAVX9OT/+ooj8CGy/gV8Imi8inwbwi357bEMWibcou5sYyYxlDmhEcBGFDsGmNq12zhIOjc86U4ZSrgQRFaxWk0UMgVxQEkAQjCDNcKLRnSoGIed6XleZhWWXuxCjRhzCD5CPixo4NBwbI3EXAInyIzWGMSMSiPjpWB5HJwo4zQ4giAlWW0XfuXqk3gefinsxcACQ8R8NDBSy2TAob6yBQaZAzI1ADAWWOZDm15zAXtoaO5kDNCJQDj+gHGqhuavFjIQ4MxSf70D3Z60KDr9c2zMAh2Z048jMUI5LVwDNPzDMFzRHhCOfvs+mPXJotW91DQe8aGfS/I0FSFwBwl34kvSSDUm/AsBQ1V/x838OwL+P2m7sB3HchuzfEJE/DXMI/vLT/gDPPWvHfDeMzo4J24M0JtQYMreJKh6zPx8QG1H4Crt93nKjiVtq/76UtNPupWXXll7YgJzcw/TQRFmc3nXAewoQDkSB1XwGqPDOEBoiNAc0w2PDf2oH4gm6aC5nBO0vVtttlwRTEYFsPnylVtdDij7r8MCb04Uf8LBo3QcQjMC86+UIRIBEo+9af47WB+CrGjwAgPrS6Vg2XYM1aYH3BmZgd9uPRxOYmTXuRYBWAMfm3URus+eAMMIJzTkgpqfCTE/oM+Vm6VtPUkBKL2ECXwvgR/zhFwD/uar+mIj8JQD/pYh8L4D/E7Xr0H8PGx78KdgQ4b/y3AtC2OuCWghnrxx1jrPZLBtfgx2z5VATSLInqN9T1N6Wjrrg33YKLzVpmHCZ+iol6ckFGu93p9Kqe8IBlp1EfSahVNTj9igGAbdnw6EXPgRBBbr0N6dQi1NBVLgq+wJEJEj06SSEP8yB/XbDPjbs1xtulytuY3Nz4Ibb8F18Llcr8dx8BGRaO8GcoeIhtI0FeN2I2C47KOo/6dhAIJhA+ATINEhfCZsD2uukKREYAExiAluwsaD0oVWzXhamp3FvtedSkdXvFMgRAaXRAfevzGn7FAyt0aQxxfc18JBiMFD1ncy8O9WiowCAau/qkUuWX5ResiHp3wbwzSfX/z6A7zi5rgD+wAvfn4lRzRfV2SIb0TbNck5gyMyFxznssrvHNf/Fwgu7tk8S+ra0lGYMzlysbHmS8kEwIHT1b8djpSsihPRK6eN5DU+8DkSAOUbO1R+0GjHtROgBDDJoxQoAVE+Us7rNtX943PfLTqaA+wRc8G9jayAwYpGLe7iHC4pkI5rge4xuWKBtpJZsIBBDg/si9Hs5CZkVgMCafQO9lYoNDBFgDNvdWeF1RUOWfjwq+hLunuKaK54E/VgCTdo/AAGK4VOLLU4gMYE5T6U2FnDznojVnxaKtwDCJ2YO/GqlRmMkaFx4vqMhFZJeD9e2SSujol2jLysQOchnmySU5gCPDHC+PEPgI9/A9yE1fNiQYUWuDs0DqUCAgDg19IUmQ52JnminACzHB3jEH3FtwQVJLUaZULg5oLMxgrmtpgD7C8x5KDLgQdYzDxCf8hV+k0BzQ24TDMEBANhpVkITIwMFCvDzBACW4Cxcr1+bw+FHN5m2IZBps7uCasdjJDR+Vrac1h1cqKuNwzQJ7e/KqJmZ4Qfo5tAIXxHchSNAzQ6sdzP/KCiIgCf42OmVgABpV/QGq6267ThGRdJJBM5527EKcPd9A/aM7RfCb1GAS/j30IQ5swvlhGk2/goEgjXfjSG4ZjCWG5qBNPLabs4MZIh5r4HcbFl86jFT/VrWCuuICQTSOjHbq4fZcYCzgIl9lKDfbgO3ccPtcnNzwM2AseF6KSZgBkDVVwwHjuTazqR8vXsIHW/A2jZl9VGAsv/DJNCaO2D2XwbiXE2CahmpOg0mAPIDiIZFmWxiZvtIPXuprzaHg47redn+5L+AOruVnIsiUv4BwE2A1nZo760RhO4g5Kv546On+TS9DhBQo4ORrHOgRgIqCnSjfo2O5ZmWFl7/AOos3eaOyD6sAZJtxP2g34I7Gj2r3evj7LmDcOqMtOHrWZU/vdO588n5X9eCEe0nWZWzA6uO+x06nG02J2PHLruBwPWG69hwfbzhcbtiyMBlbLgM22Zt23dsD7utkYjzmwVVMX8Azx0YOVJQLKAcZ3MVfDIBEgSqE1g5XRMns4sRA8SwoPkAYrRCVCFTsItvHzZG5UUtqtKc5k+aCgcJbTXNAn0U/ggl37U/9UpqrLrOysGYCMG9sKkcJqZmOcP4NeagDibqE7OWznMnvQoQUFVcr9f8bA0TaOnaIxSAa3NeYinwKDpq9NNBPmQwabaI1BDNtM0+55yYI+iwjeFO9WMAk7dGr1ICAvDED6fCqpi+GzJEoTJzajGVHGzfmfkzqvOOkwVA629zGuNzjJC7stuZCHBCCkKMFuy7hWDffHLV4/VqJsDYsI0Nu4PA2G+2C9Plhu1mAVYkwquNkUODAQRmDsSoQMQLqFGCmCdQ53YsQKzRiOjk4eSdewgUTOiHBXmxziO+uYu4ghgQtdBzUxUSzkkQRfe1HClubBZUzSF8AdGXs8bdpxTEKK6tMxCRJu+xZ0QxTfCtvWM+oQQoIPJtOx9Bej6eS68CBKaqh/KyVGAvTtEKCOauhfpeYBmCoTbzS/y+AIYZ+7uNkZSfRwJqeMw75ebXtmInWZ+smBOmGQz8zOnemNXRpy+9rUVC540kUvHlK5jHOAGBeEbYATiiAH0ODWPn0aHspqzbaTEcbrJjGzuu1xuGDLwZj+afCN+VApfrBeOy5TZscS6bbc2Wwu+jAnkEUe/wAfBfUsA6r9GVMsDSVwqbCBZRomMOidn+FvlJRozHB9DVnI19Toyp2GXGajBgqu0UqILUA+x0Tc1P/SLZZm+L6h0OvKMDupyCNxF7/y+WE4kGiHk51b7jsOaYwBx68tzz9CpAQHXi8ZFBICZPSIKB+vUagy1KJWIbkWgCgdMktbjDU53+uW3NqwYTANJGJUDIYSimgziYCAwAcRw6LXb8sKXHsX11NH52lm6juGnSo/+sHQdkDR6hJFTPU5qAfyvZiXOegBgQXMcNMdtOYNQZLrwRZXlcfGusrUDAdml2wU8wgC+WQXViFp6YADaL9mlSQHvvcGo83O8gAgyIbTByu7lyCBotuQeEAcAgJ5wpkLDJ55iQXbCLGBB4zcwJ1PqQAAJe6cmUH+2YAkgTPcJEsVmWyO3DnpLWxt0U5sR0mz98C4M66PR3D32q/Xt6JSDQzYFYMdjOEwjoL7qxe6XVPehps4HsK5WK9jOGaelmFxYIKIOBVh6zrgN/Mh39BTqN3s9Zi42s00l2nJL/oJnq896PlDEWGIXmFjcDJH8bbw+QWXwN6GyAc5126bQFW8OBYL/tuGHgOowRhGkFwAKsbgNy23LmYGzTjm00AMhptkOqA4cmJTCweivvOdNAAbJetqwjWICOWTM/o53EV5mOzTYDDa+8HW3lnoi3jW/qEf0LMjNKsrVX1VT2BTYKtIwCa5xewYkDAQDRpiAMIJOB35X9SaPvIE0M9RPNjlnQ/u6ZA1Px5s1ju6Y53zXWg1Ut8XmtyPMlmRgZE7BWZJsNWN7/GE4kDUjaP+3VyZ0VeX9nA1iUrjfUUFuKG2aIiMUSjGcSohRwhSlN8QlpQVQ0fK03UBL2npHqtl3NZDex8afKr5Y/wPT/zXfL9eco2hZt2yitn/vhbcVcigmAVhJKTmXooMrUWvzrStgAACAASURBVHOWpxTaQ4BaHDQierRvwKnTmYCtAUkmsG22elNjMlNfIDanQnzdBMDbw1Vbt9Woi81OqiZ+QBzN6jZ/i8UfEIuvuLHifHlPtZs7KZf6iklF3P5TPdjuC9KrAAFdfAIiYfVFzdO5+Gru3FILuZEGN8vpOQl+0lGce6njHKhKL/oK6riBxN5MtJxUhgXniKXG4WwqEGAAcI0XPgEP9NGZAdVZ1JWpLi8paaxmkxJzEUkXQmwVBmPftmejz+21vigFeBO+ZdeO29VZQPgrHARAqwlr5WCcE1OLd1JBUoFqfSFaNw5YVOkxBJex5fk2bB2+rQuZ2T4RUNX2hRzZ5oP6wdQJ3AQie6tTqPmphuc/lw5z7Uc+k3USs/NliAUe1X4Z9Trak+5rj8/2877noG1drvpZNKxNSZs2wpUXToLUnKRXAwK325WuCCJWgKSAh4PM6TwAkQ0BDhFjX6FegQwAfmQQsBdbZ+AhKwYCBgvXMErsYO3EQCG0iECcARjizwxXtoKADUOVz0BGNwmwOJI0BUSyrN0L0OlrXElW4B2qDIZgRDZjErBty0X3Vj6eUFQh1Wsr7DwSAygmsIBAz15TwZLHYARWnxYwxAKfXDYL0zU3A9ncQUgDDsWjTVUE6GifKKVoMZ2xxVReA2/xCU6iLJ29lo/nKxPoAFBb2TE4cHstzRePJQYYGKOJ5JwvKZ/Akrun0qsBgSuBgFm6sXFjZwJjuA06NltY5Ig/M36+3aqkDQGGgnxpq6Sga+p0fMK9+zGbLNjzWASeKSQJjIEI+RlGjUyEKRBAEPkLW280IKgpxFkOrQ6XLIJKV+WiEiqBQMxbUGNRW7IO0u5e367s0pMu7t8wOi+ATLOpmTO7674BAQoMDr2T/RTUbMLXBAA2AlkvlNcXDycj3xXM0WaaxnAtsOVog23CohjTNmSJTV/t2H0DTVtHBrV/aPa/E4hiAcFynWedPW+tGrqc/UaizqJi4q4Chnn+uNP0KkAAsIkq7YrO0AOI0QFAMlz2GBtk2zLM2IjINtlzpH9ub+vnRe99kkjajfEoe1YgfMwNKLvsCAQcL5+nxvJsRwakTIKjYzBChLm2KVbzFAhoK6gq3dFAwLd3F8HmYdvifBs2LyA3VIl6H7VLMxm8dZ4M5SQdFOkdLRjCDgF8G/MYUyfpqr/4Dgx29Rw9vDefjnK8Rn3baJJOm7Y91zBwkYWcptkzX5q/HJiH+QGMdEsVsEgzzvBOaEQR1hp+ufR7ehUgEFQzP/u19W8C1gllYIwdYw9WYJ2zOiOQSJudBnk9U3TYcFIp0EwFredUJxRECOrIO5cjjkNhYaTUAY1mprXUsqMpXNmR0DtNE3wf6lJGo8zHwge0jglgWh11oMAgj4ehSt5ViQQvQFKAXjoSRu7VZ4U/Vka/3oR9Oef7Sb0SBizvUPpJAbwVpVYdngruoZySY/n5mU25FWCW7B/TiVCn8HveFc6WeXao9R2984in0qsAAaCi7wA+bdgF0qaMa3r2c3PQ2ARjMBMogU1NFTHwVnDgDhS56LKU9DwckUXPfXtyb8W14hMMYhEJ+xb8y8pDYZNwh8yOUp0T9KxiAzM/1/tXAAhgkwQD/kkKvXdYAwQpU4HBwJmASMt4PweA8x6+0GdO0q6Ehq76oTYl8MnvvD2XJmznMTy5pgR3GWUejGHzB9SZ2JRSBC6CvRhSbyFBDxPgHLe4zE2/I3ClHIT2bc0P8bUkZB/FFOd3kglAbTpwpN2987E70E7e+tBIQUvjfBBFbZNrltDbjMzVqSLJoZPGfbEjse3A4wAEdnSdMQNNrcwAEM8tIWfwiv5drZ8aC2ZWFBiEfyGYwcEgQFwMNhV5bCCA6uDugnWNGLsOOQgEAI7R8lesftXK3MhP9cylDU5+F3k6A4IAoRUAsojdMjo8d9XW4Q+I0YFgBkxC2P23EpwW7DUA4CQEfOYfWiM1lPi5KevgsvgduXr046VXAQIKc6JFyiAXPo3VxqcVt/RKD8jYiabyBqDLRqAECp1qj9ahWKNkH5YSvwACG3ZyECJO1xWMpPDXdzxygNZJEmQOgr+SFWYVBABazsaoz/yFVh5Y+AuUYjTZyj1I19WeiNJNAs8ra6kCAsptajKS0Ds99RQ3zlhbCmPUPf9yUbVAl/475yyP3D+E2yRriArsR5FijaHf8zngI+U4H1uifrCaZP32zNyySk6roDrBi9OrAAEoajNHoMX+u91mxQG87an9kw0kMyibtU23jXn4QeEdFAaBQO73F7QQDhJwoVkAYPMdiFs0JABrVz7z2QAdAHKjTx8F6Pf1B5wK/xMg0AFAFwAg7UJaRiD5OYBpZP0dhUMPva3T5M6W0DtnCk2vvxK2YmMJ4nGd/mIeAgQ1GamJDtBkg6iCiPheEmHyuW0dk7SU26jn6ejjsIcWmICGAxfgp9KeAWMOAyd+1m+qTNquKAzE15Gx59KrAAFdfAK7RwTe94nr9YarB7W4Xm/ND9C2DCcg4ONg0KAhsOzUYyA2DOmAEBMvvAkcUMbYcLn4luQjF6cvx/vn1hkKmHKxkIMX10o/ogv9yTHqMn7GYNCcrLM+87CB0OtiqK6EowChOrJ1wrBF9UQqnlRIqT2pjljwUWysCT5rfGHxcMEPrbi8vJtu2tRulE1DScAjFMu67XsAgB6Cy8az+b4j2BOTYSJDeeSp4LJ8B6U5IdQG7fpHTC/dhuwrAfxxAP+kv+dfBfAZfELbkAm6ANj4v3tofWOMyRUHZI+emNZhZzyr1F1QZ1tHbusEIhZ/+BBKwwUAVIefrunH2DCnxziMDkMCHA3fjihtdFKfuTowdvsVAoEelDI0Og81LmGsCQS8cjrzvcsEgq/HdFQ02aDKXiDJRV6Rk1b6QOVTXVFOTznvASwImg2fBtvC/PqioGmLxjIgyvWKmUrkZkPP1G2CZifriWNqbsn9I2UIxjwT6shlp/FZLAaN7BMFePdUhd2zsD80rDp9XzPFzonFk+mlTOCHAPyYqv4eEfkAwJcD+HfwSW1DJmggsPnkn6Hqm4kMP59Uk+YZj/BVtrpKa7GITMx5FHaEsLclurV0l6/ZPgdqXvJtw/R49QEC2xbbdA80dsG7KRH/LJ+AoICk/BdDIg4B7bfnQs7x6+247Gxz4hkK6tyGQD07BgILAPhv4pytkw4C5d8oBhLfH23Xs25dYeIov6HBqb5yQrQFJ7TRojFxm7XUGqoZQHb3DVQsFJptV8/sZcScCwGZf74aZQyMCSgmdHgMwDQxbVl6CXflU1BrIgB084FmB1q54X2wlFonQ/SgrI/kOOgOydVQ+HjpJSHHfz2AfxrA7wcAVX0E8Cgin9g2ZAJg20pwpm4YvvjG/qZN2Dj0SoXtAGfQLI7e0Fj4YUAA17Tiu81AVu3fBRHis+fEthLftgsus7RtB4Et5ykEKMR01bLJj8ODZRIUIIhILtAxQavfFzCU8O++BXvcL6TmAgC4lnkItM7LZi+NY/VZC1NY/8Ls5uX6yhaepqUdAGTpv32IU/P2Cdugo/ZwDGoeodJvHlwkgowYCPDoBpx1CiyAawhQsAIDfluMFsOE0Rdig5BkpFRI/tiYAwg4GptlRiRUB+c1V5OhO/hUnSHbPZnBC3HhJUzgGwD8XQB/QkS+GcBfAfB9+Dy3IWtJbMVXpDGtAcYW0zklF15UB4t54K7lIICH3o4loobyfXZbor/UsbOBDhCA4OFyQ2x8YdktEAgn4XoelDb/fM5ALCdmk4DBoBYtGeiYoFe0n5wj79GTAxh4xVtqGi+rVzK82qyusq8dHXOsb6YzoQYIxDqOGv+o95fGXj6fklusIGhmXeTueJ86E5h77VE4o+4226hUx4CIj+pICFWxggHYUKACAyOZxvD1/2P6FGW2KRgIpI7NCejMIW7Id56YB2uKsgYw834D4SPIZ4nf2anFs+klIHAB8C0A/qCq/oSI/BCM+t9LZ28+9AWhvQg/9fDQPOPhJBtM5dxbG5y22aDqWkKl5oyHLSazzpP+9Xs6E+ggsG1bblDCTKADwYbL5YLL5SHPAdqEQrWd907SQSAEe98HbCd40DM6AOyu6fb9BrdTiK6Ke6ZZJYTwx8wyo+UcoCTuLowg+xzx26cFXU+us26X6qVNcMpyqmnROokNtZGRmjY9c4eivW1aGsFnTTR8MZGU7yXfCRLMDNUmbZJQzRWIvhQMhWuuyte0v79MwEygmuRUYuSkXhfckfZermmh8+fTS0DgswA+q6o/4Z9/GAYCn9c2ZEp7Ef66r/hy3UYxgbl52CcdGFMw9gCEDBvYOkssHkmaxR1fyIMfiNxg2O3/g1DW0TRMrAKs5b4m8BsulwdcLg94eHjA5XLBw8ODleNOxCIWVvYaB9jNudu0aM9jrGysPRPc5nX6yyBQIyDu1Brcyxz46HMFtSw7vfdthXoA1jAFJpCrGEG/ae27NvhJWs0AQbUvz7Sc0NzQU2d3jFrd7rk9me6xZ+FeE9DIvAg/T9JritEIGOtRVWBUXIfwS8WirmACjAUsfCv9b6ZAK7ucyn8BQDe7Otwo9IgKC6v7hJiAqv4dEflZEfkmVf0MbMORv+F/34NPZBsyYJBPYMw+VbUomdhmo0EVU0POWpgjghY/DztVBoNDXWOt3MftTdBv+w2xp5z9lEHABP+DDx7wwQcf4HKxI4CMdBPBONTDnfcOMg70cb/t2MYNu2usqZM2T7nlEOrtdiUQQIJIG3bUnAOI6iX0GWU6RFBzkXCICqC2PDWIWpkCJQHPjUt3QFg4dKOv8YPyoXAsyH3uZAY5I3JhV1UCgPiLpdCxAMl8T+EoTc0/4rMFpIlVetnvuL0GahcpAG1ub1w6gACSnS3NQL9hr8qx7sTrPljIOn04nsX3vxADXjw68AcB/CkfGfjbsK3FBj6xbcj6EGH6AGixSk271JzMoihKmKsOw9E1eR45gwB/tmtM72du522OyRuxgIgpkAKWk4fKHHh4eHAmINg2C3xpjs06jzIftYUJ35YBOzz/urKBvl/g7VYgMMaADg9o4uGp0zPNZlACgAl62ZpVN9W3aTSA92bntPS++0zANNhB8PPr0NIx6uFLsLUE3iaTlSkUDlI0ACgfzrYPzG2HRX+++PN9VETKFOA9yYVHAk7+ohK0fpLXjppf6rBcOmrv8wHWleR3AGiVfzh9Lr10V+K/BuBbT776BLcho3Ontaltp0W7mZfQCLaG3UKHqwWI3F3ANegkEAFL+S1E2ux+e2FjBlWYOHQHX22aUc6rDgyb2/eCMQxM9n0kjU3tmx2qOo6qWtDOHHrMem2jBDmr0iPtAposYJ2AtIJAAQDy3a3zxn0Bjuv5GL2PLWrnOVOA38dHALlRTLCAMKlqB6k9WRGzgtqdaEaLdY0r8JBe1dxZt5T7PtlqpeXli4iVg/Z8dTAw0eT+tQICV1XrcgkG59JbjuwCrXo2j3DVQjA2P55Kr2LGIICWYaayYxvYpu11d3HnmEyL2yd7RYo1LYpcfWggkBvVAEiczvcopGbGneSphL+EkMftmVZavkcDAZGBqdNHOyh6cfzEx6wZEFQVlxxq7CsVcyelWZuH3m43C8iiDAJl2sQzGh1FvJfAQPhaZycrELykg90HggCf0d5VINBNgAIDK7fmbtKm6ff9ZutOvMEzLmG9rgS+nRMb0SW/7vsBP4ufGf4ohCCHPyVe2N8JLueTAEC+Ri4AORtL+P0ckvEmYjp8VzDPp1cDAt0+qo6XjEAVug3cBHDHQKP9IfRmGojHqqwZ5C3ok6I+NwS4U2nkf6i95jXBBkp55rULHnVHY1ORESsh7V2lIep8IwBoDRnmSLIBMgluNyjgu9v6IiuZlRdmAKzpT4S+gUJqHjdPgGwXvLCDnTf08i7U81oAFgYEmjCV8yWcIeQIQHYC53yL1meZTAFdeTboel52PqDZ004tovht9TQysCS/Bn19uJZZkfp9gjizAQKCkdf7DNZ3DgRYQ8eCFbOPFYqNhH4mCxAxO1t2q5Q51feO8PDSU4AYOfC3AGxTZRU3lF5T9K3omDWDrzRFAUBNGpqz1qfzUKEJIGuRYkJz7jUJKRuSHGWxN4CbA1ffISgceQfHYA6HdXOgaWEJQOydp2v+qqfzzrXU35M2QXXoYCUhCbw9fNR1jQDt6R/osycr/jQLXQRpicouQFht9vB5FAMIgc/JSnecdk3cxa8yyND37VoVuSuhU7AgwTepT0B+7u8l6VWAQBQoPw/BUJu5tXlDZAP7FmIi4htG+LY4u0CGQmL3mrDdJiIQMQCkUyobUBetRqdHM4B9ArOckSEgo8wB0+ZsStSzSuDqpdHIc3cQiHDeeQ+DD+8YbPPlc7LQi0CgM5AU/mAKrHEWrd3Nihcmlp8AHJIUQbVBzQcg0KWyJyugNRTB6waQw6pHILN3YzAYVP5svgROZL00f/gEFvsyOgC9D72eGzrF7fcFvuqqnt3A+IwVEEC/kyDQ0bA63hgWcHTTYgq6Fs5PFTAAgEJhe8APQVvpFZ5c8+nE8w7tWM9rn6MzlnaKDhHT4GuGWUwd7iBgeQjm4CVdgGAcWID9MZOwgCu1X+DNveMigjFjqqvtuzjCibcIeRDV1IjceROgAhzq93JWWXeaNCvy5JssWwgr2Kyrup7tvC+Yqg1IYwKWTfNmgcy3huBH9qWUQeRTWnaLBRg48DfcZYn8c1vmS1F1mPcwaHDV0O+iqk7ArAn58vk5VnuWXgcIgKvSmcCynULOGNz3ZAKQvSLd7hY3P2YI2oZMCgGvPus14w5eQmY53ANmAxQnsBb3gBqimMC2bQfBXzVNChrqtZfL5eAXiB9Nj2C8zxomu91uuF6vDgI1oSWZQAr4uTA3zUWd6vibyDOps7tteZJauameCejstvK1tHO1FaNIYKj9GwTqDjEFxvA19VE+RMT6PC8mYNCT5qEumWUTIRyFccxG0ybJWT/BBKhS6ruj9hc6jy+Tvfhz2QHYwGBlCwEEL0yvBwRYuWs1Us4f8BbanH+rAjHHcEuk8Kj5arEArKNIOlkY6ssvgKxEv6vdkFoox+l9tZoLoP3VtXDWcbK+1BtFBE1rwd3CB/Ojecpr/Hue/ImomVHBotRWwdn7ukB3n0B1oNA++X10sNTgrZaeTO2uUxBYgSD1r9c7MzDUN6r9XADBtK3Q3TRIoVnX8cebJDJFTt8wO7i+yQGsXAh/Bsn8wqCo7lKDx4+o2PkcflA9I27KsPOs9bnd6J1V1pe106sBAS0YPla2J2aQ0amzsWlyhwkAWmX1l4ERAGfVlb9RuO1ZM/Zutyuu1w3Xx0c8PjxiGwMfXi64XGxoMMJ0swZtIpHXK29Rnuv1EbfrNQFm332SjNYmqjFbslGLpUN6d8hnH+obMT9wCViZNxgSH+7rJXlZOnk02uSkeiq5447CR8LPD01w4791whkBQb2DHLY06jK1lnKzo7KAiNMijCz0UfcNIFB9ruFgB4FUTB1lkP9SyDsAJHPgDvFMejUgsPaUItBaSlypwgcs9BNsnXf4D9T8gumUmwO5D50e3kIXRGjiVVVeOgP3iG50xePjowXg9J15Ac0wZjZkBV9JWFqPtcCK1ux9//DNG7x5fIPr9RFXB4M9FsNwpzwIQrwnugmqM4EFNynOoUOHoEPVt71SZyz2lNgKi5nTWQolexQXOvNZioD4oh1qcWVRJ6ccnWVZQjF4s+W6iZxyvtGWaZQL9rHkPISYHeoh7WjZdgvhJuvfqpnXa6g+INU2DQDOwCCOBDJZTkh7R2iBaptSAs+lVwMCrUOGpvOUuiIro6/UMgAQDIU5wzS2a/IoMWMaHdcg/3cAgTtJ5ANlBtxuN9yuN1y3R3uv2+7m8bff7fvuW3dvB+2UQrpSOTEwwxA8vnmDN2/e4PGxg0CuZJzECCLbYjVUmqZrE2nlq7o+E9ZcG+DCnuPV6zVdwYXbcqlSNrEReXWVDElwQbbMYha0XMeDe//IviDFAGKn5L6lWz2LJ35NB4Co55yD4SyMHcH5XqG/gdbWLPgdGCrTZS7QA9uR6r6BwGjAgCh/AkEHlufS6wCBRSLzlAqRQAAchCsaXwcw1JZ8Wkgyq8AhkrvKB6VtvPbs3clAtWxwcsbJEFy2DZftAhF4KPKRQ3URiNQ6wTjk9/RvCB4fH3G7uTmwV4gsFv4CyKPZVP2HAOBOZ6iyrnw9fqOIXXYOQ6vE1eiNh+ckUC0PT9bRXtjpfz/X03JEf2jaUgQZuo38AgeGl3MRamISL1QqFhCbxujSN58GeADlxEPlMX7fGQLVEoPAQchBz6nv0No7APodYgJHp8vR7gMIAEBr8n2utO1dOCE6fFGYWqioMaE6fOPy8hqvPTSA4bD0xTvHvu+4jRu26xWPEsIe0YB8W2uduHl4q227VFBTEQp/3jVTW7UogsfrYzKB2/UxY+XtKwtgpkQdMpteuL5O6veErlfdU/W45IuUn4UxgwX5wADWp2rkVxxfPL9B87AKfj/v5a0XlUklNXV6iyNt8y4FZNa04QisWASh/U/NAR6OJIGEoJ5PQt80+AoA0TbUWEWIGl3IOmtAwhof/Byh97xDIHAU+PIIV8+x/0TY+6sY0zzgAybsKjOZgA7B1ApPdr/b11h/rkKUEpRgAbsIrj7GFPskCOCLWIwxXK833GLCzygAiKnELa5gLl2uo/kc3uCRfAI3D5mVE2TIUbWmBEf/UN1JyqYFCVcCoSLoUomdsQCrFu+ALcRu8qp6P7Voazqub5+glduqe73HjWUCVF6YBaqSEPoLotwRTyF8AsHQRgpcvWHSSEAwgFqyXcuVuzlApikYgNCA5ui9r2ud5nNZOI+SyqkAo2v+Dv7Hdx1r/jy9EhDAMjrgKeTW+mKhLqjA9HkMmJNQeXSAKv/svahnhz0sSnkIx6DbiWPcIFcBVCsqMY31RQwBHuvn0Od9lV8/Fxk+8nDF7VqjA9ERa9FShd6Iioo4AFQhLKvNzmFN2OpfD3f4/y7sseSYgnBUyUuTLwFzT0yCvJOOJ/lb6HeyNenmRJMbiU1WpRgAmwPREdz5mcOANBO0YjnutbHsgZcUEzgK/VnfW9nAOQi0qqBOexR8Pnahl4Cnd4sJLGlRcdyHD+cEAmgOQzjFtvHjNZRzpDATYkgMMe6s5KTSGIcXnwPAQ2rWkeAaJeh77EsQ6wmiQ+b5CQiMIbjdbnjz5g3ePH6Ix8crHt0kuN1utISWVyT6ttssUKvUsVr2CzWRyT/HjboQ+yarrrmhOeOybtHDT9YUphgDSrEJIROkA0HkMxYxcZw9hqB2FgoizQwXYp0WNVgVEk7ACE6isVS54j/0rd5wEOiDbypGIYiunzkGz0cEqjytCZnBhG9GO4RWD/DnntHEO+l1gkDTXKtuqKurhltBNP6GN0Db9ziGpXJ4qgNB5oOAQKfZ/0a3FfsQ3G4BAsilrTuZA32N/8oEzAQIP8EYA7f9hsfHRzy+eYPr7WamQTgLr1caKdhpMkvksypEgcNkJO7MNeYO+n2xg9Dx4h0qPfju+GwjDPnrEsP7XXD9Jjp25Tvyym2tgth8wmeJetMN0Di+C63vG9DWe8Bmj+YTncnt66QrNrk4F1KlS+p/+KuhyJWidxYQwi3tucfaUejxImL9SQ7ZilUG72rtOb3bCpxeJwhk6trl0LGK1+al0ghI9DWNsITTZAAgIFDq5KJSTaMKhQOBeIjvfeSClav/5uJOJN41uajpIMcgxa4j+rrvew4PXm9XXB+vOVrAIcd1sVOzPmSBAwLJ1LQMq6cAEM8KgCHbX4oFCWo/QvvMoNITd0eNC4S1vX3PYb8VJ5os0K49wNsrPP9QiHr4+QCDYT/bSfgnjRasw4FIoY6P4YRkp2Q3A4Lqr74BZv9M6atNFiDvJae29T4rzs9EvDvroW2eSq8UBM50iQLZMN1/fLidUDcWEUGQm4rN0PgsLmeMIB4aWmH6/HXXjh4LmMaZL7htGy63Swm9sGOQnYDSRwpo34Hb9YqrrxC8eizBG01VnjRngDtrOP7ONXEX9GQE0KeBAAagmj2qFmCzozHfKXVfHJ5mBmeC3hmKxmO9/iw2OL3UpSXXdyzLtxUTU/1H0/Pu2jTMqzIBigUkW/L3hCmAAG0pf0Os02jzBUDC74DQAQB53bqfcrXlfx1UiYPFTeH/ERpBCJPoBeklm498E2y7sUjfAODfA/An8QltQ+YvqvNWaj30oG7HgmpMC30RQzkcK54obtOXd4BgeamtTnQQErMVVRX72LHNDfuwvRKv21Z7JJKAr+cZ3py0iU71+QE8TblCaR291pMqQehM29UzQWcAiK+583nD0JOiprpYNwA4mXOwmg7UaHU1O/+RD2i0zQBMkAXINSVIAID/KQV9mVMdNCzKszqgqUdh5mHAWKLc69Vekqwy+hCzuDvmQAMBMDvoRc86bIon2kcP4tDqyRGygwD5WV6QXhJt+DMAfqsXYgPwfwH4EVjY8U9mGzIq3FNfNu3Pidz5ibJkd4nyDDWOe++dK6gUAw7RamuMUD0jnYTGDiySj87d9jqU0VmAhN0fGoFBoAOAyDC/gg9JptNq37vdSpNYmBmxVRma+zA7MjT9AQiiPug2B8sML760RGidrLJnnFEs3sVV58k9/X/LiwLqgu9tal8HgJPmRoACHCDsNKPSA9BpUNaEnyNGhecN1KcQgkx2/sEceIoJoMwKemYVfoXAszbCAtZVnRn4WHsE4ufSRzUHvgPAT6vq/yGf4DZkwMJcjsof2dn1eG2tEccAFzS1mYPF53x9YUg5MwLUeXR8r1R7hXqoMPtOpm+aSn/GNEbrHEljl3vP/kqD0erBnCQ008mlzS8QWY+RjigXSNjYyXTsXGfsIOpAwc/zsscTm8I59OpDKx7aDsRW6DfFXBbnmAuXRpwEVVssIkThMve9aAAAFYdJREFUQ/Ddxh8zfIq+FgKlnWuOwMxh2IpTUGbIat9zENcahhz3Rwf8PMvOWn+tLer/vHipvtOl20uWC1427Q3zZPqoIPB7AfwXfv7JbUN2lhZmdKwuutIG9v2SCGqNgTNC2gRT0u6354kW7c0w1IzLqUG75srnrdDeQKG0RwMCiO+NwiDC7zoeQ/tHJ+91Ia4Vojza+0Gz98/Kdri1np0783gdndTBfaE/u67tm3OtR+UWwQivuFr4OMM6co4GIVg0+kTNbRASZgA0PXhPIIhZhNl+IABYwuDXCBCDQbGkbGcJEGDhLKoRxCPKleVuIKDpFuPP5QSk/tba5en0YhAQ23PgdwP4geduPbl26AVC25B92aceDk8ICqoR1jl20A3EC22Xmom396rKQaJ/bwRbeq7pKEIIjJaHlTe6r2p9Oqb+vatOKA5XlTbXXLVg9e2V7Zy8KzPIseg7kuq938ZPRZc81rOf6k5P259PsYEz+LD2A4DcNbnR6ejqIWCkYcMv4OeH0ZMlVwEoTLwzF/HsyGQwAgL1uJ5VvzAiafbEc6krNqG/+/0nzqyvJkC/nAQA+GhM4DsB/FVV/QX//IltQ/Ybft1XaKGXf+//99lpUblw7W4OP2soyetueiNX94VgDDVeCEUMFungflPDX5qrDu2NXW+dyOSaJHpIeWwj83I4J03O7/GWb5yE/R8Bbk+0OJm2/pw100edrqA8tnwtxfu8UiK9f7r3jhKDwY7WFl2n+gD87jClRKwtc4hYk9TYveSRt0f41mzRj1KT17WqkXDInVSIRI+t68e6z0ysF7ACAss1g0JV4fJ0PWu18zSevyXTd6NMAcC2G/seP/8e9G3Ifp9Y+ja8cBuyZhtD+uf1L+hc0DIRDyC5dAzqKMNuwRj2V3TOphubnVeboW6bL0XdBGMTbBTTfwz7vPl3vGMS76A8hr2zLzNFy6OV/U6dtPqJa7J8ppsPGrtri9ZN6Ebu3KvWS3t4+bub4Y/yB29LrM9f25yHVe/sgcB5YlOAQpgffCs602m4AsF5gasuuA3upwXUW1pMOV1vk2N10bV6+sIOeZREZ/u7l17EBETkywH8LgD/Ol3+QXxC25D5O7goIPM9gVIEbst793HeIzmfPXwAMdEnqGQNUgV1HWCveThWAlkVtVWWpKe4tbkSOku7vHyxSM2TPecpLR2MIMyWYgLxpijLmtGnCTklU5v1rBVojj94ubrpv7p3ktk4Zk2WkZRFKUCqfujvnjkQKYZ5i2nZUyTeScAU/e6IBNy+999zytiyr5QZU1+AGxZCbUmX2+3xjBe3OV6+Ddk/APDVy7W/j09yG7JWj50AdY1Y1yQbjLUqAcEQyCxHXyw7IUIF5G/dRs/ez/bx+jkKe0dss6XYf3DsKLKeH5/kcskdwsvhAlvAFs97uUHInaouep3e60UpCOfpuc63gma+ku44YiYzPEEOry7sKh6dIwO592PAQymBzC8LnUT1iYMsCToBQWEf9cXIO9sap22hx1PNgU166lHJpGmnpBi5DAuOPIN/mV7RjMGqrNgi24bkqFK8YaJyOiCUQCcQwJyCQ2I02ltZvZG1V1QH5dIr+aU/o6++W3+vDg5F4HrkHOpc/NbWYXrrpbwDCOlcZz33etTl2Auxavr4qt4ueM5ev5c+CjGQ/h8BOX8uRnbYxr2ZA9ZRog0yVsAc4FGiKi75CBYzqUYSqg2boFuHK/BhYHhC2bcJPitcsjng56SmsgMcWrT9hiZ9reV6Ir0aEJDeQtlJcxNcWbq3agFCniuhtuY+gzkRCEjJCQ7BhDpzQCfccIscokYZaVRWJYErxxSeUs7CX7H2X+unNEGjhK6Z+7Aa33XsCJydAyAkGDwjzh9F2u8+owu/nctB+Ot49mcPyEfQ0K/1kTsjHpk6VysAjLzVOxBCf1p4eXmdcPMsebh7qzd2Y2+NAVSnzEGSF1KBVwECPEae15bz1P5Ao2rWJo7UGizArw3/zSRtq8g5AX1O/J2Xp+YnNGbtnxq+g4WBU2n+aqeX9ZRmBrRrJfyA1Hg5A9czjb/moPGPJ7J376sXss56zpPavwt/zbXACQCET2ApBw0NxuzOyqkwTlTu452oL0vLSyqhNd/UK6mA/CH5offdlQFov3dt8353+5qVRwp9Hp8Dv0qvAgQA1P4CiM5fHl7xCR957lFw6rMAPhwkc0JgE4NkqoPCyWQLGI+mUcCnUzpbihKsi21S92o1fEQq4vOXvPDIGInqNcA5AtOpWN7tEOc08zw9bRg8zTnoKU3DPy/4dc9Tf8D9WI7nwBMfaoVk/EeK5F5ZtQO1qi1PGIOoa0n+oR7OuYQAw3WVD1GHcKvC1z34tdlnRUJ9M1eUa/T+eEBPrwYEGhMQIOpyuvobqOm0wyurpoVOwIN8Ri3alF6tKaVEj6wihQTXGqmBO6VkAqBn+HkJPpbrktdjchNfe4nIxbMiU/VZDu85zWs9Jct6YKLxdG139+/y5Ggo6Nl9hyv9ro+u/Zktngm+/XaM9Xsh4T9zOOK0zaMnrAyjyhLtWBGeo5lsnVLVcvUsPdSInOQrHt8uiVCfVeiEr5T0PPhU8jHs/fGcp6a1cXoVIGBj9MQEUIgn6PvRiVg4L5m0+7AzAev/XikOAOYbIES1FzQE73k5z2PN2qMrB8FHAYsLfTtfhL93i3Yx3tje2eaML+dPMYFT8yABjN7fQDGSlILMw3klPd3l/G0v1vhYhP788/EZx/OnWEeVSk/KebeoCGnV1kgB0JqjDGkC8MPz3dqBNfKdaC352BB4BCOYrvEjYIorSAxAY6HluwQCAE5AoMAABAL5OQXfPoujZQCAuFMogKIzgVWglraW9QoDADOB5TnxGUbnzgCg5PGoX5qeuCPsp4JPjISfFmXFen/rwFSm6Ktn1HXxwbwsnfOEe3Z+fHcU8vNz/sxHFv7Ibh9xWJI41CVQaJwYjB9+E74kbdesTmnZsdsMNOmVtLzkudA54jPnC6idth0MjBT46JnMXBUpc9qkWPFFci9IrwIEBB0ELOiHegBcWwloBXb6rzb7K02AsIumuk0WjiFHyEBQ8JFRfIV/6uwLve7DPN4Q8Z0/L/wz6Q9I0+M+CPA1Zh3Pa/8zgab8sjmxnBeVdU24UNuqBxaSj+IXON553873Z5+AwPp9/bZfi3d2QWcWcHKPcBlxp4xnYJDq+sQB1wFC4HUcow6xHmYRdmYDzCRsFbV0EHClplMBGR5a3+rCAunIs0u7I70KEMBiDoQjsNnwIMcgMwGY8AuBhLiDhJnAUWtrHhmGpf9HSQ8f2SmXABCPZCBY2UAvfH/smXY/PSdzpDX2HRawlDkXYeV3BUA8UYgFLD3xL2YC5+notAtmcG8UoLOHfuT84UQg1zLg/Dz/W5pfVkBIBX8ws2ytigtgglSBRXGAGDXqeZFgR1EX9FnDFwA3BQIEBAAmPNYNoLCt6T2s+0vS6wABAEPIHJAmRQUCXgEBBOkfkAgIAYslF6sIZZYpgBo2Yo3Nae3g51Wo7eyo/b2haOShzl8KBD2fPc/3v+M8nlH/zoRK+DvzWCWpOuLKCD5OOpoAT2n/50CgC3jVRa+DJ2VBeknyuWsRpWv3FQjyc/4s+iCPTvA9vZ8FwIrY1urxechwJgEfHUD6BADYUQHxiZHqi+RUpL3jqfQqQEAAyOgZHmrKKY4aaDjKJJg67XvbddRpki0LXM8bHW5aXJ2hrWzgXgV2EIBSR0CwAhI8DcFiMMi30BPpMwlhjymwzlbs39XT5HA9zRaNfEp+kAj/ra6zqL+LlPAfbe+Pns5AILfd9newl3+MeufKSvgYQFhsoAtt/e4O/JLG78XT5e5j/ZcZtTzbgYPzE5u7niFPAYEBxyBAMCbg4dGmmLCrpplgfUMQG7uoJLl7Nr0KEDhPYYcdS1JxJj10qJALJIBDpUBCqwEslZDSzxblJonaxUrWm6qB4+vS/sj3aZoEQJsrkBScn1j/sYmBJvArm1G0/1tf7CMizAbS20y/q7xp1gGImrI2fjodBdDS4vQT8YjNz/gDQrOdsJLQlNGm4gvG5OC8i+e07KTgMcnpRVzLokTyY1TBtZXb/SOeF+WB2Oj1WRnj/XTs7+F+60BGbcXtwvU2FsV6L71SEBDUhhrVFiIWjTeuBRhkw6YasL8hFX+eSNrTENnoYVV+F476vstbF2olf0AyAb+hCb4ez4uiswbXJ489rYwlzCoHEj7Pdyv/ICsjWcDCCFoVnaXTPPXn3PP6N6CgDh73tTYHavUnUEDbGFLP76r1s3wN6I7Cn2RAuH2iXAWc7D/h8hyWQS91G5mL8+pRmp8176lCHXfqHqf+kbP0SkGgkhVanKZWkNBsC6A6hFP7+iWS3iZqaqu94/uEnuspV5+p30Ca92h3kfb3m+K8T1BigUdez3IgbPVlYhCZBTUJqTrkmo94QwRM0WAP7gtgs0CL19bvV21FGvm0/pZccE+sxbTsOLsHBPS+pi1PAMFfrLnCshRzNfyxVg4sIAW3wIDrkB+i0EAehOMuH5vaP+qPAI2OmaGoU8oTv5Uw4Dzx4+hd7zgTAAqN6Yr4LEAfAU0mMATiM7ViKDGHUxxVV2L1gtczN05WUgLq7ddaTdqRhZ/pamMDi62eQn3v3I/NHCCfxNkssdD4NR2bP1Me8gdUWykcUh36DggcOm+e9Oef0foVBECC34X05Bw1QUchrpBpBKTlleZYNqFfgelQkuVzMABnVakcdClLB6yzgCiFN9Q/11Wlggaoa2wBBtPOop5PrxIEGIRVuGLgsQEHxpyYCCAYEJn1Y0VaBiEjo1WItHfcS020uFdrwIJQ4xUcGxgBqxnQ2QELNdI+t6/9zQ0dzoU/f/tECYAu9HH+5EiD90Gm7aWVXwoCytmn/BBVjs+NFdR5E9D4zcoiQHWd5WD2dswlC789dhGgtR6FP7ngMwgQmyrWAsqrnw9pdcrsKnO2gBYYzLIyje44kfBJQ8EAooQvCxz2KkEg04KSXD8MDhlB6NjisI6i8bj6PjTRIRUVj0Y5zBGn3zJFzY7rHCGAIEYJmAkchHil/XFPIoDmswwENDt6d3guZccZExCszKDYS69Dtl3BHTffsJ5EVjp70sNNKwDUeRMM7+WlUUlYCSgStELuBagJOvfy2rVmf0crTDsXf1E6VyMwhcyaDxQARs9iH0AqjdAdzyrtQlMr7rIQXuov3/UyIvBKQYCENRvZESBMgCG1vaQkE3Cd4B1AIha938UVxILLOh95xpqlzy7kmXf3hCQFloGAAUGRnajoOb1fe67uMoA8b9W2pNL88a7QXscAKUk7CuwkvN13QODknVwWZhchoA1oFtBhU4Dz0AW2A0LVTzytg9u9/J5R6HMB6qzKWw8Ru8+q1BevEQsIxXXX+ZmgwLB4591CFevdtoALEHr3eDkReKUgADStnwIrrpVRTECBFlQkQpWXBniig+WRf4vCWG+kYAIr6xaI+SMYDEQgMor+a7Qd+wesg4rTx9DqjZ5T+VWR78985LldOHRa/j0ATAfFBIJ4Rr13ZTtRP0cbllQOv5IurZNo8vxYgydtFIIPet8xL/HO+yzwmNZsn5avPbPyGp+tzqbX37Bz6bcWA1iUTyi1hSXESRGXe2Xxvr2SPvR3PeP/bumjRBv+VU1r56i6Oe+AefezJT+tvZN3vzCdPq5Tt7tPfMGLnhzmeb4odJ8+87C49SOV/iOkE5TCkYOdp5cW9Pn0fBWsgv/EPXJ+3+dVg/njexml6wdA+3ivfLUgYOnEnkuD+l76JDqxnpzdedUTgP3sW17Qt7vfoj/0pbUQGufu9Ai+fvehkt+fBmN6rryfhxy/7KdfCPB6CWi+NL0wfx+lnpZ7X9KfztIrB4GPKWGfd3/4CA+4Q8uevSkA5JmG0zbNdTFxnsgmd4iYb3DuCO156MJ8gg6yAtPxGadPeJIMffIC/PHkYf3VC/O1hma+xw7Pf/zEx/U3z5TqY1ajPBeP7lcjicivAPjM287HFyj9JgB/721n4guQ3pfr3Uv/iKr+Q+vF1+IY/IyqfuvbzsQXIonIX/5iLNv7cn3xpFduDrxP79P79IVO70HgfXqfvsTTawGB/+RtZ+ALmL5Yy/a+XF8k6VU4Bt+n9+l9envptTCB9+l9ep/eUnoPAu/T+/Qlnt46CIjIPy8inxGRnxKR73/b+fkoSUR+s4j8BRH5SRH5X0Xk+/z6V4nInxeRv+XH3+jXRUT+Iy/rXxeRb3m7JXg6icgmIv+ziPyof/5HReQnvFx/RkQ+8Ouf8s8/5d9//dvM93NJRL5SRH5YRP6mt923f7G02cdJbxUERGQD8B8D+E4AvwXAd4vIb3mbefqI6Qbg31LVfwLAtwH4A57/7wfw46r6jQB+3D8DVs5v9L9/DcAf/dXP8kdK3wfgJ+nzfwDgD3u5fgnA9/r17wXwS6r6jwH4w37fa04/BODHVPUfB/DNsDJ+sbTZR08cYOJX+w/AtwP4c/T5BwD8wNvM0+dZnv8OwO+CzX78tF/7NGwyFAD8MQDfTffnfa/tD8DXwYThnwXwo7BJqX8PwGVtOwB/DsC3+/nF75O3XYY75fr1AP73NX9fDG32cf/etjnwDwP4Wfr8Wb/2ziWnwL8NwE8A+FpV/XkA8OPX+G3vUnn/CIB/G7W57VcD+H9U9eafOe9ZLv/+l/3+15i+AcDfBfAn3NT54yLyFfjiaLOPld42CJwteXjnxixF5NcC+K8B/Juq+v8+devJtVdXXhH5FwD8oqr+Fb58cusxaMPxu9eWLgC+BcAfVdXfBuD/Q1H/s/Qule1jpbcNAp8F8Jvp89cB+Lm3lJePlUTkAQYAf0pV/xu//Asi8mn//tMAftGvvyvl/R0AfreI/AyAPw0zCf4IgK8UkVhvwnnPcvn3vwHA//2rmeGPkD4L4LOq+hP++YdhoPCut9nHTm8bBP4SgG90r/MHAH4vgD/7lvP04iQWfuY/BfCTqvof0ld/FsD3+Pn3wHwFcf33ucf52wD8clDQ15RU9QdU9etU9ethbfI/quq/DOAvAPg9fttarijv7/H7X6W2VNW/A+BnReSb/NJ3APgbeMfb7PNKb9spAeC7APxvAH4awL/7tvPzEfP+T8Go4V8H8Nf877tg9vCPA/hbfvwqv19goyE/DeB/AfCtb7sMLyjj7wTwo37+DQD+JwA/BeC/AvApv/5l/vmn/PtveNv5fqZMvxXAX/Z2+28B/MYvpjb7qH/vpw2/T+/Tl3h62+bA+/Q+vU9vOb0HgffpffoST+9B4H16n77E03sQeJ/epy/x9B4E3qf36Us8vQeB9+l9+hJP70HgfXqfvsTT/w+tCba+RPSREwAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "import mindspore.dataset.transforms.c_transforms as c_transforms\n", + "import mindspore.dataset.transforms.vision.c_transforms as C\n", + "import matplotlib.pyplot as plt\n", + "cifar10_path = \"./dataset/Cifar10Data/cifar-10-batches-bin/\"\n", + "\n", + "# create Cifar10Dataset for reading data\n", + "cifar10_dataset = ds.Cifar10Dataset(cifar10_path,num_parallel_workers=4)\n", + "transforms = C.RandomResizedCrop((800,800))\n", + "# apply the transform to the dataset through dataset.map()\n", + "cifar10_dataset = cifar10_dataset.map(input_columns=\"image\",operations=transforms,num_parallel_workers=4)\n", + "\n", + "data = next(cifar10_dataset.create_dict_iterator())\n", + "plt.imshow(data[\"image\"])\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "2. 使用自定义Python函数进行数据增强,数据增强时采用多进程优化方案,开启了4个进程并发完成任务。" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "before map:\n", + "[0 1 2 3 4]\n", + "[1 2 3 4 5]\n", + "[2 3 4 5 6]\n", + "[3 4 5 6 7]\n", + "[4 5 6 7 8]\n", + "after map:\n", + "[ 0 1 4 9 16]\n", + "[ 1 4 9 16 25]\n", + "[ 4 9 16 25 36]\n", + "[ 9 16 25 36 49]\n", + "[16 25 36 49 64]\n" + ] + } + ], + "source": [ + "def generator_func():\n", + " for i in range(5):\n", + " yield (np.array([i,i+1,i+2,i+3,i+4]),)\n", + "\n", + "ds3 = ds.GeneratorDataset(source=generator_func,column_names=[\"data\"])\n", + "print(\"before map:\")\n", + "for data in ds3.create_dict_iterator():\n", + " print(data[\"data\"])\n", + "\n", + "func = lambda x:x**2\n", + "ds4 = ds3.map(input_columns=\"data\",operations=func,python_multiprocessing=True,num_parallel_workers=4)\n", + "print(\"after map:\")\n", + "for data in ds4.create_dict_iterator():\n", + " print(data[\"data\"])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 性能优化方案总结" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 多线程优化方案" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "在数据pipeline过程中,相关算子一般都有线程数设置参数,来提升处理并发度,提升性能,例如:\n", + "- 在数据加载的过程中,内置数据加载类有`num_parallel_workers`参数用来设置线程数。\n", + "- 在数据增强的过程中,`map`函数有`num_parallel_workers`参数用来设置线程数。\n", + "- 在Batch的过程中,`batch`函数有`num_parallel_workers`参数用来设置线程数。\n", + "\n", + "具体内容请参考[内置加载算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html)。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 多进程优化方案" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "数据处理中Python实现的算子均支持多进程的模式,例如:\n", + "- `GeneratorDataset`这个类默认是多进程模式,它的`num_parallel_workers`参数表示的是开启的进程数,默认为1,具体内容请参考[GeneratorDataset](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html#mindspore.dataset.GeneratorDataset)。\n", + "- 如果使用Python自定义函数或者`py_transforms`模块进行数据增强的时候,当`map`函数的参数`python_multiprocessing`设置为True时,此时参数`num_parallel_workers`表示的是进程数,参数`python_multiprocessing`默认为False,此时参数`num_parallel_workers`表示的是线程数,具体的内容请参考[内置加载算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html)。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Compose优化方案" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Map算子可以接收Tensor算子列表,并将按照顺序应用所有的这些算子,与为每个Tensor算子使用的Map算子相比,此类“胖Map算子”可以获得更好的性能,如图所示:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/optimize_the_performance_of_data_preparation/images/compose.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 算子融合优化方案" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "提供某些融合算子,这些算子将两个或多个算子的功能聚合到一个算子中。具体内容请参考[数据增强算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.transforms.vision.html),与它们各自组件的流水线相比,这种融合算子提供了更好的性能。如图所示:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![title](https://gitee.com/mindspore/docs/raw/r0.7/tutorials/notebook/optimize_the_performance_of_data_preparation/images/operator_fusion.png)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.5" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/tutorials/notebook/quick_start.ipynb b/tutorials/notebook/quick_start.ipynb index e56f9cd1746269e946b4b8b5c80ff4ef3adf98e4..399e8b8c04b6f9750f9fb5d47672f65d8fbfdebd 100644 --- a/tutorials/notebook/quick_start.ipynb +++ b/tutorials/notebook/quick_start.ipynb @@ -34,7 +34,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "说明:
你可以在这里找到完整可运行的样例代码:。" + "说明:
你可以在这里找到完整可运行的样例代码:。" ] }, { @@ -470,7 +470,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\"LeNet5\"" + "\"LeNet5\"" ] }, { diff --git a/tutorials/notebook/synchronization_training_and_evaluation.ipynb b/tutorials/notebook/synchronization_training_and_evaluation.ipynb index 236ae433c882d620ead10a0247dc321bab8122d3..43a4b2ef890f6b3f0f8103d513f1c11ea67c7083 100644 --- a/tutorials/notebook/synchronization_training_and_evaluation.ipynb +++ b/tutorials/notebook/synchronization_training_and_evaluation.ipynb @@ -208,7 +208,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "实现思想:每隔n个epoch验证一次模型精度,由于在自定义函数中实现,如需了解自定义回调函数的详细用法,请参考[API说明](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.train.html?highlight=callback#mindspore.train.callback.Callback)。\n", + "实现思想:每隔n个epoch验证一次模型精度,由于在自定义函数中实现,如需了解自定义回调函数的详细用法,请参考[API说明](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.train.html?highlight=callback#mindspore.train.callback.Callback)。\n", "\n", "核心实现:回调函数的`epoch_end`内设置验证点,如下:\n", "\n", diff --git a/tutorials/source_zh_cn/advanced_use/images/cifar10_c_transforms.png b/tutorials/source_zh_cn/advanced_use/images/cifar10_c_transforms.png new file mode 100644 index 0000000000000000000000000000000000000000..10dc267dc650764566f6d20b7f090e20c12f8e11 Binary files /dev/null and b/tutorials/source_zh_cn/advanced_use/images/cifar10_c_transforms.png differ diff --git a/tutorials/source_zh_cn/advanced_use/images/compose.png b/tutorials/source_zh_cn/advanced_use/images/compose.png new file mode 100644 index 0000000000000000000000000000000000000000..a1dcbf92d4ce37bd9b794b6c04bc38b131cc3a40 Binary files /dev/null and b/tutorials/source_zh_cn/advanced_use/images/compose.png differ diff --git a/tutorials/source_zh_cn/advanced_use/images/data_enhancement_performance_scheme.png b/tutorials/source_zh_cn/advanced_use/images/data_enhancement_performance_scheme.png new file mode 100644 index 0000000000000000000000000000000000000000..a21caea16f4ee0852be47c3e56b32d184f06a7de Binary files /dev/null and b/tutorials/source_zh_cn/advanced_use/images/data_enhancement_performance_scheme.png differ diff --git a/tutorials/source_zh_cn/advanced_use/images/data_loading_performance_scheme.png b/tutorials/source_zh_cn/advanced_use/images/data_loading_performance_scheme.png new file mode 100644 index 0000000000000000000000000000000000000000..fd32feee9d720141fc1bfcf3bb03cd40363316e6 Binary files /dev/null and b/tutorials/source_zh_cn/advanced_use/images/data_loading_performance_scheme.png differ diff --git a/tutorials/source_zh_cn/advanced_use/images/operator_fusion.png b/tutorials/source_zh_cn/advanced_use/images/operator_fusion.png new file mode 100644 index 0000000000000000000000000000000000000000..bd3a88cfb04825f7469e76bcd48988a596ce222d Binary files /dev/null and b/tutorials/source_zh_cn/advanced_use/images/operator_fusion.png differ diff --git a/tutorials/source_zh_cn/advanced_use/images/pipeline.png b/tutorials/source_zh_cn/advanced_use/images/pipeline.png new file mode 100644 index 0000000000000000000000000000000000000000..5fb3f3defd20eb700c0e16d6dff5d57a1d2007c9 Binary files /dev/null and b/tutorials/source_zh_cn/advanced_use/images/pipeline.png differ diff --git a/tutorials/source_zh_cn/advanced_use/images/shuffle_performance_scheme.png b/tutorials/source_zh_cn/advanced_use/images/shuffle_performance_scheme.png new file mode 100644 index 0000000000000000000000000000000000000000..d09ca3dda379502827d58c1269599fa4381cbf76 Binary files /dev/null and b/tutorials/source_zh_cn/advanced_use/images/shuffle_performance_scheme.png differ diff --git a/tutorials/source_zh_cn/advanced_use/optimize_the_performance_of_data_preparation.md b/tutorials/source_zh_cn/advanced_use/optimize_the_performance_of_data_preparation.md new file mode 100644 index 0000000000000000000000000000000000000000..629333f4be6669904753f876bfec6becd426d621 --- /dev/null +++ b/tutorials/source_zh_cn/advanced_use/optimize_the_performance_of_data_preparation.md @@ -0,0 +1,389 @@ +# 优化数据准备的性能 + +`Ascend` `GPU` `CPU` `数据准备` `初级` `中级` `高级` + + + +- [优化数据准备的性能](#优化数据准备的性能) + - [概述](#概述) + - [整体流程](#整体流程) + - [准备环节](#准备环节) + - [导入模块](#导入模块) + - [下载所需数据集](#下载所需数据集) + - [数据加载性能优化](#数据加载性能优化) + - [性能优化方案](#性能优化方案) + - [代码示例](#代码示例) + - [shuffle性能优化](#shuffle性能优化) + - [性能优化方案](#性能优化方案-1) + - [代码示例](#代码示例-1) + - [数据增强性能优化](#数据增强性能优化) + - [性能优化方案](#性能优化方案-2) + - [代码示例](#代码示例-2) + - [性能优化方案总结](#性能优化方案总结) + - [多线程优化方案](#多线程优化方案) + - [多进程优化方案](#多进程优化方案) + - [Compose优化方案](#compose优化方案) + - [算子融合优化方案](#算子融合优化方案) + + + +   + + + +## 概述 + +数据是整个深度学习中最重要的一环,因为数据的好坏决定了最终结果的上限,模型的好坏只是去无限逼近这个上限,所以高质量的数据输入,会在整个深度神经网络中起到积极作用,数据在整个数据处理和数据增强的过程像经过pipeline管道的水一样,源源不断地流向训练系统,如图所示: + +![title](./images/pipeline.png) + +MindSpore为用户提供了数据处理以及数据增强的功能,在数据的整个pipeline过程中,其中的每一步骤,如果都能够进行合理的运用,那么数据的性能会得到很大的优化和提升。本次体验将基于CIFAR-10数据集来为大家展示如何在数据加载、数据处理和数据增强的过程中进行性能的优化。 + +## 整体流程 +- 准备环节。 +- 数据加载性能优化。 +- shuffle性能优化。 +- 数据增强性能优化。 +- 性能优化方案总结。 + +## 准备环节 + +### 导入模块 + +`dataset`模块提供API用来加载和处理数据集。 + + +```python +import mindspore.dataset as ds +``` + +`numpy`模块用于生成ndarray数组。 + + +```python +import numpy as np +``` + +### 下载所需数据集 + +1. 在当前工作目录下创建`./dataset/Cifar10Data`目录,本次体验所用的数据集存放在该目录下。 +2. 在当前工作目录下创建`./transform`目录,本次体验转换生成的数据集存放在该目录下。 +3. 下载[CIFAR-10二进制格式数据集](https://www.cs.toronto.edu/~kriz/cifar-10-binary.tar.gz),并将数据集文件解压到`./dataset/Cifar10Data/cifar-10-batches-bin`目录下,数据加载的时候使用该数据集。 +4. 下载[CIFAR-10 Python文件格式数据集](https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz),并将数据集文件解压到`./dataset/Cifar10Data/cifar-10-batches-py`目录下,数据转换的时候使用该数据集。 + +目录结构如下所示: + + + dataset/Cifar10Data + ├── cifar-10-batches-bin + │   ├── batches.meta.txt + │   ├── data_batch_1.bin + │   ├── data_batch_2.bin + │   ├── data_batch_3.bin + │   ├── data_batch_4.bin + │   ├── data_batch_5.bin + │   ├── readme.html + │   └── test_batch.bin + └── cifar-10-batches-py + ├── batches.meta + ├── data_batch_1 + ├── data_batch_2 + ├── data_batch_3 + ├── data_batch_4 + ├── data_batch_5 + ├── readme.html + └── test_batch + +其中: +- `cifar-10-batches-bin`目录为CIFAR-10二进制格式数据集目录。 +- `cifar-10-batches-py`目录为CIFAR-10 Python文件格式数据集目录。 + +## 数据加载性能优化 + +MindSpore为用户提供了多种数据加载方式,其中包括常用数据集加载、用户自定义数据集加载、MindSpore数据格式加载,详情内容请参考[加载数据集](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/loading_the_datasets.html)。对于数据集加载,底层实现方式的不同,会导致数据集加载的性能存在差异,如下所示: + +| | 常用数据集 | 用户自定义 | MindRecord | +| :----: | :----: | :----: | :----: | +| 底层实现 | C++ | Python | C++ | +| 性能 | 高 | 中 | 高| + +### 性能优化方案 + +![title](./images/data_loading_performance_scheme.png) + +数据加载性能优化建议如下: +- 已经支持的数据集格式优选内置加载算子,具体内容请参考[内置加载算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html),如果性能仍无法满足需求,则可采取多线程并发方案,请参考本文[多线程优化方案](#多线程优化方案)。 +- 不支持的数据集格式,优选转换为MindSpore数据格式后再使用`MindDataset`类进行加载,具体内容请参考[将数据集转换为MindSpore数据格式](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/converting_datasets.html),如果性能仍无法满足需求,则可采取多线程并发方案,请参考本文[多线程优化方案](#多线程优化方案)。 +- 不支持的数据集格式,算法快速验证场景,优选用户自定义`GeneratorDataset`类实现,如果性能仍无法满足需求,则可采取多进程并发方案,请参考本文[多进程优化方案](#多进程优化方案)。 + +### 代码示例 + +基于以上的数据加载性能优化建议,本次体验分别使用内置加载算子`Cifar10Dataset`类、数据转换后使用`MindDataset`类、使用`GeneratorDataset`类进行数据加载,代码演示如下: + +1. 使用内置算子`Cifar10Dataset`类加载CIFAR-10数据集,这里使用的是CIFAR-10二进制格式的数据集,加载数据时采取多线程优化方案,开启了4个线程并发完成任务,最后对数据创建了字典迭代器,并通过迭代器读取了一条数据记录。 + + + ```python + cifar10_path = "./dataset/Cifar10Data/cifar-10-batches-bin/" + + # create Cifar10Dataset for reading data + cifar10_dataset = ds.Cifar10Dataset(cifar10_path,num_parallel_workers=4) + # create a dictionary iterator and read a data record through the iterator + print(next(cifar10_dataset.create_dict_iterator())) + ``` + + 输出: + + {'image': array([[[235, 235, 235], + [230, 230, 230], + [234, 234, 234], + ..., + [248, 248, 248], + [248, 248, 248], + [249, 249, 249]], + ..., + [120, 120, 119], + [146, 146, 146], + [177, 174, 190]]], dtype=uint8), 'label': array(9, dtype=uint32)} + + +2. 使用`Cifar10ToMR`这个类将CIFAR-10数据集转换为MindSpore数据格式,这里使用的是CIFAR-10 python文件格式的数据集,然后使用`MindDataset`类加载MindSpore数据格式数据集,加载数据采取多线程优化方案,开启了4个线程并发完成任务,最后对数据创建了字典迭代器,并通过迭代器读取了一条数据记录。 + + + ```python + from mindspore.mindrecord import Cifar10ToMR + + cifar10_path = './dataset/Cifar10Data/cifar-10-batches-py/' + cifar10_mindrecord_path = './transform/cifar10.record' + + cifar10_transformer = Cifar10ToMR(cifar10_path,cifar10_mindrecord_path) + # executes transformation from Cifar10 to MindRecord + cifar10_transformer.transform(['label']) + + # create MindDataset for reading data + cifar10_mind_dataset = ds.MindDataset(dataset_file=cifar10_mindrecord_path,num_parallel_workers=4) + # create a dictionary iterator and read a data record through the iterator + print(next(cifar10_mind_dataset.create_dict_iterator())) + ``` + + 输出: + + {'data': array([255, 216, 255, ..., 63, 255, 217], dtype=uint8), 'id': array(30474, dtype=int64), 'label': array(2, dtype=int64)} + + +3. 使用`GeneratorDataset`类加载自定义数据集,并且采取多进程优化方案,开启了4个进程并发完成任务,最后对数据创建了字典迭代器,并通过迭代器读取了一条数据记录。 + + + ```python + def generator_func(num): + for i in range(num): + yield (np.array([i]),) + + # create GeneratorDataset for reading data + dataset = ds.GeneratorDataset(source=generator_func(5),column_names=["data"],num_parallel_workers=4) + # create a dictionary iterator and read a data record through the iterator + print(next(dataset.create_dict_iterator())) + ``` + + 输出: + + {'data': array([0], dtype=int64)} + + +## shuffle性能优化 + +shuffle操作主要是对有序的数据集或者进行过repeat的数据集进行混洗,MindSpore专门为用户提供了`shuffle`函数,其中设定的`buffer_size`参数越大,混洗程度越大,但时间、计算资源消耗也会大。该接口支持用户在整个pipeline的任何时候都可以对数据进行混洗,具体内容请参考[shuffle处理](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/data_processing_and_augmentation.html#shuffle)。但是因为底层的实现方式不同,该方式的性能不如直接在[内置加载算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html)中设置`shuffle`参数直接对数据进行混洗。 + +### 性能优化方案 + +![title](./images/shuffle_performance_scheme.png) + +shuffle性能优化建议如下: +- 直接使用内置加载算子的`shuffle`参数进行数据的混洗。 +- 如果使用的是`shuffle`函数,当性能仍无法满足需求,可通过调大`buffer_size`参数的值来优化提升性能。 + +### 代码示例 + +基于以上的shuffle性能优化建议,本次体验分别使用内置加载算子`Cifar10Dataset`类的`shuffle`参数和`Shuffle`函数进行数据的混洗,代码演示如下: + +1. 使用内置算子`Cifar10Dataset`类加载CIFAR-10数据集,这里使用的是CIFAR-10二进制格式的数据集,并且设置`shuffle`参数为True来进行数据混洗,最后对数据创建了字典迭代器,并通过迭代器读取了一条数据记录。 + + + ```python + cifar10_path = "./dataset/Cifar10Data/cifar-10-batches-bin/" + + # create Cifar10Dataset for reading data + cifar10_dataset = ds.Cifar10Dataset(cifar10_path,shuffle=True) + # create a dictionary iterator and read a data record through the iterator + print(next(cifar10_dataset.create_dict_iterator())) + ``` + + 输出: + + {'image': array([[[254, 254, 254], + [255, 255, 254], + [255, 255, 254], + ..., + [232, 234, 244], + [226, 230, 242], + [228, 232, 243]], + ..., + [ 64, 61, 63], + [ 63, 58, 60], + [ 61, 56, 58]]], dtype=uint8), 'label': array(9, dtype=uint32)} + + +2. 使用`shuffle`函数进行数据混洗,参数`buffer_size`设置为3,数据采用`GeneratorDataset`类自定义生成。 + + + ```python + def generator_func(): + for i in range(5): + yield (np.array([i,i+1,i+2,i+3,i+4]),) + + ds1 = ds.GeneratorDataset(source=generator_func,column_names=["data"]) + print("before shuffle:") + for data in ds1.create_dict_iterator(): + print(data["data"]) + + ds2 = ds1.shuffle(buffer_size=3) + print("after shuffle:") + for data in ds2.create_dict_iterator(): + print(data["data"]) + ``` + + 输出: + + before shuffle: + [0 1 2 3 4] + [1 2 3 4 5] + [2 3 4 5 6] + [3 4 5 6 7] + [4 5 6 7 8] + after shuffle: + [2 3 4 5 6] + [0 1 2 3 4] + [4 5 6 7 8] + [1 2 3 4 5] + [3 4 5 6 7] + + +## 数据增强性能优化 + +在图片分类的训练中,尤其是当数据集比较小的时候,用户可以使用数据增强的方式来预处理图片,从而丰富数据集。MindSpore为用户提供了多种数据增强的方式,其中包括: +- 使用内置C算子(`c_transforms`模块)进行数据增强。 +- 使用内置Python算子(`py_transforms`模块)进行数据增强。 +- 用户可根据自己的需求,自定义Python函数进行数据增强。 + +具体的内容请参考[数据增强](https://www.mindspore.cn/tutorial/zh-CN/r0.7/use/data_preparation/data_processing_and_augmentation.html#id3)。因为底层的实现方式不同,所以性能还是有一定的差异,如下所示: + +| 模块 | 底层接口 | 说明 | +| :----: | :----: | :----: | +| c_transforms | C++(基于OpenCV)| 性能高 | +| py_transforms | Python(基于PIL) | 该模块提供了多种图像增强功能,并提供了PIL Image和Numpy数组之间的传输方法 | + + +### 性能优化方案 + +![title](./images/data_enhancement_performance_scheme.png) + + +数据增强性能优化建议如下: +- 优先使用`c_transforms`模块进行数据增强,因为性能最高,如果性能仍无法满足需求,可采取[多线程优化方案](#多线程优化方案)、[Compose优化方案](#Compose优化方案)或者[算子融合优化方案](#算子融合优化方案)。 +- 如果使用了`py_transforms`模块进行数据增强,当性能仍无法满足需求,可采取[多线程优化方案](#多线程优化方案)、[多进程优化方案](#多进程优化方案)、[Compose优化方案](#Compose优化方案)或者[算子融合优化方案](#算子融合优化方案)。 +- `c_transforms`模块是在C++内维护buffer管理,`py_transforms`模块是在Python内维护buffer管理。因为Python和C++切换的性能成本,建议不要混用算子。 +- 如果用户使用了自定义Python函数进行数据增强,当性能仍无法满足需求,可采取[多线程优化方案](#多线程优化方案)或者[多进程优化方案](#多进程优化方案),如果还是无法提升性能,就需要对自定义的Python代码进行优化。 + +### 代码示例 + +基于以上的数据增强性能优化建议,本次体验分别使用`c_transforms`模块和自定义Python函数进行了数据增强,演示代码如下所示: + +1. 使用`c_transforms`模块进行数据增强,数据增强时采用多线程优化方案,开启了4个线程并发完成任务,并且采用了算子融合优化方案,使用`RandomResizedCrop`融合类替代`RandomResize`类和`RandomCrop`类。 + + + ```python + import mindspore.dataset.transforms.c_transforms as c_transforms + import mindspore.dataset.transforms.vision.c_transforms as C + import matplotlib.pyplot as plt + cifar10_path = "./dataset/Cifar10Data/cifar-10-batches-bin/" + + # create Cifar10Dataset for reading data + cifar10_dataset = ds.Cifar10Dataset(cifar10_path,num_parallel_workers=4) + transforms = C.RandomResizedCrop((800,800)) + # apply the transform to the dataset through dataset.map() + cifar10_dataset = cifar10_dataset.map(input_columns="image",operations=transforms,num_parallel_workers=4) + + data = next(cifar10_dataset.create_dict_iterator()) + plt.imshow(data["image"]) + plt.show() + ``` + + 输出: + + ![png](./images/cifar10_c_transforms.png) + + +2. 使用自定义Python函数进行数据增强,数据增强时采用多进程优化方案,开启了4个进程并发完成任务。 + + + ```python + def generator_func(): + for i in range(5): + yield (np.array([i,i+1,i+2,i+3,i+4]),) + + ds3 = ds.GeneratorDataset(source=generator_func,column_names=["data"]) + print("before map:") + for data in ds3.create_dict_iterator(): + print(data["data"]) + + func = lambda x:x**2 + ds4 = ds3.map(input_columns="data",operations=func,python_multiprocessing=True,num_parallel_workers=4) + print("after map:") + for data in ds4.create_dict_iterator(): + print(data["data"]) + ``` + + 输出: + + before map: + [0 1 2 3 4] + [1 2 3 4 5] + [2 3 4 5 6] + [3 4 5 6 7] + [4 5 6 7 8] + after map: + [ 0 1 4 9 16] + [ 1 4 9 16 25] + [ 4 9 16 25 36] + [ 9 16 25 36 49] + [16 25 36 49 64] + + +## 性能优化方案总结 + +### 多线程优化方案 + +在数据pipeline过程中,相关算子一般都有线程数设置参数,来提升处理并发度,提升性能,例如: +- 在数据加载的过程中,内置数据加载类有`num_parallel_workers`参数用来设置线程数。 +- 在数据增强的过程中,`map`函数有`num_parallel_workers`参数用来设置线程数。 +- 在Batch的过程中,`batch`函数有`num_parallel_workers`参数用来设置线程数。 + +具体内容请参考[内置加载算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html)。 + +### 多进程优化方案 + +数据处理中Python实现的算子均支持多进程的模式,例如: +- `GeneratorDataset`这个类默认是多进程模式,它的`num_parallel_workers`参数表示的是开启的进程数,默认为1,具体内容请参考[GeneratorDataset](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html#mindspore.dataset.GeneratorDataset)。 +- 如果使用Python自定义函数或者`py_transforms`模块进行数据增强的时候,当`map`函数的参数`python_multiprocessing`设置为True时,此时参数`num_parallel_workers`表示的是进程数,参数`python_multiprocessing`默认为False,此时参数`num_parallel_workers`表示的是线程数,具体的内容请参考[内置加载算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.html)。 + +### Compose优化方案 + +Map算子可以接收Tensor算子列表,并将按照顺序应用所有的这些算子,与为每个Tensor算子使用的Map算子相比,此类“胖Map算子”可以获得更好的性能,如图所示: + +![title](./images/compose.png) + +### 算子融合优化方案 + +提供某些融合算子,这些算子将两个或多个算子的功能聚合到一个算子中。具体内容请参考[数据增强算子](https://www.mindspore.cn/api/zh-CN/r0.7/api/python/mindspore/mindspore.dataset.transforms.vision.html),与它们各自组件的流水线相比,这种融合算子提供了更好的性能。如图所示: + +![title](./images/operator_fusion.png) diff --git a/tutorials/source_zh_cn/index.rst b/tutorials/source_zh_cn/index.rst index c1551225334e0208d3589dd2bfec89b39b26cbaa..b02b7e90950818c9ab8f91397dd30ab0e347e645 100644 --- a/tutorials/source_zh_cn/index.rst +++ b/tutorials/source_zh_cn/index.rst @@ -55,6 +55,7 @@ MindSpore教程 advanced_use/graph_kernel_fusion advanced_use/quantization_aware advanced_use/gradient_accumulation + advanced_use/optimize_the_performance_of_data_preparation .. toctree:: :glob: