diff --git "a/notebook/notebook_ch/4.ppocr_system_strategy/PP-OCR\347\263\273\347\273\237\345\217\212\344\274\230\345\214\226\347\255\226\347\225\245.ipynb" "b/notebook/notebook_ch/4.ppocr_system_strategy/PP-OCR\347\263\273\347\273\237\345\217\212\344\274\230\345\214\226\347\255\226\347\225\245.ipynb"
new file mode 100644
index 0000000000000000000000000000000000000000..4160b077b3ef2939ffd2ac88867bb12c93438c61
--- /dev/null
+++ "b/notebook/notebook_ch/4.ppocr_system_strategy/PP-OCR\347\263\273\347\273\237\345\217\212\344\274\230\345\214\226\347\255\226\347\225\245.ipynb"
@@ -0,0 +1,3491 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "# 1. PP-OCR系统简介与总览\n",
+ "\n",
+ "前两章主要介绍了DBNet文字检测算法以及CRNN文字识别算法。然而对于我们实际场景中的一张图像,想要单独基于文字检测或者识别模型,是无法同时获取文字位置与文字内容的,因此,我们将文字检测算法以及文字识别算法进行串联,构建了PP-OCR文字检测与识别系统。在实际使用过程中,检测出的文字方向可能不是我们期望的方向,最终导致文字识别错误,因此我们在PP-OCR系统中也引入了方向分类器。\n",
+ "\n",
+ "本章主要介绍PP-OCR文字检测与识别系统以及该系统中涉及到的优化策略。通过本节课的学习,您可以获得:\n",
+ "\n",
+ "* PaddleOCR策略调优技巧\n",
+ "* 文本检测、识别、方向分类器模型的优化技巧和优化方法\n",
+ "\n",
+ "PP-OCR系统共经历了2次优化,下面对PP-OCR系统和这2次优化进行简单介绍。\n",
+ "\n",
+ "## 1.1 PP-OCR系统与优化策略简介\n",
+ "\n",
+ "PP-OCR中,对于一张图像,如果希望提取其中的文字信息,需要完成以下几个步骤:\n",
+ "\n",
+ "* 使用文本检测的方法,获取文本区域多边形信息(PP-OCR中文本检测使用的是DBNet,因此获取的是四点信息)。\n",
+ "* 对上述文本多边形区域进行裁剪与透视变换校正,将文本区域转化成矩形框,再使用方向分类器对方向进行校正。\n",
+ "* 基于包含文字区域的矩形框进行文本识别,得到最终识别结果。\n",
+ "\n",
+ "上面便完成了对于一张图像的文本检测与识别过程。\n",
+ "\n",
+ "PP-OCR的系统框图如下所示。\n",
+ "\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
PP-OCR系统框图\n",
+ "\n",
+ "文本检测基于后处理方案比较简单的DBNet,文字区域校正主要使用几何变换以及方向分类器,文本识别使用了基于融合了卷积特征与序列特征的CRNN模型,使用CTC loss解决预测结果与标签不一致的问题。\n",
+ "\n",
+ "PP-OCR从骨干网络、学习率策略、数据增广、模型裁剪量化等方面,共使用了19个策略,对模型进行优化瘦身,最终打造了面向服务器端的PP-OCR server系统以及面向移动端的PP-OCR mobile系统。\n",
+ "\n",
+ "## 1.2 PP-OCRv2系统与优化策略简介\n",
+ "\n",
+ "相比于PP-OCR, PP-OCRv2 在骨干网络、数据增广、损失函数这三个方面进行进一步优化,解决端侧预测效率较差、背景复杂以及相似字符的误识等问题,同时引入了知识蒸馏训练策略,进一步提升模型精度。具体地:\n",
+ "\n",
+ "* 检测模型优化: (1) 采用 CML 协同互学习知识蒸馏策略;(2) CopyPaste 数据增广策略;\n",
+ "* 识别模型优化: (1) PP-LCNet 轻量级骨干网络;(2) U-DML 改进知识蒸馏策略; (3) Enhanced CTC loss 损失函数改进。\n",
+ "\n",
+ "从效果上看,主要有三个方面提升:\n",
+ "\n",
+ "* 在模型效果上,相对于 PP-OCR mobile 版本提升超7%;\n",
+ "* 在速度上,相对于 PP-OCR server 版本提升超过220%;\n",
+ "* 在模型大小上,11.6M 的总大小,服务器端和移动端都可以轻松部署。\n",
+ "\n",
+ "PP-OCRv2 模型与之前 PP-OCR 系列模型的精度、预测耗时、模型大小对比图如下所示。\n",
+ "\n",
+ "\n",
+ " \n",
+ "
\n",
+ "
PP-OCRv2与PP-OCR的速度、精度、模型大小对比\n",
+ "\n",
+ "PP-OCRv2的系统框图如下所示。\n",
+ "\n",
+ "\n",
+ " \n",
+ "
\n",
+ "
PP-OCRv2系统框图\n",
+ " \n",
+ "\n",
+ "本章将对上述PP-OCR以及PP-OCRv2系统优化策略进行详细的解读。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "# 2. PP-OCR 优化策略\n",
+ "\n",
+ "PP-OCR系统包括文本检测器、方向分类器以及文本识别器。本节针对这三个方向的模型优化策略进行详细介绍。\n",
+ "\n",
+ "## 2.1 文本检测\n",
+ "\n",
+ "PP-OCR中的文本检测基于DBNet (Differentiable Binarization)模型,它基于分割方案,后处理简单。DBNet的具体模型结构如下图。\n",
+ "\n",
+ "\n",
+ "
\n",
+ "
\n",
+ "DBNet框图\n",
+ "\n",
+ "DBNet通过骨干网络(backbone)提取特征,使用DBFPN的结构(neck)对各阶段的特征进行融合,得到融合后的特征。融合后的特征经过卷积等操作(head)进行解码,生成概率图和阈值图,二者融合后计算得到一个近似的二值图。计算损失函数时,对这三个特征图均计算损失函数,这里把二值化的监督也也加入训练过程,从而让模型学习到更准确的边界。\n",
+ "\n",
+ "DBNet中使用了6种优化策略用于提升模型精度与速度,包括骨干网络、特征金字塔网络、头部结构、学习率策略、模型裁剪等策略。在验证集上,不同模块的消融实验结论如下所示。\n",
+ "\n",
+ "\n",
+ "
\n",
+ "
\n",
+ "DBNet消融实验\n",
+ "\n",
+ "\n",
+ "下面进行详细说明。\n",
+ "\n",
+ "### 2.1.1 轻量级骨干网络\n",
+ "\n",
+ "骨干网络的大小对文本检测器的模型大小有重要影响。因此,在构建超轻量检测模型时,应选择轻量的骨干网络。随着图像分类技术的发展,MobileNetV1、MobileNetV2、MobileNetV3和ShuffleNetV2系列常用作轻量骨干网络。每个系列都有不同的模型大小和性能表现。[PaddeClas](https://github.com/PaddlePaddle/PaddleClas)提供了20多种轻量级骨干网络。他们在ARM上的`精度-速度`曲线如下图所示。\n",
+ "\n",
+ "\n",
+ "
\n",
+ "
\n",
+ "PaddleClas中骨干网络的\"速度-精度\"曲线\n",
+ "\n",
+ "在预测时间相同的情况下,MobileNetV3系列可以实现更高的精度。作者在设计的时候为了覆盖尽可能多的场景,使用scale这个参数来调整特征图通道数,标准为1x,如果是0.5x,则表示该网络中部分特征图通道数为1x对应网络的0.5倍。为了进一步平衡准确率和效率,在V3的尺寸选择上,我们采用了MobileNetV3_large 0.5x的结构。\n",
+ "\n",
+ "下面打印出DBNet中MobileNetV3各个阶段的特征图尺寸。\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "fatal: destination path 'PaddleOCR' already exists and is not an empty directory.\n",
+ "Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple\n",
+ "Requirement already satisfied: pip in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (21.3.1)\n",
+ "Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple\n",
+ "Requirement already satisfied: shapely in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 1)) (1.8.0)\n",
+ "Requirement already satisfied: scikit-image in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 2)) (0.19.1)\n",
+ "Requirement already satisfied: imgaug==0.4.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 3)) (0.4.0)\n",
+ "Requirement already satisfied: pyclipper in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 4)) (1.3.0.post2)\n",
+ "Requirement already satisfied: lmdb in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 5)) (1.2.1)\n",
+ "Requirement already satisfied: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 6)) (4.27.0)\n",
+ "Requirement already satisfied: numpy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 7)) (1.20.3)\n",
+ "Requirement already satisfied: visualdl in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 8)) (2.2.0)\n",
+ "Requirement already satisfied: python-Levenshtein in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 9)) (0.12.2)\n",
+ "Requirement already satisfied: opencv-contrib-python==4.4.0.46 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 10)) (4.4.0.46)\n",
+ "Requirement already satisfied: cython in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 11)) (0.29)\n",
+ "Requirement already satisfied: lxml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 12)) (4.7.1)\n",
+ "Requirement already satisfied: premailer in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 13)) (3.10.0)\n",
+ "Requirement already satisfied: openpyxl in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 14)) (3.0.5)\n",
+ "Requirement already satisfied: fasttext==0.9.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 15)) (0.9.1)\n",
+ "Requirement already satisfied: imageio in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (2.6.1)\n",
+ "Requirement already satisfied: scipy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (1.6.3)\n",
+ "Requirement already satisfied: Pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (7.1.2)\n",
+ "Requirement already satisfied: opencv-python in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (4.1.1.26)\n",
+ "Requirement already satisfied: six in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (1.15.0)\n",
+ "Requirement already satisfied: matplotlib in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (2.2.3)\n",
+ "Requirement already satisfied: pybind11>=2.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from fasttext==0.9.1->-r requirements.txt (line 15)) (2.8.1)\n",
+ "Requirement already satisfied: setuptools>=0.7.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from fasttext==0.9.1->-r requirements.txt (line 15)) (56.2.0)\n",
+ "Requirement already satisfied: packaging>=20.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->-r requirements.txt (line 2)) (20.9)\n",
+ "Requirement already satisfied: networkx>=2.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->-r requirements.txt (line 2)) (2.4)\n",
+ "Requirement already satisfied: tifffile>=2019.7.26 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->-r requirements.txt (line 2)) (2021.11.2)\n",
+ "Requirement already satisfied: PyWavelets>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->-r requirements.txt (line 2)) (1.2.0)\n",
+ "Requirement already satisfied: protobuf>=3.11.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (3.14.0)\n",
+ "Requirement already satisfied: flake8>=3.7.9 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (3.8.2)\n",
+ "Requirement already satisfied: Flask-Babel>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (1.0.0)\n",
+ "Requirement already satisfied: pandas in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (1.1.5)\n",
+ "Requirement already satisfied: pre-commit in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (1.21.0)\n",
+ "Requirement already satisfied: shellcheck-py in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (0.7.1.1)\n",
+ "Requirement already satisfied: requests in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (2.22.0)\n",
+ "Requirement already satisfied: flask>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (1.1.1)\n",
+ "Requirement already satisfied: bce-python-sdk in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (0.8.53)\n",
+ "Requirement already satisfied: cssutils in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from premailer->-r requirements.txt (line 13)) (2.3.0)\n",
+ "Requirement already satisfied: cachetools in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from premailer->-r requirements.txt (line 13)) (4.0.0)\n",
+ "Requirement already satisfied: cssselect in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from premailer->-r requirements.txt (line 13)) (1.1.0)\n",
+ "Requirement already satisfied: et-xmlfile in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from openpyxl->-r requirements.txt (line 14)) (1.0.1)\n",
+ "Requirement already satisfied: jdcal in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from openpyxl->-r requirements.txt (line 14)) (1.4.1)\n",
+ "Requirement already satisfied: pycodestyle<2.7.0,>=2.6.0a1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->-r requirements.txt (line 8)) (2.6.0)\n",
+ "Requirement already satisfied: importlib-metadata in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->-r requirements.txt (line 8)) (0.23)\n",
+ "Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->-r requirements.txt (line 8)) (0.6.1)\n",
+ "Requirement already satisfied: pyflakes<2.3.0,>=2.2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->-r requirements.txt (line 8)) (2.2.0)\n",
+ "Requirement already satisfied: Werkzeug>=0.15 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->-r requirements.txt (line 8)) (0.16.0)\n",
+ "Requirement already satisfied: click>=5.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->-r requirements.txt (line 8)) (7.0)\n",
+ "Requirement already satisfied: Jinja2>=2.10.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->-r requirements.txt (line 8)) (2.11.0)\n",
+ "Requirement already satisfied: itsdangerous>=0.24 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->-r requirements.txt (line 8)) (1.1.0)\n",
+ "Requirement already satisfied: Babel>=2.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask-Babel>=1.0.0->visualdl->-r requirements.txt (line 8)) (2.8.0)\n",
+ "Requirement already satisfied: pytz in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask-Babel>=1.0.0->visualdl->-r requirements.txt (line 8)) (2019.3)\n",
+ "Requirement already satisfied: decorator>=4.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from networkx>=2.2->scikit-image->-r requirements.txt (line 2)) (4.4.2)\n",
+ "Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from packaging>=20.0->scikit-image->-r requirements.txt (line 2)) (2.4.2)\n",
+ "Requirement already satisfied: future>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from bce-python-sdk->visualdl->-r requirements.txt (line 8)) (0.18.0)\n",
+ "Requirement already satisfied: pycryptodome>=3.8.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from bce-python-sdk->visualdl->-r requirements.txt (line 8)) (3.9.9)\n",
+ "Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->imgaug==0.4.0->-r requirements.txt (line 3)) (1.1.0)\n",
+ "Requirement already satisfied: cycler>=0.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->imgaug==0.4.0->-r requirements.txt (line 3)) (0.10.0)\n",
+ "Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->imgaug==0.4.0->-r requirements.txt (line 3)) (2.8.0)\n",
+ "Requirement already satisfied: aspy.yaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (1.3.0)\n",
+ "Requirement already satisfied: cfgv>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (2.0.1)\n",
+ "Requirement already satisfied: pyyaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (5.1.2)\n",
+ "Requirement already satisfied: toml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (0.10.0)\n",
+ "Requirement already satisfied: nodeenv>=0.11.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (1.3.4)\n",
+ "Requirement already satisfied: virtualenv>=15.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (16.7.9)\n",
+ "Requirement already satisfied: identify>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (1.4.10)\n",
+ "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl->-r requirements.txt (line 8)) (1.25.6)\n",
+ "Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl->-r requirements.txt (line 8)) (2.8)\n",
+ "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl->-r requirements.txt (line 8)) (3.0.4)\n",
+ "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl->-r requirements.txt (line 8)) (2019.9.11)\n",
+ "Requirement already satisfied: MarkupSafe>=0.23 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Jinja2>=2.10.1->flask>=1.1.1->visualdl->-r requirements.txt (line 8)) (1.1.1)\n",
+ "Requirement already satisfied: zipp>=0.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from importlib-metadata->flake8>=3.7.9->visualdl->-r requirements.txt (line 8)) (3.6.0)\n"
+ ]
+ }
+ ],
+ "source": [
+ "import os\n",
+ "import sys\n",
+ "\n",
+ "# 下载代码\n",
+ "os.chdir(\"/home/aistudio/\")\n",
+ "!git clone https://gitee.com/paddlepaddle/PaddleOCR.git\n",
+ "# 切换工作目录\n",
+ "os.chdir(\"/home/aistudio/PaddleOCR/\")\n",
+ "!pip install -U pip\n",
+ "!pip install -r requirements.txt"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "the shape of 0 stage: [1, 16, 160, 160]\n",
+ "the shape of 1 stage: [1, 24, 80, 80]\n",
+ "the shape of 2 stage: [1, 56, 40, 40]\n",
+ "the shape of 3 stage: [1, 480, 20, 20]\n"
+ ]
+ }
+ ],
+ "source": [
+ "# 具体代码实现位于:\n",
+ "# https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/ppocr/modeling/backbones/det_mobilenet_v3.py\n",
+ "import numpy as np\n",
+ "import paddle\n",
+ "\n",
+ "# 设置随机输入\n",
+ "inputs = np.random.rand(1, 3, 640, 640).astype(np.float32)\n",
+ "x = paddle.to_tensor(inputs)\n",
+ "\n",
+ "# 导入MobileNetV3库\n",
+ "from ppocr.modeling.backbones.det_mobilenet_v3 import MobileNetV3\n",
+ "\n",
+ "# 模型定义\n",
+ "backbone_mv3 = MobileNetV3(scale=0.5, model_name='large')\n",
+ "\n",
+ "# 模型forward\n",
+ "bk_out = backbone_mv3(x)\n",
+ "\n",
+ "# 模型中间层打印\n",
+ "for i, stage_out in enumerate(bk_out):\n",
+ " print(\"the shape of \",i,'stage: ',stage_out.shape)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 2.1.2 轻量级特征金字塔网络DBFPN结构\n",
+ "\n",
+ "文本检测器的特征融合(neck)部分DBFPN与目标检测任务中的FPN结构类似,融合不同尺度的特征图,以提升不同尺度的文本区域检测效果。\n",
+ "\n",
+ "为了方便合并不同通道的特征图,这里使用`1×1`的卷积将特征图减少到相同数量的通道。\n",
+ "\n",
+ "概率图和阈值图是由卷积融合的特征图生成的,卷积也与inner_channels相关联。因此,inner_channels对模型尺寸有很大的影响。当inner_channels由256减小到96时,模型尺寸由7M减小到4.1M,速度提升48%,但精度只是略有下降。\n",
+ "\n",
+ "下面打印DBFPN的结构以及对于骨干网络特征图的融合结果。"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "DBFPN(\n",
+ " (in2_conv): Conv2D(16, 96, kernel_size=[1, 1], data_format=NCHW)\n",
+ " (in3_conv): Conv2D(24, 96, kernel_size=[1, 1], data_format=NCHW)\n",
+ " (in4_conv): Conv2D(56, 96, kernel_size=[1, 1], data_format=NCHW)\n",
+ " (in5_conv): Conv2D(480, 96, kernel_size=[1, 1], data_format=NCHW)\n",
+ " (p5_conv): Conv2D(96, 24, kernel_size=[3, 3], padding=1, data_format=NCHW)\n",
+ " (p4_conv): Conv2D(96, 24, kernel_size=[3, 3], padding=1, data_format=NCHW)\n",
+ " (p3_conv): Conv2D(96, 24, kernel_size=[3, 3], padding=1, data_format=NCHW)\n",
+ " (p2_conv): Conv2D(96, 24, kernel_size=[3, 3], padding=1, data_format=NCHW)\n",
+ ")\n",
+ "the shape of output of DBFPN: [1, 96, 160, 160]\n"
+ ]
+ }
+ ],
+ "source": [
+ "# 具体代码实现位于:\n",
+ "# https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/ppocr/modeling/necks/db_fpn.py\n",
+ "from ppocr.modeling.necks.db_fpn import DBFPN\n",
+ "\n",
+ "neck_bdfpn = DBFPN(in_channels=[16, 24, 56, 480], out_channels=96)\n",
+ "# 打印 DBFPN结构\n",
+ "print(neck_bdfpn)\n",
+ "\n",
+ "# 先对原始的通道数降到96,再降到24,最后4个feature map进行concat\n",
+ "fpn_out = neck_bdfpn(bk_out)\n",
+ "\n",
+ "print('the shape of output of DBFPN: ', fpn_out.shape)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 2.1.3 骨干网络中SE模块分析\n",
+ "\n",
+ "SE是`squeeze-and-excitation`的缩写(Hu, Shen, and Sun 2018)。如图所示\n",
+ "\n",
+ "\n",
+ "
\n",
+ "
\n",
+ "SE模块示意图\n",
+ "\n",
+ "SE块显式地建模通道之间的相互依赖关系,并自适应地重新校准通道特征响应。在网络中使用SE块可以明显提高视觉任务的准确性,因此MobileNetV3的搜索空间包含了SE模块,最终MobileNetV3中也包含很多个SE模块。然而,当输入分辨率较大时,例如`640×640`,使用SE模块较难估计通道的特征响应,精度提高有限,但SE模块的时间成本非常高。在DBNet中,**我们将SE模块从骨干网络中移除**,模型大小从`4.1M`降到`2.6M`,但精度没有影响。\n",
+ "\n",
+ "PaddleOCR中可以通过设置`disable_se=True`来移除骨干网络中的SE模块,使用方法如下所示。"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "the shape of 0 stage: [1, 16, 160, 160]\n",
+ "the shape of 1 stage: [1, 24, 80, 80]\n",
+ "the shape of 2 stage: [1, 56, 40, 40]\n",
+ "the shape of 3 stage: [1, 480, 20, 20]\n"
+ ]
+ }
+ ],
+ "source": [
+ "# 具体代码实现位于:\n",
+ "# https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/ppocr/modeling/backbones/det_mobilenet_v3.py\n",
+ "\n",
+ "x = paddle.rand([1, 3, 640, 640])\n",
+ "\n",
+ "from ppocr.modeling.backbones.det_mobilenet_v3 import MobileNetV3\n",
+ "\n",
+ "# 定义模型\n",
+ "backbone_mv3 = MobileNetV3(scale=0.5, model_name='large', disable_se=True)\n",
+ "\n",
+ "# 模型forward\n",
+ "bk_out = backbone_mv3(x)\n",
+ "# 输出\n",
+ "for i, stage_out in enumerate(bk_out):\n",
+ " print(\"the shape of \",i,'stage: ',stage_out.shape)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 2.1.4 学习率策略优化\n",
+ "\n",
+ "* Cosine 学习率下降策略\n",
+ "\n",
+ "梯度下降算法需要我们设置一个值,用来控制权重更新幅度,我们将其称之为学习率。它是控制模型学习速度的超参数。学习率越小,loss的变化越慢。虽然使用较低的学习速率可以确保不会错过任何局部极小值,但这也意味着模型收敛速度较慢。\n",
+ "\n",
+ "因此,在训练前期,权重处于随机初始化状态,我们可以设置一个相对较大的学习速率以加快收敛速度。在训练后期,权重接近最优值,使用相对较小的学习率可以防止模型在收敛的过程中发生震荡。\n",
+ "\n",
+ "Cosine学习率策略也就应运而生,Cosine学习率策略指的是学习率在训练的过程中,按照余弦的曲线变化。在整个训练过程中,Cosine学习率衰减策略使得在网络在训练初期保持了较大的学习速率,在后期学习率会逐渐衰减至0,其收敛速度相对较慢,但最终收敛精度较好。下图比较了两种不同的学习率衰减策略`piecewise decay`和`cosine decay`。\n",
+ "\n",
+ "\n",
+ "
\n",
+ "
\n",
+ "Cosine与Piecewise学习率下降策略\n",
+ "\n",
+ "* 学习率预热策略\n",
+ "\n",
+ "模型刚开始训练时,模型权重是随机初始化的,此时若选择一个较大的学习率,可能造成模型训练不稳定的问题,因此**学习率预热**的概念被提出,用于解决模型训练初期不收敛的问题。\n",
+ "\n",
+ "学习率预热指的是将学习率从一个很小的值开始,逐步增加到初始较大的学习率。它可以保证模型在训练初期的稳定性。使用学习率预热策略有助于提高图像分类任务的准确性。在DBNet中,实验表明该策略也是有效的。学习率预热策略与Cosine学习率结合时,学习率的变化趋势如下代码演示。\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4IAAAGDCAYAAAB+yq7tAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3XeY1NX5v/H70AWJImADBERBQbAhVgSsEKNogt+o0WjsiSVGjRJNbLHFaDSxJDGW2BI1Go297lAERUClKQhiAYwGETtIO78/zvBjQwAX2NnPlPt1XXPN7sxndp9dUPa955znCTFGJEmSJEmVo17WBUiSJEmS6pZBUJIkSZIqjEFQkiRJkiqMQVCSJEmSKoxBUJIkSZIqjEFQkiRJkiqMQVCStEIhhEkhhL4F+Lh/DSFcWtsfd7nPcVEI4e5a+lh9Qwgza/va2hRCeDKEcPQqni/491ySVFoMgpJUBkIIR4QQxoQQvggh/DsfDPZYm48ZY+wWYxxSSyWqgGKMA2KMdwCEEI4JIbywph8rhPB0COHcau+3CSHElTy28dpVLknKikFQkkpcCOFM4DrgcmAjYDPgJmBglnWtiRBCg6xryEoRfe3DgD2rvb8nMHkFj02NMX6wOh84JP7sIUlFwP8ZS1IJCyGsB1wCnBJj/GeM8csY48IY46Mxxp/nr2kcQrguhPB+/nZdCKFx/rlWIYTHQgifhBA+DiEMX/qDegjhnRDCPvm3Lwoh3B9CuDOE8Hl+22jPanVsGkJ4MIQwO4Twdgjh9BrW3zeEMDOEcG4I4QPg9hVcs0UIYWgI4dMQwkchhPuqPdcthPBsvvYPQwjnVXtpozWpN4SwTn4r5dwQwuvATsvVE0MIW1R7f6XbLr/h81wUQngghHB3COEz4JjlXtsx/+ey9M/jLyGE/1R7/q4Qwhn5t4eEEI4PIWwN/AnYNb86/Em1D9kihPB4/vsxKoTQaUU1k4Lg7tUCW2/SLxp6LvfYsPznbpH/OzQ7/z17LITQtlqdQ0IIl4UQRgBfAZvnH7s0hDAyX+ejIYSWIYR7QgifhRBGhxA65F/fIf89b7Dcxzw+//YxIYQRIYQb8n9HJocQ9l7J1yZJyjMISlJp2xVoAjy0imvOB3YBtgO2BXoBv8w/dxYwE2hNWk08D4gr+TgHAfcC6wOPADcA5MPBo8A4oA2wN3BGCGH/Gn4NGwMbAO2BE1fw/K+BZ4AWQFvg+vznbQ48BzwFbApsATxfC/VeCHTK3/YHVnr2blVq+H0ZCDyQr/Ge6q+PMb4NfAZsn39oT+CLfNgD6AMMXe41bwAnAy/GGNeNMa5f7enDgItJ38dpwGUrKf1loDHp78rSz/ts/jXVHxuWf7seKcC3J61GzyP/va7mKNKfbXPg3Wr1HEX63nQCXsx/nA2AN0h/DjW1M/AW0Cr/un+GEDZYjddLUsUxCEpSaWsJfBRjXLSKa34AXBJj/E+McTYpDByVf24hsAnQPr+SODzGuLIg+EKM8YkY42LgLpaFgp2A1jHGS2KMC2KM04G/kH7Qr4klwIUxxq9jjPNW8PxCUsjYNMY4P8a49Pzbd4APYozX5B//PMY4qhbq/T/gshjjxzHGGcAfavh1LK8m35cXY4wPxxiXrORrHwr0CcvO4j2Qf78j8C1SyKyph2KML+f/rtxD+sXA/4gxfg2MAvbMh6n18rUPr/ZY13xtxBjnxBgfjDF+FWP8nBQw+yz3Yf8aY5wUY1wUY1yYf+z2GONbMcZPgSeBt2KMz+Xr+wfLAnBN/Ae4Lv93+D5gCnDAarxekipOsZxHkCStmTlAqxBCg1WEwU1ZtgpD/u1N82//FrgIeCaEAHBzjPHKlXyc6ufBvgKa5LfrtQc2XW4bYn1ScKiJ2THG+at4/hzSquDLIYS5wDUxxtuAdqRVoJVZ03o3BWZUe67692511OT7MoNVG0pa2ZxJWoEbQgrx84HhMcYlq1HP8t+PdVdx7dJzgu8AI/KPvQD8KP/YjBjjuwAhhKbAtUB/0mojQPMQQv18CIcVf50fVnt73greX1V9y5u13C8wqv8dlyStgCuCklTaXgS+Bg5exTXvk0LJUpvlHyO/inZWjHFzUuA4cw3OV80A3o4xrl/t1jzG+O0avn5lK5Dka/wgxnhCjHFT4CTgpvwZvRnA5qtZa03q/TcpZC612XKv/wpoWu39lXXOrMn3ZZVfOykI9gb65t9+AdidFWwLXY2PWRPD8p93T5YF1xH5z119Wyik7cVdgJ1jjN9iWVOZUEs1fZm/X9X3vE3I/yYj7///HZckrZhBUJJKWH5b3QXAjSGEg0MITUMIDUMIA0IIV+Uv+zvwyxBC6xBCq/z1dwOEEL6Tb8YSgE+BxaStmqvjZeDzfMOXdUII9UMI24QQdvrGV9ZACOHQas1H5pJCxRLgMWCTEMIZITXEaR5C2LkW6r0f+EW+CUpb4LTlXv8acET+df35322QNf083yjGOJW0OnYkMDTG+Blp5ex7rDwIfgi0DSE0qunnWYEXSecWjyQfBGOMc4HZ+ceqB8Hm+Ro/yW8bXZ2zfd8ov515FnBk/nt4LOlMYXUbAqfn/+4fCmwNPFGbdUhSuTEISlKJizFeA5xJagAzm7QSdSrwcP6SS4ExwHhgAvBK/jGALUkNV74g/fB/U4wxt5qffzHpvN52wNvAR8AtwHpr/EX9t52AUSGEL0hNX34aY5yeP4+2L3AgadvjVKBfLdR7MWlr4dukJjV3Lfchfpr/nJ+Qzl8+zArU4vdlKDAnf15x6fuB9Oe4IlXAJOCDEMJHq/m5AIgxfgmMBRoBE6s9NZwUuqoHweuAdUhf30uk5j217QTg56St0N2Akcs9P4r0d/kj0hnFQTHGOQWoQ5LKRlh5TwBJkqTiFkI4Bjg+xrhH1rVIUilxRVCSJEmSKoxBUJIkSZIqjFtDJUmSJKnCuCIoSZIkSRXGIChJkiRJFaZB1gXUllatWsUOHTpkXYYkSZIkZWLs2LEfxRhb1+TasgmCHTp0YMyYMVmXIUmSJEmZCCG8W9Nr3RoqSZIkSRXGIChJkiRJFcYgKEmSJEkVpmzOCEqSJEnS2li4cCEzZ85k/vz5WZeySk2aNKFt27Y0bNhwjT+GQVCSJEmSgJkzZ9K8eXM6dOhACCHrclYoxsicOXOYOXMmHTt2XOOP49ZQSZIkSQLmz59Py5YtizYEAoQQaNmy5VqvWhoEJUmSJCmvmEPgUrVRo0FQkiRJkorEuuuuWyefxyAoSZIkSUVs0aJFtf4xCxoEQwj9QwhTQgjTQgiDV/D8niGEV0IIi0IIg5Z77ugQwtT87ehC1ilJkiRJxWTIkCH07t2bgw46iK5du9b6xy9Y19AQQn3gRmBfYCYwOoTwSIzx9WqXvQccA5y93Gs3AC4EegIRGJt/7dxC1StJkiRJ/98ZZ8Brr9Xux9xuO7juuhpf/sorrzBx4sS16g66MoUcH9ELmBZjnA4QQrgXGAj8/yAYY3wn/9yS5V67P/BsjPHj/PPPAv2BvxewXmXt44/h5ZehXj1o0ADq11/xrUEDaNIE1lkn3Zo2hcaNoQQO9kqSJEk11atXr4KEQChsEGwDzKj2/kxg57V4bZvlLwohnAicCLDZZputWZUqHqecAvfeu2avDWFZOGzaNN03awbrrQfrr7/i+6Vvt2oFG26Y7hs1qt2vSZIkSaVpNVbuCqVZs2YF+9glPVA+xngzcDNAz549Y8blaG0sWQLPPgsHHQTnnAOLFsHixSu+LVoE8+fDvHnw1Vfpfumt+vtffAGffgrTp6f7Tz+Fzz6DuIq/Kuuvn0Jh69b/fb/RRtC2LbRpk+433DCtTkqSJEklqJBBcBbQrtr7bfOP1fS1fZd77ZBaqUrFaeJEmDMHvvtd2H33wn2eJUvg889TKPzkk3T76COYPRv+8590W/r21KkwYkR6fslyu5fr14dNN/3vcNi2LbRvD5tvnm7rr1+4r0OSJElaC4UMgqOBLUMIHUnB7jDgiBq+9mng8hBCi/z7+wG/qP0SVTRyuXTfr19hP0+9esu2hdZ0O/HixSkMzpoFM2cuu196mzABnngirUZW16JFCoQdOy4Lh5tvDp06pcDoiqIkSZKW88UXXwDQt29f+vbtW7DPU7AgGGNcFEI4lRTq6gO3xRgnhRAuAcbEGB8JIewEPAS0AA4MIVwcY+wWY/w4hPBrUpgEuGRp4xiVqaqqFJCK8axn/fppa+hGG8EOO6z4mhjT6uI778Dbb6ftqNOnp7fHj4dHHoEFC5Zd37gxbLklbLVVunXpsuy+efM6+bIkSZJUuQp6RjDG+ATwxHKPXVDt7dGkbZ8reu1twG2FrE9FYvFiGDoUDj0060rWXAhpBbBFC9h++/99fskSeP/9FA6nToUpU2DyZBg3Dh56KH0PlmrTJgXCbbaB7t2hRw/o1i01v5EkSZJqQUk3i1GZePXVdGav0NtCs1Sv3rJzhHvu+d/PLVgAb72VguHkySkkvvEG3HorfPlluiaEtGLao0e6LQ2Im2+ePrYkSZK0GgyCyl5dnQ8sVo0awdZbp1t1S5Ys21o6fnw6izh+fFpBXNr5tFmztF21Z89023HHtOXUcChJkrRGYoyEIp9PHVfVBb+GDILKXi6XzsdtsknWlRSXevXSKmCnTnDIIcse//JLeP31FApffRXGjoU//jGN1IB0xnDHHZcFw54908co8v+hSZIkZa1JkybMmTOHli1bFm0YjDEyZ84cmjRpslYfxyCobC1cCMOGwdFHZ11J6WjWDHbaKd2WWrQohcOxY2HMmHS7/nr4+uv0fIsWsOuu6bbbbtCrF6y7bjb1S5IkFam2bdsyc+ZMZs+enXUpq9SkSRPatl1hq5UaMwgqW2PGpBWuSt0WWlsaNFh2fvBHP0qPLVwIkyal7/HLL8PIkWnMBaTVxh49UijcbbcUEDt2dNVQkiRVtIYNG9KxY8esy6gTBkFlq6oq3RdwRkrFatgQttsu3Y4/Pj02dy6MGgUvvpiC4Z13wk03pec22gj22AP69El/Ht26edZQkiSpTBkEla1cLq1MtWqVdSWVoUUL6N8/3SCNrZg4MYXCkSNh+HB48MH0XMuWqcNp377pts02BkNJkqQyYRBUdr7+GkaMgJNOyrqSylW/Pmy7bbr9+MfpsXfeSXMdhwxJ9w89lB7fYINlwbBfvzTCwq2kkiRJJckgqOy89FLqdLnXXllXouo6dEi3pQ183nvvv4Phww+nxzfeGPbZB/bbL93b9VWSJKlkGASVnVwubTVcfsC6istmm8FRR6UbwIwZ8Pzz8Oyz8PTTcPfd6fFttkmhcN99059p06bZ1SxJkqRVCrUxjLAY9OzZM44ZMybrMrQ6+vRJHUP9cytdS5bAuHEpFD7zDLzwQtry26hRajzTvz8ccABsvbXbSCVJkgoshDA2xtizJtfa+UHZ+Oqr1LnSbaGlrV492H57OOcceO45+PhjeOopOO00+M9/0uPduqWB9qedlp5bOvhekiRJmTEIKhsjR6Y5d84PLC9Nm8L++8PVV8OECfDuu/DHP6YweOutMGBA6kY6cCDcfDPMmpV1xZIkSRXJM4LKRlVVGoK+xx5ZV6JC2mwzOPnkdJs3LzWceewxePxxeOSRdM1228F3vgOHHJJWF91CKkmSVHCeEVQ2dt01/cA/cmTWlSgLMcLrr6dA+NhjaYzIkiXQvj0cfHAKhXvskcZbSJIkqUY8I6ji9vnnMHq020IrWQhpu+g558CwYfDhh3DbbdCjB/zpT2lW4cYbw3HHpbDouUJJkqRaZRBU3Rs+HBYvtlGMlmnVCn70o7Rd9KOP4P770xiKBx5I20Zbt4bvfx/uuw+++CLraiVJkkqeQVB1L5dL4wV22y3rSlSM1l0XDj0U/vY3mD0bnnwSjjgiDbM/7DDYcMP0/AMPpO6zkiRJWm0GQdW9qqp0RnCddbKuRMWuUaM0i/DPf04dRocNg2OPTavKhx6aQuHhh8PDD7t9VJIkaTUYBFW35s6FV1/1fKBWX/360Ls33HBDCoXPPw9HHpnmFx5yCGy0Efzwh6n5zIIFWVcrSZJU1AyCqlvDhqWOkQZBrY369dMZ0z/9Cd5/H55+GgYNgkcfhQMPTKHwuOPSuIolS7KuVpIkqegYBFW3qqrSltCdd866EpWLhg1hv/3SwPoPP0wrggcemBrO9OsHHTrAL36RxlVIkiQJMAiqruVysPvu0Lhx1pWoHDVqBAccAHfemULh3/4G3bvDb3+bxlXssANcey188EHWlUqSJGXKIKi6M3s2TJjg2AjVjaZNUyOZxx9PZwp///u0pfTMM6FNm9SE5u674csvs65UkiSpzhkEVXeGDEn3ng9UXdtoIzj9dBg9Gt54I20VnTwZjjoqPfejH6VOpDFmXakkSVKdMAiq7uRyaUbcjjtmXYkq2VZbwaWXwvTpKfwdcQQ8+CDsuSd07gxXXJFWECVJksqYQVB1p6oq/bDdsGHWlUhQrx7ssQfcfDP8+9/pXGGbNnDeebDZZums4YMPOopCkiSVJYOg6sb778OUKW4LVXFq1ixtEx0yBKZOTVtHx41LIynatEnnCidOzLpKSZKkWmMQVN3I5dK9jWJU7LbYIm0dffddePLJ9MuLG25I3Ud79UoriF98kXWVkiRJa8UgqLqRy8H668O222ZdiVQz9eunzqL3359WtK+7DubPh5NOgk03hR//OK0aSpIklSCDoOpGLgd9+qQfrqVS06oV/PSnKfiNHAnf/S789a+w3Xaw667p7a++yrpKSZKkGjMIqvDefTd1aHRbqEpdCMuC36xZaTj9J5+k8RNt2sAZZ6TxFJIkSUXOIKjCW3o+0EYxKicbbJCC3+uvpyYz/fvDTTdB165p9ftvf4Ovv866SkmSpBUyCKrwqqqgdWvo1i3rSqTaF0IKfn//O8ycCb/5Tbr/wQ/SGIpf/cq5hJIkqegYBFVYMaYVwb5909w2qZxtuCGcc04aQfH007DzznDZZdChA3z/+/DCC+m/CUmSpIz5k7kK66230uqI20JVSerVg/32g0cegWnT4PTT4ZlnoHdv2GEHuO02mDcv6yolSVIFMwiqsKqq0r2NYlSpNt8crrkm/ULkz3+GRYvguOOgXTsYPBjeey/rCiVJUgUyCKqwcjnYZBPo3DnrSqRsNWsGJ54I48cvG6fy299Cx45pHMWQIW4blSRJdcYgqMJZej5wr71SQw1J6b+Fvn3hwQfh7bfTmcJhw9L26R13hLvvhgULsq5SkiSVOYOgCueNN+DDDz0fKK3MZpvBFVfAjBnwl7/A/Plw1FFplfDKK2Hu3KwrlCRJZcogqMJxfqBUM+usA8cfDxMnwhNPpFmEv/gFtG0Lp52WGs5IkiTVIoOgCqeqCtq3T6sbkr5ZvXowYAA8+yy89hocemhqMNO5czpH6PgJSZJUSwyCKowlS1Lzi379PB8orYltt4W//hXefTetDg4ZksZP7Lwz3Hdf6j4qSZK0hgyCKozx4+Hjjx0bIa2tTTZJQ+lnzICbboJPPoHDDoMttoDrr4evvsq6QkmSVIIMgioMzwdKtatZM/jxj2HyZPjXv6BNmzSovn17uOQSmDMn6wolSVIJMQiqMHK5tGLRtm3WlUjlpV49OOggGDEChg9PW0UvvDAFwp/9LK0cSpIkfQODoGrfokUwdKjbQqVC22MPeOyxtBX7kEPSVtHNN4djjoHXX8+6OkmSVMQMgqp9r74Kn33mtlCprnTvDnfdBW+9BT/5Cdx/P3TrBgMHwosvZl2dJEkqQgZB1b6qqnRvEJTqVvv28Pvfw3vvwQUXpHETu+0Ge+6Z5hM6ekKSJOUZBFX7crk0EHujjbKuRKpMrVrBxRen0RPXXgvvvAMHHAA9e8JDD6XxLpIkqaIZBFW7FixIqxCuBkrZW3ddOOMMmDYNbr0VPv00Dabfdlu4915YvDjrCiVJUkYKGgRDCP1DCFNCCNNCCINX8HzjEMJ9+edHhRA65B9vGEK4I4QwIYTwRgjhF4WsU7Vo9Gj48ksbxUjFpFEjOPbYNHri7rtTADz88LRyf8cdsHBh1hVKkqQ6VrAgGEKoD9wIDAC6AoeHELoud9lxwNwY4xbAtcBv8o8fCjSOMXYHdgROWhoSVeRyOQgB+vTJuhJJy2vQAH7wA5g4Ef7xD1hnndRhtEsX+Mtf0oq+JEmqCIVcEewFTIsxTo8xLgDuBQYud81A4I782w8Ae4cQAhCBZiGEBsA6wALgswLWqtpSVZW2nbVsmXUlklamXj0YNCh1+P3Xv9J/ryeeCJ06wQ03wLx5WVcoSZIKrJBBsA1QfbLxzPxjK7wmxrgI+BRoSQqFXwL/Bt4Dro4xfrz8JwghnBhCGBNCGDN79uza/wq0eubPh5EjPR8olYoQ0nD6l1+Gp55KXUdPOy3NIrzmmrTNW5IklaVibRbTC1gMbAp0BM4KIWy+/EUxxptjjD1jjD1bt25d1zVqeS+9BF9/bRCUSk0IsP/+MHz4sq6/Z5+9LBB+9VXWFUqSpFpWyCA4C2hX7f22+cdWeE1+G+h6wBzgCOCpGOPCGON/gBFAzwLWqtpQVZW2nO25Z9aVSFoTIUDfvvD886n7b48eywLhdde5ZVSSpDJSyCA4GtgyhNAxhNAIOAx4ZLlrHgGOzr89CKiKMUbSdtC9AEIIzYBdgMkFrFW1IZeDHXeE9dbLuhJJa2v33eHZZ2HYsLRC+LOfpTOE11+ftoFLkqSSVrAgmD/zdyrwNPAGcH+McVII4ZIQwkH5y24FWoYQpgFnAktHTNwIrBtCmEQKlLfHGMcXqlbVgi+/hFGjHBshlZvevdNqfy4HW24Jp58OW2wBf/xj2gouSZJKUkgLcKWvZ8+eccyYMVmXUbmeeSadMXrqqXQvqfzEmALhBRfAiBHQrh2cfz786EdpVqEkScpUCGFsjLFGR+qKtVmMSk0ul2aU7b571pVIKpQQ0qr/8OHplz9t2sDJJ0PnznDLLQ6mlySphBgEVTuqqmDnnWHddbOuRFKhhQD77pvGxTz5JGy0EZxwQhpMf8cdsHhx1hVKkqRvYBDU2vv0UxgzxrERUqUJAfr3T6NjHnsMWrSAY46B7t3hn/9MW0klSVJRMghq7Q0fDkuW2ChGqlQhwAEHpF8IPfBACoDf+x706pW2kBoIJUkqOgZBrb1cDho3hl13zboSSVkKIQXACRPg9tth9uzUPKpfv7SNVJIkFQ2DoNZeLpdCYJMmWVciqRg0aJC2iE6ZkuYOTp6cGkkdeCCMG5d1dZIkCYOg1tbHH8Nrr7ktVNL/atwYTj0V3noLrrgCXngBttsOjjgCpk7NujpJkiqaQVBrZ+jQdP7HRjGSVqZZMxg8GKZPh/POg3/9C7beGk48EWbMyLo6SZIqkkFQa6eqCpo2TU0hJGlVWrSAyy5LgfCUU9KoiS23hDPPhI8+yro6SZIqikFQayeXgz32gEaNsq5EUqnYaCP4/e/hzTfTNtHf/x46dYLLL4evvsq6OkmSKoJBUGvuww9h0iS3hUpaM+3bw223wfjx0LcvnH9+WiG85RZYtCjr6iRJKmsGQa25IUPSvY1iJK2Nbt3SucFhw1I4POEE6NEjPeYMQkmSCsIgqDWXy0Hz5rDDDllXIqkc9O4NI0bAP/8JS5bAwQenx5xBKElSrTMIas1VVUGfPmlmmCTVhhDgkENg4kT405/S6Indd4fvfjfNI5QkSbXCIKg1M2tWmgPm+UBJhdCgAZx0EkybBr/+NTz3HGyzTXrs3//OujpJkkqeQVBrJpdL9wZBSYXUrBn88pdpZfCUU+D222GLLdJjn32WdXWSJJUsg6DWTFVVmgm27bZZVyKpErRuncZMvPEGHHRQmkfYqRNcfz0sXJh1dZIklRyDoNZMLpfavdfzr5CkOtSpE/z97zBmTOosevrpacuoHUYlSVot/hSv1ff22/DOO46NkJSdHXdM5wYfeyz9Qurgg9P/k8aOzboySZJKgkFQq8/zgZKKQQhwwAFpIP2NN6ZOoz17wg9/CDNmZF2dJElFzSCo1ZfLwYYbQteuWVciSdCwIfzkJ6nD6Lnnwv33Q+fOqaHM559nXZ0kSUXJIKjVE2NqFNOvX/ptvCQVi/XWgyuvTPMGDzkkNZTZcku4+WZYtCjr6iRJKioGQa2eqVPh/ffdFiqpeHXoAH/7G4walUZNnHQSbL89PP101pVJklQ0DIJaPVVV6d5GMZKKXa9eMHw4PPAAzJsH/fun24QJWVcmSVLmDIJaPbkctGmTfssuScUuBPje9+D11+F3v4OXX4bttoMTToAPPsi6OkmSMmMQVM3FmIKg5wMllZpGjeBnP0sNZU4/He64I50fvPJKmD8/6+okSapzBkHV3KRJMHu220Illa4NNoBrr03/P9t7b/jFL1IH5H/+04H0kqSKYhBUzTk/UFK52HJLePjhNJS+WbO0fXSvvWDcuKwrkySpThgEVXNVVdCxY+rIJ0nlYO+94dVX00D6CRNghx1Sl9H//CfryiRJKiiDoGpmyRIYOtTVQEnlp0GDNJB+6tR0fvC229KK4TXXwIIFWVcnSVJBGARVM+PGwdy5BkFJ5atFi3R+cMIE2H13OPts2GYbePRRzw9KksqOQVA1s3R+oEFQUrnbait44ol0q18fDjoI9t8/NZiRJKlMGARVM7kcdO6cZghKUiUYMADGj4frroPRo2HbbeHUU2HOnKwrkyRprRkE9c0WLYJhwxwbIanyNGwIP/1pOj940knwxz+m84N/+AMsXJh1dZIkrTGDoL7Z2LHw+eduC5VUuVq1Sp1Fx42DHXdM4XDbbeGZZ7KuTJKkNWIQ1DdbOj+wb99My5CkzG2zTQp///pXWhHcf384+GCYPj3ryiRJWi0GQX2zqqr0w8+GG2ZdiSRlL4TUQGbiRLjiijSUvmtXuOAC+OqrrKuTJKlGDIJatQUL4IUX3BYqSctr3BgGD4YpU+B734Nf/zp1HP3HPxw3IUkqegZBrdqoUTBvno1iJGll2rSBe+5JTbU22AD+7/9g773TiqEkSUXKIKhVy+XSNqg+fbKuRJKKW+/eqbnWTTelpjLbbZeaynz/Dr+XAAAgAElEQVTySdaVSZL0PwyCWrVcLv0w06JF1pVIUvGrXx9+/GN480044QS4/vo0buKWW2DJkqyrkyTp/zMIauXmzYORI90WKkmrq2XLNHNw7Fjo0iWFwp13hpdeyroySZIAg6BW5cUXU7MYG8VI0prZfnsYPhzuvhtmzYJdd4VjjoEPPsi6MklShTMIauWqqtI2p969s65EkkpXCPCDH6TuoueeC3/7G3TuDL/7XZpFKElSBgyCWrlcDnr2hG99K+tKJKn0NW8OV16ZuonusQecdRb06AHPPpt1ZZKkCmQQ1Ip98QW8/LLbQiWptnXuDI8/Do8+mlYE99sPvvtdePfdrCuTJFUQg6BW7IUXYNEiG8VIUiGEAN/5TlodvOwyeOop2HpruPxy+PrrrKuTJFUAg6BWLJeDhg1h992zrkSSyleTJnDeeTB5MgwYAOefD927w9NPZ12ZJKnMGQS1YlVVsMsu0LRp1pVIUvnbbDN48MG0MgjQvz8MGgTvvZdtXZKksmUQ1P/69FN45RXPB0pSXdt/f5gwIW0XfeKJtF30iivcLipJqnUFDYIhhP4hhCkhhGkhhMEreL5xCOG+/POjQggdqj3XI4TwYghhUghhQgihSSFrVTXDhsGSJQZBScpC48Zpu+gbb6RgeN55qbvoM89kXZkkqYwULAiGEOoDNwIDgK7A4SGErstddhwwN8a4BXAt8Jv8axsAdwMnxxi7AX0Bhy3VlaqqdG5ll12yrkSSKlf79vDPf6aVwSVLUig89FCYMSPryiRJZaCQK4K9gGkxxukxxgXAvcDA5a4ZCNyRf/sBYO8QQgD2A8bHGMcBxBjnxBgXF7BWVZfLwW67pTAoScrWgAFpu+ill6axE1ttBb/5DSxYkHVlkqQSVsgg2Aao/mvLmfnHVnhNjHER8CnQEugMxBDC0yGEV0II5xSwTlX30UcwbpxjIySpmDRpkjqKvv56mjs4eHDaLvrcc1lXJkkqUcXaLKYBsAfwg/z9ISGEvZe/KIRwYghhTAhhzOzZs+u6xvI0dGi693ygJBWfDh3goYfSyuCiRbDvvvD978PMmVlXJkkqMYUMgrOAdtXeb5t/bIXX5M8FrgfMIa0eDosxfhRj/Ap4Athh+U8QY7w5xtgzxtizdevWBfgSKlAuB82awU47ZV2JJGllvv3tNIz+kkvgkUfSdtGrrnK7qCSpxgoZBEcDW4YQOoYQGgGHAY8sd80jwNH5twcBVTHGCDwNdA8hNM0HxD7A6wWsVUtVVUHv3mmYvCSpeDVpAr/6Vdouus8+cO65sO228PzzWVcmSSoBBQuC+TN/p5JC3RvA/THGSSGES0IIB+UvuxVoGUKYBpwJDM6/di7wO1KYfA14Jcb4eKFqVd4HH6R25W4LlaTS0bEjPPwwPPZYWhHcZx847DCYtfwmHEmSlglpAa709ezZM44ZMybrMkrb3/8ORxwBo0dDz55ZVyNJWl3z56ctoldcAfXrw8UXw+mnu8tDkipECGFsjLFGP8gXa7MYZSGXg/XWg+23z7oSSdKaaNIELrggbRft1w/OPht23BFGjMi6MklSkTEIaplcDvbcM/0WWZJUujp2TE1kHn4YPv0U9tgDjjsujQiSJAmDoJaaMQOmTXN+oCSVixBg4MC0OnjuuXDnndClC9xyCyxZknV1kqSMGQSV5HLp3kYxklRemjWDK6+EceOge3c44YS0QjhuXNaVSZIyZBBUUlUFLVumHxIkSeWna9f0S78770w7QHbYAX72M/jss6wrkyRlwCAoiDH9cNC3L9Tzr4Qkla0Q4KijYMoUOOkk+P3vYeut4f77078FkqSK4U/9grffhvfec1uoJFWKFi3gpptg1CjYZBP4/vehf3+YOjXryiRJdcQgqLQtFGwUI0mVZqedUhi84QZ46SXYZhu48EKYNy/ryiRJBWYQVNoWuvHGsNVWWVciSapr9evDKaek7aKHHgqXXJLOiz/1VNaVSZIKyCBY6WJMK4L9+qWzI5KkyrTxxnD33enfhIYNYcAAGDQIZs7MujJJUgEYBCvdlCnwwQeeD5QkJf36pdESl18OTzyRdotccw0sXJh1ZZKkWmQQrHTOD5QkLa9RI/jFL2DSpPTvw9lnw447wogRWVcmSaolBsFKV1UF7dpBp05ZVyJJKjYdO8Kjj8LDD8Onn6ZB9MceCx99lHVlkqS1ZBCsZEuWwJAhng+UJK3awIHw+usweDDcdRd06QJ/+Uv6d0SSVJIMgpVs4sT0W13HRkiSvkmzZnDFFen8YPfucOKJsPvu8NprWVcmSVoDBsFK5vlASdLq6to1/ftx550wfXo6O3jGGfDZZ1lXJklaDQbBSpbLweabw2abZV2JJKmUhABHHQWTJ8PJJ8Mf/pC6i953XxpLJEkqegbBSrV4cTof6LZQSdKaatECbrwRRo2CTTeFww6D/feHqVOzrkyS9A0MgpXqtddSBzi3hUqS1tZOO6UweMMN6X6bbeDCC2H+/KwrkySthEGwUlVVpXuDoCSpNtSvD6ecAlOmwKBBcMklKRA+9VTWlUmSVsAgWKlyuXSeY5NNsq5EklRONt4Y7rkHnn8eGjSAAQPg0ENh1qysK5MkVWMQrEQLF8Lw4a4GSpIKZ6+90qiJyy6Dxx5Lv3z83e9g0aKsK5MkYRCsTGPGwBdf2ChGklRYjRvDeeelYfR9+sBZZ6VxEyNHZl2ZJFW8bwyCIYT6IYTJdVGM6sjS+YF9+2ZahiSpQnTsCI8+Cg89BHPnpkH0xx8Pc+ZkXZkkVaxvDIIxxsXAlBCCw+bKRVUV9OgBrVplXYkkqVKEAAcfnFYHf/5zuOMO6NIFbrsNlizJujpJqjg13RraApgUQng+hPDI0lshC1OBfP01jBjh+UBJUjbWXReuugpefRW6doXjjoPevWH8+Kwrk6SK0qCG1/2qoFWo7owaleY6GQQlSVnaZhsYOjStDP7857DDDvDTn8JFF0Hz5llXJ0llr0YrgjHGoSu6Fbo4FUBVFdSrlw7tS5KUpRDgmGPS7MHjjktdRbfeGh58EGLMujpJKmurDIIhhM9DCJ+t4PZ5COGzuipStSiXg+23h/XXz7oSSZKSDTaAP/8ZXnwxnV8fNAi+/W14662sK5OksrXKIBhjbB5j/NYKbs1jjN+qqyJVS776Kv0j69gISVIx2mWXNOLouuvSefZttoFf/zqdb5ck1SrnCFaSkSPTMHnPB0qSilWDBums4OTJMHAgXHABdO8Ozz2XdWWSVFYMgpUkl4P69WGPPbKuRJKkVdt0U7j3Xnj66XRecN994fDD4f33s65MksqCQbCSVFVBr152Y5MklY799oMJE+Dii9NA+q22gj/8ARYtyroySSppBsFK8fnnMHq020IlSaWnSZO0RXTiRNhtt7R1tFevNBJJkrRGDIKVYvhwWLzYRjGSpNK1xRbw5JPwj3/Ahx/CrrvCySfD3LlZVyZJJccgWClyOWjUKP0mVZKkUhVCGi8xeTKccQbccgt06ZIG0zt7UJJqzCBYKXK51JZ7nXWyrkSSpLXXvHkaQD92bFopPOYY6NMHJk3KujJJKgkGwUowdy688orbQiVJ5WfbbeGFF+Avf0khcLvt4Nxz4csvs65MkoqaQbASDBuWtsvYKEaSVI7q1YPjj4cpU+CHP4SrroKuXeHhh90uKkkrYRCsBFVVaUvozjtnXYkkSYXTqhXcemtqkPatb8Ehh8BBB8E772RdmSQVHYNgJcjlYPfdoXHjrCuRJKnw9tgjHYm4+ur0b2DXrnD55bBgQdaVSVLRMAiWu9mz0yBet4VKkipJw4Zw1lnwxhswYACcf346T5jLZV2ZJBUFg2C5GzIk3dsoRpJUidq1gwcfhMcfTyuCe+0FRx6Z5hBKUgUzCJa7XA7WXRd23DHrSiRJys63vw0TJ8KvfpUG0nfpAjfdBIsXZ12ZJGXCIFjuqqpgzz3TFhlJkirZOuvAJZfA+PHQsyecckqasTtmTNaVSVKdMwiWs/ffT620PR8oSdIyXbrAs8/C3/8OM2dCr15w6qnwySdZVyZJdcYgWM6Wng80CEqS9N9CgMMOg8mTUwj84x9hq63gnnucPSipIhgEy1lVFay/Pmy3XdaVSJJUnNZbD/7wBxg9Gtq3T41k9t47BURJKmMGwXKWy0GfPlC/ftaVSJJU3HbYAUaOTCuDr74KPXqkkRNffZV1ZZJUEAbBcvXuuzB9umMjJEmqqfr14eST0/n6ww9PQ+i7dYPHHsu6MkmqdQbBcrV0YK7nAyVJWj0bbgh33JHO2q+zDhx4IBxyCLz3XtaVSVKtKWgQDCH0DyFMCSFMCyEMXsHzjUMI9+WfHxVC6LDc85uFEL4IIZxdyDrLUi4HrVql32RKkqTV16cPvPYaXHklPPMMbL01XHVVGkwvSSWuYEEwhFAfuBEYAHQFDg8hdF3usuOAuTHGLYBrgd8s9/zvgCcLVWPZijE1iunXD+q56CtJ0hpr1AjOPRdefx323Te9ve228PzzWVcmSWulkCmhFzAtxjg9xrgAuBcYuNw1A4E78m8/AOwdQggAIYSDgbeBSQWssTy99Vaai+S2UEmSakf79vDww+m84IIFsM8+8H//l/69laQSVMgg2AaYUe39mfnHVnhNjHER8CnQMoSwLnAucPGqPkEI4cQQwpgQwpjZs2fXWuElr6oq3dsoRpKk2nXAATBpElx8MTz6aJo9+JvfuF1UUskp1n2DFwHXxhi/WNVFMcabY4w9Y4w9W7duXTeVlYJcDjbZBDp3zroSSZLKT5MmcMEFabvoPvvA4MFp3MSzz2ZdmSTVWCGD4CygXbX32+YfW+E1IYQGwHrAHGBn4KoQwjvAGcB5IYRTC1hr+YgxBcF+/SDtspUkSYXQsWPaLvrEE7B4Mey3HwwaZHdRSSWhkEFwNLBlCKFjCKERcBjwyHLXPAIcnX97EFAVk94xxg4xxg7AdcDlMcYbClhr+XjjDfjwQ7eFSpJUVwYMgAkT4NJLUyjceus0g/Drr7OuTJJWqmBBMH/m71TgaeAN4P4Y46QQwiUhhIPyl91KOhM4DTgT+J8RE1pNzg+UJKnuNWkC55+ffiHbv396u3t3eOqprCuTpBUKMcasa6gVPXv2jGPGjMm6jOx973swdiy8/bZbQyVJysozz8Bpp8Gbb6Zh9NdemzqPSlIBhRDGxhh71uTaYm0WozWxZAkMGeL5QEmSsrbffjB+PFxxBTz9dNoueumlMH9+1pVJEmAQLC8TJsDHH7stVJKkYtC4ceooOnkyfOc78KtfwTbbpHOEkpQxg2A5WTo/0CAoSVLxaNcO7r8/jZdo0CDNIhw4MB3jkKSMGATLSS4HW2yR/sGRJEnFZZ990nbRq66C55+Hrl3TYPp587KuTFIFMgiWi0WLYOhQx0ZIklTMGjWCn/8cpkyBgw+Giy6Cbt3g0UfTLGBJqiMGwXLx6qvw2WduC5UkqRS0aQN//3s61rHOOnDQQfDtb6eAKEl1wCBYLpbOD+zbN9MyJEnSaujXD157LY2XePHF1Ezm7LPTL3clqYAMguWiqiqdNdh446wrkSRJq6NhQzjjjDRz8Jhj4He/g86d4fbb02goSSoAg2A5WLAAXnjBbaGSJJWyDTeEv/wFRo+GzTeHY4+FXXeFUaOyrkxSGTIIloPRo+HLL20UI0lSOdhxRxgxAu66C2bMgF12SSuFH3yQdWWSyohBsBzkchAC9OmTdSWSJKk2hABHHpmaxwwenBrLdO4MV1+ddgJJ0loyCJaDXA569ICWLbOuRJIk1abmzeGKK2DixPQL35//HLp3hyeeyLoySSXOIFjq5s9P20fcFipJUvnacss0a3BpADzgAPjOd2Dq1GzrklSyDIKl7qWX4OuvbRQjSVIlGDAAJkyA3/4Whg1Lw+jPPRc+/zzryiSVGINgqauqgnr1YM89s65EkiTVhUaN0qzBN9+EH/wArroqnR+8807HTUiqMYNgqcvlUnex9dbLuhJJklSXNt44zRp86SVo1w6OPhp2391xE5JqxCBYyr78Mv3P3m2hkiRVrp13TmHw9tvh7bfTuImjjoKZM7OuTFIRMwiWshEjYOFCG8VIklTp6tVLswanToXzzoN//CNtF73oovSLY0lajkGwlOVy0KBB2gYiSZLUvDlcdhlMngwHHggXXwxduqTh9J4flFSNQbCUVVWl7SDrrpt1JZIkqZh06AD33QfDh8Mmm8APf5i2jI4cmXVlkoqEQbBUffYZjB3r+UBJkrRye+yR+gnccQfMmpV2ER1+OLz7btaVScqYQbBUDR8OixcbBCVJ0qrVq5dWBN98Ey64AB5+GLbaCn75S/jii6yrk5QRg2CpqqqCxo1h112zrkSSJJWCZs3SmcEpU+C7301nCbfcMnUb9fygVHEMgqUql0shcJ11sq5EkiSVks02g3vugRdfhPbt4dhjYaedYNiwrCuTVIcMgqXo44/htdccGyFJktbcLrukMHjPPTB7NvTpA4MGwfTpWVcmqQ4YBEvR0KEQo+cDJUnS2gkBjjgijZu45BJ48knYems4+2yYOzfr6iQVkEGwFOVy0LQp9OqVdSWSJKkcNG0Kv/pVaihz5JHwu99Bp07p/uuvs65OUgEYBEtRVVVqB92oUdaVSJKkctKmDdx6azqC0qsXnHVWWiG87760G0lS2TAIlpoPP4RJk9wWKkmSCqdHD3jqKXj6aWjeHA47LDWpe+GFrCuTVEsMgqVmyJB0b6MYSZJUaPvtB6+8kkZMzJgBvXvDIYekLaSSSppBsNTkcuk3czvskHUlkiSpEtSvD8ccA1OnwqWXwnPPQbducOqpqduopJJkECw1uRzsuSc0aJB1JZIkqZI0bQrnnw/TpsEJJ8Cf/pQaylxxBcybl3V1klaTQbCUzJqVtmK4LVSSJGVlo43gpptg4sTUs+C886BzZ7jzTliyJOvqJNWQQbCU5HLp3kYxkiQpa1ttBf/6V+pfsPHGcPTRsOOOqcGMHUalomcQLCVVVdCiBWy7bdaVSJIkJX36wKhR8Le/wSefQP/+sPfe8PLLWVcmaRUMgqUkl4O+faGef2ySJKmI1KsHhx8OkyfDH/6Qto3uvDMMGgRTpmRdnaQVMFGUirffhnfecVuoJEkqXo0bw2mnwVtvwUUXpW2i3brBiSemXgeSioZBsFQsPR9ooxhJklTsmjeHCy9MgfCUU+Cvf4UttoDBg2Hu3Kyrk4RBsHTkcrDhhtC1a9aVSJIk1cyGG8Lvf5+2hw4aBFddlUZOXHWVIyekjBkES0GMqVFMv34QQtbVSJIkrZ6OHeGuu+DVV2GXXeDcc2HLLeGWW2DRoqyrkyqSQbAUTJ0K77/v+UBJklTatt0WnngijZxo1y4Npu/eHf75T0dOSHXMIFgKnB8oSZLKSZ8+MHIkPPRQ2u30ve+lLqPPPGMglOqIQbAUVFVBmzZpC4UkSVI5CAEOPhjGj4dbb4UPP4T9908hcdiwrKuTyp5BsNjFmFYEPR8oSZLKUYMGcOyx8OabcOONMG1aCoP77+9QeqmADILFbtIkmD3bsRGSJKm8NW4MP/lJCoJXXw2vvJK2iw4cCOPGZV2dVHYMgsXO84GSJKmSNG0KZ50F06fDpZfC0KGw3Xbw/e/D5MlZVyeVDYNgscvloEOHdJMkSaoUzZvD+efD22+n+8cfh27d4OijU0iUtFYMgsVsyZLUXtltoZIkqVK1aJFWBt9+G372M7j/fujSBU4+GWbOzLo6qWQZBIvZuHEwd67bQiVJklq3TmcHp02DE0+E226DLbaA006DWbOyrk4qOQbBYlZVle4NgpIkSUmbNqm76JtvwpFHwp/+BJtvDqee6gqhtBoKGgRDCP1DCFNCCNNCCINX8HzjEMJ9+edHhRA65B/fN4QwNoQwIX9fmXsjczno3Dn9D0+SJEnLdOgAt9ySAuHRR8Of/wydOqXOozNmZF2dVPQKFgRDCPWBG4EBQFfg8BBC1+UuOw6YG2PcArgW+E3+8Y+AA2OM3YGjgbsKVWfRWrQoDVN1NVCSJGnlOnaEm2+GqVPhmGNSOOzUCX78Y3jvvayrk4pWIVcEewHTYozTY4wLgHuBgctdMxC4I//2A8DeIYQQY3w1xvh+/vFJwDohhMYFrLX4jB0Ln39uoxhJkqSa6NAhrQpOnZoG1N96azpDePLJ8O67WVcnFZ1CBsE2QPV1+Zn5x1Z4TYxxEfAp0HK5a74HvBJj/LpAdRanpfMD+/bNtAxJkqSS0r59Ojc4bRocf3xqKrPllnDSSQZCqZqibhYTQuhG2i560kqePzGEMCaEMGb27Nl1W1yhVVXBNtvAhhtmXYkkSVLp2WwzuOkmeOstOOEE+Otf0wrhCSc4h1CisEFwFtCu2vtt84+t8JoQQgNgPWBO/v22wEPAD2OMb63oE8QYb44x9owx9mzdunUtl5+hBQtgxAjPB0qSJK2tdu1Sl9G33krbRO+6K60QHnkkTJyYdXVSZgoZBEcDW4YQOoYQGgGHAY8sd80jpGYwAIOAqhhjDCGsDzwODI4xjihgjcXp5Zfhq68MgpIkSbWlbVu4/vo0mP7MM+Hhh6F7dzjkEBg9OuvqpDpXsCCYP/N3KvA08AZwf4xxUgjhkhDCQfnLbgVahhCmAWcCS0dMnApsAVwQQngtf6ucPZJVVRAC9OmTdSWSJEnlZZNN4Le/TecFL7wQhg6FXr1g331Tj4YYs65QqhMhlslf9p49e8YxY8ZkXUbt6NcPPv0UXnkl60okSZLK2+efp26j11wDH3wAu+4K550HBxyQfjEvlZAQwtgYY8+aXFvUzWIq0rx5MHKkYyMkSZLqQvPmcPbZacvoTTfBv/8NBx4I220H990HixdnXaFUEAbBYvPii6lZjOcDJUmS6k6TJmkI/Ztvwp13wsKFcNhhsPXWaUj9/PlZVyjVKoNgscnloH596N0760okSZIqT8OGcNRRqaPogw/Ct76VRk506ACXXw5z52ZdoVQrDILFpqoKevZM/9ORJElSNurVg+9+N3UUfe65tFX0/PPTOIozznA4vUqeQbCYfPFFGh3htlBJkqTiEALsvTc89RSMG5fC4Y03QqdOcMQR8OqrWVcorRGDYDF54QVYtMhGMZIkScWoR490fnD69LQq+NhjsMMOafTEM884ekIlxSBYTHK5tC99992zrkSSJEkr064dXH01vPce/OY38PrrsP/+sP32cPfdqdGMVOQMgsUkl4Odd4amTbOuRJIkSd9k/fXhnHPSCuFtt6UAeNRRsPnmKSB+/HHWFUorZRAsFp9+CmPHui1UkiSp1DRuDD/6EUyYkLaLdukCgwdD27Zw8snwxhtZVyj9D4NgsRg2DJYssVGMJElSqapXDw44IHUZHT8+NZP561+ha1fo3x+efDL9vCcVAYNgsaiqSoNMd9kl60okSZK0trp3T4PoZ8yAX/86BcNvfzuFwj/+Eb78MusKVeEMgsUil4PddkthUJIkSeWhdWv45S/hnXdSI5nmzeEnP0nbRs85JzWckTJgECwGc+akuTRuC5UkSSpPjRrBD36QZka/8EIaOXHNNamxzKBBaVHA8ROqQwbBYjBkSLq3UYwkSVJ5CyGNCrv//tRt9MwzUwjcay/o1g1uuCE1EZQKzCBYDHI5aNYMdtop60okSZJUV9q3h6uugpkz4fbbYd114bTToE2b1G10woSsK1QZMwgWg6oq6N07DZOXJElSZVlnHTjmmLRt9OWX4dBD4Y47oEeP9DPivffCggVZV6kyYxDM2gcfpNkyng+UJEnSTjul1cGZM+Hqq+Hf/4bDD4d27VLTGZvLqJYYBLO29HygQVCSJElLtWwJZ50Fb76Z5g/uvDNcfjl07Ajf+Q7861+waFHWVaqEGQSzVlUF660H22+fdSWSJEkqNvXqpWH0jzySmssMHgyvvAIHHwybbQbnn58el1aTQTBruRzsuSc0aJB1JZIkSSpmHTrAZZel7aEPPww77ABXXgmdOsE++8B998HXX2ddpUqEQTBLM2bAtGmOjZAkSVLNNWgAAwfCY4/Bu+/CJZeknykPOyx1HD3zzNSDQloFg2CWcrl07/lASZL0/9q7+yC7yvqA49+fCYlIIYBCQEJDoqEQwIG4ASYG2YGGBKoEW6VYR0MLgogvtGoNrShqR3mZUiqjODhBoSoYFW1mtAngRgsqCQsmQICQQHAUMLxEAQV5SX794zmbXNNsyIa9e/be+/3MnNlzn3vuzm+f8+Tk/O55XqTtMW4cnHce3H8/LFwI3d1w2WUweTJMnw7z5sFTT9UdpYYhE8E6LV5cBgIfckjdkUiSJKmVjRgBM2fCd74DDz1U1id8/HE4/XTYay9417vg+uth/fq6I9UwYSJYl8wyUUx3dxkELEmSJA2GPfeEj32sdA/9+c9hzhz44Q9Lojh+PJx7Ltx7b91RqmZmIHVZs6YM9LVbqCRJkpohAo48Ei6/vKxHOH8+HHooXHwxHHhgWZLi8sth3bq6I1UNTATr0tNTfjpRjCRJkprtla+Ed7yjTDDTt1j9s8/C+98Pe+9d3vv+9511tIOYCNZl8eLSX/uAA+qORJIkSZ1kr73KYvXLl5c1Cc86C37yE3jb22DsWDjtNLjxRscTtjkTwTpklkSwu7s8spckSZKGWgQcdhhceik8/HCZdXT2bPj2t2HGjDIj6TnnwJIl5f5VbcVEsA4rV5Z+2nYLlSRJ0nAwcmSZTOaqq2Dt2pIMTpsGX/5yGWf4+tfDJz4BK1bUHakGiYlgHVw/UJIkScPVjjvC298O3/1uSQq/+lV43evg85+Hgw+GN7yhLGJ/9911R6qXwUSwDj09sO++5R+UJEmSNFyNGQOnnlrWIHz4YfjCF0rZ+efDQQeVhevPO6+MN7T7aEsxERxqGzbAj39cngY6PlCSJEmtYuxY+AJVA6cAAA10SURBVOAH4aabyqL1X/ximXjmc58ry1Lsvz/MnQu9vSaFLcBEcKitWAGPP263UEmSJLWuvfcuS0/09JS5L664AiZOLMtSTJ0KEyaUmUl/+lNnHx2mTASHWt/6gSaCkiRJagd77gnvfS8sWgSPPlrGFB58MFx2GUyfXp4annoqXHcd/P73dUerSmSbPLbt6urK3t7eusN4aSedBHfeCfffX3ckkiRJUvM8+WRJDhcsgB/8AH73Oxg1Co49Fk48Ed76Vthnn7qjbCsRcVtmdm3LsT4RHErr15fxgS4bIUmSpHY3ZgycfDJ8/evlSeHixXD22XDffWUR+3HjoKsLPv3psrD9hg11R9xRTASH0rJl5ZsRu4VKkiSpk+ywA3R3wyWXwKpVZd6MCy6A0aNLIvjGN8JrXwtz5sA115Q5NdRUI+sOoKO4fqAkSZI6XURZdmLyZPj4x8tahYsWwcKFpQvp1VeXY6ZOhVmzyjZ1aln0XoPGMYJD6YQTYM0auOeeuiORJEmShp/16+G220pSuHAhLFlSuozuthvMmAEzZ5YxhuPH1x3psDSQMYImgkPlhRdg993h3e+GL32p7mgkSZKk4W/dOrjxxk2J4SOPlPKJE0tCeMwxpbfd2LH1xjlMDCQR9PnqUOntLdPlOlGMJEmStG12371MOHPyyWWR+rvuKsOtenpg/nz4ylfKcQcdVO6zjz0Wjj4adt213rhbgIngUOkbH9jdXWsYkiRJUkuKgEMOKduHPlS6kf7iFyUp7OmBefPK2oWveAUcdhgcdVRZx3D6dJ8YboFdQ4fKjBll2tzly+uORJIkSWo/zz8PS5fCj35UlmxbsgSefba8N2nSpqRw+vTyOqLWcJvBMYLDzXPPlcfTZ54Jl15adzSSJElS+3v++fLE8Oab4aabys8nnijv7blnSQinTYMjjoApU+BVr6o33kHgGMHhZskS+OMfXTZCkiRJGiqjRpUk74gj4CMfKWMMV64sCWFfcnjddeXYESNKl9PDDy/bEUfAgQeW8jZlIjgUenpKX+Wjj647EkmSJKkzRcABB5Tt9NNL2dq1cOut5cHN0qVlAporrijv7bQTdHWVxHDq1PLUcOLEtulSatfQoXD00fCHP5SZQyVJkiQNT5mwalVJCpcuLQnismWlmynAmDFlIpopU+BTn4Jddqk33s04RnA4eeaZsgDmhz8MF11UdzSSJEmSBuK552DFCrj99k3bqlXlaeLI4dXB0jGCw8nPfla+QXB8oCRJktR6Ro8uTwCnTNlUtmFDGfrVwlo7+laweHEZZDp9et2RSJIkSRoMLZ4Egolg8/X0lAGmO+9cdySSJEmSBDQ5EYyIWRGxMiJWR8TcLbw/OiK+Vb2/JCL2a3jv3Kp8ZUTMbGacTfP002UWIruFSpIkSRpGmpYIRsQI4IvA8cBk4J0RMXmzw04DfpuZrwf+A7iw+uxk4BTgIGAW8KXq97WWm2+G9evhmGPqjkSSJEmSNmrmE8HDgdWZ+UBmPg9cC8ze7JjZwFXV/neAYyMiqvJrM/O5zFwDrK5+X+t505tg2rS6o5AkSZKkjZqZCO4D/Krh9a+rsi0ek5kvAk8Cr97GzxIRZ0REb0T0PvbYY4MY+iA5/vjyVHDHHeuORJIkSZI2aunJYjLziszsysyuPfbYo+5wJEmSJKklNDMRfAjYt+H1uKpsi8dExEhgDPDENn5WkiRJkrQdmpkI3gpMiogJETGKMvnLgs2OWQDMqfbfDvRkZlblp1Szik4AJgFLmxirJEmSJHWMkc36xZn5YkR8AFgEjACuzMwVEfEZoDczFwDzgP+KiNXAOkqySHXcfOBu4EXg7Mxc36xYJUmSJKmTRHkA1/q6urqyt7e37jAkSZIkqRYRcVtmdm3LsS09WYwkSZIkaeBMBCVJkiSpw5gISpIkSVKHMRGUJEmSpA5jIihJkiRJHcZEUJIkSZI6jImgJEmSJHUYE0FJkiRJ6jBts6B8RDwG/LLuOLbgNcDjdQfRwaz/+lj39bHu62Pd18e6r491Xx/rvl7Dsf7HZ+Ye23Jg2ySCw1VE9GZmV91xdCrrvz7WfX2s+/pY9/Wx7utj3dfHuq9Xq9e/XUMlSZIkqcOYCEqSJElShzERbL4r6g6gw1n/9bHu62Pd18e6r491Xx/rvj7Wfb1auv4dIyhJkiRJHcYngpIkSZLUYUwEmygiZkXEyohYHRFz646n3UTEvhGxOCLujogVEfHhqvz8iHgoIpZV2wkNnzm3Oh8rI2JmfdG3voh4MCLurOq4tyrbPSJuiIhV1c/dqvKIiC9UdX9HREypN/rWFRF/0dC2l0XEUxFxju2+eSLiyoh4NCLuaigbcFuPiDnV8asiYk4df0ur6afuL46Ie6v6/V5E7FqV7xcRzzb8G/hyw2feWF2vVlfnJ+r4e1pJP3U/4OuM90ID10/df6uh3h+MiGVVue1+EG3l3rI9r/mZ6daEDRgB3A9MBEYBy4HJdcfVThuwNzCl2t8ZuA+YDJwPfHQLx0+uzsNoYEJ1fkbU/Xe06gY8CLxms7KLgLnV/lzgwmr/BOB/gACOBJbUHX87bNV15jfAeNt9U+v5zcAU4K6GsgG1dWB34IHq527V/m51/23Dfeun7o8DRlb7FzbU/X6Nx232e5ZW5yOq83N83X/bcN/6qfsBXWe8Fxq8ut/s/X8HPlnt2+4Ht+77u7dsy2u+TwSb53BgdWY+kJnPA9cCs2uOqa1k5iOZeXu1/zRwD7DPVj4yG7g2M5/LzDXAasp50uCZDVxV7V8FnNRQfnUWtwC7RsTedQTYZo4F7s/MX27lGNv9y5SZ/wus26x4oG19JnBDZq7LzN8CNwCzmh99a9tS3Wfm9Zn5YvXyFmDc1n5HVf+7ZOYtWe7QrmbT+VI/+mn3/envOuO90HbYWt1XT/VOBq7Z2u+w3W+frdxbtuU130SwefYBftXw+tdsPUnRyxAR+wGHAUuqog9Uj+iv7Ht8j+dksCVwfUTcFhFnVGVjM/ORav83wNhq37pvjlP405sB2/3QGWhb9zw0xz9Qvo3vMyEifhERP4mIo6qyfSj13ce6f3kGcp2x3Q++o4C1mbmqocx23wSb3Vu25TXfRFAtLyL+DPgucE5mPgVcDrwOOBR4hNKFQoNvemZOAY4Hzo6INze+WX0D6bTETRIRo4ATgW9XRbb7mtjW6xER/wq8CHyjKnoE+PPMPAz4J+CbEbFLXfG1Ka8z9Xsnf/oFoO2+CbZwb7lRO13zTQSb5yFg34bX46oyDaKI2IHyD/UbmXkdQGauzcz1mbkB+AqbusF5TgZRZj5U/XwU+B6lntf2dfmsfj5aHW7dD77jgdszcy3Y7msw0LbueRhEEXEq8BbgXdVNGVW3xCeq/dsoY9P2p9RzY/dR6347bcd1xnY/iCJiJPDXwLf6ymz3g29L95a06TXfRLB5bgUmRcSE6pv7U4AFNcfUVqp+8vOAezLzkobyxrFnbwP6Zt1aAJwSEaMjYgIwiTKQWgMUETtFxM59+5TJG+6i1HHfzFhzgP+u9hcA76lm1zoSeLKhi4W2z598K2y7H3IDbeuLgOMiYreqO91xVZkGKCJmAf8MnJiZzzSU7xERI6r9iZS2/kBV/09FxJHV/xvvYdP50gBsx3XGe6HB9ZfAvZm5scun7X5w9XdvSZte80fWHUC7yswXI+IDlJM+ArgyM1fUHFa7eRPwbuDOqKZRBv4FeGdEHEp5bP8gcCZAZq6IiPnA3ZTuRGdn5vohj7o9jAW+V66XjAS+mZkLI+JWYH5EnAb8kjKgHeCHlJm1VgPPAH8/9CG3jyr5nkHVtisX2e6bIyKuAbqB10TEr4FPARcwgLaemesi4rOUG2OAz2Tmtk7E0bH6qftzKbNT3lBdg27JzPdRZlr8TES8AGwA3tdQx+8HvgbsSBlT2DiuUFvQT913D/Q6473QwG2p7jNzHv9/XDjY7gdbf/eWbXnNj6pHhSRJkiSpQ9g1VJIkSZI6jImgJEmSJHUYE0FJkiRJ6jAmgpIkSZLUYUwEJUmSJKnDmAhKkvQSIuJn1c/9IuLv6o5HkqSXy0RQkqSXkJnTqt39gAElghHhmr2SpGHHRFCSpJcQEb+vdi8AjoqIZRHxjxExIiIujohbI+KOiDizOr47Im6KiAXA3RGxU0T8ICKWR8RdEfG3tf0xkiQBfkspSdK2mwt8NDPfAhARZwBPZubUiBgN/DQirq+OnQIcnJlrIuJvgIcz86+qz42pI3hJkvr4RFCSpO13HPCeiFgGLAFeDUyq3luamWuq/TuBGRFxYUQclZlP1hCrJEkbmQhKkrT9AvhgZh5abRMys++J4B/6DsrM+yhPCO8E/i0iPllDrJIkbWQiKEnStnsa2Lnh9SLgrIjYASAi9o+InTb/UES8FngmM78OXExJCiVJqo1jBCVJ2nZ3AOsjYjnwNeA/KTOJ3h4RATwGnLSFzx0CXBwRG4AXgLOGJFpJkvoRmVl3DJIkSZKkIWTXUEmSJEnqMCaCkiRJktRhTAQlSZIkqcOYCEqSJElShzERlCRJkqQOYyIoSZIkSR3GRFCSJEmSOoyJoCRJkiR1mP8DBboTdwrh1rUAAAAASUVORK5CYII=\n",
+ "text/plain": [
+ "