1-Introduction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 介绍\n",
    "PGL 是一个用paddlepaddle实现的图神经网络(GNN)框架，它可以方便用户快速构建自己的图神经网络模型。\n",
    "\n",
    "为了让用户快速上手，本教程的主要目的是：\n",
    "* 理解PGL是如何在图网络上进行计算的。\n",
    "* 使用PGL实现一个简单的图神经网络模型，用于对图网络中的节点进行二分类。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 第一步：使用PGL创建一个图网络"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pgl\n",
    "from pgl import graph  # 导入pgl的图模块\n",
    "import numpy as np\n",
    "\n",
    "def build_graph():\n",
    "    # 定义节点的个数；每个节点用一个数字表示，即从0~9\n",
    "    num_node = 10\n",
    "    # 添加节点之间的边，每条边用一个tuple表示为: (src, dst)\n",
    "    edge_list = [(2, 0), (2, 1), (3, 1),(4, 0), (5, 0), \n",
    "             (6, 0), (6, 4), (6, 5), (7, 0), (7, 1),\n",
    "             (7, 2), (7, 3), (8, 0), (9, 7)]\n",
    "\n",
    "    # 每个节点可以用一个d维的特征向量作为表示，这里随机产生节点的向量表示.\n",
    "    # 在PGL中，我们可以使用numpy来添加节点的向量表示。\n",
    "    d = 16\n",
    "    feature = np.random.randn(num_node, d).astype(\"float32\")\n",
    "    #feature = np.array(feature,  dtype=\"float32\")\n",
    "    # 对于边，也同样可以用边的权重作为边特征\n",
    "    edge_feature = np.random.randn(len(edge_list), 1).astype(\"float32\")\n",
    " \n",
    "    # 根据节点，边以及对应的特征向量，创建一个完整的图网络。\n",
    "    # 在PGL中，节点特征和边特征都是存储在一个dict中。\n",
    "    g = graph.Graph(num_nodes = num_node,\n",
    "                    edges = edge_list, \n",
    "                    node_feat = {'feature':feature}, \n",
    "                    edge_feat ={'edge_feature': edge_feature})\n",
    "\n",
    "    return g"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 创建一个图对象，用于保存图网络的各种数据。\n",
    "g = build_graph()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 10 nodes in the graph.\n",
      "There are 14 edges in the graph.\n"
     ]
    }
   ],
   "source": [
    "# 打印图的节点的数量和边的数量\n",
    "print('There are %d nodes in the graph.'%g.num_nodes)\n",
    "print('There are %d edges in the graph.'%g.num_edges)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "除了打印节点，我们也可以可视化整个图网络，下面演示如何绘图显示整个图网络。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcUAAAE1CAYAAACWU/udAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3Xd4VNXWx/HvnCnJpJCE3kMNJCA1BEEpigUUpGuiIoKCvCqKBAtXLNer2BARFJEiqGACAakWQGlKCwQInQBKDSUQSG8zc94/gAhSzEymZWZ97pPnGsjea6mYX/Ype2tUVVURQgghBIqrGxBCCCHchYSiEEIIcZmEohBCCHGZhKIQQghxmYSiEEIIcZmEohBCCHGZhKIQQghxmYSiEEIIcZmEohBCCHGZhKIQQghxmYSiEEIIcZmEohBCCHGZhKIQQghxmc7VDQjhDlRVZffZ3WxJ3cL64+tJSk0ipygHVVXx1/vTslpL7qh1B21qtKF5leZoNBpXtyyEcACNHB0lvFl2YTbfJX/Hxxs+5mzOWQByinJu+LV+ej80aChvLM/L7V/mieZPEOQb5Mx2hRAOJqEovJKqqnyb/C3Dfx6ORbXcNAhvxl/vD8C4+8bxTOtnZOUohIeQUBRe53T2aR5d8CiJJxOtDsN/8tf707xqc+L7xlMrqJadOhRCuIqEovAqB88f5M6Zd5Kel47JYrLLnDpFRzlDOdYNWkeTyk3sMqcQwjUkFIXXOHLxCJFTI0nPS0fFvn/sNWgI8gli85DNhFUIs+vcQgjnkVAUXiHflE/4F+EcyziGRbU4pIYGDVUDqnJw+EH8Df4OqSGEcCx5T1F4hTGrxnA256zDAhFAReVi/kVGLh/psBpCCMeSlaLweFtTt9JxZkfyTHlOqWfUGVn++HI6hHZwSj0hhP3ISlF4vP/89h+nBSJAnimPV3991Wn1hBD2I6EoPNqJzBOsO7rO6XW3n97OwfMHnV5XCFE6EorCo03eMtm2gWnALOB94DNgn3XDzRYzExMn2lZbCOEyEorCoy1LWUaBucC6QWYgDggDXgV6AD8A50o+RZGliJ8P/mxdXSGEy0koCo9ltphJOZ9i/cBzQBbQjkv/hdQDagE7rZvmWMYx8k351tcXQriMhKLwWCnnU9Br9fab8Kx1X27UG9l1Zpf96gshHE5CUXisU9mn0Gq01g+sCPgD67l0KfUQcAQosm4aDRpOZ5+2vr4QwmXkPEXhsQrNhbYN1ALRwM9cCsbqQBOs/q9FRbX+fqYQwqUkFIXHMmgNtg+uCgy66vPpQAvrptCgwUfrY3sPQgink8unwmNVC6iGWTXbNvg0ly6XFnJptZiN1aGoolI1oKpt9YUQLiErReGxwiqEUWS28kbgFTuBbVy6pxgKDMDq/1ryivJoVqWZbfWFEC4hoSg8llbRElYhjF1nbXgC9L7LH6VQO6g2Pjq5fCpEWSKXT4VHe6jRQy65r6dX9HRr2M3pdYUQpSOhKDzasMhhLqmrVbS8EPWCS2oLIWwnoSg8Ws1yNekY2tHpdVtWbUnDCg2dXlcIUToSisLjvd/lfYw6o9PqGXVGPrr3I6fVE0LYj4Si8Hitq7fmuajn8NP7ObyWUWdkYPOB3Fn7TofXEkLYn0ZVVdXVTQjhaAWmAsK/COdoxlEsqsUhNTRoqBZYjZTnU/A3+DukhhDCsWSlKLyCj86HVQNXEeIbgqKx/x97DRqCfIJYPXC1BKIQZZiEovAadYLrsOnpTVT0q4hesd/pGTpFR4gxhPVPrSesQpjd5hVCOJ+EovAqDco3IHlYMp1CO+GvL/2Kzl/vz+01byd5WDIRlSLs0KEQwpXknqLwSqqq8l3ydwxaMAi9Xk+Bat1pFlcCdfz94xnSaggajcYRbQohnExWisIraTQaKpysQKMljZjQbQL1Qurhr/e/5erxyu+HBoXy4T0fkhqbytDWQyUQhfAgslIUXslisRAZGcmYMWPo06cPqqqyN20vW1K3sOH4BrambiWnMAcVFX+9Py2rteSOWncQWT2SZlWaSRAK4aEkFIVXWrBgAWPHjmXr1q0ScEKIYm4biqp66dRyRaOgV/TyjUvYjdlsplmzZowbN45u3WTTbiHE39zm6Ki/LvzFwv0LWXNkDVtTt3I6+zSKRkFFRdEo1A2uS/ta7elcpzN9wvtQzqecq1sWZVRcXBzBwcF07drV1a0IIdyMS1eKqqqy/PByPvjjAzaf3Fy8OryVAH0AZtVMdNNoRrUfJY/BC6sUFRURHh7OtGnTuOuuu1zdjhDCzbgsFE9mnmTAwgFsObmF7KJsq8frFB16Rc9zUc/x7l3vymGuokSmT59OXFwcv/32m6tbEUK4IZeEYvzueIYsHUK+KR+TxVSqufx0flQOqMzSmKU0rdzUTh0KT1RQUEDDhg2ZO3cu7dq1c3U7Qgg35PT3FD/d9CmDFw8muzC71IEIkGvK5cjFI7Sf0Z7Ek4l26FB4qmnTptGsWTMJRCHETTl1pThl6xRiV8SSW5TrkPkDDYH8Puh3mldt7pD5RdmVm5tLgwYNWLZsGa1atXJ1O0IIN+W0leL2U9sZuXykwwIRIKswi25zujm0hiibvvjiC9q3by+BKIS4JaesFAvNhUR8EcHhC4cdXar4kNcvu3/p8FqibMjMzKRBgwasWbOGiAh5WlkIcXNOWSm+t+49TmWfckYp8kx5fJP8DZtPbHZKPeH+PvvsM+6//34JRCHEv3L4SrHAVECljyuRVZjlyDLX0KDhwbAHWRqz1Gk1hXtKT08nLCyMTZs20aBBA1e3I4Rwcw5fKS7YtwAV5771oaKy8vBKTmU5Z3Uq3Ne4cePo3bu3BKIQokQcvs3b+I3jyS60/uV8FgB/AYVAAHAH0LrkwzVomLVjFqM7jLa+tvAIZ8+e5auvvmL79u2ubkUIUUY4dKVospjYdXaXbYM7ACOA/wAxwCogteTD8835rPhzhW21hUf44IMPeOyxx6hdu7arWxFClBEOXSnuP7cfg9ZAobnQ+sGVr/przeWPdKB6yadIPp1sfV3hEU6cOMGsWbPYs2ePq1sRQpQhDg3FHad3lG6CZcAOwARUBRpaNzy3KJe0nDQq+VcqXR+izHnvvfd4+umnqVatmqtbEUKUIQ4NxQt5FygyF9k+QXfgAeA4cASru9Vr9VzMvyih6GX++usv5s2bR0pKiqtbEUKUMQ6/p1jqJ08VIBTIBLZYN1SDhiJLKUJZlEnvvPMOzz//PBUqVHB1K0KIMsahK0VfnS9ajdY+k1mAC1YOUS346nztU1+UCQcOHGDZsmUcPHjQ1a0IIcogh64U64XUQ6/VWz8wG9gFFHApDA8Bu4G61k1TaC6kRmAN6+uLMuutt95i5MiRBAcHu7oVIUQZ5NCVYuvqrck35Vs/UANs5dKDNioQDHQFGls3TWhwqBw+7EV27tzJmjVrmDFjhqtbEUKUUQ4NxYp+FQk0BHI+77x1A/2BQaWv37Jiy9JPIsqMN998k9deew1/f39XtyKEKKMcvs1b3/C+6DQO3zjnOjqzjh8/+pG77rqLyZMnc/r0aaf3IJxny5YtbN26lWHDhrm6FSFEGebwUBxx+wjb7iuWUkhgCGf+OMOLL77Ihg0bCA8Pp1OnTnz++eecOiV7onqaMWPGMGbMGHx95cEqIYTtHB6K4ZXCaVK5iaPLXMOoMzLi9hEE+AfQq1cvZs+ezalTp4iNjWXz5s1ERETQsWNHJk2aRGqqFXvHCbe0bt06Dh48yODBg13dihCijHPKIcNbTm6h06xO5JnyHF0KgCr+VTj0wiECDAE3/P2CggJWrFjB/PnzWbp0KREREfTv359+/fpRo4Y8rVqWqKpKp06deOqppxg4cKCr2xFClHFOCUWA2BWxTNkyhVxTrkPrGHVGfnz0R+6qe1eJvr6goIBff/2VhIQElixZQnh4eHFA1qxZ06G9itJbuXIlw4cPZ/fu3eh0zr93LYTwLE4LxQJTAU2/bMqRi0cwWUwOqeGn92NAswFM6T7FpvGFhYXXBGSjRo3o378/ffv2lZMW3JCqqrRt25bY2FgeeeQRV7cjhPAATgtFgFNZp4icGsnZ3LN2D0ajzsjdde9mcfRitErpd9EpLCxk1apVJCQksHjxYho0aFC8ggwNDbVDx6K0lixZwhtvvMH27dtRFIffHhdCeAGnhiJAalYqd359J6ezT9vtHqO/3p9uDbsR1zcOnWL/S2hFRUXFAblo0SLq169fHJB16tSxez3x7ywWCy1btuR///sfDz30kKvbEUJ4CKeHIkBOYQ4jV4zku+TvShWMOkWHj9aHCV0n8FTLp9BoNHbs8saKiopYvXo18+fPZ+HChdSpU4f+/fvTv39/6ta1ch86YbN58+Yxbtw4Nm/e7JR/70II7+CSULxi3dF1DFo8iDPZZ8gtyi3xiRoGrQFFo9CxdkemPzSdWkG1HNzpjZlMJtasWUNCQgILFy6kdu3axQFZr149l/TkDUwmE02bNmXixIncd999rm5HCOFBXBqKcOlhiQ3HN/Dxho/56eBPxXuVZhdmF3+NggJFoPfR46P3YUirITzX5jnqhrjPysxkMrF27Vrmz5/PDz/8QI0aNYoDskGDBq5uz6N88803zJgxg7Vr18oqUQhhVy4PxauZLCb2pu0lKTWJvWl7ySrMQqfoCPENYdOiTXRs2JExz49x+2+EZrOZdevWkZCQwA8//EC1atWKA7Jhw4aubq9MKywspHHjxsyaNYuOHTu6uh0hhIdxq1C8lcmTJ7Njxw6mTp3q6lasYjab+f3334sDskqVKsUP6TRq1MjV7ZU5X331FT/88APLly93dStCCA9UZkJxw4YNjBgxgsTERFe3YjOz2cz69etJSEhgwYIFVKxYsXgF2bixledieaH8/HwaNmzIggULiIqKcnU7QggPVGZCMSsri6pVq5KRkeERO5dYLJZrArJ8+fLFARkeHu7q9tzShAkTWL16NYsXL3Z1K0IID1VmQhEgLCyMRYsWERER4epW7MpisbBhwwbmz5/P/PnzCQoKKg7IJk2cu5m6u8rOzqZBgwasWLGCZs2aubodIYSHKlPbgLRo0YIdO3a4ug27UxSFO++8kwkTJnDs2DGmTZtGRkYGXbt2JSIigrfeeovdu3dThn5+sbvPP/+czp07SyAKIRyqTK0Ux44dy8WLF/noo49c3YpTWCwWNm/eTEJCAvPnz8ff359+/frRv39/brvtNrd/CtdeMjIyaNCgAb///rvcexVCOJSsFN2Yoii0a9eO8ePHc/ToUWbNmkVeXh49evSgcePGjBkzhuTkZI9fQY4fP54HH3xQAlEI4XBlaqWYmppK8+bNOXv2rNeskm5EVVW2bNlSvILU6/XFK8gWLVq49T+b1KxUNp3YxOYTm1l/fD3peemYLCZ8db6EVQijY2hHIqtHElk9EoPWwLlz52jUqBFbt26VbfSEEA5XpkJRVVWqVKnC9u3b5TDgy1RVJSkpiYSEBBISEtBqtcUB2bJlS7cISItqYfmh5Xy0/iM2ndyEQWsguyAbC5brvtZX54tBa0CDhmGRw8j4NQMy4Msvv3RB50IIb1OmQhHgvvvu48UXX+TBBx90dStuR1VVtm3bVhyQQPFTrK1atXJJQCalJvHw/Ic5m3P2mq37SsKgNVBYUMjA2wYyufdk/PR+DupSCCEuKXOh+MorrxAUFMTrr7/u6lbcmqqqbN++nfnz55OQkIDZbC5eQUZGRjo8IAvNhYxZNYbPEz8v9RFhRp2R8sbyJPRPoF2tdnbqUAghrlfmQvH7779n4cKFxSsh8e9UVSU5Obl4BVlUVFQckG3atLF7QOYW5dJtdje2ntpKblGu3eb10/sxq+cs+jfpb7c5hRDiamUuFPfu3UvPnj05ePCgq1spk1RVZefOncUBmZ+fXxyQbdu2LXVAFpgK6PJtF5JOJZFvyrdT138z6ox83/d7ejXuZfe5hRCizIWiyWQiKCiI06dPExgY6Op2yjRVVdm9e3dxQObk5FwTkIpi/Rs7gxYNYu6euaW+ZHorfno/Nj+9maaVmzqshhDCO5W5UASIiori9Q9eJ7heMHmmPLQaLeV8ytG0clP8Df6ubq9MUlWVPXv2FAdkVlYWffv2pX///rRr165EAbny8Ep6ze1l10umN6JBQ6OKjdg5bCd6rd6htYQQ3qXMhGJeUR7xu+OZlTyLjX9tRFVU/H3+DkAVldyiXKoFVKND7Q482+ZZ2tdq7xavJJRFe/fuLQ7IixcvFgdk+/btbxiQ2YXZ1P2sLudyzzmlPz+9H6/e8SpvdnrTKfWEEN7B7UPxfO553l77NjO3z0Sj0ZTosX4NGvz0flT2r8wbHd9gYIuBKJoytXmPW9m3b19xQKanp18TkFqtFoBJmyfx2m+vOXyVeDV/vT9nXz4rr2oIIezGrUNx8f7FPLnoSXJNuRSaC22aw1/vT9PKTYnvF0+d4Dr2bdAL7d+/v/g1j7S0NPr06UO/fv0YsH0AJzJPOLUXf70/n3X9jKdaPeXUukIIz+WWoVhkLuLJRU+y6MAiu6w8tBotPjofvun1Df0i+tmhQwGQkpJCQkICM1fN5M/b/0Q1WPFH6b1/fG4C2gAPWNdDWIUwDjx/wLpBQghxE24XioXmQrp/350/jv1h9ycYjTojkx+YzJMtn7TrvN7u9d9e54M/Prjhtm0lUgCMAx4D6lg3VK/oSXs5jSDfINtqCyHEVdzqRpuqqkTPj2b9sfUOeaQ/z5THsz89y6L9i+w+tzdbe3St7YEIsA/wB0KtH2rUG9l2apvttYUQ4ipuFYozd8xkxeEV5Joc97BGnimPJxY+wens0w6r4W12nd1Vugl2AM0BGx4UzjflszV1a+nqCyHEZW4TiiczT/LiLy+SU5Tj8Fr5pnwGLBzg8ecQOoNFtZBZkGn7BBeBo0AL24YXmgs5lnHM9vpCCHEVtwnFZ3981iHbgt1IkaWIjcc38uPBH51Sz5MVmYvQarS2T5AM1AZCbJ/CkbvnCCG8i87VDcClg2eXH16OyWJyWs2cohzG/j6W7mHdnVbT3ZnNZjIyMkhPT+fChQv/+pGenk76hXTMT5ptuvQJXArFO0vXt1FnLN0EQghxmVuE4pStU1yy88z209s5eP4gDSs0dHptR7kSbDcLsVuFXHZ2NoGBgYSEhNzwo3z58tSvX5/y5ctf8+vN45uTXWTdWYkAHAOygCa2//0atAZqBdWyfQIhhLiKW4TijG0zrL90uplLD2icBZoCva2va7KY+G7nd7xz1zvWD3Ygi8Vi9Yrtyl9nZWVdE2z/DLArwXaj0AsKCireocYazao2Y8PxDdb/jSYD4YCP9UOvMOqMRFaPtH0CIYS4istD8WL+RdJy06wfGAh0BA4DRbbVNllMrD6y2rbB/+JKsNmyYsvKyiIgIOCWK7Z69erZNdhKo1NoJzad2IRFtfK1jB6lr51blEuraq1KP5EQQuAGobjt1Db89H5kFGRYNzDi8v+nYnMoAuw6c/PXCSwWC5mZmSUKsn9+TWZm5i0vRYaEhFC3bt0bruhcEWylcW+9e5mUOKlE+9LaW2hwKMG+wU6vK4TwTC4PxT1n91BgLnBZ/ez8bAYOG0heet51IZeZmYm/v/91q7SrP69Tp84NAy84OLhMBVtpdK7TmSCfIKeHYoA+gJfbv+zUmkIIz+byUMwuzKbIXIqlXikpGoWwpmE0qNTghpcidTqX/yNyexqNhlHtR/H6qtedekqGBQuP3faY0+oJITyfy7/jq5f/5yoGvYHHH3+c0GAb9hgTxQa3HMy76951Wij66f2IbRcrh0oLIezK5S/vBxgC0CuuOz3dZDHJN1Y7KOdTjtl9ZjvlbEMNGmoE1uCNjm84vJYQwru4PBQjKkXgo7PhmXwzlx6wUS9/FF3+NSsZtAYqGCtYP1Bcp2uDrvRu3NvhL9OrRSqDAwej17ruhykhhGdy+eXTVtVa2ba92zpg7VWf7wQ6AXdZN81tVW5zycYBnmr6Q9M5lH6I5NPJ5Jvtv22fUWfkv23+y+TnJpNxIIN3333Xax5oEkI4nlucp1h1XFXO5Jxxel2douOV9q/wXpd/nngrSiO7MJv7v7uf7ae323VfUj+dH9Mfmk7MbTGcO3eO/v374+/vz5w5cwgKkvMUhRCl5/LLpwCDWgzCR1uKbU1spFN0DGg+wOl1PV2AIYBVA1cxtPVQu1xK9dX5UtW/Kj8//jMxt8UAULFiRVasWEFoaCi33347Bw8eLHUdIYRwi1B8Luo5l9Q1nTQRNymOs2fPuqS+J/PR+TCh6wRWDVxFrXK18NNa/wCOQWvAV+fLgGYDOPTCITqGdrzm9/V6PV988QUvvfQSd955J8uXL7dX+0IIL+UWoVizXE3uqnMXOsV5tzj99f582vdTzpw5Q+PGjRkyZAj79u1zWn1vcXvN2/nzxT9pe7ItoZpQfHW+BBoC0dzkWA2D1kA5n3IEGAJ4rs1z7H12L1N7TL3lE8JDhw5l/vz5DBo0iPHjx8s5mUIIm7nFPUWAoxeP0mRyE6ccMqxTdLSv1Z41A9eg0WhIS0tjypQpTJ48mZYtWxIbG8vdd98tD+DYSVpaGmFhYRw6dIgcXQ4bj29k08lNrD+2ngv5FzCZTfjofGhYviEdQzsSWT2SdrXa4avztarOsWPH6NmzJ82aNeOrr77C19e68UII4TahCPDlli95eeXLDg/GAH0A+57fR81yNa/59fz8fObMmcP48ePR6/WMHDmS6OhoDAaDQ/vxdGPHjuXw4cPMmDHD4bVycnIYPHgwR44cYeHChVSvXt3hNYUQnsOtQlFVVbrN6ca6o+scdpq6n86PGT1nEN00+pZ9LF++nE8++YS9e/cyfPhwnnnmGUJCSnE8vJcymUzUrVuXJUuW0LJlS6fUVFWV999/n8mTJ7NgwQLatm3rlLpCiLLPLe4pXqHRaFgUvYjW1Vo75AVwP70fH9/78S0D8UofXbt2ZeXKlfz000/s37+f+vXrM3z4cA4fPmz3vjzZ4sWLCQ0NdVogwqV/f//5z3/48ssv6dGjB99++63Tagshyja3CkW49Pj9r0/8yn3177PblmGKRsGoMzL5wck8G/WsVWObN2/OrFmz2L17N4GBgdx+++307duX9evXywMdJfD5558zfPhwl9Tu0aMHa9as4Z133iE2NhaTyeSSPoQQZYdbXT69mqqqxO2OY9iyYeSb8imy2HaShr/enwblGzC331waVWxU6r5ycnKYNWsWn376KRUrViQ2NpbevXvLaRo3sGvXLrp27cqRI0fQ6123JVt6ejqPPPIIiqIQHx8vl8GFEDfltqF4xZnsM4z+bTTxu+NRNEqJH8IJMAQQaAjkPx3+w/9F/h9axb5bgZnNZpYsWcL48eM5ceIEL774Ik899RSBgYF2rVOWPfPMM9SoUYM333zT1a1gMpl4+eWXWbZsGUuWLCE8PNzVLQkh3JDbh+IVmQWZfJv8LbN2zGJv2l40Gg16RV987JQGDXmmPIJ8gri95u0MjxpOl3pdUDSOv0KcmJjIJ598wm+//cbgwYMZPnw4tWrVcnhdd3bhwgXq1avHvn37qFq1qqvbKTZr1ixeeeUVvv76a7p37+7qdoQQbqbMhOLVVFXlzwt/knI+hTxTHlqNlnI+5WhetTnljeVd1teRI0eYOHEi33zzDV27diU2NpZWrVq5rB9XGj9+PElJScyZM8fVrVxn48aN9OvXj+eff57XXntN3kcVQhQrk6Ho7jIyMpg2bRoTJ06kfv36jBw5kgcffBBFcbvnmhzCYrHQsGFD5syZw+233+7qdm7o5MmT9O7dm/r16zNjxgz8/Bx/DqQQwv15x3dpJwsKCmLUqFEcPnyYoUOH8t///peIiAi++uorcnOdczK9K/3888+EhIS49fuBNWrUYO3ateh0Ojp06MDx48dd3ZIQwg1IKDqQXq8nJiaGLVu28NVXX/HTTz9Rp04d3nzzTc6ccf5RWc4yadIkhg8f7vaXJY1GI99++y0xMTG0bduW9evXu7olIYSLSSg6gUajoVOnTixevJg//viDtLQ0wsPDefrpp9mzZ4+r27OrlJQUtm3bxiOPPOLqVkpEo9EwatQovv76a3r37s306dNd3ZIQwoUkFJ0sLCyML7/8kpSUFOrUqcM999xDt27d+PXXXz1iM4AvvviCp59+usxtxt21a1d+//13xo0bx/Dhwykqsu29WCFE2SYP2rhYfn4+33//PePHj0er1TJy5EhiYmLK5CbkWVlZhIaGkpycXGZfSbl48SKPPvoo+fn5JCQkUKFCBVe3JIRwIlkpupivry+DBw9m165dfPTRR8yZM4e6devy/vvvk56e7ur2rPLdd99x1113ldlABAgODmbp0qW0adOGNm3asGvXLle3JIRwIlkpuqGdO3fy6aefsnjxYh599FFGjBhBgwYNXNpTobmQszlnyTflo1f0lDeWJ9Dn7917VFWlSZMmTJ48mc6dO7uuUTuaM2cOI0aMYOrUqfTu3dvV7QghnEBC0Y2dOnWKzz//nKlTp9KhQwdGjhzJHXfc4ZSnOk0WEz+m/MgP+39g4/GNHLl4BL1Wj6JRUFWVQnMhFf0qElk9km4NulE9vTpjRo1h586dbv/UqTW2bt1K7969GTJkCGPGjPGad02F8FYSimVATk4O33zzDZ9++inly5cnNjaWPn36OGQT8ov5F5m4eSITN0+k0FxIVmHWv47x0/uRX5BPlF8UM5+aSeOKje3elyudOnWKvn37Ur16dWbNmkVAQICrWxJCOIiEYhliNptZunQp48eP59ixY8WbkJcrV84u8/908CcGLBxAblEu+aZ8q8drNVoMWgOj7xzN6A6j0Smec3JIQUEBzz77LFu3bmXx4sXUqVPH1S0JIRxAQrGMSkxMZPz48axcuZLBgwfzwgsv2PyAS4GpgMGLB7PowCJyi0q/446/3p/aQbX55fFfqB1Uu9TzuQtVVZk0aRJjx44lPj7eY+6dCiH+JjdIyqioqCji4+PZtm0bFouFFi1a8Oijj5KUlGTVPPmmfO797l4W7l9ol0AEyCnKIeWuKmHNAAAgAElEQVR8Cq2ntubg+YN2mdMdaDQaXnjhBWbPns0jjzzC5MmTPeLdUiHE32Sl6CEyMjKYPn06n332GXXr1iU2Npbu3bvf8sEQk8VEt9nd+OP4HzZdLv03GjRU9KvItme2UbNcTbvP70qHDx+mZ8+e3HHHHUyaNKlMvlcqhLiehKKHKSoqYsGCBXzyySdkZmby0ksv8cQTT9zwFIixv4/lvd/fs9sK8Ua0Gi2R1SPZ8NQGp5xt6UxZWVk8/vjjXLhwgfnz51O5cmVXtySEKCXP+i4l0Ov1REdHk5iYyLRp0/jll1+oU6cOb7zxBqdPny7+un1p+3h33bsODUQAs2pm99ndTNk6xaF1XCEwMJCFCxfSqVMnoqKi2L59u6tbEkKUkqwUvUBKSgoTJkwgPj6e3r1789JLLzHgjwEkn0lGxTn/+v30fvz5wp9UCajilHrOlpCQwLPPPsvnn39eZjZDF0JcT1aKXiAsLIzJkydz8OBB6tWrR+dHO7MzdafTAhHAolr4Kukrp9Vztv79+7Ny5UpeffVVxowZg8VicXVLQggbyErRCz0872Hm75vv1FAEqGCswOlRpz3q/cV/Onv2LP369SM4OJjZs2fb7R1SIYRzyErRy+QV5bEkZYn1gWgCFgOfAmOBLwEr37YoNBey6q9V1g0qYypXrsyvv/5K9erVadeuHYcOHXJ1S0IIK0goepmdZ3bio/OxfqAFKAc8CbwG3A0kABdKPkVeUR4bj2+0vnYZYzAYmDJlCsOHD+eOO+7g119/dXVLQogSklD0Mkmnkigy23CArgG4Cwjh0p+aRkAwcKrkU5hUE2uOrrG+dhk1bNgw5s2bx4ABA5gwYYK86C9EGSCh6GU2n9xMnimv9BNlA+eBStYN231md+lrlyGdOnVi48aNzJw5k8GDB1NQUODqloQQtyCh6GXO5Z4r/SRmYAHQAqtDMdfk2Pci3VGdOnXYsGED2dnZdO7cmVOnrFheCyGcSkLRy5T6VQEL8AOgBR6wYbjqna8q+Pv7M2/ePB544AGioqLYsmWLq1sSQtyAhKKX8Tf42z5YBZYAOcAjXApGKxm03rtHqEaj4Y033mDSpEk88MADzJ4929UtCSH+QULRyzSv0tz29wSXAWlADKC3bYo6wXVsG+hBevXqxerVq3nrrbd45ZVXMJvNrm5JCHGZhKKXaVOjDX766zcH/1cXgSTgNDAOeO/yx07rpulQu4P1tT1Q06ZNSUxMJCkpiR49enDx4kVXtySEQHa08TppOWnU/LQmheZCp9fWFGronN2Zl+55iS5dutzw5A5vU1RURGxsLMuXL2fJkiU0atTI1S0J4dVkpehlKvlXokmlJi6pbfA1cHftuxk/fjxVq1alR48eTJ06lZMnT7qkH3eg1+uZOHEir7zyCh06dOCnn35ydUtCeDVZKXqhubvnMmTpELIKs5xWU9EoPBzxMHH94gC4cOECv/zyC0uXLuWXX36hbt269OjRgx49etCyZctbHo7sqdavX0///v0ZMWIEL7/8MhqNxtUtCeF1JBS9UKG5kCrjqnAx33n3sfz0fqx9ci2R1SOv+z2TycT69etZunQpS5cuJTs7mwcffJAePXp43WXW48eP06tXL8LDw5k2bRpGo9HVLQnhVbzvx3GBQWvgiwe+wF9fitczrOCj9aF7WPcbBiKATqejU6dOjBs3jgMHDrB69WoaN27slZdZa9Wqxe+//47FYqFjx46cOHHC1S0J4VVkpeilVFWl25xurPprFUUWG/ZCtUKIbwiHXzhMiDHE6rG3uszaqlUrj73EqKoqH330ERMnTmT+/Pm0a9fO1S0J4RUkFL3YmewzNJnchPS8dIedrWjUGVn4yELub3B/qefyxsusP/74I4MGDeLDDz9k0KBBrm5HCI8noejl9qbtpf2M9mQWZNo9GI06I5MfnMyTLZ6067xXpKSksGzZMpYuXUpSUhKdOnWiR48ePPjgg9SoUcMhNV1h37599OzZkwceeIBx48ah03nuIc1CuJqEomD/uf10nNmRzIJMCsylP8VBgwZfnS9f9/ya6KbRdujw33n6ZdYLFy4QExOD2Wxm7ty5lC9f3tUtCeGRJBQFABfzL/Lsj8+y+MBicotsP8nCT+9HneA6JPRPIKJShB07LLkbXWbt3r073bt3L9OXWc1mM6+++iqLFi1i8eLFNGnimvdNhfBkEoriGj8f/JnhPw/ndPZpcotyS3xJNdAQiKJReO3O1xjVfpTt+6s6gKddZv3222+JjY1l+vTp9OzZ09XtCOFRJBTFdVRVZdOJTXyy8RN+OfQLFtWCXqsn35SP2WJG0Sj4aH1AAwWmAlpUbcGo9qPo1biX25+C4SmXWRMTE+nTpw/Dhg3j9ddfLzN9C+HuJBTFLamqyrGMYySdSuLoxaMUmAvQK3oq+lWkVbVWhFcKd6tVoTXK+mXW1NRUevfuTWhoKDNnzsTf3znvnQphC1VVsVjyUdVCNBoDiuLrlj/MSSgKcVlKSgpLly5l2bJl11xm7d69O9WrV3d1ezeUn5/PM888w86dO1m0aBGhoaGubkkIAFTVwoULq0hP/4WMjHXk5OzBYilAo9ECFjQaHX5+4QQFdaB8+fsoX77r5d9zLQlFIW6gLF1mVVWVCRMm8NFHHzF37lw6duzo6paEFzOZMjl1ahrHj3+C2ZyF2ZwLWG4xQoNWG4BG40PNmi9So8b/oddXcFa713cjoSjErZWVy6wrVqxgwIABvPPOOzzzzDOubkd4ofT05ezb9zhmcw4WS57V4zUaXxTFh8aNZ1CpUl8HdFiCHiQUhbCOO19mPXjwID179qRz58589tln6PV6l/YjvIPFUsCBA0+TlvYDFovtr3RdoSh+hIR0ITz8e3S6ADt0WHISikKUwj8vs9arV4/u3bu79DJrZmYmjz32GJmZmcyfP59KlSqVar4rD0hYLAUoigFFMbrV5WPhWmZzLsnJ95Kdvd2m1eHNaDQ++PmF0aLFOvT6YLvN+691JRSFsA93usxqNpt54403iIuLY9GiRTRv3rzEY1XVwsWLqzl//ucbPiABWvz8GhMUdOflByQeQCmjTyCL0rFYCklOvpesrEQslny7z6/RGPDzC6dVq/Votc55ulpCUQgHcYfLrPHx8QwfPpwpU6bQt++t79GYTFlXPSCRacUDEgZq1nyB6tWfxWCoaNf+hXs7fPg1Tp6cZJdLpjejKL5UrhxD48ZfO6zG1SQUhXACV15m3bZtG71792bgwIG8/fbbKMr1x6imp69k375HMZtzbfoGd+kBCQONGk2lUqWH5fKqF8jK2sb27Xfa9ZLpzSiKH7fdtpSQkLsdXktCUQgnKyoqYsOGDdddZu3Rowd33323Qy6znjlzhr59+1KpUiW+/fZbAgMDgUuXvw4ceIa0tHl2e0AiOLgzERFx6HTlSj2fcE+qamHz5kbk5x9yWk29vgrt2h1DURy7a5aEohAu5qzLrIWFhTz33HNs2rSJxYsXExpajZ077ycra6vdH5AwGuvRsuUf6PVymocnSk9fyZ49fTCbs51WU6sNJCxsKlWqOPbkHQlFIdyIoy+zqqrK5MmTGTv2HWbPro5Wu9+BD0iE0bLlRqc/Ui8cLzn5Xi5c+NXpdQMCWhAZud2hNSQUhXBTRUVFrF+/vviEj6svs3bp0gWj0Wjz3KtXD6CwcDY+PnZs+B80Gl8qV+5PePi3jisinK6w8CwbN9ZGVW07e/XECRg8GDp1gtdft26sohiJjNyBn1+YTbVLVMNhMwshSkWv19O5c2fGjRvHgQMHWL16NWFhYXzyySdUqVKFHj16MHXqVFJTU62aNzs7Ga12gUMDEUBV80lLW0B6+grHFhJOlZm5GUWx/Q/PZ59B48a2jtaSmbnR5tolIaEoRBkRFhZGbGwsq1ev5ujRozz66KOsWbOGpk2bEhkZydtvv01SUhK3uvijqip79kQ75JLpjVgsuezb9xhms3PqCcfLzEzEbM6xaeyqVeDvD61a2VbbYskmI2ODbYNLSEJRiDIoJCSEmJgYvv/+e86cOcO4cePIzs7m0UcfpWbNmjzzzDMsW7aMvLxrH6C5eHENhYUnoISHR9uDxZJPWlqC0+oJx8rMXA+YrR6XkwMzZ8Jzz5W2vqwUhRC3YM1l1uPHP7b5p3xbmc3ZHDv2oVNrCscpKkq3adzXX8MDD0Apdx3EZMoo3QT/Qh60EcKDXf006x9//MSMGZno9db/J5+ZCR9/DFu3QlAQPP003HNPyccrih+tWyfi79/E6trCdqqqUlRURF5eHnl5eeTn5xf/9Y0+L8nXREevoFIl617FOHQI3n0Xpk0DvR5mzYKTJ61/0AbAYKhB+/YnrB9YQrJhoRAe7Mpl1piYGM6eXcbevdGA9SvFzz4DnQ5++OHSN7jRo6F+fahbt6QzaMjI2OjVoaiqKgUFBSUKHlvC6mZjFEXBaDQWf/j6+pb48woVKlz3+yEhOwHrXtrfsQPOnIFHHrn0eV4eWCxw9ChMnWrdP0dHv7wvoSiEl8jN3Q4UWj0uLw/Wrbt0+ctohNtug/btYeVKGDq0ZHNYLDlkZKynevWnra7vCBaLhfz8fKeE05XP8/Pz0ev1JQ6nf/5auXLlrAq0K5/rdPb9Nr9v3xLOnLEuFLt3h7uv2qFt7lw4fRpeesn6+kZjQ+sHWUFCUQgvkZGxHiiyetyJE6DVQq1af/9a/fqQnGzdPFlZm2/462az2eagsXVMYWEhPj4+Nq2eLq2WQqwe4+Pjg1artfqfv7sJCrqTtLQFVm0L6Ot76eMKoxEMBgi28kQojUZPcHBn6wZZSUJRCC9RVHTOpnF5efDP7Vj9/SHXyq1ST506SHh4+HVhZTKZbL68FxAQQMWKFa1acV0JKNm03DaBga0vHyNmuyeftG2cohgJDIwsVe1/I6EohNew/jF6uPRT/T8DMDf3+qD8N8HB5ViwYMF1YWUwGCSgypCAgBZoND5AltNrq6qZoKA7HVpDXskQwksoim3bwtWsCWbzpcuoVxw6BHXqWDePwRBAREQEdevWpWrVqgQFBcmKrQzSaLTUrPkiGo3vv3+xXevqqVZtMFqt7dsbloSEohBews8vwqZxRiN06HDpxeu8PNi1CzZsgHvvtXae+jbVF+6nevWhOPtnGY1GS40aLzi8joSiEF4iKOgOFMXfprEjRkBBAfTpc+l9sxEjrHkdA0BHcHAnm2oL92MwVKZmzZdQFPuf/XkjiuJL5cqP4efXwOG15OV9IbxEdvZOtm+/w6ln4F2h0QTQtOlcKlR4wOm1hWNYLIX8/nt9zOYTKA5eXun1VWjb9pBTjiGTlaIQXsLfv6nNK8XSysvLpk+f/zJx4kROnz7tkh6EfX377feMGpUNOPa4FUUx0qTJXKedyymhKISX0GiUy5e8HPugwvV0hIYOYfTo/5KUlER4eDhdunRh+vTppKfbto+mcJ2CggL+7//+j/fff59Zs/6gefNFDvszpShGGjWa4dRL73L5VAgvUlR0no0bazrt6Ci49I2tdett+PtfOkQvPz+fn376ifj4eFasWEGHDh2Ijo6mZ8+eBAQ4ZzUgbHPixAn69etHtWrVmDVrFkFBQQCkp69g9+4+WCx5gMUOlTQoipHGjWdRuXJ/O8xXcrJSFMKL6PUVqFXrVac9IKHR+FKp0sPFgQjg6+tLnz59mDdvHsePHyc6Opq4uDhq1KjBww8/zMKFC8nPl/MX3c2aNWuIioqiZ8+eLFiwoDgQAcqXv4/IyB0EBLQo9SV6RfHDaAyjVatNTg9EkJWiEF7HYiliy5bbyMtLwdHnKur1lS4/IFHuX782PT2dBQsWEB8fz/bt23nooYeIjo6mS5cu6PV6h/Ypbk5VVcaPH8/HH3/Md999x723eBdHVS2cODGRI0feAixWPdSlKAGASq1arxAaOhpFcc2/cwlFIbxQdvYutm273ar9K62lKEZuu20ZISF3//sX/8OpU6dISEggLi6Ow4cP07dvX6Kjo+nQoQOKox91FMWys7N56qmnOHz4MAsWLCA0NLRE4yyWQtLSfuD48Y/IydmFovhhsRShqn8feq3R+KAoPlgseRiNDahV6xUqV37E4S/n/xsJRSG8VHr6r+ze3dMhwagoRsLCplC16hOlnuuvv/5i7ty5xMfHc+7cOR5++GFiYmKIjIyU3XAc6MCBA/Tp04fbb7+dL774Al9f23awMZtzyc5OJisriYKC45jNuWi1fhgM1QgMjCQgoIXTniwtCQlFIbzYhQur2LXrocsP3ti2N+q1NCiKL40afU2VKtF2mO9a+/btIz4+nri4OCwWC9HR0URHR9O0aVO71/JmixYtYujQobz77rsMGTLEq374kFAUwsvl5f3J3r3R5OTsxWKx/gDiKwoKFIKD69OkyXwCAprZscPrqarK9u3biYuLY+7cuQQFBRUHZP36sp2crcxmM2+88QazZ89m/vz5REVFubolp5NQFEKgqhZOnvyCv/56EzBjNpf8BIQrD0isXBlMvXpvMWjQEIf1eSMWi4WNGzcSFxdHQkICoaGhxMTE8PDDD1OjRg2n9lKWnTt3jkcffRSTyUR8fDyVK1d2dUsuIaEohChmsRRy7twijh37kJycnSiKH7m5mfhctWnJ1Q9I+PrWp3btl6lcOZpt2/bQs2dP9u/fT7ly//60qSOYTCZWr15NXFwcixYtolmzZkRHR9OvXz8qVqzokp7KgqSkJPr27csjjzzCe++9h07nvacKSigKIW7IbM4jLW0jL7zQlfffH4nFkoei+F5+QKL15QckAq8ZM2jQICpXrsyHH37ooq7/VlBQwPLly4mLi+Pnn3+mXbt2xMTE0KtXL5eFtjv6+uuvefXVV5kyZQp9+/Z1dTsuJ6EohLipnTt3EhMTw549e0r09adOneK2225j48aNNGzY0MHdlVxOTg5Lly4lLi6ONWvWcM899xAdHU337t0xGl37CsCtmEzZZGdvJysricLCk5jN+Wi1vvj41CQgoHWpntwsKCjghRdeYO3atSxcuJDw8HA7d182SSgKIW5q8eLFTJs2jWXLlpV4zIcffsiGDRtYvHixAzuz3YULF1i0aBFxcXEkJibSvXt3YmJiuPfeezEYDK5u7/IKfR7Hjn1EXl7K5Xf8ClDVguKvURRfNBoDFksufn6NqVXrFSpV6o9WW7LXJo4fP06/fv2oWbMmM2fOlJXzVeQtWCHETf3111/Ute7gREaMGMGePXtYsWKFg7oqnZCQEAYNGsSKFSs4cOAA7dq144MPPqB69eoMHTqUVatWYTbb4/UU61gsJo4eHcuGDZU5ePB5cnP3oqomzObMawLx0tfmX/51Ezk5uzl48Fk2bKjMsWMfoqq37n3VqlVERUXRp08f5s+fL4H4D7JSFELc1IsvvkhoaCgjR460atySJUsYPXo0O3bsKDNbtB07dox58+YRFxdHampq8SYBbdu2dfh7ejk5e9iz52Hy84+W6rUYRfHHaKxHRMS8a/abhUuvsYwbN45PPvmEOXPm0KVLl9K27ZFkpSiEuClbVooAPXr0oEaNGkyZMsUBXTlG7dq1GTVqFElJSaxdu5YKFSowaNAg6tWrx+jRo0lOTsYRa4i0tEUkJUWRm7uvVIEIYLHkkJOzh6Sk1pw79/cl76ysLB5++GHmzZtHYmKiBOItyEpRCHFTt912G9999x0tWrSweuyePXu466672Lt3b5l9HUJVVXbu3El8fDzx8fH4+voSExNDdHQ0YWFhpZ7/7Nn57N//xOUjl+xLUYxERMRz7lwYffr04Y477mDSpEk2b9fmLSQUhRA3pKoqgYGBnDx58ppjgqzxwgsvYDKZmDx5sp27cz5VVdm8eTNxcXHMmzeP6tWrF28SULt2bavny8jYQHLyPQ4JxCtU1cB//mPkqafG8fTTTzusjieRUBRC3FBaWhqNGzfm/PnzNs+Rnp5OeHg4K1eupFkzx2795kxms5m1a9cSHx/PDz/8QOPGjYmJiaFfv35UqVKlBONz2by5IYWFqQ7vVaOpwp13Hinxk6neTu4pCiFuyNb7iVcrX748b731FiNGjHDI/ThX0Wq13H333UydOpXU1FRGjx7Nxo0badSoEffddx9ff/01Fy9evOn4w4dHYTJdcEqvGk0mf/452im1PIGEohDihuwRigBDhw4lLS2NhQsX2qEr92MwGHjwwQeZPXs2qampDBkyhB9//JHQ0FB69uxJfHw8OTl/P0CTn3+U06dnOvSy6dUsljxOnZpCQcFJp9Qr6yQUhRA3ZK9Q1Ol0TJgwgdjYWPLz8+3Qmfvy8/Ojf//+LFiwgGPHjtG3b1+++eYbatSoQUxMDIsXL+bo0c9QVYtT+1JVlZMnv3RqzbJKQlEIcUP2CkWALl260LJlS8aPH2+X+cqCoKAgnnjiCX7++WcOHTpEp06dmDjxEw4f/hRVLXRqL6paQGrqF1gsRU6tWxZJKAohbsieoQgUvzh+8qT3XcarWLEiw4YNY8GC/xEQEPjvA25i1SoYOBC6dYPHHoOdO0s+VlUtZGVttbm2t5BQFELc0J9//mnXUKxXrx7PPPMMo0d770MfWVlJNq8St26FqVPh1Vfhxx9hwgSoVq3k4y2WIrKykmyq7U0kFIUQ1zGbzRw/fpzQ0FC7zjt69Gh+++03Nm3aZNd5y4qLF9det49pSc2aBQMGQEQEKApUqnTpo6RUNY+MjHU21fYmEopCiOucPHmSihUr2n33k8DAQN5//31efPFFLBbnPmziDvLyUmwaZzbDgQOQkXHpsmn//vDZZ1BgZb7m5u63qb43kVAUQlzH3vcTr/b4448DMHv2bIfM784sFtuevr1wAUwmWLsWJk6E6dPh4EH47jtr69u2SvUmEopCiOs4MhQVRWHixImMHj2arKwsh9RwXzqbRvn4XPr/3r2hQgUICrq0Wty82bp5NBqtTfW9iYSiEOI6jgxFgLZt23LPPfcwduxYh9VwRzpdsE3jAgMv3T+8+gQrW06z0ulCbKrvTSQUhRDXcXQoArz//vtMmzaNw4cPO7SOOzCZTCQnJ3P8eDlsvZXatSssXHjpUmpWFsyfD+3aWTODQlDQHbYV9yISikKI6zgjFKtXr05sbCyjRo1yaB1nU1WVv/76i7lz5xIbG0uHDh0IDg4mOjqa5OQiVNXHpnmfeAIaNbr0BOrAgdCgAVy+PVsiWm0A5cq1tam2N5FTMoQQ16lZsybr16+3+ysZ/5Sfn09ERARTp07lnnvucWgtR0lLS2PLli0kJiaSmJjIli1bMBgMREVFFX9ERkYSFBREXt4RtmwJt/mBm9LQaHxp2/Ygvr41nV67LLHtrq8QosxTVZX8/L/IykoiMzORwsITWCyFgIH77juFn99OCgt9MRj+/SgkW/n6+vLJJ58wYsQIduzYgU7n3t+ScnNz2bZtW3EAJiYmcv78edq0aUNUVBRDhw5l2rRp1KhR44bjjcY6+Ps3Iysr0cmdQ7lybSQQS0BWikJ4mYKC06SmTuHkyc+xWPLQaHSYzdnA3ze7TCbw8SmHxVKAj08NatV6hSpVHkOnC7B7P6qqcs8999CnTx+ee+45u89vK5PJxN69e68JwJSUFJo2bXrNKjAsLAxFKfmdqLS0hezfPxCz2XlP3mq1gURExFGhwoNOq1lWSSgK4SVMpkwOHhzO2bNz0Wg0Vl3CUxR/QKV27dHUrv0aimLfFd2uXbvo0qUL+/bto0KFCnaduyRUVeXIkSPXBOD27dupVasWUVFRxSvB5s2b4+Nj2z3BKywWE5s21aGw0Hl7wPr4hHL77YfllYwSkFAUwgukp69g377HMJuzS3U/S1H88fUNpUmTBPz9I+zYITz33HMoisKkSZPsOu+NnDt37pr7gImJiej1etq2bVu8AmzdujXBwba9QvFvMjLWk5x8r1POVFQUIy1arKFcuSiH1/IEEopCeLijR8dy9Oh7WCy5dppRg6IYadJkPhUqdLPTnHD+/HnCw8NZtWoVTZs2tdu8V98HvBKE586dIzIy8prLoDe7D+goKSnPOfywYUUxUr36MBo08J4ju0pLQlEID3bkyDscO/ahHQPxb38H4wN2m3PSpEksXryYlStXorHh7XRH3Qd0BLM5nx07OpCdvcvmTcJvRaPxJTCwFS1arEJRSnfJ15tIKArhoU6fnkNKylCHBOIViuJHq1YbCQhoZpf5ioqKaNGiBWPHjqVnz563/FpVVTl69Oh19wFr1KhxTQDa4z6go5hMmezYcRe5uXvt+pqGovji79+M5s1/c8jDUZ5MQlEID1RQkEpiYqPLT5U6kgajsSFt2uxGUfR2mXHlypUMGzaMvXv3XhNm/7wPuGXLFnQ6XfF9wDZt2hAZGemw+4COYjbnsm/fQNLTf7LLDzCK4kfFig/RqNFMtFr7nnLiDSQUhfAwqqqSnHwPFy+uA0wOr6coftSsOZJ69f5ntzm7d+9OrVq1CAsLKw5Bd7gP6Ejnzi1h//4nsVjybbrPqChGFMVIePh3dr2k7W0kFIXwMBcv/sHOnV2xWHKcVlNRfGnXLhW93voNp81m83X3AQ8cOEBhYSGPP/44d911F1FRUTRq1Mjl9wEdrajoIqmpUzlxYjwWS+7llf6tvkVr0Gr90WoDqFkzlurVh6DTBTmrXY8koSiEh9m1qyfnzy/l1t9M7UtR/Khb93/UqjXyll9nzX3At956i9OnTzNr1izn/E24EVW1cOHCStLTl3Px4u/k5u7BYilEo1FQVQuK4oO/fxOCgjpSvvz9hIR0QaPx7B8YnEVCUQgPUlh4lo0bQ1FV5++taTBUp127E9c8NXr+/Pnr3gfUarXXvA94s/uAWVlZNGrUiEWLFhEV5d3v2KmqiqoWYrEUoCg+aDQGm57OFf9OQlEID3LmzBxSUobZ9IDNiPkvwb0AAAldSURBVBGwdy9oL296UqkSfPttyccrih9a7dds2XKqOADT0tKuuQ/Ypk0batSoUeJv6LNmzWLKlCls2LDB4y+dCvcgoSiEB0lJeZ7U1MnYcul0xAi491540MbtMXNz4Ycf6uHj07U4BEt7H9BisdC2bVtefPFFHrfmnCQhbOTeW9ILIaySkfEHzryXeDU/Pw2vvtqdhg0/s9uciqIwceJE+vfvT69evQgIkHfuhGPJ9QghPEhBwYlSjZ82DXr2hOefhx07rB2tkpu7r1T1b6Rdu3Z07tyZDz74wO5zC/FPcvlUCA/y++9BmM2ZNo3duxfq1AGdDlatgokTL4WkNa8CBgV1oGXLdTbVv5UTJ07QvHlztm7dSt26de0+vxBXyEpRCA9SmsfyIyLAzw8MBujaFZo2hc2bra1vn11t/qlmzZq89NJLvPzyyw6ZX4grJBSF8CA6nf22ONNowNrrSAZDFbvV/6fY2Fi2bt3K6tWrHVZDCAlFITxIQECkTeOysyExEQoLwWyGlSth506w5vVARTESFHSnTfVLwmg0Mm7cOEaMGIHJ5Pjt64R3klAUwoMEB3dEUazfBNpkgq+/hl69Lj1os3Ah/O9/UKtWyefQaPQEBra2urY1+vbtS0hICNOnT3doHeG95EEbITxIVtYOtm+/06n7nl6hKEbuuOM8Wq3RoXWSk5O577772L9/PyEh1u+1KsStyEpRCA8SGNgCH5+aLqispXLlGIcHIkDz5s3p06cPb7/9tsNrCe8jK0UhPMypUzM5ePAFLBZHn6X4N0Ux0qpVIgEBTZ1SLy0tjYiICNauXUtERIRTagrvICtFITxM5crRTlmxXaHR6AgMjHJaIAJUqlSJMWPGMGLECOTnemFPEopCeBit1kh4+BwUxc8p9TQaH8LDrdg53E6effZZjh8/zrJly5xeW3guCUUhPFD58vdSqVI/m55EtYai+NOgwXh8fWs7tM6N6PV6JkyYwMiRIykoKHB6feGZJBSF8FBhYV/i59cYjcbHIfMrih+VKvWjWrUhDpm/JO6//34aN27MxIkTXdaD8CzyoI0QHsxkymD79k7k5h6w68HDiuJHxYq9CQ//Bo1Ga7d5bZGSkkL79u3ZvXs3VatWdWkvouyTUBTCw5nNOezf/xTnzy/FYskt5WwaFMWXWrVepU6dN93m9PeXX36Z9PR0ZsyY4epWRBknoSiElzh//if27RuAxZJvUzhqtQEYDDVo0mQeAQHNHNCh7TIyMmjcuDFLly4lMvL6re5UVSU//ygm0wVU1YxWa8TXty5arXMeRhJlh4SiEF7EZMri9OlvOH78Y0ymdMzmfODm+4hqNP/f3v2FSFUFcBz/nXN3x9nZGV1t0xZ1NfFPrcUKxfZQlEQgUdFmFIXUc2QFRaQIUfgSPQQKZVjqU1AQglQgiv2R7C/9k1YQDRGz1nXF2WHGXXd27j097O0hyHJmd+7s2f1+3s/53af97Z17/sySMVaZTJc6Ozepvb1X1tbnJoyJ2r17t/bs2aMjR47IGKPh4d/U3/+u8vlD8T2PRsb8fa+6UxSNKJXqUC53mxYs2KBrrrlf1nLv+kxHKQIzkHNOhcIRDQ19oaGhwyqVjioMC3IulDHNam5uVy53q9ra7tK8eevU2jr1N8iHYaienh5t2bJO119/WKXST3IulHNj/zs2CHIyplkLFz6nxYtfUFNTLoEnxlREKQKYFsbG8vr66/UaGTmsdLq2P2vWphUEs9XV9b7mzr17kp8QPqAUAXhvaOhL9fU9qDAclnMT37NobYsWLHhSK1e+1fDVtUgWpQjAaxcvHlRf30OTsLL2n6zNaO7ce7R69V6+Nc4gbN4H4K1C4Zu6FKIkRdGw8vlDOn78Cc5XnUEoRQBeqlSK6ut7sC6F+LcoGtaFCx9rYOC9umVgaqEUAXjp5MlnVakU654TRZd08uTTGh3tr3sWGo9SBOCdQuFbDQ5+OKlH1/2XMBzViRNPJZKFxqIUAXjnzJnXFEUjCSaOKZ8/qNHRPxPMRCNQigC8Ui4P6OLFg5KSXfzinNMff7ydaCaSRykC8Mr58x/UdBD5uXPS5s3SAw9I69dL27dLYXj1450bVX//O1Xnwi+UIgCv5POf1vTT6bZtUlubtHevtGuXdPSotG9fdXNUKkMqlwerzoY/KEUAXikWf6hpXH+/tHatlEpJ8+ZJPT3S6dPVzWFti4rFH2vKhx8oRQDeCMPLKpfP1zT24Yelzz6TLl+WBgel774bL8ZqRNGwLl36taZ8+IGziwB4IwxLMqZJzlXxMTDW3S198ol0331SFEnr1kl33FHdHM6NqVIpVJ0Nf/CmCMAjrqZFNlEkbdok3XmntH//+LfEYlHaubOWZ6i+kOEPShGAN6xtkXNXvhT5SopFaWBA6u0d/6Y4Z450773jP6FWJ1AQcNfidEYpAvBGU1O2plKaM0fq6JA++mh8G0apJB04IC1bVt08QdCqTOaGqvPhD0oRgFey2e6axm3dKn3//fjb4oYNUhBIGzdWN4dzFeVyt9SUDz+w0AaAV9ra1qpQ+ErOjVU1bvny8b2KE2GM1axZnRObBFMab4oAvDJ//mMyphH/zwe69tpHa1roA39QigC8ksmsUmvrzYnnWpvSokXPJ56LZFGKALzT2blZ1rYmmGiUyXQpm70pwUw0AqUIwDvt7b3KZrsT+xnV2rRWrdqVSBYai1IE4B1jjLq63pcxs+qeZW1Gixa9oFxuTd2z0HiUIgAvpdOdWrlyh6zN1C3DmJRaWlZo6dJX6paBqYVSBOCt6657UkuXvlqXYjQmpXR6idas+VzWNk/6/JiajHMu2eurAWCSnT27Q6dOvVjTPYv/xtqMMpkb1d19SM3NbZMyJ/xAKQKYForFX3Ts2CMql/9UFA3XOIuRtWktWfKyOjtfkjHBpD4jpj5KEcC0EUVlnTnzun7//Q1JocKwdFXjjEnJGKvZs2/XihVvqrWV801nKkoRwLQTRWO6cGGfzp7dplLpZ0kmvoexIslJsjImUBQNK5Xq0Pz5j2vhwmeUTi9u8JOj0ShFANOac5FGRk6pVPpFlcqQnKsoCFrU0rJC2Wy3giDJQwAw1VGKAADE2JIBAECMUgQAIEYpAgAQoxQBAIhRigAAxChFAABilCIAADFKEQCAGKUIAECMUgQAIEYpAgAQoxQBAIhRigAAxP4CFbmhaac9TskAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import networkx as nx # networkx是一个常用的绘制复杂图形的Python包。\n",
    "\n",
    "def display_graph(g):\n",
    "    nx_G = nx.Graph()\n",
    "    nx_G.add_nodes_from(range(g.num_nodes))\n",
    "    for line in g.edges:\n",
    "        nx_G.add_edge(*line)\n",
    "    nx.draw(nx_G, with_labels=True,\n",
    "            node_color=['y','g','g','g','y','y','y','g','y','g'], node_size=1000)\n",
    "    foo_fig = plt.gcf() # 'get current figure'\n",
    "    foo_fig.savefig('gcn.png', format='png', dpi=1000)\n",
    "    #foo_fig.savefig('./foo.pdf', format='pdf')  # 也可以保存成pdf\n",
    "\n",
    "    plt.show()\n",
    "\n",
    "display_graph(g)# 创建一个GraphWrapper作为图数据的容器，用于构建图神经网络。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在PGL中，图对象用于保存各种图数据。我们还需要用到GraphWrapper作为图数据的容器，用于构建图神经网络。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import paddle.fluid as fluid\n",
    "use_cuda = False  \n",
    "place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()\n",
    "\n",
    "gw = pgl.graph_wrapper.GraphWrapper(name='graph',\n",
    "                place = place,\n",
    "                node_feat=g.node_feat_info(),\n",
    "                edge_feat=g.edge_feat_info())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 第二步：构建一个图卷积网络模型(GCN)\n",
    "\n",
    "在本教程中，我们使用图卷积网络模型([Kipf和Welling](https://arxiv.org/abs/1609.02907))来实现节点分类器。为了方便，这里我们使用最简单的GCN结构。如果读者想更加深入了解GCN，可以参考原始论文。\n",
    "\n",
    "* 在第$l$层中，每个节点$u_i^l$都有一个特征向量$h_i^l$;\n",
    "* 在每一层中，GCN的想法是下一层的每个节点$u_i^{l+1}$的特征向量$h_i^{l+1}$是由该节点的所有邻居节点的特征向量加权后经过一个非线性变换后得到的。\n",
    "\n",
    "GCN模型符合消息传递模式(message-passing paradigm)，当一个节点的所有邻居节点把消息发送出来后，这个节点就可以根据上面的定义更新自己的特征向量了。\n",
    "\n",
    "在PGL中，我们可以很容易实现一个GCN层。如下所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 自定义GCN层函数\n",
    "def gcn_layer(gw, nfeat, efeat, hidden_size, name, activation):\n",
    "    # gw是一个GraphWrapper；feature是节点的特征向量。\n",
    "    \n",
    "    # 定义message函数，\n",
    "    def send_func(src_feat, dst_feat, edge_feat): \n",
    "        # 注意： 这里三个参数是固定的，虽然我们只用到了第一个参数。\n",
    "        # 在本教程中，我们直接返回源节点的特征向量作为message。用户也可以自定义message函数的内容。\n",
    "        return src_feat['h'] * edge_feat['e']\n",
    "\n",
    "    # 定义reduce函数，参数feat其实是从message函数那里获得的。\n",
    "    def recv_func(feat):\n",
    "        # 这里通过将源节点的特征向量进行加和。\n",
    "        # feat为LodTensor，关于LodTensor的介绍参照Paddle官网。\n",
    "        return fluid.layers.sequence_pool(feat, pool_type='sum')\n",
    "\n",
    "    # send函数触发message函数，发送消息，并将返回消息。\n",
    "    msg = gw.send(send_func, nfeat_list=[('h', nfeat)], efeat_list=[('e', efeat)])\n",
    "    # recv函数接收消息，并触发reduce函数，对消息进行处理。\n",
    "    output = gw.recv(msg, recv_func) \n",
    "    # 以activation为激活函数的全连接输出层。\n",
    "    output = fluid.layers.fc(output, size=hidden_size, bias_attr=False, act=activation, name=name)\n",
    "    return output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在定义好GCN层之后，我们可以构建一个更深的GCN模型，如下我们定一个两层GCN。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 第一层GCN将特征向量从16维映射到8维，激活函数使用relu。\n",
    "output = gcn_layer(gw, gw.node_feat['feature'], gw.edge_feat['edge_feature'], \n",
    "                   hidden_size=8, name='gcn_layer_1', activation='relu')\n",
    "# 第二层GCN将特征向量从8维映射导2维，对应我们的二分类。不使用激活函数。\n",
    "output = gcn_layer(gw, output, gw.edge_feat['edge_feature'], \n",
    "                   hidden_size=1, name='gcn_layer_2', activation=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 第三步：数据预处理\n",
    "\n",
    "由于我们实现一个节点二分类器，所以我们可以使用0，1分别表示两个类。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = [0,1,1,1,0,0,0,1,0,1]\n",
    "label = np.array(y, dtype=\"float32\")\n",
    "label = np.expand_dims(label, -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 第四步：设置训练程序\n",
    "GCN的训练过程跟训练其它基于paddlepaddle的模型是一样的。\n",
    "* 首先我们构建损失函数；\n",
    "* 接着创建一个优化器；\n",
    "* 最后创建执行器并执行训练过程。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0 | Loss: 0.629119\n",
      "Epoch 1 | Loss: 0.614591\n",
      "Epoch 2 | Loss: 0.602767\n",
      "Epoch 3 | Loss: 0.593824\n",
      "Epoch 4 | Loss: 0.587454\n",
      "Epoch 5 | Loss: 0.581866\n",
      "Epoch 6 | Loss: 0.576963\n",
      "Epoch 7 | Loss: 0.572337\n",
      "Epoch 8 | Loss: 0.567905\n",
      "Epoch 9 | Loss: 0.563806\n",
      "Epoch 10 | Loss: 0.559831\n",
      "Epoch 11 | Loss: 0.555969\n",
      "Epoch 12 | Loss: 0.552211\n",
      "Epoch 13 | Loss: 0.548553\n",
      "Epoch 14 | Loss: 0.544992\n",
      "Epoch 15 | Loss: 0.541524\n",
      "Epoch 16 | Loss: 0.538145\n",
      "Epoch 17 | Loss: 0.534852\n",
      "Epoch 18 | Loss: 0.531641\n",
      "Epoch 19 | Loss: 0.528505\n",
      "Epoch 20 | Loss: 0.525442\n",
      "Epoch 21 | Loss: 0.522446\n",
      "Epoch 22 | Loss: 0.519513\n",
      "Epoch 23 | Loss: 0.516638\n",
      "Epoch 24 | Loss: 0.513819\n",
      "Epoch 25 | Loss: 0.511053\n",
      "Epoch 26 | Loss: 0.508336\n",
      "Epoch 27 | Loss: 0.505668\n",
      "Epoch 28 | Loss: 0.503046\n",
      "Epoch 29 | Loss: 0.500472\n"
     ]
    }
   ],
   "source": [
    "# 创建一个标签层作为节点类别标签的容器。\n",
    "node_label = fluid.layers.data(\"node_label\", shape=[None, 1], dtype=\"float32\", append_batch_size=False)\n",
    "# 使用带sigmoid的交叉熵函数作为损失函数\n",
    "loss = fluid.layers.sigmoid_cross_entropy_with_logits(x=output, label=node_label)\n",
    "# 计算平均损失\n",
    "loss = fluid.layers.mean(loss)\n",
    "\n",
    "# 选择Adam优化器，学习率设置为0.01\n",
    "adam = fluid.optimizer.Adam(learning_rate=0.01)\n",
    "adam.minimize(loss)\n",
    "\n",
    "# 创建执行器\n",
    "exe = fluid.Executor(place)\n",
    "exe.run(fluid.default_startup_program())\n",
    "feed_dict = gw.to_feed(g) # 获取图数据\n",
    "\n",
    "for epoch in range(30):\n",
    "    feed_dict['node_label'] = label\n",
    "    \n",
    "    train_loss = exe.run(fluid.default_main_program(), feed=feed_dict, fetch_list=[loss], return_numpy=True)\n",
    "    print('Epoch %d | Loss: %f'%(epoch, train_loss[0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}