dynamic_graph.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 动态图\n",
    "\n",
    "从飞桨开源框架2.0beta版本开始，飞桨默认为用户开启了动态图模式。在这种模式下，每次执行一个运算，可以立即得到结果（而不是事先定义好网络结构，然后再执行）。\n",
    "\n",
    "在动态图模式下，您可以更加方便的组织代码，更容易的调试程序，本示例教程将向你介绍飞桨的动态图的使用。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 设置环境\n",
    "\n",
    "我们将使用飞桨2.0beta版本，并确认已经开启了动态图模式。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0.0\n",
      "7f2aa2db3c69cb9ebb8bae9e19280e75f964e1d0\n"
     ]
    }
   ],
   "source": [
    "import paddle\n",
    "import paddle.nn.functional as F\n",
    "import numpy as np\n",
    "\n",
    "paddle.disable_static()\n",
    "print(paddle.__version__)\n",
    "print(paddle.__git_commit__)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 基本用法\n",
    "\n",
    "在动态图模式下，您可以直接运行一个飞桨提供的API，它会立刻返回结果到python。不再需要首先创建一个计算图，然后再给定数据去运行。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.40741017  0.2083312 ]\n",
      " [-1.7567089   0.72117436]\n",
      " [ 0.8870686  -1.1389219 ]\n",
      " [ 1.1233491   0.34348443]]\n",
      "[1. 2.]\n",
      "[[ 1.4074101   2.208331  ]\n",
      " [-0.75670886  2.7211742 ]\n",
      " [ 1.8870686   0.86107814]\n",
      " [ 2.1233492   2.3434844 ]]\n",
      "[ 0.8240726  -0.31436014 -1.3907751   1.810318  ]\n"
     ]
    }
   ],
   "source": [
    "a = paddle.randn([4, 2])\n",
    "b = paddle.arange(1, 3, dtype='float32')\n",
    "\n",
    "print(a.numpy())\n",
    "print(b.numpy())\n",
    "\n",
    "c = a + b\n",
    "print(c.numpy())\n",
    "\n",
    "d = paddle.matmul(a, b)\n",
    "print(d.numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 使用python的控制流\n",
    "\n",
    "动态图模式下，您可以使用python的条件判断和循环，这类控制语句来执行神经网络的计算。（不再需要`cond`, `loop`这类OP)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 +> [5 6 7]\n",
      "1 +> [5 7 9]\n",
      "2 +> [ 5  9 15]\n",
      "3 -> [-3  3 21]\n",
      "4 +> [ 5 21 87]\n",
      "5 +> [  5  37 249]\n",
      "6 -> [ -3  59 723]\n",
      "7 +> [   5  133 2193]\n",
      "8 -> [  -3  251 6555]\n",
      "9 -> [   -3   507 19677]\n"
     ]
    }
   ],
   "source": [
    "a = paddle.to_tensor(np.array([1, 2, 3]))\n",
    "b = paddle.to_tensor(np.array([4, 5, 6]))\n",
    "\n",
    "for i in range(10):\n",
    "    r = paddle.rand([1,])\n",
    "    if r > 0.5:\n",
    "        c = paddle.pow(a, i) + b\n",
    "        print(\"{} +> {}\".format(i, c.numpy()))\n",
    "    else:\n",
    "        c = paddle.pow(a, i) - b\n",
    "        print(\"{} -> {}\".format(i, c.numpy()))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 构建更加灵活的网络\n",
    "\n",
    "- 使用动态图可以用来创建更加灵活的网络，比如根据控制流选择不同的分支网络，和方便的构建权重共享的网络。接下来我们来看一个具体的例子，在这个例子中，第二个线性变换只有0.5的可能性会运行。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MyModel(paddle.nn.Layer):\n",
    "    def __init__(self, input_size, hidden_size):\n",
    "        super(MyModel, self).__init__()\n",
    "        self.linear1 = paddle.nn.Linear(input_size, hidden_size)\n",
    "        self.linear2 = paddle.nn.Linear(hidden_size, hidden_size)\n",
    "        self.linear3 = paddle.nn.Linear(hidden_size, 1)\n",
    "\n",
    "    def forward(self, inputs):\n",
    "        x = self.linear1(inputs)\n",
    "        x = F.relu(x)\n",
    "\n",
    "        if paddle.rand([1,]) > 0.5: \n",
    "            x = self.linear2(x)\n",
    "            x = F.relu(x)\n",
    "\n",
    "        x = self.linear3(x)\n",
    "        \n",
    "        return x     "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 [2.0915627]\n",
      "200 [0.67530334]\n",
      "400 [0.52042854]\n",
      "600 [0.28010666]\n",
      "800 [0.09739777]\n",
      "1000 [0.09307177]\n",
      "1200 [0.04252927]\n",
      "1400 [0.03095707]\n",
      "1600 [0.03022156]\n",
      "1800 [0.01616007]\n",
      "2000 [0.01069116]\n",
      "2200 [0.0055158]\n",
      "2400 [0.00195092]\n",
      "2600 [0.00101116]\n",
      "2800 [0.00192219]\n"
     ]
    }
   ],
   "source": [
    "total_data, batch_size, input_size, hidden_size = 1000, 64, 128, 256\n",
    "\n",
    "x_data = np.random.randn(total_data, input_size).astype(np.float32)\n",
    "y_data = np.random.randn(total_data, 1).astype(np.float32)\n",
    "\n",
    "model = MyModel(input_size, hidden_size)\n",
    "\n",
    "loss_fn = paddle.nn.MSELoss(reduction='mean')\n",
    "optimizer = paddle.optimizer.SGD(learning_rate=0.01, \n",
    "                                 parameters=model.parameters())\n",
    "\n",
    "for t in range(200 * (total_data // batch_size)):\n",
    "    idx = np.random.choice(total_data, batch_size, replace=False)\n",
    "    x = paddle.to_tensor(x_data[idx,:])\n",
    "    y = paddle.to_tensor(y_data[idx,:])\n",
    "    y_pred = model(x)\n",
    "\n",
    "    loss = loss_fn(y_pred, y)\n",
    "    if t % 200 == 0:\n",
    "        print(t, loss.numpy())\n",
    "\n",
    "    loss.backward()\n",
    "    optimizer.minimize(loss)\n",
    "    model.clear_gradients()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The end\n",
    "\n",
    "可以看到使用动态图带来了更灵活易用的方式来组网和训练。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}