Skip to content

P
PaddleSlim

PaddlePaddle / PaddleSlim
1 年多 前同步成功

通知 51
Star 1434
Fork 344
P
PaddleSlim
切换分支/标签
查找文件 普通视图历史永久链接Permalink
index.html 15.6 KB
Newer Older
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 
<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
  <meta charset="utf-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  
  
  <link rel="shortcut icon" href="../../img/favicon.ico">
  <title>SA搜索 - PaddleSlim Docs</title>
  <link href='https://fonts.googleapis.com/css?family=Lato:400,700|Roboto+Slab:400,700|Inconsolata:400,700' rel='stylesheet' type='text/css'>

  <link rel="stylesheet" href="../../css/theme.css" type="text/css" />
  <link rel="stylesheet" href="../../css/theme_extra.css" type="text/css" />
  <link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/github.min.css">
B
Deployed 405a468 with MkDocs version: 1.0.4  
baiyfbupt 已提交
17 
  <link href="../../extra.css" rel="stylesheet">
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 
  
  <script>
    // Current page data
    var mkdocs_page_name = "SA\u641c\u7d22";
    var mkdocs_page_input_path = "tutorials/nas_demo.md";
    var mkdocs_page_url = null;
  </script>
  
  <script src="../../js/jquery-2.1.1.min.js" defer></script>
  <script src="../../js/modernizr-2.8.3.min.js" defer></script>
  <script src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/highlight.min.js"></script>
  <script>hljs.initHighlightingOnLoad();</script> 
  
</head>

<body class="wy-body-for-nav" role="document">

  <div class="wy-grid-for-nav">

    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side stickynav">
      <div class="wy-side-nav-search">
        <a href="../.." class="icon icon-home"> PaddleSlim Docs</a>
        <div role="search">
  <form id ="rtd-search-form" class="wy-form" action="../../search.html" method="get">
    <input type="text" name="q" placeholder="Search docs" title="Type search term here" />
  </form>
</div>
      </div>

      <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
	<ul class="current">
	  
          
            <li class="toctree-l1">
		
    <a class="" href="../..">Home</a>
	    </li>
          
            <li class="toctree-l1">
B
Deployed 482d368 with MkDocs version: 1.0.4  
baiyfbupt 已提交
59 60 61 62 63 
    <a class="" href="../../model_zoo/">模型库</a>
	    </li>
          
            <li class="toctree-l1">
B
Deployed 55210fe with MkDocs version: 1.0.4  
baiyfbupt 已提交
64 
    <span class="caption-text">教程</span>
B
Deployed 9bcff00 with MkDocs version: 1.0.4  
baiyfbupt 已提交
65 66 67 
    <ul class="subnav">
                <li class="">
B
Deployed 55210fe with MkDocs version: 1.0.4  
baiyfbupt 已提交
68 
    <a class="" href="../quant_post_demo/">离线量化</a>
B
Deployed 9bcff00 with MkDocs version: 1.0.4  
baiyfbupt 已提交
69 70 71 
                </li>
                <li class="">
B
Deployed 55210fe with MkDocs version: 1.0.4  
baiyfbupt 已提交
72 
    <a class="" href="../quant_aware_demo/">量化训练</a>
B
Deployed 9bcff00 with MkDocs version: 1.0.4  
baiyfbupt 已提交
73 74 75 
                </li>
                <li class="">
B
Deployed 55210fe with MkDocs version: 1.0.4  
baiyfbupt 已提交
76 
    <a class="" href="../quant_embedding_demo/">Embedding量化</a>
B
Deployed 9bcff00 with MkDocs version: 1.0.4  
baiyfbupt 已提交
77 
                </li>
B
Deployed 55210fe with MkDocs version: 1.0.4  
baiyfbupt 已提交
78 
                <li class=" current">
B
Deployed 9bcff00 with MkDocs version: 1.0.4  
baiyfbupt 已提交
79
B
Deployed 55210fe with MkDocs version: 1.0.4  
baiyfbupt 已提交
80 81 82 83 84 85 86 87 88 89 90 91 92 
    <a class="current" href="./">SA搜索</a>
    <ul class="subnav">
            
    <li class="toctree-l3"><a href="#_1">网络结构搜索示例</a></li>
    
        <ul>
        
            <li><a class="toctree-l4" href="#_2">接口介绍</a></li>
        
        </ul>
    

    </ul>
B
Deployed 9bcff00 with MkDocs version: 1.0.4  
baiyfbupt 已提交
93 94 95 
                </li>
                <li class="">
W
Deployed 7790b33 with MkDocs version: 1.0.4  
wanghaoshuang 已提交
96 
    <a class="" href="../../search_space/">搜索空间</a>
W
Deployed 7790b33 with MkDocs version: 1.0.4  
wanghaoshuang 已提交
97 98 99 
                </li>
                <li class="">
B
Deployed 55210fe with MkDocs version: 1.0.4  
baiyfbupt 已提交
100 
    <a class="" href="../distillation_demo/">知识蒸馏</a>
B
Deployed 9bcff00 with MkDocs version: 1.0.4  
baiyfbupt 已提交
101 102 103 104 105 106 
                </li>
    </ul>
	    </li>
          
            <li class="toctree-l1">
B
Deployed 55210fe with MkDocs version: 1.0.4  
baiyfbupt 已提交
107 
    <span class="caption-text">API</span>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
108 109 110 
    <ul class="subnav">
                <li class="">
B
Deployed 55210fe with MkDocs version: 1.0.4  
baiyfbupt 已提交
111 
    <a class="" href="../../api/quantization_api/">量化</a>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
112 113 114 
                </li>
                <li class="">
B
Deployed 55210fe with MkDocs version: 1.0.4  
baiyfbupt 已提交
115 
    <a class="" href="../../api/prune_api/">剪枝与敏感度</a>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
116 117 118 
                </li>
                <li class="">
B
Deployed 55210fe with MkDocs version: 1.0.4  
baiyfbupt 已提交
119 
    <a class="" href="../../api/analysis_api/">模型分析</a>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
120 
                </li>
B
Deployed 55210fe with MkDocs version: 1.0.4  
baiyfbupt 已提交
121 
                <li class="">
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
122
B
Deployed 55210fe with MkDocs version: 1.0.4  
baiyfbupt 已提交
123 
    <a class="" href="../../api/single_distiller_api/">知识蒸馏</a>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
124 
                </li>
B
Deployed 0f9c3ce with MkDocs version: 1.0.4  
baiyfbupt 已提交
125 126 
                <li class="">
B
Deployed 55210fe with MkDocs version: 1.0.4  
baiyfbupt 已提交
127 128 129 130 131 
    <a class="" href="../../api/nas_api/">SA搜索</a>
                </li>
                <li class="">
                    
    <a class="" href="../../table_latency/">硬件延时评估表</a>
B
Deployed 0f9c3ce with MkDocs version: 1.0.4  
baiyfbupt 已提交
132 
                </li>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
133 134 135 136 137 
    </ul>
	    </li>
          
            <li class="toctree-l1">
B
Deployed 9bcff00 with MkDocs version: 1.0.4  
baiyfbupt 已提交
138 
    <a class="" href="../../algo/algo/">算法原理</a>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 
	    </li>
          
        </ul>
      </div>
      &nbsp;
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" role="navigation" aria-label="top navigation">
        <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
        <a href="../..">PaddleSlim Docs</a>
      </nav>

      
      <div class="wy-nav-content">
        <div class="rst-content">
          <div role="navigation" aria-label="breadcrumbs navigation">
  <ul class="wy-breadcrumbs">
    <li><a href="../..">Docs</a> &raquo;</li>
    
      
        
          <li>教程 &raquo;</li>
        
      
    
    <li>SA搜索</li>
    <li class="wy-breadcrumbs-aside">
B
Deployed 09a1807 with MkDocs version: 1.0.4  
baiyfbupt 已提交
170 
        <a href="https://github.com/PaddlePaddle/PaddleSlim/edit/master/docs/tutorials/nas_demo.md"
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
171 172 173 174 175 176 177 178 179 180 181 182 
          class="icon icon-github"> Edit on GitHub</a>
      
    </li>
  </ul>
  <hr/>
</div>
          <div role="main">
            <div class="section">
              
                <h1 id="_1">网络结构搜索示例<a class="headerlink" href="#_1" title="Permanent link">#</a></h1>
<p>本示例介绍如何使用网络结构搜索接口，搜索到一个更小或者精度更高的模型，该文档仅介绍paddleslim中SANAS的使用及如何利用SANAS得到模型结构，完整示例代码请参考sa_nas_mobilenetv2.py或者block_sa_nas_mobilenetv2.py。</p>
<h2 id="_2">接口介绍<a class="headerlink" href="#_2" title="Permanent link">#</a></h2>
W
Deployed 7790b33 with MkDocs version: 1.0.4  
wanghaoshuang 已提交
183 
<p>请参考<a href='../api/nas_api.md'>神经网络搜索API介绍</a>。</p>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
184 
<h3 id="1">1. 配置搜索空间<a class="headerlink" href="#1" title="Permanent link">#</a></h3>
W
Deployed 7790b33 with MkDocs version: 1.0.4  
wanghaoshuang 已提交
185 186 
<p>详细的搜索空间配置可以参考<a href='../search_space.md'>搜索空间</a>。
<div class="codehilite"><pre><span></span><span class="err">config = [(&#39;MobileNetV2Space&#39;)]</span>
B
Deployed d95da67 with MkDocs version: 1.0.4  
baiyfbupt 已提交
187 
</pre></div></p>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
188 
<h3 id="2-sanas">2. 利用搜索空间初始化SANAS实例<a class="headerlink" href="#2-sanas" title="Permanent link">#</a></h3>
B
Deployed d95da67 with MkDocs version: 1.0.4  
baiyfbupt 已提交
189 
<div class="codehilite"><pre><span></span><span class="kn">from</span> <span class="nn">paddleslim.nas</span> <span class="kn">import</span> <span class="n">SANAS</span>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
190
B
Deployed 405a468 with MkDocs version: 1.0.4  
baiyfbupt 已提交
191 192 193 194 195 196 197 
<span class="n">sa_nas</span> <span class="o">=</span> <span class="n">SANAS</span><span class="p">(</span>
    <span class="n">config</span><span class="p">,</span>
    <span class="n">server_addr</span><span class="o">=</span><span class="p">(</span><span class="s2">&quot;&quot;</span><span class="p">,</span> <span class="mi">8881</span><span class="p">),</span>
    <span class="n">init_temperature</span><span class="o">=</span><span class="mf">10.24</span><span class="p">,</span>
    <span class="n">reduce_rate</span><span class="o">=</span><span class="mf">0.85</span><span class="p">,</span>
    <span class="n">search_steps</span><span class="o">=</span><span class="mi">300</span><span class="p">,</span>
    <span class="n">is_server</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
198 199 200 
</pre></div>

<h3 id="3-nas">3. 根据实例化的NAS得到当前的网络结构<a class="headerlink" href="#3-nas" title="Permanent link">#</a></h3>
B
Deployed d95da67 with MkDocs version: 1.0.4  
baiyfbupt 已提交
201 
<div class="codehilite"><pre><span></span><span class="n">archs</span> <span class="o">=</span> <span class="n">sa_nas</span><span class="o">.</span><span class="n">next_archs</span><span class="p">()</span>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
202 203 204 
</pre></div>

<h3 id="4-program">4. 根据得到的网络结构和输入构造训练和测试program<a class="headerlink" href="#4-program" title="Permanent link">#</a></h3>
B
Deployed d95da67 with MkDocs version: 1.0.4  
baiyfbupt 已提交
205 
<div class="codehilite"><pre><span></span><span class="kn">import</span> <span class="nn">paddle.fluid</span> <span class="k">as</span> <span class="nn">fluid</span>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
206
B
Deployed 405a468 with MkDocs version: 1.0.4  
baiyfbupt 已提交
207 208 209 
<span class="n">train_program</span> <span class="o">=</span> <span class="n">fluid</span><span class="o">.</span><span class="n">Program</span><span class="p">()</span>
<span class="n">test_program</span> <span class="o">=</span> <span class="n">fluid</span><span class="o">.</span><span class="n">Program</span><span class="p">()</span>
<span class="n">startup_program</span> <span class="o">=</span> <span class="n">fluid</span><span class="o">.</span><span class="n">Program</span><span class="p">()</span>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
210
B
Deployed 405a468 with MkDocs version: 1.0.4  
baiyfbupt 已提交
211 212 213 214 215 216 217 218 219 220 
<span class="k">with</span> <span class="n">fluid</span><span class="o">.</span><span class="n">program_guard</span><span class="p">(</span><span class="n">train_program</span><span class="p">,</span> <span class="n">startup_program</span><span class="p">):</span>
    <span class="n">data</span> <span class="o">=</span> <span class="n">fluid</span><span class="o">.</span><span class="n">data</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">&#39;data&#39;</span><span class="p">,</span> <span class="n">shape</span><span class="o">=</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">32</span><span class="p">,</span> <span class="mi">32</span><span class="p">],</span> <span class="n">dtype</span><span class="o">=</span><span class="s1">&#39;float32&#39;</span><span class="p">)</span>
    <span class="n">label</span> <span class="o">=</span> <span class="n">fluid</span><span class="o">.</span><span class="n">data</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">&#39;label&#39;</span><span class="p">,</span> <span class="n">shape</span><span class="o">=</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="n">dtype</span><span class="o">=</span><span class="s1">&#39;int64&#39;</span><span class="p">)</span>
    <span class="k">for</span> <span class="n">arch</span> <span class="ow">in</span> <span class="n">archs</span><span class="p">:</span>
        <span class="n">data</span> <span class="o">=</span> <span class="n">arch</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
    <span class="n">output</span> <span class="o">=</span> <span class="n">fluid</span><span class="o">.</span><span class="n">layers</span><span class="o">.</span><span class="n">fc</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span>
    <span class="n">softmax_out</span> <span class="o">=</span> <span class="n">fluid</span><span class="o">.</span><span class="n">layers</span><span class="o">.</span><span class="n">softmax</span><span class="p">(</span><span class="nb">input</span><span class="o">=</span><span class="n">output</span><span class="p">,</span> <span class="n">use_cudnn</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
    <span class="n">cost</span> <span class="o">=</span> <span class="n">fluid</span><span class="o">.</span><span class="n">layers</span><span class="o">.</span><span class="n">cross_entropy</span><span class="p">(</span><span class="nb">input</span><span class="o">=</span><span class="n">softmax_out</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="n">label</span><span class="p">)</span>
    <span class="n">avg_cost</span> <span class="o">=</span> <span class="n">fluid</span><span class="o">.</span><span class="n">layers</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">cost</span><span class="p">)</span>
    <span class="n">acc_top1</span> <span class="o">=</span> <span class="n">fluid</span><span class="o">.</span><span class="n">layers</span><span class="o">.</span><span class="n">accuracy</span><span class="p">(</span><span class="nb">input</span><span class="o">=</span><span class="n">softmax_out</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="n">label</span><span class="p">,</span> <span class="n">k</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
221
B
Deployed 405a468 with MkDocs version: 1.0.4  
baiyfbupt 已提交
222 223 224 
    <span class="n">test_program</span> <span class="o">=</span> <span class="n">train_program</span><span class="o">.</span><span class="n">clone</span><span class="p">(</span><span class="n">for_test</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
    <span class="n">sgd</span> <span class="o">=</span> <span class="n">fluid</span><span class="o">.</span><span class="n">optimizer</span><span class="o">.</span><span class="n">SGD</span><span class="p">(</span><span class="n">learning_rate</span><span class="o">=</span><span class="mf">1e-3</span><span class="p">)</span>
    <span class="n">sgd</span><span class="o">.</span><span class="n">minimize</span><span class="p">(</span><span class="n">avg_cost</span><span class="p">)</span>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
225 226 227 
</pre></div>

<h3 id="5-program">5. 根据构造的训练program添加限制条件<a class="headerlink" href="#5-program" title="Permanent link">#</a></h3>
B
Deployed d95da67 with MkDocs version: 1.0.4  
baiyfbupt 已提交
228 
<div class="codehilite"><pre><span></span><span class="kn">from</span> <span class="nn">paddleslim.analysis</span> <span class="kn">import</span> <span class="n">flops</span>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
229
B
Deployed 405a468 with MkDocs version: 1.0.4  
baiyfbupt 已提交
230 231 
<span class="k">if</span> <span class="n">flops</span><span class="p">(</span><span class="n">train_program</span><span class="p">)</span> <span class="o">&gt;</span> <span class="mi">321208544</span><span class="p">:</span>
    <span class="k">continue</span>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
232 233 234 
</pre></div>

<h3 id="6-score">6. 回传score<a class="headerlink" href="#6-score" title="Permanent link">#</a></h3>
B
Deployed d95da67 with MkDocs version: 1.0.4  
baiyfbupt 已提交
235 
<div class="codehilite"><pre><span></span><span class="n">sa_nas</span><span class="o">.</span><span class="n">reward</span><span class="p">(</span><span class="n">score</span><span class="p">)</span>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
236 237 238 239 240 241 242 243 
</pre></div>
              
            </div>
          </div>
          <footer>
  
    <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
W
Deployed 7790b33 with MkDocs version: 1.0.4  
wanghaoshuang 已提交
244 
        <a href="../../search_space/" class="btn btn-neutral float-right" title="搜索空间">Next <span class="icon icon-circle-arrow-right"></span></a>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 
      
      
        <a href="../quant_embedding_demo/" class="btn btn-neutral" title="Embedding量化"><span class="icon icon-circle-arrow-left"></span> Previous</a>
      
    </div>
  

  <hr/>

  <div role="contentinfo">
    <!-- Copyright etc -->
    
  </div>

  Built with <a href="http://www.mkdocs.org">MkDocs</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
      
        </div>
      </div>

    </section>

  </div>

  <div class="rst-versions" role="note" style="cursor: pointer">
    <span class="rst-current-version" data-toggle="rst-current-version">
      
          <a href="https://github.com/PaddlePaddle/PaddleSlim/" class="fa fa-github" style="float: left; color: #fcfcfc"> GitHub</a>
      
      
        <span><a href="../quant_embedding_demo/" style="color: #fcfcfc;">&laquo; Previous</a></span>
W
Deployed 7790b33 with MkDocs version: 1.0.4  
wanghaoshuang 已提交
278 
        <span style="margin-left: 15px"><a href="../../search_space/" style="color: #fcfcfc">Next &raquo;</a></span>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
279 280 281 282 283 284 
      
    </span>
</div>
    <script>var base_url = '../..';</script>
    <script src="../../js/theme.js" defer></script>
      <script src="../../mathjax-config.js" defer></script>
B
Deployed 0f9c3ce with MkDocs version: 1.0.4  
baiyfbupt 已提交
285 
      <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" defer></script>
B
Deployed 09f99c9 with MkDocs version: 1.0.4  
baiyfbupt 已提交
286 287 288 289 
      <script src="../../search/main.js" defer></script>

</body>
</html>