Skip to content

P
Paddle

BaiXuePrincess / Paddle
与 Fork 源项目一致

Fork自 PaddlePaddle / Paddle

通知 1
Star 1
Fork 0
P
Paddle
切换分支/标签
查找文件 普通视图历史永久链接Permalink
index.html 7.4 KB
Newer Older
Y
Update docs  
Yu Yang 已提交
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">


<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    
    <title>Paddle的数据提供(DataProvider)介绍 &mdash; PADDLE  documentation</title>
    
    <link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
    <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
    
    <script type="text/javascript">
      var DOCUMENTATION_OPTIONS = {
        URL_ROOT:    '../../',
        VERSION:     '',
        COLLAPSE_INDEX: false,
        FILE_SUFFIX: '.html',
        HAS_SOURCE:  true
      };
    </script>
    <script type="text/javascript" src="../../_static/jquery.js"></script>
    <script type="text/javascript" src="../../_static/underscore.js"></script>
    <script type="text/javascript" src="../../_static/doctools.js"></script>
    <script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
    <link rel="top" title="PADDLE  documentation" href="../../index.html" />
    <link rel="up" title="配置" href="../index.html" />
    <link rel="next" title="PyDataProvider2的使用" href="pydataprovider2.html" />
    <link rel="prev" title="配置" href="../index.html" /> 
  </head>
  <body role="document">
    <div class="related" role="navigation" aria-label="related navigation">
      <h3>Navigation</h3>
      <ul>
        <li class="right" style="margin-right: 10px">
          <a href="../../genindex.html" title="General Index"
             accesskey="I">index</a></li>
        <li class="right" >
          <a href="pydataprovider2.html" title="PyDataProvider2的使用"
             accesskey="N">next</a> |</li>
        <li class="right" >
          <a href="../index.html" title="配置"
             accesskey="P">previous</a> |</li>
        <li class="nav-item nav-item-0"><a href="../../index.html">PADDLE  documentation</a> &raquo;</li>
          <li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">配置</a> &raquo;</li> 
      </ul>
    </div>  

    <div class="document">
      <div class="documentwrapper">
        <div class="bodywrapper">
          <div class="body" role="main">
            
  <div class="section" id="paddle-dataprovider">
<h1>Paddle的数据提供(DataProvider)介绍<a class="headerlink" href="#paddle-dataprovider" title="Permalink to this headline"></a></h1>
<p>数据提供(DataProvider，后用DataProvider代替)是Paddle负责提供数据的模块。其作用是将训练数据
传入内存或者显存，让神经网络可以进行训练。简单的使用，用户可以使用Python的
<code class="code docutils literal"><span class="pre">PyDataProvider</span></code> 来自定义传数据的过程。如果有更复杂的使用，或者需要更高的效率，
用户也可以在C++端自定义一个 <code class="code docutils literal"><span class="pre">DataProvider</span></code> 。</p>
<p>Paddle需要用户在网络配置(trainer_config.py)中定义使用什么DataProvider，和DataProvider
的一些参数，训练文件列表(train.list)和测试文件列表(test.list)。</p>
<p>其中，train.list和test.list均为本地的两个文件(推荐直接放置到训练目录，以相对路径引用)。如果
test.list不设置，或者设置为None的话，那么在训练过程中，不会执行测试操作。否则，则会根据命令行
参数指定的测试方式，在训练过程中进行测试，从而防止过拟合。</p>
<p>一般情况下，train.list和test.list为纯文本文件，其每一行对应这每一个数据文件。数据文件存放在
本地磁盘中，将文件的绝对路径或相对路径(相对于Paddle程序运行时的路径)的方式写在train.list和
test.list中。当然，train.list和test.list也可以放置hdfs文件路径，或者数据库连接地址等等。
用户在DataProvider中需要实现如何访问其中每一个文件。</p>
<p>DataProvider的具体用法和如何实现一个新的DataProvider，请参考下述文章:</p>
<div class="toctree-wrapper compound">
<ul>
<li class="toctree-l1"><a class="reference internal" href="pydataprovider2.html">PyDataProvider2的使用</a><ul>
<li class="toctree-l2"><a class="reference internal" href="pydataprovider2.html#id1">简单的使用场景</a></li>
<li class="toctree-l2"><a class="reference internal" href="pydataprovider2.html#id3">序列模型数据提供</a></li>
<li class="toctree-l2"><a class="reference internal" href="pydataprovider2.html#reference">参考(Reference)</a><ul>
<li class="toctree-l3"><a class="reference internal" href="pydataprovider2.html#provider">&#64;provider</a></li>
<li class="toctree-l3"><a class="reference internal" href="pydataprovider2.html#input-types">input_types</a></li>
<li class="toctree-l3"><a class="reference internal" href="pydataprovider2.html#init-hook">init_hook</a></li>
<li class="toctree-l3"><a class="reference internal" href="pydataprovider2.html#cache">cache</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="write_new_dataprovider.html">自定义一个DataProvider</a></li>
</ul>
</div>
</div>


          </div>
        </div>
      </div>
      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
        <div class="sphinxsidebarwrapper">
  <h4>Previous topic</h4>
  <p class="topless"><a href="../index.html"
                        title="previous chapter">配置</a></p>
  <h4>Next topic</h4>
  <p class="topless"><a href="pydataprovider2.html"
                        title="next chapter">PyDataProvider2的使用</a></p>
  <div role="note" aria-label="source link">
    <h3>This Page</h3>
    <ul class="this-page-menu">
      <li><a href="../../_sources/ui/data_provider/index.txt"
            rel="nofollow">Show Source</a></li>
    </ul>
   </div>
<div id="searchbox" style="display: none" role="search">
  <h3>Quick search</h3>
    <form class="search" action="../../search.html" method="get">
      <input type="text" name="q" />
      <input type="submit" value="Go" />
      <input type="hidden" name="check_keywords" value="yes" />
      <input type="hidden" name="area" value="default" />
    </form>
    <p class="searchtip" style="font-size: 90%">
    Enter search terms or a module, class or function name.
    </p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
        </div>
      </div>
      <div class="clearer"></div>
    </div>
    <div class="related" role="navigation" aria-label="related navigation">
      <h3>Navigation</h3>
      <ul>
        <li class="right" style="margin-right: 10px">
          <a href="../../genindex.html" title="General Index"
             >index</a></li>
        <li class="right" >
          <a href="pydataprovider2.html" title="PyDataProvider2的使用"
             >next</a> |</li>
        <li class="right" >
          <a href="../index.html" title="配置"
             >previous</a> |</li>
        <li class="nav-item nav-item-0"><a href="../../index.html">PADDLE  documentation</a> &raquo;</li>
          <li class="nav-item nav-item-1"><a href="../index.html" >配置</a> &raquo;</li> 
      </ul>
    </div>
    <div class="footer" role="contentinfo">
        &copy; Copyright 2016, PADDLE developers.
      Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5.
    </div>
  </body>
</html>