Skip to content

P
Paddle

PaddlePaddle / Paddle
大约 1 年 前同步成功

通知 2298
Star 20931
Fork 5422
P
Paddle
提交 aa2f76fd 编写于 作者: X Xin Pan
浏览文件
操作

move trainer

上级 da158c22
隐藏空白更改
内联 并排
Showing with 85 addition and 80 deletion
doc/v2/design/cluster_train/large_model_dist_train.md
浏览文件 @ aa2f76fd
......@@ -52,7 +52,7 @@ In `trainer_internal.cpp:L93 trainOneBatch`:
When doing actual network forward and backward, at the beginning of each batch, the trainer will try to download one row of data from pserver.
In `trainer/RemoteParameterUpdater.cpp`: `parameterUpdater_->getParametersRemote();`:
In `legacy/trainer/RemoteParameterUpdater.cpp`: `parameterUpdater_->getParametersRemote();`:
```c++
if (fullSize) {
......
doc/v2/design/mkl/mkldnn.md
浏览文件 @ aa2f76fd
......@@ -18,20 +18,20 @@ Figure 1. PaddlePaddle on IA
具体的完成状态可以参见[这里](https://github.com/PaddlePaddle/Paddle/projects/21)。
## Contents
- [Overview](#overview)
- [Actions](#actions)
- [CMake](#cmake)
- [Matrix](#matrix)
- [Layers](#layers)
- [Activations](#activations)
- [Parameters](#parameters)
- [Gradients](#gradients)
- [Unit Tests](#unit-tests)
- [Python API](#python-api)
- [Benchmarking](#benchmarking)
- [Others](#others)
- [Design Concerns](#design-concerns)
- [Overview](#overview)
- [Actions](#actions)
- [CMake](#cmake)
- [Matrix](#matrix)
- [Layers](#layers)
- [Activations](#activations)
- [Parameters](#parameters)
- [Gradients](#gradients)
- [Unit Tests](#unit-tests)
- [Python API](#python-api)
- [Benchmarking](#benchmarking)
- [Others](#others)
- [Design Concerns](#design-concerns)
## Overview
......@@ -218,20 +218,20 @@ if use_mkldnn
我们总结出一些特别需要注意的点：
1. 使用**deviceId_**。为了尽可能少的在父类Layer中添加变量或者函数，
我们决定使用已有的`deviceId_`变量来区分layer的属性，定义`-2`为`MKLDNNLayer`特有的设备ID。
2. 重写父类Layer的**init**函数，修改`deviceId_`为`-2`，代表这个layer是用于跑在MKL-DNN的环境下。
我们决定使用已有的`deviceId_`变量来区分layer的属性，定义`-2`为`MKLDNNLayer`特有的设备ID。
2. 重写父类Layer的**init**函数，修改`deviceId_`为`-2`，代表这个layer是用于跑在MKL-DNN的环境下。
3. 创建`MKLDNNBase`，定义一些除了layer和memory相关的类和函数。
包括MKL-DNN会用到`MKLDNNStream`和`CPUEngine`，和未来可能还会用到`FPGAEngine`等。
包括MKL-DNN会用到`MKLDNNStream`和`CPUEngine`，和未来可能还会用到`FPGAEngine`等。
4. 如果MKL-DNN layer的后面接有cpu device，那么就会使`output_.value`与`extOutVal_`共享内存，
同时数据格式就是`NCHW`，这样下一个cpu device就能拿到正确的数据。
在有普通的CPU layer时， `extOutVal_`和`extOutGrad_`的格式始终是`NCHW`或者`NC`。
## References
1. [MKL small library](https://github.com/01org/mkl-dnn#linking-your-application)是[Intel MKL](https://software.intel.com/en-us/mkl)的一个子集。
主要包括了深度学习相关的数学原语与操作，一般由MKL-DNN在发布[新版本](https://github.com/01org/mkl-dnn/releases)时一起更新。
主要包括了深度学习相关的数学原语与操作，一般由MKL-DNN在发布[新版本](https://github.com/01org/mkl-dnn/releases)时一起更新。
2. [MKL-DNN System Requirements](https://github.com/01org/mkl-dnn#system-requirements)。
目前在PaddlePaddle中，仅会在支持AVX2指令集及以上的机器才使用MKL-DNN。
3. [原来的方案](https://github.com/PaddlePaddle/Paddle/pull/3096)会引入**nextLayer**的信息。
但是在PaddlePaddle中，无论是重构前的layer还是重构后的op，都不会想要知道next layer/op的信息。
但是在PaddlePaddle中，无论是重构前的layer还是重构后的op，都不会想要知道next layer/op的信息。
4. MKL-DNN的高性能格式与PaddlePaddle原有的`NCHW`不同(PaddlePaddle中的cuDNN部分使用的也是`NCHW`，所以不存在这个问题)。
所以需要引入一个转换方法，并且只需要在必要的时候转换这种格式，才能更好的发挥MKL-DNN的性能。
所以需要引入一个转换方法，并且只需要在必要的时候转换这种格式，才能更好的发挥MKL-DNN的性能。
doc/v2/dev/new_layer_en.rst
浏览文件 @ aa2f76fd
......@@ -339,7 +339,7 @@ If you are creating a new file for the test, such as :code:`paddle/legacy/gserve
Implement Python Wrapper
========================
Implementing Python wrapper allows us to use the added layer in configuration files. All the Python wrappers are in file :code:`python/paddle/trainer/config_parser.py`. An example of the Python wrapper for fully connected layer is listed below. It has the following steps:
Implementing Python wrapper allows us to use the added layer in configuration files. All the Python wrappers are in file :code:`python/paddle/legacy/trainer/config_parser.py`. An example of the Python wrapper for fully connected layer is listed below. It has the following steps:
- Use :code:`@config_layer('fc')` at the decorator for all the Python wrapper class. :code:`fc` is the identifier of the layer.
- Implements :code:`__init__` constructor function.
......
paddle/CMakeLists.txt
浏览文件 @ aa2f76fd
......@@ -10,7 +10,7 @@ if(NOT WITH_FLUID_ONLY)
add_subdirectory(legacy/capi)
else()
add_subdirectory(legacy/pserver)
add_subdirectory(trainer)
add_subdirectory(legacy/trainer)
add_subdirectory(scripts)
if(WITH_C_API)
......
paddle/legacy/api/ConfigParser.cpp
浏览文件 @ aa2f76fd
......@@ -14,7 +14,7 @@ limitations under the License. */
#include "PaddleAPI.h"
#include "PaddleAPIPrivate.h"
#include "paddle/trainer/Trainer.h"
#include "paddle/legacy/trainer/Trainer.h"
struct ParameterConfigPrivate {
paddle::ParameterPtr parameter;
......
paddle/legacy/api/PaddleAPIPrivate.h
浏览文件 @ aa2f76fd
......@@ -17,7 +17,7 @@ limitations under the License. */
#include "paddle/legacy/gserver/evaluators/Evaluator.h"
#include "paddle/legacy/gserver/gradientmachines/GradientMachine.h"
#include "paddle/legacy/parameter/ParameterUpdaterBase.h"
#include "paddle/trainer/TrainerConfigHelper.h"
#include "paddle/legacy/trainer/TrainerConfigHelper.h"
struct GradientMachinePrivate {
std::shared_ptr<paddle::GradientMachine> machine;
......
paddle/legacy/api/ParameterUpdater.cpp
浏览文件 @ aa2f76fd
......@@ -16,10 +16,10 @@ limitations under the License. */
#include "PaddleAPIPrivate.h"
#ifndef PADDLE_WITHOUT_GOLANG
#include "paddle/trainer/NewRemoteParameterUpdater.h"
#include "paddle/legacy/trainer/NewRemoteParameterUpdater.h"
#endif
#include "paddle/trainer/RemoteParameterUpdater.h"
#include "paddle/trainer/ThreadParameterUpdater.h"
#include "paddle/legacy/trainer/RemoteParameterUpdater.h"
#include "paddle/legacy/trainer/ThreadParameterUpdater.h"
ParameterUpdater::ParameterUpdater() : m(new ParameterUpdaterPrivate()) {}
......
paddle/legacy/api/Trainer.cpp
浏览文件 @ aa2f76fd
......@@ -20,9 +20,9 @@ limitations under the License. */
#include <memory>
#include "paddle/legacy/gserver/gradientmachines/NeuralNetwork.h"
#include "paddle/trainer/ParamUtil.h"
#include "paddle/trainer/Trainer.h"
#include "paddle/trainer/TrainerInternal.h"
#include "paddle/legacy/trainer/ParamUtil.h"
#include "paddle/legacy/trainer/Trainer.h"
#include "paddle/legacy/trainer/TrainerInternal.h"
#include "paddle/utils/Flags.h"
using paddle::real;
......
paddle/legacy/capi/Main.cpp
浏览文件 @ aa2f76fd
......@@ -18,7 +18,7 @@ limitations under the License. */
#include <vector>
#include "capi_private.h"
#include "main.h"
#include "paddle/trainer/TrainerConfigHelper.h"
#include "paddle/legacy/trainer/TrainerConfigHelper.h"
#include "paddle/utils/Excepts.h"
#include "paddle/utils/PythonUtil.h"
......
paddle/legacy/capi/tests/test_GradientMachine.cpp
浏览文件 @ aa2f76fd
......@@ -14,7 +14,7 @@ limitations under the License. */
#include <gtest/gtest.h>
#include <paddle/legacy/gserver/gradientmachines/GradientMachine.h>
#include <paddle/trainer/TrainerConfigHelper.h>
#include <paddle/legacy/trainer/TrainerConfigHelper.h>
#include <stdlib.h>
#include <string.h>
#include <type_traits>
......
paddle/legacy/gserver/tests/MKLDNNTester.cpp
浏览文件 @ aa2f76fd
......@@ -15,7 +15,7 @@ limitations under the License. */
#include "MKLDNNTester.h"
#include "paddle/legacy/gserver/layers/MKLDNNBase.h"
#include "paddle/legacy/gserver/layers/MKLDNNLayer.h"
#include "paddle/trainer/Trainer.h"
#include "paddle/legacy/trainer/Trainer.h"
namespace paddle {
......
paddle/legacy/gserver/tests/test_CompareSparse.cpp
浏览文件 @ aa2f76fd
......@@ -14,7 +14,7 @@ limitations under the License. */
#include <paddle/utils/PythonUtil.h>
#include "paddle/trainer/Trainer.h"
#include "paddle/legacy/trainer/Trainer.h"
#include <gtest/gtest.h>
#include <paddle/legacy/pserver/ParameterServer2.h>
......
paddle/legacy/gserver/tests/test_CompareTwoNets.cpp
浏览文件 @ aa2f76fd
......@@ -17,7 +17,7 @@ limitations under the License. */
#include <algorithm>
#include <cstdlib>
#include "paddle/trainer/Trainer.h"
#include "paddle/legacy/trainer/Trainer.h"
using namespace paddle; // NOLINT
using namespace std; // NOLINT
......
paddle/legacy/gserver/tests/test_Evaluator.cpp
浏览文件 @ aa2f76fd
......@@ -15,8 +15,8 @@ limitations under the License. */
#include <gtest/gtest.h>
#include <vector>
#include "ModelConfig.pb.h"
#include "paddle/legacy/trainer/Trainer.h"
#include "paddle/testing/TestUtil.h"
#include "paddle/trainer/Trainer.h"
using namespace paddle; // NOLINT
using namespace std; // NOLINT
......
paddle/legacy/gserver/tests/test_NetworkCompare.cpp
浏览文件 @ aa2f76fd
......@@ -18,8 +18,8 @@ limitations under the License. */
#include <algorithm>
#include <cstdlib>
#include "paddle/legacy/trainer/Trainer.h"
#include "paddle/testing/TestUtil.h"
#include "paddle/trainer/Trainer.h"
#include "paddle/utils/Stat.h"
using namespace paddle; // NOLINT
......
paddle/legacy/gserver/tests/test_RecurrentGradientMachine.cpp
浏览文件 @ aa2f76fd
......@@ -15,8 +15,8 @@ limitations under the License. */
#include <gtest/gtest.h>
#include <paddle/legacy/gserver/gradientmachines/GradientMachine.h>
#include <paddle/legacy/parameter/ParameterUpdateFunctions.h>
#include <paddle/trainer/Trainer.h>
#include <paddle/trainer/TrainerInternal.h>
#include <paddle/legacy/trainer/Trainer.h>
#include <paddle/legacy/trainer/TrainerInternal.h>
#include <paddle/utils/PythonUtil.h>
#include <paddle/utils/Util.h>
#include <paddle/utils/Version.h>
......
paddle/trainer/tests/CMakeLists.txt → paddle/legacy/trainer/tests/CMakeLists.txt
浏览文件 @ aa2f76fd
......@@ -5,7 +5,7 @@ add_custom_target(copy_trainer_conf ALL DEPENDS sample_trainer_config.conf)
set(PYTHON_PATH
${PADDLE_SOURCE_DIR}/paddle/.set_python_path.sh -d
${PADDLE_BINARY_DIR}/python/:${PADDLE_BINARY_DIR}/paddle/trainer/tests)
${PADDLE_BINARY_DIR}/python/:${PADDLE_BINARY_DIR}/paddle/legacy/trainer/tests)
function(trainer_test TARGET)
add_unittest_without_exec(${TARGET} ${TARGET}.cpp)
add_test(NAME ${TARGET}
......@@ -33,5 +33,5 @@ endif()
#################### test_config_parser #########################
add_test(NAME test_config_parser
COMMAND ${PYTHON_PATH} ${PYTHON_EXECUTABLE}
${PADDLE_SOURCE_DIR}/paddle/trainer/tests/config_parser_test.py
${PADDLE_SOURCE_DIR}/paddle/legacy/trainer/tests/config_parser_test.py
WORKING_DIRECTORY ${PADDLE_BINARY_DIR}/paddle/)
paddle/trainer/tests/config_parser_test.py → paddle/legacy/trainer/tests/config_parser_test.py
浏览文件 @ aa2f76fd
......@@ -15,9 +15,9 @@
from paddle.trainer.config_parser import parse_config_and_serialize
if __name__ == '__main__':
parse_config_and_serialize('trainer/tests/test_config.conf', '')
parse_config_and_serialize('legacy/trainer/tests/test_config.conf', '')
parse_config_and_serialize(
'trainer/tests/sample_trainer_config.conf',
'legacy/trainer/tests/sample_trainer_config.conf',
'extension_module_name=paddle.trainer.config_parser_extension')
parse_config_and_serialize(
'legacy/gserver/tests/pyDataProvider/trainer.conf', '')
paddle/legacy/trainer/tests/pydata_provider_wrapper_dir/test_pydata_provider_wrapper.list 0 → 100644
浏览文件 @ aa2f76fd
legacy/trainer/tests/pydata_provider_wrapper_dir/test_pydata_provider_wrapper.data
paddle/legacy/trainer/tests/sample_filelist.txt 0 → 100644
浏览文件 @ aa2f76fd
legacy/trainer/tests/sample_data.txt
paddle/trainer/tests/sample_trainer_config.conf → paddle/legacy/trainer/tests/sample_trainer_config.conf
浏览文件 @ aa2f76fd
......@@ -16,13 +16,13 @@
from paddle.trainer_config_helpers import *
TrainData(SimpleData(
files = "trainer/tests/sample_filelist.txt",
files = "legacy/trainer/tests/sample_filelist.txt",
feat_dim = 3,
context_len = 0,
buffer_capacity = 1000000))
TestData(SimpleData(
files = "trainer/tests/sample_filelist.txt",
files = "legacy/trainer/tests/sample_filelist.txt",
feat_dim = 3,
context_len = 0,
buffer_capacity = 1000000))
......
paddle/trainer/tests/sample_trainer_config_hsigmoid.conf → paddle/legacy/trainer/tests/sample_trainer_config_hsigmoid.conf
浏览文件 @ aa2f76fd
......@@ -17,7 +17,7 @@
from paddle.trainer_config_helpers import *
TrainData(SimpleData(
files = "trainer/tests/sample_filelist.txt",
files = "legacy/trainer/tests/sample_filelist.txt",
feat_dim = 3,
context_len = 0,
buffer_capacity = 1000000,
......
paddle/trainer/tests/sample_trainer_config_parallel.conf → paddle/legacy/trainer/tests/sample_trainer_config_parallel.conf
浏览文件 @ aa2f76fd
......@@ -16,13 +16,13 @@
from paddle.trainer_config_helpers import *
TrainData(SimpleData(
files = "trainer/tests/sample_filelist.txt",
files = "legacy/trainer/tests/sample_filelist.txt",
feat_dim = 3,
context_len = 0,
buffer_capacity = 1000000))
TestData(SimpleData(
files = "trainer/tests/sample_filelist.txt",
files = "legacy/trainer/tests/sample_filelist.txt",
feat_dim = 3,
context_len = 0,
buffer_capacity = 1000000))
......
paddle/trainer/tests/sample_trainer_nest_rnn_gen.conf → paddle/legacy/trainer/tests/sample_trainer_nest_rnn_gen.conf
浏览文件 @ aa2f76fd
......@@ -63,8 +63,8 @@ beam_gen_concat = recurrent_group(name="rnn_gen_concat",
seqtext_printer_evaluator(input=beam_gen_concat,
id_input=sent_id,
dict_file="./trainer/tests/test_gen_dict.txt",
result_file="./trainer/tests/dump_text.test")
dict_file="./legacy/trainer/tests/test_gen_dict.txt",
result_file="./legacy/trainer/tests/dump_text.test")
#outputs(beam_gen_concat)
# In this config, as dummy_data_input doesn't work on beam_gen (we can find dummy_memory
# is read-only memory, and isn't used by other layers of step), we show the Inputs and Outputs
......
paddle/trainer/tests/sample_trainer_rnn_gen.conf → paddle/legacy/trainer/tests/sample_trainer_rnn_gen.conf
浏览文件 @ aa2f76fd
......@@ -56,8 +56,8 @@ beam_gen = beam_search(name="rnn_gen",
seqtext_printer_evaluator(input=beam_gen,
id_input=sent_id,
dict_file="./trainer/tests/test_gen_dict.txt",
result_file="./trainer/tests/dump_text.test")
dict_file="./legacy/trainer/tests/test_gen_dict.txt",
result_file="./legacy/trainer/tests/dump_text.test")
#outputs(beam_gen)
# In this config, as dummy_data_input doesn't work on beam_gen (we can find dummy_memory
# is read-only memory, and isn't used by other layers of step), we show the Inputs and Outputs
......
paddle/trainer/tests/simple_sparse_neural_network.py → paddle/legacy/trainer/tests/simple_sparse_neural_network.py
浏览文件 @ aa2f76fd
......@@ -16,7 +16,7 @@ from paddle.trainer_config_helpers import *
settings(batch_size=17, learning_method=AdaGradOptimizer(), learning_rate=1e-4)
file_list = 'trainer/tests/fake_file_list.list'
file_list = 'legacy/trainer/tests/fake_file_list.list'
define_py_data_sources2(
train_list=file_list,
......
paddle/trainer/tests/test_Compare.cpp → paddle/legacy/trainer/tests/test_Compare.cpp
浏览文件 @ aa2f76fd
......@@ -14,7 +14,7 @@ limitations under the License. */
#include <paddle/utils/PythonUtil.h>
#include "paddle/trainer/Trainer.h"
#include "paddle/legacy/trainer/Trainer.h"
#include <gtest/gtest.h>
#include <cstdlib>
......@@ -22,7 +22,8 @@ limitations under the License. */
using namespace paddle; // NOLINT
using namespace std; // NOLINT
static const string& configFile = "trainer/tests/sample_trainer_config.conf";
static const string& configFile =
"/legacy/trainer/tests/sample_trainer_config.conf";
DECLARE_int32(gpu_id);
DECLARE_bool(use_gpu);
......
paddle/trainer/tests/test_PyDataProviderWrapper.cpp → paddle/legacy/trainer/tests/test_PyDataProviderWrapper.cpp
浏览文件 @ aa2f76fd
......@@ -26,7 +26,7 @@ limitations under the License. */
#include "picojson.h"
void checkValue(std::vector<paddle::Argument>& arguments, picojson::array& arr);
const std::string kDir = "./trainer/tests/pydata_provider_wrapper_dir/";
const std::string kDir = "./legacy/trainer/tests/pydata_provider_wrapper_dir/";
TEST(PyDataProviderWrapper, SequenceData) {
paddle::DataConfig conf;
......
paddle/trainer/tests/test_Trainer.cpp → paddle/legacy/trainer/tests/test_Trainer.cpp
浏览文件 @ aa2f76fd
......@@ -14,18 +14,19 @@ limitations under the License. */
#include <paddle/utils/PythonUtil.h>
#include <paddle/utils/Version.h>
#include "paddle/trainer/Trainer.h"
#include "paddle/legacy/trainer/Trainer.h"
#include <gtest/gtest.h>
using namespace paddle; // NOLINT
using namespace std; // NOLINT
static const string& configFile1 = "trainer/tests/sample_trainer_config.conf";
static const string& configFile1 =
"legacy/trainer/tests/sample_trainer_config.conf";
static const string& configFile2 =
"trainer/tests/sample_trainer_config_hsigmoid.conf";
"legacy/trainer/tests/sample_trainer_config_hsigmoid.conf";
static const string& configFile4 =
"trainer/tests/sample_trainer_config_parallel.conf";
"legacy/trainer/tests/sample_trainer_config_parallel.conf";
DECLARE_bool(use_gpu);
DECLARE_string(config);
......
paddle/trainer/tests/test_TrainerOnePass.cpp → paddle/legacy/trainer/tests/test_TrainerOnePass.cpp
浏览文件 @ aa2f76fd
......@@ -14,8 +14,8 @@ limitations under the License. */
#include <paddle/utils/GlobalConstants.h>
#include <paddle/utils/PythonUtil.h>
#include "paddle/trainer/Trainer.h"
#include "paddle/trainer/TrainerInternal.h"
#include "paddle/legacy/trainer/Trainer.h"
#include "paddle/legacy/trainer/TrainerInternal.h"
#include <gtest/gtest.h>
#include <paddle/legacy/pserver/ParameterServer2.h>
......@@ -23,12 +23,13 @@ limitations under the License. */
using namespace paddle; // NOLINT
using namespace std; // NOLINT
static const string& configFile1 = "trainer/tests/sample_trainer_config.conf";
static const string& configFile1 =
"legacy/trainer/tests/sample_trainer_config.conf";
static const string& configFile2 =
"trainer/tests/sample_trainer_config_parallel.conf";
"legacy/trainer/tests/sample_trainer_config_parallel.conf";
static const string& configFileSimpleSparse =
"trainer/tests/simple_sparse_neural_network.py";
"legacy/trainer/tests/simple_sparse_neural_network.py";
DECLARE_bool(use_gpu);
DECLARE_string(config);
......
paddle/trainer/tests/test_config.conf → paddle/legacy/trainer/tests/test_config.conf
浏览文件 @ aa2f76fd
......@@ -16,7 +16,7 @@
from paddle.trainer_config_helpers import *
TrainData(SimpleData(
files = "trainer/tests/sample_filelist.txt",
files = "legacy/trainer/tests/sample_filelist.txt",
feat_dim = 3,
context_len = 0,
buffer_capacity = 1000000,
......
paddle/trainer/tests/test_recurrent_machine_generation.cpp → paddle/legacy/trainer/tests/test_recurrent_machine_generation.cpp
浏览文件 @ aa2f76fd
......@@ -14,7 +14,7 @@ limitations under the License. */
#include <fstream>
#include <paddle/trainer/Trainer.h>
#include <paddle/legacy/trainer/Trainer.h>
#include <paddle/utils/PythonUtil.h>
#include <gtest/gtest.h>
......@@ -22,13 +22,15 @@ limitations under the License. */
using namespace paddle; // NOLINT
using namespace std; // NOLINT
static const string& CONFIG_FILE = "trainer/tests/sample_trainer_rnn_gen.conf";
static const string& CONFIG_FILE =
"legacy/trainer/tests/sample_trainer_rnn_gen.conf";
static const string& NEST_CONFIG_FILE =
"trainer/tests/sample_trainer_nest_rnn_gen.conf";
static const string& OUTPUT_DIR = "trainer/tests/dump_text.test";
static string modelDir = "trainer/tests/rnn_gen_test_model_dir/t1"; // NOLINT
static string expectFile = // NOLINT
"trainer/tests/rnn_gen_test_model_dir/r1.test"; // NOLINT
"legacy/trainer/tests/sample_trainer_nest_rnn_gen.conf";
static const string& OUTPUT_DIR = "legacy/trainer/tests/dump_text.test";
static string modelDir =
"legacy/trainer/tests/rnn_gen_test_model_dir/t1"; // NOLINT
static string expectFile = // NOLINT
"legacy/trainer/tests/rnn_gen_test_model_dir/r1.test"; // NOLINT
DECLARE_string(config_args);
......
paddle/trainer/tests/pydata_provider_wrapper_dir/test_pydata_provider_wrapper.list 已删除 100644 → 0
浏览文件 @ da158c22
trainer/tests/pydata_provider_wrapper_dir/test_pydata_provider_wrapper.data
python/paddle/trainer/config_parser.py
浏览文件 @ aa2f76fd
......@@ -67,7 +67,7 @@ extension_module_name=[MODULE_NAME], then config_parser will call
MODULE_NAME.get_config_funcs(g_config)
MODULE_NAME.get_config_funcs() should return a dictionary of name to functions,
those functions will be available in the config file.
See trainer/tests/config_parser_test.py for example
See legacy/trainer/tests/config_parser_test.py for example
To use this from paddle_trainer, paddle_trainer should be called with
--config_args=extension_module_name=[MODULE_NAME]
......
python/setup.py.in
浏览文件 @ aa2f76fd
......@@ -93,8 +93,8 @@ if '${CMAKE_SYSTEM_PROCESSOR}' not in ['arm', 'armv7-a', 'aarch64']:
paddle_bins = ''
if '${WITH_FLUID_ONLY}'== 'OFF':
paddle_bin_dir = 'opt/paddle/bin'
paddle_bins = ['${PADDLE_BINARY_DIR}/paddle/trainer/paddle_trainer',
'${PADDLE_BINARY_DIR}/paddle/trainer/paddle_merge_model',
paddle_bins = ['${PADDLE_BINARY_DIR}/paddle/legacy/trainer/paddle_trainer',
'${PADDLE_BINARY_DIR}/paddle/legacy/trainer/paddle_merge_model',
'${PADDLE_BINARY_DIR}/paddle/legacy/pserver/paddle_pserver_main',
'${PADDLE_BINARY_DIR}/paddle/scripts/paddle']
......
tools/codestyle/cpplint_pre_commit.hook
浏览文件 @ aa2f76fd
......@@ -4,7 +4,7 @@ TOTAL_ERRORS=0
# The trick to remove deleted files: https://stackoverflow.com/a/2413151
for file in $(git diff --cached --name-status | awk '$1 != "D" {print $2}'); do
if [[ $file =~ ^(paddle/api/.*|paddle/capi/.*|paddle/contrib/.*|paddle/legacy/cuda/.*|paddle/legacy/function/.*|paddle/legacy/gserver/.*|paddle/legacy/math/.*|paddle/legacy/optimizer/.*|paddle/legacy/parameter/.*|paddle/legacy/pserver/.*|paddle/trainer/.*|paddle/utils/.*|paddle/testing/TestUtil.*) ]]; then
if [[ $file =~ ^(paddle/legacy/api/.*|paddle/legacy/capi/.*|paddle/contrib/.*|paddle/legacy/cuda/.*|paddle/legacy/function/.*|paddle/legacy/gserver/.*|paddle/legacy/math/.*|paddle/legacy/optimizer/.*|paddle/legacy/parameter/.*|paddle/legacy/pserver/.*|paddle/legacy/trainer/.*|paddle/utils/.*|paddle/testing/TestUtil.*) ]]; then
continue;
else
cpplint --filter=-readability/fn_size $file;
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或