Skip to content

P
Paddle

SummerGao. / Paddle
与 Fork 源项目一致

Fork自 PaddlePaddle / Paddle

通知 1
Star 0
Fork 0
P
Paddle

体验新版 GitCode，发现更多精彩内容 >>

提交 60d827a8 编写于 作者: Y Yancey1989
浏览文件
操作

Merge branch 'develop' of github.com:PaddlePaddle/Paddle into overlap_send_op

上级 20c24c05 8f7b020b
展开全部 隐藏空白更改
内联 并排
Showing with 4154 addition and 1158 deletion
.clang-format
浏览文件 @ 60d827a8
......@@ -19,7 +19,7 @@ BasedOnStyle: Google
IndentWidth: 2
TabWidth: 2
ContinuationIndentWidth: 4
AccessModifierOffset: -2 # The private/protected/public has no indent in class
AccessModifierOffset: -1 # The private/protected/public has no indent in class
Standard: Cpp11
AllowAllParametersOfDeclarationOnNextLine: true
BinPackParameters: false
......
.pre-commit-config.yaml
浏览文件 @ 60d827a8
......@@ -34,6 +34,14 @@ repos:
entry: bash ./tools/codestyle/cpplint_pre_commit.hook
language: system
files: \.(c|cc|cxx|cpp|cu|h|hpp|hxx)$
- repo: local
hooks:
- id: pylint-doc-string
name: pylint
description: Check python docstring style using docstring_checker.
entry: bash ./tools/codestyle/pylint_pre_commit.hook
language: system
files: \.(py)$
- repo: https://github.com/PaddlePaddle/pre-commit-golang
sha: 8337620115c25ff8333f1b1a493bd031049bd7c0
hooks:
......
.travis.yml
浏览文件 @ 60d827a8
......@@ -18,6 +18,8 @@ env:
addons:
ssh_known_hosts: 13.229.163.131
before_install:
# For pylint dockstring checker
- sudo pip install pylint pytest astroid isort
- |
function timeout() { perl -e 'alarm shift; exec @ARGV' "$@"; }
script:
......
Dockerfile
浏览文件 @ 60d827a8
......@@ -79,6 +79,9 @@ RUN pip install pre-commit 'ipython==5.3.0' && \
pip install 'ipykernel==4.6.0' 'jupyter==1.0.0' && \
pip install opencv-python
#For docstring checker
RUN pip install pylint pytest astroid isort
COPY ./python/requirements.txt /root/
RUN pip install -r /root/requirements.txt
......
benchmark/fluid/README.md
浏览文件 @ 60d827a8
......@@ -24,22 +24,22 @@ Currently supported `--model` argument include:
* Run the following command to start a benchmark job locally:
```bash
python fluid_benchmark.py --model mnist --parallel 1 --device GPU --with_test
python fluid_benchmark.py --model mnist --device GPU
```
You can choose to use GPU/CPU training. With GPU training, you can specify
`--parallel 1` to run multi GPU training.
`--gpus <gpu_num>` to run multi GPU training.
* Run distributed training with parameter servers:
* start parameter servers:
```bash
PADDLE_TRAINING_ROLE=PSERVER PADDLE_PSERVER_PORT=7164 PADDLE_PSERVER_IPS=127.0.0.1 PADDLE_TRAINERS=1 PADDLE_CURRENT_IP=127.0.0.1 PADDLE_TRAINER_ID=0 python fluid_benchmark.py --model mnist --parallel 0 --device GPU --update_method pserver
PADDLE_TRAINING_ROLE=PSERVER PADDLE_PSERVER_PORT=7164 PADDLE_PSERVER_IPS=127.0.0.1 PADDLE_TRAINERS=1 PADDLE_CURRENT_IP=127.0.0.1 PADDLE_TRAINER_ID=0 python fluid_benchmark.py --model mnist --device GPU --update_method pserver
```
* start trainers:
```bash
PADDLE_TRAINING_ROLE=PSERVER PADDLE_PSERVER_PORT=7164 PADDLE_PSERVER_IPS=127.0.0.1 PADDLE_TRAINERS=1 PADDLE_CURRENT_IP=127.0.0.1 PADDLE_TRAINER_ID=0 python fluid_benchmark.py --model mnist --parallel 0 --device GPU --update_method pserver
PADDLE_TRAINING_ROLE=TRAINER PADDLE_PSERVER_PORT=7164 PADDLE_PSERVER_IPS=127.0.0.1 PADDLE_TRAINERS=1 PADDLE_CURRENT_IP=127.0.0.1 PADDLE_TRAINER_ID=0 python fluid_benchmark.py --model mnist --device GPU --update_method pserver
```
* Run distributed training using NCCL2
```bash
PADDLE_PSERVER_PORT=7164 PADDLE_TRAINER_IPS=192.168.0.2,192.168.0.3 PADDLE_CURRENT_IP=127.0.0.1 PADDLE_TRAINER_ID=0 python fluid_benchmark.py --model mnist --parallel 0 --device GPU --update_method nccl2
PADDLE_PSERVER_PORT=7164 PADDLE_TRAINER_IPS=192.168.0.2,192.168.0.3 PADDLE_CURRENT_IP=127.0.0.1 PADDLE_TRAINER_ID=0 python fluid_benchmark.py --model mnist --device GPU --update_method nccl2
```
## Run Distributed Benchmark on Kubernetes Cluster
......@@ -48,7 +48,7 @@ We provide a script `kube_gen_job.py` to generate Kubernetes yaml files to submi
distributed benchmark jobs to your cluster. To generate a job yaml, just run:
```bash
python kube_gen_job.py --jobname myjob --pscpu 4 --cpu 8 --gpu 8 --psmemory 20 --memory 40 --pservers 4 --trainers 4 --entry "python fluid_benchmark.py --model mnist --parallel 1 --device GPU --update_method pserver --with_test" --disttype pserver
python kube_gen_job.py --jobname myjob --pscpu 4 --cpu 8 --gpu 8 --psmemory 20 --memory 40 --pservers 4 --trainers 4 --entry "python fluid_benchmark.py --model mnist --parallel 1 --device GPU --update_method pserver " --disttype pserver
```
Then the yaml files are generated under directory `myjob`, you can run:
......
doc/v2/build_and_install/docker_install_cn.rst
浏览文件 @ 60d827a8
......@@ -98,7 +98,7 @@ PaddlePaddle Book是为用户和开发者制作的一个交互式的Jupyter Note
国内用户可以使用下面的镜像源来加速访问：
.. code-block: bash
.. code-block:: bash
docker run -p 8888:8888 docker.paddlepaddlehub.com/book
......
doc/v2/build_and_install/docker_install_en.rst
浏览文件 @ 60d827a8
......@@ -105,7 +105,7 @@ We provide a packaged book image, simply issue the command:
For users in China, we provide a faster mirror:
.. code-block: bash
.. code-block:: bash
docker run -p 8888:8888 docker.paddlepaddlehub.com/book
......
paddle/api/GradientMachine.cpp
浏览文件 @ 60d827a8
......@@ -94,7 +94,7 @@ void UpdateCallback::apply(Parameter* p) {
}
class UpdateCallbackWrapper {
public:
public:
explicit UpdateCallbackWrapper(const UpdateCallback& callback)
: callback(const_cast<UpdateCallback&>(callback)) {}
......@@ -105,7 +105,7 @@ public:
delete p;
}
private:
private:
UpdateCallback& callback;
};
......
paddle/api/PaddleAPI.h
浏览文件 @ 60d827a8
......@@ -59,9 +59,10 @@ class RangeError {};
/// Not support Error, such as access GPU memory directly, etc.
class UnsupportError : public std::runtime_error {
public:
UnsupportError() : std::runtime_error(" "){};
UnsupportError(const std::string& message) : std::runtime_error(message){};
public:
UnsupportError() : std::runtime_error(" ") {}
explicit UnsupportError(const std::string& message)
: std::runtime_error(message) {}
};
/// This type will map to python's list of float.
......@@ -105,7 +106,7 @@ class Matrix {
DISABLE_COPY(Matrix);
static Matrix* createByPaddleMatrixPtr(void* sharedPtr);
public:
public:
virtual ~Matrix();
/**
......@@ -231,7 +232,7 @@ public:
bool isGpu() const;
private:
private:
void* getSharedPtr() const;
MatrixPrivate* m;
......@@ -248,7 +249,7 @@ class Vector {
void* getSharedPtr();
public:
public:
~Vector();
/// Create Vector filled with zero.
......@@ -310,10 +311,10 @@ public:
/// __len__ in python
size_t getSize() const;
private:
private:
VectorPrivate* m;
private:
private:
friend class Parameter;
friend class ParameterOptimizer;
friend struct ParameterTraverseCallbackPrivate;
......@@ -325,7 +326,7 @@ class IVector {
DISABLE_COPY(IVector);
static IVector* createByPaddleVectorPtr(void* ptr);
public:
public:
/// Create IVector filled with zero
static IVector* createZero(size_t sz, bool useGpu = isUsingGpu());
......@@ -389,7 +390,7 @@ public:
/// This method will map to python __len__();
size_t getSize() const;
private:
private:
void* getSharedPtr() const;
friend class Arguments;
......@@ -400,11 +401,11 @@ struct ArgumentsPrivate;
/// The Arguments is actual a std::vector<paddle::Argument> in paddle.
class Arguments {
private:
private:
Arguments(); // Internal Create.
DISABLE_COPY(Arguments);
public:
public:
/**
* Create a arguments with size.
* Note that it can be zero.
......@@ -475,12 +476,12 @@ public:
float sum() const;
private:
private:
static Arguments* createByPaddleArgumentVector(void* ptr);
static Arguments* createByPaddleArgument(const void* ptr);
void* getInternalArgumentsPtr() const;
private:
private:
ArgumentsPrivate* m;
friend class Trainer;
friend class GradientMachine;
......@@ -507,7 +508,7 @@ class ParameterConfig {
static ParameterConfig* createParameterConfigFromParameterPtr(void* ptr);
void* getRawPtr();
public:
public:
~ParameterConfig();
/**
......@@ -515,10 +516,10 @@ public:
*/
std::string toProtoString() const;
private:
private:
ParameterConfigPrivate* m;
private:
private:
friend class Parameter;
friend class ParameterOptimizer;
friend struct ParameterTraverseCallbackPrivate;
......@@ -529,7 +530,7 @@ class OptimizationConfig {
DISABLE_COPY(OptimizationConfig);
OptimizationConfig();
public:
public:
static OptimizationConfig* createFromProtoString(const std::string& str);
~OptimizationConfig();
......@@ -538,7 +539,7 @@ public:
*/
std::string toProtoString();
private:
private:
OptimizationConfigPrivate* m;
friend class TrainerConfig;
......@@ -549,11 +550,11 @@ private:
struct ParameterPrivate;
class Parameter {
private:
private:
Parameter();
DISABLE_COPY(Parameter);
public:
public:
virtual ~Parameter();
/**
......@@ -580,11 +581,11 @@ public:
size_t getSize() const;
private:
private:
static Parameter* createFromRawPtr(void* ptr);
static Parameter* createFromSharedPtr(void* ptr);
private:
private:
ParameterPrivate* m;
friend class UpdateCallbackWrapper;
friend class GradientMachine;
......@@ -598,14 +599,14 @@ struct ModelConfigPrivate;
* It is used by GradientMachine.
*/
class ModelConfig {
private:
private:
ModelConfig();
DISABLE_COPY(ModelConfig);
public:
public:
virtual ~ModelConfig();
private:
private:
ModelConfigPrivate* m;
friend class TrainerConfig;
friend struct TrainerConfigPrivate;
......@@ -619,11 +620,11 @@ struct TrainerConfigPrivate;
* It is used by GradientMachine.
*/
class TrainerConfig {
private:
private:
TrainerConfig();
DISABLE_COPY(TrainerConfig);
public:
public:
virtual ~TrainerConfig();
static TrainerConfig* createFromTrainerConfigFile(
......@@ -634,7 +635,7 @@ public:
OptimizationConfig* getOptimizationConfig() const;
private:
private:
TrainerConfigPrivate* m;
friend class Trainer;
};
......@@ -654,7 +655,7 @@ private:
* @endcode
*/
class UpdateCallback {
public:
public:
virtual ~UpdateCallback();
virtual void apply(Parameter* p);
};
......@@ -664,14 +665,14 @@ class ParameterTraverseCallback {
DISABLE_COPY(ParameterTraverseCallback);
ParameterTraverseCallback();
public:
public:
~ParameterTraverseCallback();
void apply(const std::vector<Vector*>& vecs,
const ParameterConfig& config,
size_t sparseId);
private:
private:
ParameterTraverseCallbackPrivate* m;
friend class ParameterOptimizer;
};
......@@ -686,7 +687,7 @@ class ParameterOptimizer {
DISABLE_COPY(ParameterOptimizer);
ParameterOptimizer();
public:
public:
static ParameterOptimizer* create(OptimizationConfig* config);
~ParameterOptimizer();
......@@ -710,7 +711,7 @@ public:
ParameterTraverseCallback* needSpecialTraversal(
const ParameterConfig& config) const;
private:
private:
ParameterOptimizerPrivate* m;
};
......@@ -718,11 +719,11 @@ class SequenceGenerator;
class Evaluator;
struct GradientMachinePrivate;
class GradientMachine {
private:
private:
GradientMachine();
DISABLE_COPY(GradientMachine);
public:
public:
virtual ~GradientMachine();
/**
......@@ -817,7 +818,7 @@ public:
void eval(Evaluator* evaluator);
private:
private:
GradientMachinePrivate* m;
static GradientMachine* createFromPaddleModelPtr(
......@@ -833,10 +834,10 @@ private:
struct ParameterUpdaterPrivate;
class ParameterUpdater {
private:
private:
ParameterUpdater();
public:
public:
static ParameterUpdater* createLocalUpdater(OptimizationConfig* config);
static ParameterUpdater* createRemoteUpdater(OptimizationConfig* config,
int passCount,
......@@ -911,17 +912,17 @@ public:
*/
void catchUpWith();
private:
private:
ParameterUpdaterPrivate* m;
};
struct EvaluatorPrivate;
class Evaluator {
private:
private:
Evaluator();
DISABLE_COPY(Evaluator);
public:
public:
~Evaluator();
/**
......@@ -945,7 +946,7 @@ public:
double getValue(const std::string name) const;
private:
private:
EvaluatorPrivate* m;
friend class GradientMachine;
......@@ -953,13 +954,13 @@ private:
struct TrainerPrivate;
class Trainer {
private:
private:
TrainerPrivate* m;
Trainer();
Trainer(TrainerConfig* optConfig, GradientMachine* gm);
DISABLE_COPY(Trainer);
public:
public:
virtual ~Trainer();
/// Create A Trainer By TrainerConfig. using paddle command line.
......@@ -1002,7 +1003,7 @@ public:
/// the N-Best results generated from one input sequence.
class ISequenceResults {
public:
public:
virtual ~ISequenceResults();
/// Number of result.
......@@ -1026,7 +1027,7 @@ class SequenceGenerator {
DISABLE_COPY(SequenceGenerator);
SequenceGenerator();
public:
public:
virtual ~SequenceGenerator();
/**
......@@ -1044,10 +1045,10 @@ public:
void setMaxLength(size_t maxlength);
void setBeamSize(size_t beamSize);
private:
private:
static SequenceGenerator* createByGradientMachineSharedPtr(void* ptr);
friend class GradientMachine;
private:
private:
SequenceGeneratorPrivate* m;
};
paddle/api/SequenceGenerator.cpp
浏览文件 @ 60d827a8
......@@ -138,7 +138,7 @@ struct SequenceGeneratorPrivate {
maxLength(0UL),
feedback(__create_feedback__()) {}
private:
private:
static paddle::Argument __create_feedback__() {
paddle::Argument feedback;
feedback.ids = paddle::IVector::create(/* size= */ 1, FLAGS_use_gpu);
......@@ -157,7 +157,7 @@ SequenceGenerator::~SequenceGenerator() { delete m; }
class PathSequenceResults : public ISequenceResults {
// ISequenceResults interface
public:
public:
PathSequenceResults(const std::shared_ptr<std::vector<Path>>& path,
const std::shared_ptr<std::vector<std::string>>& dict)
: path_(path), dict_(dict) {}
......@@ -196,7 +196,7 @@ public:
}
}
private:
private:
std::shared_ptr<std::vector<Path>> path_;
std::shared_ptr<std::vector<std::string>> dict_;
};
......
paddle/capi/gradient_machine.cpp
浏览文件 @ 60d827a8
......@@ -26,7 +26,7 @@ enum GradientMatchineCreateMode {
namespace paddle {
class MyNeuralNetwork : public NeuralNetwork {
public:
public:
MyNeuralNetwork(const std::string& name, NeuralNetwork* network)
: NeuralNetwork(name, network) {}
};
......
paddle/contrib/inference/CMakeLists.txt
浏览文件 @ 60d827a8
......@@ -13,10 +13,45 @@
# limitations under the License.
#
function(inference_api_test TARGET_NAME TEST_SRC DEP_TEST)
set(options "")
set(oneValueArgs "")
set(multiValueArgs ARGS)
cmake_parse_arguments(inference_test "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})
set(PYTHON_TESTS_DIR ${PADDLE_BINARY_DIR}/python/paddle/fluid/tests)
set(arg_list "")
if(inference_test_ARGS)
foreach(arg ${inference_test_ARGS})
list(APPEND arg_list "_${arg}")
endforeach()
else()
list(APPEND arg_list "_")
endif()
foreach(arg ${arg_list})
string(REGEX REPLACE "^_$" "" arg "${arg}")
cc_test(${TARGET_NAME}
SRCS ${TEST_SRC}
DEPS paddle_fluid_api paddle_inference_api paddle_inference_api_impl
ARGS --dirname=${PYTHON_TESTS_DIR}/book/)
# set_tests_properties(${TARGET_NAME}
# PROPERTIES DEPENDS ${DEP_TEST})
endforeach()
endfunction(inference_api_test)
cc_library(paddle_inference_api
SRCS paddle_inference_api.cc
DEPS ${FLUID_CORE_MODULES} ${GLOB_OP_LIB})
cc_library(paddle_inference_api_impl
SRCS paddle_inference_api_impl.cc
DEPS paddle_inference_api paddle_fluid_api)
cc_test(test_paddle_inference_api
SRCS test_paddle_inference_api.cc
DEPS paddle_inference_api)
inference_api_test(test_paddle_inference_api_impl
test_paddle_inference_api_impl.cc
test_word2vec)
paddle/contrib/inference/paddle_inference_api.h
浏览文件 @ 60d827a8
......@@ -27,29 +27,38 @@
namespace paddle {
enum PaddleDType {
FLOAT32,
INT64,
};
struct PaddleBuf {
void* data; // pointer to the data memory.
size_t length; // number of memory bytes.
};
struct PaddleTensor {
std::string name; // variable name.
std::vector<int> shape;
std::vector<unsigned char> data; // bytes of data.
size_t type{typeid(float).hash_code()}; // hash of type
PaddleBuf data; // blob of data.
PaddleDType dtype;
};
/*
* A simple Inference API for Paddle. Currently this API might just be used by
* non-sequence scenerios.
* TODO(Superjomn) Prepare another API for NLP-related usages.
*/
* A simple Inference API for Paddle. Currently this API might just be used by
* non-sequence scenerios.
* TODO(Superjomn) Prepare another API for NLP-related usages.
*/
class PaddlePredictor {
public:
public:
struct Config;
PaddlePredictor() = default;
PaddlePredictor(const PaddlePredictor&) = delete;
// One drived class should has such a constructor
// PaddlePredictor(const XConfig& config);
// The XConfig is a derived class of Config.
// Predict an record.
// The caller should be responsible for allocating and releasing the memory of
// `inputs`. `inputs` should be alive until Run returns. caller should be
// responsible for releasing the memory of `output_data`.
virtual bool Run(const std::vector<PaddleTensor>& inputs,
std::vector<PaddleTensor>* output_data) = 0;
......@@ -57,6 +66,7 @@ public:
// be thread-safe.
virtual std::unique_ptr<PaddlePredictor> Clone() = 0;
virtual bool InitShared() { return false; }
// Destroy the Predictor.
virtual ~PaddlePredictor() {}
......
paddle/contrib/inference/paddle_inference_api_impl.cc 0 → 100644
浏览文件 @ 60d827a8
/* Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. */
#include <sys/time.h>
#include <algorithm>
#include <map>
#include <set>
#include <sstream>
#include <string>
#include <utility>
#include <vector>
#include "paddle/contrib/inference/paddle_inference_api_impl.h"
namespace paddle {
namespace {
// Timer for timer
class Timer {
public:
double start;
double startu;
void tic() {
struct timeval tp;
gettimeofday(&tp, NULL);
start = tp.tv_sec;
startu = tp.tv_usec;
}
double toc() {
struct timeval tp;
gettimeofday(&tp, NULL);
double used_time_ms =
(tp.tv_sec - start) * 1000.0 + (tp.tv_usec - startu) / 1000.0;
return used_time_ms;
}
};
template <class T>
std::string num2str(T a) {
std::stringstream istr;
istr << a;
return istr.str();
}
} // namespace
bool PaddlePredictorImpl::Init() {
VLOG(3) << "Predictor::init()";
// TODO(panyx0718): Should CPU vs GPU device be decided by id?
if (config_.device >= 0) {
place_ = paddle::platform::CUDAPlace(config_.device);
} else {
place_ = paddle::platform::CPUPlace();
}
paddle::framework::InitDevices(false);
executor_.reset(new paddle::framework::Executor(place_));
scope_.reset(new paddle::framework::Scope());
// Initialize the inference program
if (!config_.model_dir.empty()) {
// Parameters are saved in separate files sited in
// the specified `dirname`.
inference_program_ = paddle::inference::Load(
executor_.get(), scope_.get(), config_.model_dir);
} else if (!config_.prog_file.empty() && !config_.param_file.empty()) {
// All parameters are saved in a single file.
// The file names should be consistent with that used
// in Python API `fluid.io.save_inference_model`.
inference_program_ = paddle::inference::Load(
executor_.get(), scope_.get(), config_.prog_file, config_.param_file);
} else {
LOG(ERROR) << "fail to load inference model.";
return false;
}
ctx_ = executor_->Prepare(*inference_program_, 0);
// Create variables
// TODO(panyx0718): Why need to test share_variables here?
if (config_.share_variables) {
executor_->CreateVariables(*inference_program_, scope_.get(), 0);
}
// Get the feed_target_names and fetch_target_names
feed_target_names_ = inference_program_->GetFeedTargetNames();
fetch_target_names_ = inference_program_->GetFetchTargetNames();
return true;
}
bool PaddlePredictorImpl::Run(const std::vector<PaddleTensor> &inputs,
std::vector<PaddleTensor> *output_data) {
VLOG(3) << "Predictor::predict";
Timer timer;
timer.tic();
// set feed variable
std::map<std::string, const paddle::framework::LoDTensor *> feed_targets;
std::vector<paddle::framework::LoDTensor> feeds;
if (!SetFeed(inputs, &feeds)) {
LOG(ERROR) << "fail to set feed";
return false;
}
for (size_t i = 0; i < feed_target_names_.size(); ++i) {
feed_targets[feed_target_names_[i]] = &feeds[i];
}
// get fetch variable
std::map<std::string, paddle::framework::LoDTensor *> fetch_targets;
std::vector<paddle::framework::LoDTensor> fetchs;
fetchs.resize(fetch_target_names_.size());
for (size_t i = 0; i < fetch_target_names_.size(); ++i) {
fetch_targets[fetch_target_names_[i]] = &fetchs[i];
}
// Run the inference program
// if share variables, we need not create variables
executor_->RunPreparedContext(ctx_.get(),
scope_.get(),
&feed_targets,
&fetch_targets,
!config_.share_variables);
if (!GetFetch(fetchs, output_data)) {
LOG(ERROR) << "fail to get fetchs";
return false;
}
VLOG(3) << "predict cost: " << timer.toc() << "ms";
return true;
}
std::unique_ptr<PaddlePredictor> PaddlePredictorImpl::Clone() {
VLOG(3) << "Predictor::clone";
std::unique_ptr<PaddlePredictor> cls(new PaddlePredictorImpl(config_));
if (!cls->InitShared()) {
LOG(ERROR) << "fail to call InitShared";
return nullptr;
}
return cls;
}
// TODO(panyx0718): Consider merge with Init()?
bool PaddlePredictorImpl::InitShared() {
VLOG(3) << "Predictor::init_shared";
// 1. Define place, executor, scope
if (this->config_.device >= 0) {
place_ = paddle::platform::CUDAPlace();
} else {
place_ = paddle::platform::CPUPlace();
}
this->executor_.reset(new paddle::framework::Executor(this->place_));
this->scope_.reset(new paddle::framework::Scope());
// Initialize the inference program
if (!this->config_.model_dir.empty()) {
// Parameters are saved in separate files sited in
// the specified `dirname`.
this->inference_program_ = paddle::inference::Load(
this->executor_.get(), this->scope_.get(), this->config_.model_dir);
} else if (!this->config_.prog_file.empty() &&
!this->config_.param_file.empty()) {
// All parameters are saved in a single file.
// The file names should be consistent with that used
// in Python API `fluid.io.save_inference_model`.
this->inference_program_ =
paddle::inference::Load(this->executor_.get(),
this->scope_.get(),
this->config_.prog_file,
this->config_.param_file);
}
this->ctx_ = this->executor_->Prepare(*this->inference_program_, 0);
// 3. create variables
// TODO(panyx0718): why test share_variables.
if (config_.share_variables) {
this->executor_->CreateVariables(
*this->inference_program_, this->scope_.get(), 0);
}
// 4. Get the feed_target_names and fetch_target_names
this->feed_target_names_ = this->inference_program_->GetFeedTargetNames();
this->fetch_target_names_ = this->inference_program_->GetFetchTargetNames();
return true;
}
bool PaddlePredictorImpl::SetFeed(
const std::vector<PaddleTensor> &inputs,
std::vector<paddle::framework::LoDTensor> *feeds) {
VLOG(3) << "Predictor::set_feed";
if (inputs.size() != feed_target_names_.size()) {
LOG(ERROR) << "wrong feed input size.";
return false;
}
for (size_t i = 0; i < feed_target_names_.size(); ++i) {
paddle::framework::LoDTensor input;
paddle::framework::DDim ddim =
paddle::framework::make_ddim(inputs[i].shape);
void *input_ptr;
if (inputs[i].dtype == PaddleDType::INT64) {
input_ptr =
input.mutable_data<int64_t>(ddim, paddle::platform::CPUPlace());
} else if (inputs[i].dtype == PaddleDType::FLOAT32) {
input_ptr = input.mutable_data<float>(ddim, paddle::platform::CPUPlace());
} else {
LOG(ERROR) << "unsupported feed type " << inputs[i].dtype;
return false;
}
// TODO(panyx0718): Init LoDTensor from existing memcpy to save a copy.
std::memcpy(static_cast<void *>(input_ptr),
inputs[i].data.data,
inputs[i].data.length);
feeds->push_back(input);
LOG(ERROR) << "Actual feed type " << feeds->back().type().name();
}
return true;
}
bool PaddlePredictorImpl::GetFetch(
const std::vector<paddle::framework::LoDTensor> &fetchs,
std::vector<PaddleTensor> *outputs) {
VLOG(3) << "Predictor::get_fetch";
outputs->resize(fetchs.size());
for (size_t i = 0; i < fetchs.size(); ++i) {
// TODO(panyx0718): Support fetch of other types.
if (fetchs[i].type() != typeid(float)) {
LOG(ERROR) << "only support fetching float now.";
return false;
}
std::vector<int> shape;
auto dims_i = fetchs[i].dims();
auto lod = fetchs[i].lod();
const float *output_ptr = fetchs[i].data<float>();
// const int64_t* output_ptr = fetchs[i].data<int64_t>();
auto num = fetchs[i].numel();
std::vector<float> data;
if (0 == lod.size()) {
std::copy(output_ptr, output_ptr + num, std::back_inserter(data));
for (int j = 0; j < dims_i.size(); ++j) {
shape.push_back(dims_i[j]);
}
} else {
// for batch detection
// image[0] -> output[0] shape {145, 6}
// image[1] -> output[1] shape {176, 6}
// then,
// the batch output shape {321, 6}
// the lod {{0, 145, 321}}
// so we should append output[0] to {176, 6}
size_t max_dim = 0;
for (size_t j = 1; j < lod[0].size(); j++) {
max_dim = std::max(max_dim, lod[0][j] - lod[0][j - 1]);
}
size_t common_dim = lod[0].back() == 0 ? 0 : num / lod[0].back();
if (max_dim > 0) {
data.resize((lod[0].size() - 1) * max_dim * common_dim, 0);
}
for (size_t j = 1; j < lod[0].size(); j++) {
size_t start = lod[0][j - 1] * common_dim;
size_t end = lod[0][j] * common_dim;
if (end > start) {
std::copy(output_ptr + start,
output_ptr + end,
data.begin() + (j - 1) * max_dim * common_dim);
}
}
shape.push_back(lod[0].size() - 1);
shape.push_back(max_dim);
for (int j = 1; j < dims_i.size(); ++j) {
shape.push_back(dims_i[j]);
}
}
outputs->at(i).shape = shape;
outputs->at(i).data.length = sizeof(float) * data.size();
outputs->at(i).data.data = malloc(outputs->at(i).data.length);
std::memcpy(
outputs->at(i).data.data, data.data(), outputs->at(i).data.length);
outputs->at(i).dtype = PaddleDType::FLOAT32;
// TODO(panyx0718): support other types? fill tensor name? avoid a copy.
}
return true;
}
std::unique_ptr<PaddlePredictorImpl> CreatePaddlePredictorImpl(
const VisConfig &config) {
VLOG(3) << "create PaddlePredictorImpl";
// 1. GPU memeroy
std::vector<std::string> flags;
if (config.fraction_of_gpu_memory >= 0.0f ||
config.fraction_of_gpu_memory <= 0.95f) {
flags.push_back("dummpy");
std::string flag = "--fraction_of_gpu_memory_to_use=" +
num2str<float>(config.fraction_of_gpu_memory);
flags.push_back(flag);
VLOG(3) << "set flag: " << flag;
framework::InitGflags(flags);
}
std::unique_ptr<PaddlePredictorImpl> predictor(
new PaddlePredictorImpl(config));
if (!predictor->Init()) {
return nullptr;
}
return predictor;
}
} // namespace paddle
paddle/contrib/inference/paddle_inference_api_impl.h 0 → 100644
浏览文件 @ 60d827a8
/* Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. */
#pragma once
#include <glog/logging.h>
#include <memory>
#include <string>
#include <vector>
#include "paddle/contrib/inference/paddle_inference_api.h"
#include "paddle/fluid/framework/ddim.h"
#include "paddle/fluid/framework/init.h"
#include "paddle/fluid/framework/lod_tensor.h"
#include "paddle/fluid/inference/io.h"
#include "paddle/fluid/platform/profiler.h"
namespace paddle {
struct VisConfig : public PaddlePredictor::Config {
int device;
float fraction_of_gpu_memory;
std::string prog_file;
std::string param_file;
bool share_variables;
};
/*
* Do not use this, just a demo indicating how to customize a Predictor.
*/
class PaddlePredictorImpl : public PaddlePredictor {
public:
explicit PaddlePredictorImpl(const VisConfig &config) : config_(config) {}
bool Init();
bool Run(const std::vector<PaddleTensor> &inputs,
std::vector<PaddleTensor> *output_data) override;
std::unique_ptr<PaddlePredictor> Clone() override;
~PaddlePredictorImpl() override{};
private:
bool InitShared();
bool SetFeed(const std::vector<PaddleTensor> &input_datas,
std::vector<paddle::framework::LoDTensor> *feeds);
bool GetFetch(const std::vector<paddle::framework::LoDTensor> &fetchs,
std::vector<PaddleTensor> *output_data);
VisConfig config_;
paddle::platform::Place place_;
std::unique_ptr<paddle::framework::Executor> executor_;
std::unique_ptr<paddle::framework::Scope> scope_;
std::unique_ptr<paddle::framework::ExecutorPrepareContext> ctx_;
std::unique_ptr<paddle::framework::ProgramDesc> inference_program_;
std::vector<std::string> feed_target_names_;
std::vector<std::string> fetch_target_names_;
};
std::unique_ptr<PaddlePredictorImpl> CreatePaddlePredictorImpl(
const VisConfig &config);
} // namespace paddle
paddle/contrib/inference/test_paddle_inference_api.cc
浏览文件 @ 60d827a8
......@@ -31,7 +31,7 @@ struct DemoConfig : public PaddlePredictor::Config {
* Do not use this, just a demo indicating how to customize a Predictor.
*/
class DemoPredictor : public PaddlePredictor {
public:
public:
explicit DemoPredictor(const DemoConfig &config) {
LOG(INFO) << "I get other_config " << config.other_config;
}
......
paddle/contrib/inference/test_paddle_inference_api_impl.cc 0 → 100644
浏览文件 @ 60d827a8
/* Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. */
#include <glog/logging.h>
#include <gtest/gtest.h>
#include "gflags/gflags.h"
#include "paddle/contrib/inference/paddle_inference_api_impl.h"
#include "paddle/fluid/inference/tests/test_helper.h"
DEFINE_string(dirname, "", "Directory of the inference model.");
namespace paddle {
PaddleTensor LodTensorToPaddleTensor(framework::LoDTensor* t) {
PaddleTensor pt;
pt.data.data = t->data<void>();
if (t->type() == typeid(int64_t)) {
pt.data.length = t->numel() * sizeof(int64_t);
pt.dtype = PaddleDType::INT64;
} else if (t->type() == typeid(float)) {
pt.data.length = t->numel() * sizeof(float);
pt.dtype = PaddleDType::FLOAT32;
} else {
LOG(FATAL) << "unsupported type.";
}
pt.shape = framework::vectorize2int(t->dims());
return pt;
}
TEST(paddle_inference_api_impl, word2vec) {
VisConfig config;
config.model_dir = FLAGS_dirname + "word2vec.inference.model";
LOG(INFO) << "dirname " << config.model_dir;
config.fraction_of_gpu_memory = 0.15;
config.device = 0;
config.share_variables = true;
std::unique_ptr<PaddlePredictorImpl> predictor =
CreatePaddlePredictorImpl(config);
framework::LoDTensor first_word, second_word, third_word, fourth_word;
framework::LoD lod{{0, 1}};
int64_t dict_size = 2073; // The size of dictionary
SetupLoDTensor(&first_word, lod, static_cast<int64_t>(0), dict_size - 1);
SetupLoDTensor(&second_word, lod, static_cast<int64_t>(0), dict_size - 1);
SetupLoDTensor(&third_word, lod, static_cast<int64_t>(0), dict_size - 1);
SetupLoDTensor(&fourth_word, lod, static_cast<int64_t>(0), dict_size - 1);
std::vector<PaddleTensor> cpu_feeds;
cpu_feeds.push_back(LodTensorToPaddleTensor(&first_word));
cpu_feeds.push_back(LodTensorToPaddleTensor(&second_word));
cpu_feeds.push_back(LodTensorToPaddleTensor(&third_word));
cpu_feeds.push_back(LodTensorToPaddleTensor(&fourth_word));
std::vector<PaddleTensor> outputs;
ASSERT_TRUE(predictor->Run(cpu_feeds, &outputs));
ASSERT_EQ(outputs.size(), 1);
for (size_t i = 0; i < outputs.size(); ++i) {
size_t len = outputs[i].data.length;
float* data = static_cast<float*>(outputs[i].data.data);
for (int j = 0; j < len / sizeof(float); ++j) {
ASSERT_LT(data[j], 1.0);
ASSERT_GT(data[j], -1.0);
}
free(outputs[i].data.data);
}
}
} // namespace paddle
paddle/cuda/include/hl_activation_functions.h
浏览文件 @ 60d827a8
......@@ -31,7 +31,7 @@ namespace hppl {
*/
template <class T>
class Active {
public:
public:
typedef T (*forward)(T);
typedef T (*backward)(T, T);
};
......
paddle/cuda/include/hl_tensor_ops.h
浏览文件 @ 60d827a8
......@@ -23,128 +23,128 @@ namespace unary {
template <class T>
class add_scale {
private:
private:
const T p;
public:
public:
INLINE add_scale(const T s) : p(s) {}
INLINE T operator()(const T a) const { return a + p; }
};
template <class T>
class sub_scale {
private:
private:
const T p;
public:
public:
INLINE sub_scale(const T s) : p(s) {}
INLINE T operator()(const T a) const { return a - p; }
};
template <class T>
class mul_scale {
private:
private:
const T p;
public:
public:
INLINE mul_scale(const T s) : p(s) {}
INLINE T operator()(const T a) const { return a * p; }
};
template <class T>
class div_scale {
private:
private:
const T p;
public:
public:
INLINE div_scale(const T s) : p(s) {}
INLINE T operator()(const T a) const { return a / p; }
};
template <class T>
class neg {
public:
public:
INLINE T operator()(const T a) const { return -a; }
};
template <class T>
class exp_op {
public:
public:
INLINE T operator()(const T a) const { return std::exp(a); }
};
template <class T>
class log_op {
public:
public:
INLINE T operator()(const T a) const { return std::log(a); }
};
template <class T>
class sqrt_op {
public:
public:
INLINE T operator()(const T a) const { return std::sqrt(a); }
};
template <class T>
class square {
public:
public:
INLINE T operator()(const T a) const { return a * a; }
};
template <class T>
class reciprocal {
public:
public:
INLINE T operator()(const T a) const { return T(1) / a; }
};
template <class T>
class abs {
public:
public:
INLINE T operator()(const T a) const { return a > 0 ? a : -a; }
};
template <class T>
class sign {
public:
public:
INLINE T operator()(const T a) const { return (a > 0) - (a < 0); }
};
template <class T>
class min {
private:
private:
const T p;
public:
public:
INLINE min(const T s) : p(s) {}
INLINE T operator()(const T a) const { return a > p ? p : a; }
};
template <class T>
class max {
private:
private:
const T p;
public:
public:
INLINE max(const T s) : p(s) {}
INLINE T operator()(const T a) const { return a < p ? p : a; }
};
template <class T>
class pow_op {
private:
private:
const T p;
public:
public:
INLINE pow_op(const T s) : p(s) {}
INLINE T operator()(const T a) const { return std::pow(a, p); }
};
template <class T>
class constant {
private:
private:
const T p;
public:
public:
INLINE constant(const T s) : p(s) {}
INLINE T operator()(int i) const { return p; }
INLINE T operator()(int i, int j) const { return p; }
......@@ -152,80 +152,80 @@ public:
template <class T>
class cmp_eq {
private:
private:
const T p;
public:
public:
INLINE cmp_eq(const T s) : p(s) {}
INLINE bool operator()(const T a) const { return a == p; }
};
template <class T>
class cmp_ne {
private:
private:
const T p;
public:
public:
INLINE cmp_ne(const T s) : p(s) {}
INLINE bool operator()(const T a) const { return a != p; }
};
template <class T>
class cmp_le {
private:
private:
const T p;
public:
public:
INLINE cmp_le(const T s) : p(s) {}
INLINE bool operator()(const T a) const { return a <= p; }
};
template <class T>
class cmp_lt {
private:
private:
const T p;
public:
public:
INLINE cmp_lt(const T s) : p(s) {}
INLINE bool operator()(const T a) const { return a < p; }
};
template <class T>
class cmp_ge {
private:
private:
const T p;
public:
public:
INLINE cmp_ge(const T s) : p(s) {}
INLINE bool operator()(const T a) const { return a >= p; }
};
template <class T>
class cmp_gt {
private:
private:
const T p;
public:
public:
INLINE cmp_gt(const T s) : p(s) {}
INLINE bool operator()(const T a) const { return a > p; }
};
template <class T>
class and_op {
private:
private:
const T p;
public:
public:
INLINE and_op(const T s) : p(s) {}
INLINE bool operator()(const T a) const { return a && p; }
};
template <class T>
class or_op {
private:
private:
const T p;
public:
public:
INLINE or_op(const T s) : p(s) {}
INLINE bool operator()(const T a) const { return a || p; }
};
......@@ -235,96 +235,96 @@ public:
namespace binary {
template <class T>
class add {
public:
public:
INLINE T operator()(const T a, const T b) const { return a + b; }
};
template <class T>
class add_scale {
private:
private:
const T p1;
const T p2;
public:
public:
INLINE add_scale(const T s1, const T s2) : p1(s1), p2(s2) {}
INLINE T operator()(const T a, const T b) const { return p1 * a + p2 * b; }
};
template <class T>
class sub {
public:
public:
INLINE T operator()(const T a, const T b) const { return a - b; }
};
template <class T>
class mul {
public:
public:
INLINE T operator()(const T a, const T b) const { return a * b; }
};
template <class T>
class div {
public:
public:
INLINE T operator()(const T a, const T b) const { return a / b; }
};
template <class T>
class cmp_eq {
public:
public:
INLINE bool operator()(const T a, const T b) const { return a == b; }
};
template <class T>
class cmp_ne {
public:
public:
INLINE bool operator()(const T a, const T b) const { return a != b; }
};
template <class T>
class cmp_le {
public:
public:
INLINE bool operator()(const T a, const T b) const { return a <= b; }
};
template <class T>
class cmp_lt {
public:
public:
INLINE bool operator()(const T a, const T b) const { return a < b; }
};
template <class T>
class cmp_ge {
public:
public:
INLINE bool operator()(const T a, const T b) const { return a >= b; }
};
template <class T>
class cmp_gt {
public:
public:
INLINE bool operator()(const T a, const T b) const { return a > b; }
};
template <class T>
class and_op {
public:
public:
INLINE bool operator()(const T a, const T b) const { return a && b; }
};
template <class T>
class or_op {
public:
public:
INLINE bool operator()(const T a, const T b) const { return a || b; }
};
template <class T>
class min {
public:
public:
INLINE T operator()(const T a, const T b) const { return a > b ? b : a; }
};
template <class T>
class max {
public:
public:
INLINE T operator()(const T a, const T b) const { return a < b ? b : a; }
};
......@@ -332,7 +332,7 @@ public:
#ifndef PADDLE_TYPE_DOUBLE
template <>
class add<__m128> {
public:
public:
INLINE __m128 operator()(const __m128 a, const __m128 b) const {
return _mm_add_ps(a, b);
}
......@@ -340,11 +340,11 @@ public:
template <>
class add_scale<__m128> {
private:
private:
const __m128 p1;
const __m128 p2;
public:
public:
INLINE add_scale(const __m128 s1, const __m128 s2) : p1(s1), p2(s2) {}
INLINE __m128 operator()(const __m128 a, const __m128 b) const {
return _mm_add_ps(_mm_mul_ps(p1, a), _mm_mul_ps(p2, b));
......@@ -353,7 +353,7 @@ public:
template <>
class sub<__m128> {
public:
public:
INLINE __m128 operator()(const __m128 a, const __m128 b) const {
return _mm_sub_ps(a, b);
}
......@@ -361,7 +361,7 @@ public:
template <>
class mul<__m128> {
public:
public:
INLINE __m128 operator()(const __m128 a, const __m128 b) const {
return _mm_mul_ps(a, b);
}
......@@ -369,7 +369,7 @@ public:
template <>
class div<__m128> {
public:
public:
INLINE __m128 operator()(const __m128 a, const __m128 b) const {
return _mm_div_ps(a, b);
}
......@@ -377,7 +377,7 @@ public:
template <>
class min<__m128> {
public:
public:
INLINE __m128 operator()(const __m128 a, const __m128 b) const {
return _mm_min_ps(a, b);
}
......@@ -385,7 +385,7 @@ public:
template <>
class max<__m128> {
public:
public:
INLINE __m128 operator()(const __m128 a, const __m128 b) const {
return _mm_max_ps(a, b);
}
......@@ -393,7 +393,7 @@ public:
#else
template <>
class add<__m128d> {
public:
public:
INLINE __m128d operator()(const __m128d a, const __m128d b) const {
return _mm_add_pd(a, b);
}
......@@ -401,11 +401,11 @@ public:
template <>
class add_scale<__m128d> {
private:
private:
const __m128d p1;
const __m128d p2;
public:
public:
INLINE add_scale(const __m128d s1, const __m128d s2) : p1(s1), p2(s2) {}
INLINE __m128d operator()(const __m128d a, const __m128d b) const {
return _mm_add_pd(_mm_mul_pd(p1, a), _mm_mul_pd(p2, b));
......@@ -414,7 +414,7 @@ public:
template <>
class sub<__m128d> {
public:
public:
INLINE __m128d operator()(const __m128d a, const __m128d b) const {
return _mm_sub_pd(a, b);
}
......@@ -422,7 +422,7 @@ public:
template <>
class mul<__m128d> {
public:
public:
INLINE __m128d operator()(const __m128d a, const __m128d b) const {
return _mm_mul_pd(a, b);
}
......@@ -430,7 +430,7 @@ public:
template <>
class div<__m128d> {
public:
public:
INLINE __m128d operator()(const __m128d a, const __m128d b) const {
return _mm_div_pd(a, b);
}
......@@ -438,7 +438,7 @@ public:
template <>
class min<__m128d> {
public:
public:
INLINE __m128d operator()(const __m128d a, const __m128d b) const {
return _mm_min_pd(a, b);
}
......@@ -446,7 +446,7 @@ public:
template <>
class max<__m128d> {
public:
public:
INLINE __m128d operator()(const __m128d a, const __m128d b) const {
return _mm_max_pd(a, b);
}
......@@ -458,7 +458,7 @@ public:
#ifndef PADDLE_TYPE_DOUBLE
template <>
class add<float32x4_t> {
public:
public:
INLINE float32x4_t operator()(const float32x4_t a,
const float32x4_t b) const {
return vaddq_f32(a, b);
......@@ -467,11 +467,11 @@ public:
template <>
class add_scale<float32x4_t> {
private:
private:
const float32x4_t p1;
const float32x4_t p2;
public:
public:
INLINE add_scale(const float32x4_t s1, const float32x4_t s2)
: p1(s1), p2(s2) {}
INLINE float32x4_t operator()(const float32x4_t a,
......@@ -482,7 +482,7 @@ public:
template <>
class sub<float32x4_t> {
public:
public:
INLINE float32x4_t operator()(const float32x4_t a,
const float32x4_t b) const {
return vsubq_f32(a, b);
......@@ -491,7 +491,7 @@ public:
template <>
class mul<float32x4_t> {
public:
public:
INLINE float32x4_t operator()(const float32x4_t a,
const float32x4_t b) const {
return vmulq_f32(a, b);
......@@ -500,7 +500,7 @@ public:
template <>
class div<float32x4_t> {
public:
public:
INLINE float32x4_t operator()(const float32x4_t a,
const float32x4_t b) const {
float32x4_t tmp = vrecpeq_f32(b);
......@@ -510,7 +510,7 @@ public:
template <>
class min<float32x4_t> {
public:
public:
INLINE float32x4_t operator()(const float32x4_t a,
const float32x4_t b) const {
return vminq_f32(a, b);
......@@ -519,7 +519,7 @@ public:
template <>
class max<float32x4_t> {
public:
public:
INLINE float32x4_t operator()(const float32x4_t a,
const float32x4_t b) const {
return vmaxq_f32(a, b);
......
paddle/cuda/src/hl_cuda_lstm.cu
浏览文件 @ 60d827a8
......@@ -30,7 +30,7 @@ bool hl_lstm_sequence_parallel(int frameSize) {
}
class frameValue {
public:
public:
real *value_;
__device__ frameValue(real *value) : value_(value) {}
template <int reversed, int frameSize>
......
paddle/fluid/inference/CMakeLists.txt
浏览文件 @ 60d827a8
set(FLUID_CORE_MODULES proto_desc memory lod_tensor executor init)
# TODO(panyx0718): Should this be called paddle_fluid_inference_api_internal?
cc_library(paddle_fluid_api
SRCS io.cc
DEPS ${FLUID_CORE_MODULES} ${GLOB_OP_LIB})
......
paddle/fluid/operators/fill_constant_batch_size_like_op.h
浏览文件 @ 60d827a8
......@@ -24,6 +24,14 @@ class FillConstantBatchSizeLikeOpKernel : public framework::OpKernel<T> {
public:
void Compute(const framework::ExecutionContext& ctx) const override {
auto* out = ctx.Output<framework::Tensor>("Out");
auto* in = ctx.Input<framework::LoDTensor>("Input");
if (in->lod().size() && ctx.Attr<int>("input_dim_idx") == 0) {
// set the correct batch size for the LoDTensor.
auto odims = out->dims();
int output_dim_idx = ctx.Attr<int>("output_dim_idx");
odims[output_dim_idx] = static_cast<int>(in->lod().back().size()) - 1;
out->mutable_data<T>(odims, ctx.GetPlace());
}
out->mutable_data<T>(ctx.GetPlace());
auto value = ctx.Attr<float>("value");
......
paddle/fluid/operators/math/cross_entropy.cc
浏览文件 @ 60d827a8
......@@ -46,7 +46,10 @@ class CrossEntropyFunctor<platform::CPUDeviceContext, T> {
const int64_t* label_data = labels->data<int64_t>();
for (int i = 0; i < batch_size; ++i) {
int index = i * class_num + label_data[i];
int lbl = label_data[i];
PADDLE_ENFORCE_GE(lbl, 0);
PADDLE_ENFORCE_LT(lbl, class_num);
int index = i * class_num + lbl;
loss_data[i] = -math::TolerableValue<T>()(std::log(prob_data[index]));
}
}
......
paddle/function/BlockExpandOp.cpp
浏览文件 @ 60d827a8
......@@ -33,7 +33,7 @@ namespace paddle {
* \param outputs[0] Image data of NCHW format.
*/
class BlockExpandFunction : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override {
// function arguments
strides_ = config.get<std::vector<size_t>>("strides");
......@@ -81,7 +81,7 @@ public:
(size_t)blockW()});
}
protected:
protected:
std::vector<size_t> strides_;
std::vector<size_t> paddings_;
std::vector<size_t> blocks_;
......@@ -101,7 +101,7 @@ protected:
template <DeviceType Device>
class BlockExpandForward : public BlockExpandFunction {
public:
public:
void init(const FuncConfig& config) override {
BlockExpandFunction::init(config);
}
......@@ -149,7 +149,7 @@ public:
template <DeviceType Device>
class BlockExpandBackward : public BlockExpandFunction {
public:
public:
void init(const FuncConfig& config) override {
BlockExpandFunction::init(config);
}
......
paddle/function/BufferArg.h
浏览文件 @ 60d827a8
......@@ -63,12 +63,12 @@ enum ArgType {
ADD_TO = 2,
};
class BufferArg {
public:
public:
void setArgType(ArgType argType) { argType_ = argType; }
ArgType getArgType() const { return argType_; }
public:
public:
BufferArg(ValueType valueType,
const TensorShape& shape,
ArgType argType = UNSPECIFIED)
......@@ -169,7 +169,7 @@ public:
const SequenceArg& sequence() const;
const SparseMatrixArg& sparse() const;
protected:
protected:
void* buf_;
ValueType valueType_;
TensorShape shape_;
......@@ -185,7 +185,7 @@ protected:
// valueType_ = int32
// if a < b then value_.buf_[a] < value_.buf_[b]
class SequenceIdArg : public BufferArg {
public:
public:
SequenceIdArg(const TensorShape& shape, ArgType argType = UNSPECIFIED)
: BufferArg(VALUE_TYPE_INT32, shape, argType) {
bufferType_ = TENSOR_SEQUENCE_ID;
......@@ -212,7 +212,7 @@ public:
size_t numSeqs() const { return numSeqs_; }
private:
private:
size_t numSeqs_;
};
......@@ -222,7 +222,7 @@ private:
// SequenceArg can be used to represent sequences that contain multiple
// unequal lengths.
class SequenceArg : public BufferArg {
public:
public:
SequenceArg(ValueType valueType,
const TensorShape& shape,
ArgType argType = UNSPECIFIED)
......@@ -255,7 +255,7 @@ public:
SequenceIdArg& getSequenceId() { return startPositions_; }
const SequenceIdArg& getSequenceId() const { return startPositions_; }
private:
private:
SequenceIdArg startPositions_;
};
......@@ -263,7 +263,7 @@ private:
// valueType_ == float or double
// shape_.ndims() == 2
class SparseMatrixArg : public BufferArg {
public:
public:
SparseMatrixArg(void* buf,
ValueType valueType,
const TensorShape& shape,
......@@ -353,7 +353,7 @@ public:
SparseDataType dataType() const { return type_; }
private:
private:
BufferArg row_;
BufferArg col_;
size_t nnz_;
......
paddle/function/ContextProjectionOp.cpp
浏览文件 @ 60d827a8
......@@ -100,7 +100,7 @@ void ContextProjectionForward<DEVICE_TYPE_CPU>(CpuMatrix& out_mat,
*/
template <DeviceType Device>
class ContextProjectionForwardFunc : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override {
context_length_ = config.get<size_t>("context_length");
context_start_ = config.get<int>("context_start");
......@@ -146,7 +146,7 @@ public:
begin_pad_);
}
private:
private:
size_t context_length_;
int context_start_;
size_t begin_pad_;
......@@ -223,7 +223,7 @@ void ContextProjectionBackward<DEVICE_TYPE_CPU>(const CpuMatrix& out_grad_mat,
*/
template <DeviceType Device>
class ContextProjectionBackwardFunc : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override {
context_length_ = config.get<size_t>("context_length");
context_start_ = config.get<int>("context_start");
......@@ -278,7 +278,7 @@ public:
total_pad_);
}
private:
private:
size_t context_length_;
int context_start_;
size_t begin_pad_;
......@@ -299,7 +299,7 @@ private:
*/
template <DeviceType Device>
class ContextProjectionBackwardDataFunc : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override {
context_length_ = config.get<size_t>("context_length");
context_start_ = config.get<int>("context_start");
......@@ -331,7 +331,7 @@ public:
out_grad_mat, in_grad_mat, seq_vec, context_length_, context_start_);
}
private:
private:
size_t context_length_;
int context_start_;
};
......@@ -348,7 +348,7 @@ private:
*/
template <DeviceType Device>
class ContextProjectionBackwardWeightFunc : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override {
context_length_ = config.get<size_t>("context_length");
context_start_ = config.get<int>("context_start");
......@@ -382,7 +382,7 @@ public:
begin_pad_);
}
private:
private:
size_t context_length_;
int context_start_;
size_t begin_pad_;
......
paddle/function/ConvOp.h
浏览文件 @ 60d827a8
......@@ -56,7 +56,7 @@ namespace paddle {
* H and W is height and width of filter.
*/
class ConvFunctionBase : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override {
// function arguments
strides_ = config.get<std::vector<size_t>>("strides");
......@@ -101,7 +101,7 @@ public:
}
}
protected:
protected:
size_t getFilterHeight(const TensorShape& filter) const {
return filter[filter.ndims() - 2];
}
......
paddle/function/CosSimOp.cpp
浏览文件 @ 60d827a8
......@@ -97,7 +97,7 @@ class CosSimForwardFunc : public FunctionBase {
CosSimForward<Device>(out_mat, in1_mat, in2_mat, scale_);
}
private:
private:
real scale_;
};
......@@ -227,7 +227,7 @@ class CosSimBackwardFunc : public FunctionBase {
out_grad, out_val, in1_val, in2_val, in1_grad, in2_grad, scale_);
}
private:
private:
real scale_;
};
......
paddle/function/CropOp.cpp
浏览文件 @ 60d827a8
......@@ -112,7 +112,7 @@ void CropGrad<DEVICE_TYPE_CPU>(const real* inGrad,
*/
template <DeviceType Device>
class CropFunc : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override { conf_ = config; }
void calc(const BufferArgs& inputs, const BufferArgs& outputs) override {
......@@ -130,7 +130,7 @@ public:
conf_);
}
private:
private:
FuncConfig conf_;
};
......@@ -145,7 +145,7 @@ private:
template <DeviceType Device>
class CropGradFunc : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override { conf_ = config; }
void calc(const BufferArgs& inputs, const BufferArgs& outputs) override {
......@@ -163,7 +163,7 @@ public:
conf_);
}
private:
private:
FuncConfig conf_;
};
......
paddle/function/CrossMapNormalOp.cpp
浏览文件 @ 60d827a8
......@@ -160,7 +160,7 @@ void CrossMapNormalGrad<DEVICE_TYPE_CPU>(real* inputsGrad,
*/
template <DeviceType Device>
class CrossMapNormalFunc : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override {
// function arguments
size_ = config.get<size_t>("size");
......@@ -220,7 +220,7 @@ public:
return ops;
}
private:
private:
size_t size_;
real scale_;
real pow_;
......@@ -260,7 +260,7 @@ private:
*/
template <DeviceType Device>
class CrossMapNormalGradFunc : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override {
// function arguments
size_ = config.get<size_t>("size");
......@@ -328,7 +328,7 @@ public:
return ops;
}
private:
private:
size_t size_;
real scale_;
real pow_;
......
paddle/function/DepthwiseConvOp.cpp
浏览文件 @ 60d827a8
......@@ -19,7 +19,7 @@ namespace paddle {
template <class T>
class DepthwiseConvFunctor<DEVICE_TYPE_CPU, T> {
public:
public:
void operator()(const T* inputData,
const T* filterData,
int batchSize,
......@@ -43,7 +43,7 @@ public:
template <class T>
class DepthwiseConvGradInputFunctor<DEVICE_TYPE_CPU, T> {
public:
public:
void operator()(const T* outputGrad,
const T* filterData,
int batchSize,
......@@ -66,7 +66,7 @@ public:
template <class T>
class DepthwiseConvGradFilterFunctor<DEVICE_TYPE_CPU, T> {
public:
public:
void operator()(const T* outputGrad,
const T* inputData,
int batchSize,
......@@ -93,7 +93,7 @@ public:
*/
template <DeviceType Device>
class DepthwiseConvFunction : public ConvFunctionBase {
public:
public:
void init(const FuncConfig& config) override {
ConvFunctionBase::init(config);
}
......@@ -156,7 +156,7 @@ public:
*/
template <DeviceType Device>
class DepthwiseConvGradInputFunction : public ConvFunctionBase {
public:
public:
void init(const FuncConfig& config) override {
ConvFunctionBase::init(config);
}
......@@ -220,7 +220,7 @@ public:
*/
template <DeviceType Device>
class DepthwiseConvGradFilterFunction : public ConvFunctionBase {
public:
public:
void init(const FuncConfig& config) override {
ConvFunctionBase::init(config);
}
......
paddle/function/DepthwiseConvOp.h
浏览文件 @ 60d827a8
......@@ -44,7 +44,7 @@ namespace paddle {
*/
template <DeviceType Device, class T>
class DepthwiseConvFunctor {
public:
public:
void operator()(const T* inputData,
const T* filterData,
int batchSize,
......@@ -89,7 +89,7 @@ public:
*/
template <DeviceType Device, class T>
class DepthwiseConvGradInputFunctor {
public:
public:
void operator()(const T* outputGrad,
const T* filterData,
int batchSize,
......@@ -135,7 +135,7 @@ public:
*/
template <DeviceType Device, class T>
class DepthwiseConvGradFilterFunctor {
public:
public:
void operator()(const T* outputGrad,
const T* inputData,
int batchSize,
......
paddle/function/DepthwiseConvOpGpu.cu
浏览文件 @ 60d827a8
......@@ -199,7 +199,7 @@ __global__ void ConvolutionDepthwiseFilterBackward(const int num_i,
template <class T>
class DepthwiseConvFunctor<DEVICE_TYPE_GPU, T> {
public:
public:
void operator()(const T* inputData,
const T* filterData,
int batchSize,
......@@ -249,7 +249,7 @@ public:
template <class T>
class DepthwiseConvGradInputFunctor<DEVICE_TYPE_GPU, T> {
public:
public:
void operator()(const T* outputGrad,
const T* filterData,
int batchSize,
......@@ -300,7 +300,7 @@ public:
template <class T>
class DepthwiseConvGradFilterFunctor<DEVICE_TYPE_GPU, T> {
public:
public:
void operator()(const T* outputGrad,
const T* inputData,
int batchSize,
......
paddle/function/EigenThreadDevice.h
浏览文件 @ 60d827a8
......@@ -46,7 +46,7 @@ int GetCpuCount() { return 1; }
#endif
class EigenDeviceWarpper {
public: // NOLINT
public: // NOLINT
#if EIGEN_USE_THREADS
static Eigen::ThreadPoolDevice* device() {
const int num_cpus = GetCpuCount();
......
paddle/function/Function.h
浏览文件 @ 60d827a8
......@@ -29,7 +29,7 @@ namespace paddle {
* The argument type of Function::init.
*/
class FuncConfig {
public:
public:
template <typename T>
T get(const std::string& key, Error* err = nullptr) const {
try {
......@@ -59,7 +59,7 @@ public:
return *this;
}
protected:
protected:
mutable std::unordered_map<std::string, any> valueMap_;
};
......@@ -77,7 +77,7 @@ protected:
* in the BufferArgs life time.
*/
class BufferArgs {
public:
public:
BufferArgs() {}
~BufferArgs() {
......@@ -137,7 +137,7 @@ public:
void addArg(SparseMatrixArg& arg) { args_.push_back(&arg); }
private:
private:
std::vector<BufferArg*> args_;
// The BufferArg object is constructed and freed by BufferArgs.
std::vector<BufferArg*> _args_;
......@@ -163,7 +163,7 @@ private:
* If Function has more than one output, each output can have different modes.
*/
class FunctionBase {
public:
public:
virtual ~FunctionBase() {}
virtual void init(const FuncConfig& config) {}
......@@ -192,7 +192,7 @@ public:
static ClassRegistrar<FunctionBase> funcRegistrar_;
protected:
protected:
// numInputs_ and numOutputs_ represents the maximum
// input and output supported by Function.
// Some functions are optimized for input and output,
......
paddle/function/FunctionTest.h
浏览文件 @ 60d827a8
......@@ -39,7 +39,7 @@ struct Allocator<DEVICE_TYPE_GPU> {
// Copy argument1 to argument2
template <DeviceType DType1, DeviceType DType2>
class CopyArgument {
public:
public:
void operator()(const BufferArg& arg1, BufferArg& arg2) {
CHECK_EQ(arg1.valueType(), arg2.valueType());
CHECK_LE(arg1.shape().getElements(), arg2.shape().getElements());
......@@ -95,7 +95,7 @@ public:
*/
template <DeviceType DType1, DeviceType DType2>
class Compare2Function {
public:
public:
typedef typename test::Allocator<DType1>::type Allocator1;
typedef typename test::Allocator<DType2>::type Allocator2;
typedef typename Tensor<real, DType1>::Vector Vector1;
......@@ -305,7 +305,7 @@ public:
std::shared_ptr<FunctionBase> getFunction2() const { return function2_; }
protected:
protected:
// only init cpu argument, gpu argument copy from cpu argument.
void initArg(BufferArg& arg) {
Vector1 vector(arg.shape().getElements(), (real*)arg.data());
......@@ -381,7 +381,7 @@ protected:
}
}
protected:
protected:
std::shared_ptr<FunctionBase> function1_;
std::shared_ptr<FunctionBase> function2_;
std::vector<std::shared_ptr<Allocator1>> func1Memory_;
......@@ -400,7 +400,7 @@ protected:
class CpuGpuFuncCompare
: public Compare2Function<DEVICE_TYPE_CPU, DEVICE_TYPE_GPU> {
public:
public:
CpuGpuFuncCompare(const std::string& name, const FuncConfig& config)
: Compare2Function(name + "-CPU", name + "-GPU", config) {}
......
paddle/function/GemmConvOp.cpp
浏览文件 @ 60d827a8
......@@ -24,7 +24,7 @@ namespace paddle {
*/
template <DeviceType Device>
class GemmConvFunction : public ConvFunctionBase {
public:
public:
void init(const FuncConfig& config) override {
ConvFunctionBase::init(config);
}
......@@ -136,7 +136,7 @@ public:
*/
template <DeviceType Device>
class GemmConvMobileFunction : public ConvFunctionBase {
public:
public:
void init(const FuncConfig& config) override {
ConvFunctionBase::init(config);
}
......@@ -297,7 +297,7 @@ public:
*/
template <DeviceType Device>
class GemmConvGradInputFunction : public ConvFunctionBase {
public:
public:
void init(const FuncConfig& config) override {
ConvFunctionBase::init(config);
}
......@@ -404,7 +404,7 @@ public:
*/
template <DeviceType Device>
class GemmConvGradFilterFunction : public ConvFunctionBase {
public:
public:
void init(const FuncConfig& config) override {
ConvFunctionBase::init(config);
}
......
paddle/function/Im2Col.h
浏览文件 @ 60d827a8
......@@ -70,7 +70,7 @@ enum ColFormat { kCFO = 0, kOCF = 1 };
*/
template <ColFormat Format, DeviceType Device, class T>
class Im2ColFunctor {
public:
public:
void operator()(const T* imData,
const TensorShape& imShape,
T* colData,
......@@ -85,7 +85,7 @@ public:
template <ColFormat Format, DeviceType Device, class T>
class Col2ImFunctor {
public:
public:
void operator()(T* imData,
const TensorShape& imShape,
const T* colData,
......@@ -100,7 +100,7 @@ public:
template <class T>
class Im2ColMobileFunctor {
public:
public:
void operator()(const T* imData,
const TensorShape& imShape,
T* colData,
......
paddle/function/Im2ColOp.cpp
浏览文件 @ 60d827a8
......@@ -23,7 +23,7 @@ namespace paddle {
*/
template <class T>
class Im2ColFunctor<kCFO, DEVICE_TYPE_CPU, T> {
public:
public:
void operator()(const T* imData,
const TensorShape& imShape,
T* colData,
......@@ -75,7 +75,7 @@ public:
*/
template <class T>
class Col2ImFunctor<kCFO, DEVICE_TYPE_CPU, T> {
public:
public:
void operator()(T* imData,
const TensorShape& imShape,
const T* colData,
......@@ -130,7 +130,7 @@ template class Col2ImFunctor<kCFO, DEVICE_TYPE_CPU, double>;
*/
template <class T>
class Im2ColFunctor<kOCF, DEVICE_TYPE_CPU, T> {
public:
public:
void operator()(const T* imData,
const TensorShape& imShape,
T* colData,
......@@ -188,7 +188,7 @@ public:
*/
template <class T>
class Col2ImFunctor<kOCF, DEVICE_TYPE_CPU, T> {
public:
public:
void operator()(T* imData,
const TensorShape& imShape,
const T* colData,
......
paddle/function/Im2ColOpGpu.cu
浏览文件 @ 60d827a8
......@@ -71,7 +71,7 @@ __global__ void im2col(const T* data_im,
*/
template <class T>
class Im2ColFunctor<kCFO, DEVICE_TYPE_GPU, T> {
public:
public:
void operator()(const T* imData,
const TensorShape& imShape,
T* colData,
......@@ -184,7 +184,7 @@ __global__ void col2im(size_t n,
*/
template <class T>
class Col2ImFunctor<kCFO, DEVICE_TYPE_GPU, T> {
public:
public:
void operator()(T* imData,
const TensorShape& imShape,
const T* colData,
......@@ -292,7 +292,7 @@ __global__ void im2colOCF(const T* imData,
*/
template <class T>
class Im2ColFunctor<kOCF, DEVICE_TYPE_GPU, T> {
public:
public:
void operator()(const T* imData,
const TensorShape& imShape,
T* colData,
......@@ -399,7 +399,7 @@ __global__ void col2imOCF(T* imData,
*/
template <class T>
class Col2ImFunctor<kOCF, DEVICE_TYPE_GPU, T> {
public:
public:
void operator()(T* imData,
const TensorShape& imShape,
const T* colData,
......
paddle/function/MulOp.cpp
浏览文件 @ 60d827a8
......@@ -240,7 +240,7 @@ void MulOp<DEVICE_TYPE_CPU>(CpuMatrix& out,
*/
template <DeviceType Device>
class MulFunc : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override {
aTrans_ = config.get<bool>("aTrans");
bTrans_ = config.get<bool>("bTrans");
......@@ -335,7 +335,7 @@ public:
}
}
private:
private:
bool aTrans_;
bool bTrans_;
};
......
paddle/function/NaiveConvOp.cpp
浏览文件 @ 60d827a8
......@@ -24,7 +24,7 @@ namespace paddle {
*/
template <class T>
class NaiveConvFunctor {
public:
public:
void operator()(const T* inputData,
size_t batchSize,
size_t inputChannels,
......@@ -85,7 +85,7 @@ public:
template <DeviceType Device>
class NaiveConvFunction : public ConvFunctionBase {
public:
public:
void init(const FuncConfig& config) override {
ConvFunctionBase::init(config);
}
......
paddle/function/PadOp.cpp
浏览文件 @ 60d827a8
......@@ -132,7 +132,7 @@ static inline PadConf castToPadConf(const FuncConfig& conf) {
template <DeviceType Device>
class PadFunc : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override { pad_ = castToPadConf(config); }
void calc(const BufferArgs& inputs, const BufferArgs& outputs) override {
......@@ -157,7 +157,7 @@ public:
pad_);
}
private:
private:
PadConf pad_;
};
......@@ -173,7 +173,7 @@ private:
template <DeviceType Device>
class PadGradFunc : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override { pad_ = castToPadConf(config); }
void calc(const BufferArgs& inputs, const BufferArgs& outputs) override {
......@@ -201,7 +201,7 @@ public:
pad_);
}
private:
private:
PadConf pad_;
};
......
paddle/function/RowConvOp.cpp
浏览文件 @ 60d827a8
......@@ -129,7 +129,7 @@ void RowConvGrad<DEVICE_TYPE_CPU>(const CpuMatrix& outG,
template <DeviceType Device>
class RowConvFunc : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override {}
void calc(const BufferArgs& inputs, const BufferArgs& outputs) override {
......@@ -176,7 +176,7 @@ public:
template <DeviceType Device>
class RowConvGradFunc : public FunctionBase {
// TODO(qingqing): split into RowConvDataFunc and RowConvWeightFunc
public:
public:
void init(const FuncConfig& config) override {}
void calc(const BufferArgs& inputs, const BufferArgs& outputs) override {
......
paddle/function/ScaleSubRegionOp.cpp
浏览文件 @ 60d827a8
......@@ -92,7 +92,7 @@ void ScaleSubRegionGrad<DEVICE_TYPE_CPU>(const real* inGrad,
*/
template <DeviceType Device>
class ScaleSubRegionFunc : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override { conf_ = config; }
void calc(const BufferArgs& inputs, const BufferArgs& outputs) override {
......@@ -109,7 +109,7 @@ public:
conf_);
}
private:
private:
FuncConfig conf_;
};
......@@ -124,7 +124,7 @@ private:
template <DeviceType Device>
class ScaleSubRegionGradFunc : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override { conf_ = config; }
void calc(const BufferArgs& inputs, const BufferArgs& outputs) override {
......@@ -141,7 +141,7 @@ public:
conf_);
}
private:
private:
FuncConfig conf_;
};
......
paddle/function/SwitchOp.cpp
浏览文件 @ 60d827a8
......@@ -75,7 +75,7 @@ void NHWC2NCHW<DEVICE_TYPE_CPU>(real* outputs,
*/
template <DeviceType Device>
class NCHW2NHWCFunc : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override {}
void calc(const BufferArgs& inputs, const BufferArgs& outputs) override {
......@@ -108,7 +108,7 @@ public:
*/
template <DeviceType Device>
class NHWC2NCHWFunc : public FunctionBase {
public:
public:
void init(const FuncConfig& config) override {}
void calc(const BufferArgs& inputs, const BufferArgs& outputs) override {
......
paddle/function/TensorShape.h
浏览文件 @ 60d827a8
......@@ -22,7 +22,7 @@ namespace paddle {
* TensorShape used to represent shape of normal tensor.
*/
class TensorShape {
public:
public:
TensorShape() : ndims_(0), nelements_(0) { initDims(0); }
TensorShape(size_t ndims) : ndims_(ndims), nelements_(1) { initDims(ndims); };
......@@ -80,7 +80,7 @@ public:
bool operator!=(const TensorShape& t) const { return !(*this == t); }
private:
private:
// compute number of elements
void numElements() {
nelements_ = 1;
......
paddle/function/neon/NeonDepthwiseConv.cpp
浏览文件 @ 60d827a8
......@@ -21,7 +21,7 @@ namespace paddle {
template <DeviceType Device>
class NeonDepthwiseConvFunction : public ConvFunctionBase {
public:
public:
void init(const FuncConfig& config) override {
ConvFunctionBase::init(config);
}
......
paddle/function/neon/NeonDepthwiseConvTranspose.cpp
浏览文件 @ 60d827a8
......@@ -21,7 +21,7 @@ namespace paddle {
template <DeviceType Device>
class NeonDepthwiseConvTransposeFunction : public ConvFunctionBase {
public:
public:
void init(const FuncConfig& config) override {
ConvFunctionBase::init(config);
}
......
paddle/function/nnpack/NNPACKConvOp.cpp
浏览文件 @ 60d827a8
......@@ -46,7 +46,7 @@ nnp_convolution_algorithm get_nnp_convolution_algorithm(
template <DeviceType Device>
class NNPACKConvFunction : public ConvFunctionBase {
public:
public:
void init(const FuncConfig& config) override {
ConvFunctionBase::init(config);
algorithm_ = get_nnp_convolution_algorithm(config.get<std::string>("algo"));
......@@ -231,7 +231,7 @@ public:
}
}
private:
private:
nnp_convolution_algorithm algorithm_;
nnp_convolution_transform_strategy transform_strategy_;
void* workspaceBuffer_;
......
paddle/gserver/activations/ActivationFunction.cpp
浏览文件 @ 60d827a8
......@@ -44,10 +44,10 @@ static ClassRegistrar<ActivationFunction> gActivationRegistrar;
*/
#define BEGIN_DEFINE_ACTIVATION(ACTIVATION_NAME) \
class ACTIVATION_CLASS_NAME(ACTIVATION_NAME) : public ActivationFunction { \
private: \
private: \
static const std::string name; \
\
public: \
public: \
const std::string& getName() const { return name; }
/**
* @def END_DEFINE_ACTIVATION
......@@ -70,7 +70,7 @@ static ClassRegistrar<ActivationFunction> gActivationRegistrar;
* Do nothing when forward/backward.
*/
class IdentityActivation : public ActivationFunction {
public:
public:
static const std::string name;
Error __must_check forward(Argument& act) {
(void)act;
......
paddle/gserver/activations/ActivationFunction.h
浏览文件 @ 60d827a8
......@@ -31,7 +31,7 @@ struct Argument;
*
*/
class ActivationFunction {
public:
public:
static ActivationFunction* create(const std::string& type);
static std::vector<std::string> getAllRegisteredTypes();
......
paddle/gserver/activations/MKLDNNActivation.cpp
浏览文件 @ 60d827a8
......@@ -35,10 +35,10 @@ static ClassRegistrar<ActivationFunction> gMKLDNNActivationRegistrar;
* @def END_MKLDNN_ACTIVATION
*/
#define END_MKLDNN_ACTIVATION(ACT_TYPE) \
private: \
private: \
static const std::string name; \
\
public: \
public: \
const std::string& getName() const { return name; } \
} \
; \
......@@ -63,11 +63,11 @@ public: \
#define DEFINE_MKLDNN_ELTWISE_ACTIVATION( \
ACT_TYPE, BASE_CLASS, ALPHA, BWD_ALPHA) \
BEGIN_MKLDNN_ACTIVATION(ACT_TYPE, BASE_CLASS) \
private: \
private: \
static const float alpha; \
static const float bwdAlpha; \
\
public: \
public: \
float getAlpha() const { return alpha; } \
float getBwdAlpha() const { return bwdAlpha; } \
END_MKLDNN_ACTIVATION(ACT_TYPE) \
......
paddle/gserver/activations/MKLDNNActivation.h
浏览文件 @ 60d827a8
......@@ -27,7 +27,7 @@ namespace paddle {
* including mkldnn_relu, mkldnn_elu, mkldnn_tanh, mkldnn_softmax
*/
class MKLDNNActivation : public ActivationFunction {
protected:
protected:
// input value element count
size_t cnt_;
// should not merge the resetBwd into resetFwd,
......@@ -43,7 +43,7 @@ protected:
std::vector<mkldnn::primitive> pipelineFwd_;
std::vector<mkldnn::primitive> pipelineBwd_;
public:
public:
MKLDNNActivation() : cnt_(0), needResetBwd_(true) {}
~MKLDNNActivation() {}
static ActivationFunction* create(const std::string& type);
......@@ -72,7 +72,7 @@ class MKLDNNEltwiseActivation : public MKLDNNActivation {
typedef mkldnn::eltwise_backward eltwise_bwd;
typedef mkldnn::algorithm algorithm;
protected:
protected:
// save the forward primitive desc, which can be used backward
std::shared_ptr<eltwise_fwd::primitive_desc> fwdPD_;
// eltwise_bwd need src input value
......@@ -80,7 +80,7 @@ protected:
// use for copy data
std::shared_ptr<mkldnn::reorder> copyInVal_;
public:
public:
MKLDNNEltwiseActivation() {}
~MKLDNNEltwiseActivation() {}
virtual const std::string& getName() const = 0;
......@@ -102,12 +102,12 @@ public:
class MKLDNNSoftmaxActivation : public MKLDNNActivation {
typedef mkldnn::softmax_forward softmax_fwd;
private:
private:
// for backward
MatrixPtr sftMaxSum_;
MatrixPtr sftMaxDot_;
public:
public:
MKLDNNSoftmaxActivation() {}
~MKLDNNSoftmaxActivation() {}
virtual const std::string& getName() const = 0;
......
paddle/gserver/dataproviders/DataProvider.h
浏览文件 @ 60d827a8
......@@ -71,7 +71,7 @@ typedef std::shared_ptr<BufferBatch> BufferBatchPtr;
* @brief Data for batch training a neural network
*/
class DataBatch {
public:
public:
DataBatch() : size_(0) { data_.clear(); }
/**
* @brief Get batch size
......@@ -181,7 +181,7 @@ public:
}
}
protected:
protected:
/**
* @brief batch size
*/
......@@ -194,7 +194,7 @@ protected:
};
class BufferBatch {
public:
public:
BufferBatch() {
hlStream_ = HPPL_STREAM_DEFAULT;
hlEvent_ = NULL;
......@@ -235,7 +235,7 @@ public:
void swap(BufferBatch* bufBatch);
void clone(DataBatch* srcBatch, bool useGpu);
protected:
protected:
DataBatch* batchData_;
hl_stream_t hlStream_;
hl_event_t hlEvent_;
......@@ -247,7 +247,7 @@ typedef std::shared_ptr<DataProvider> DataProviderPtr;
typedef Queue<BufferBatch*> BufferBatchQueue;
class DoubleBuffer {
public:
public:
DoubleBuffer(DataProvider* dataPool, bool useGpu, int64_t batchSize = 0);
virtual ~DoubleBuffer();
void removeOneBatch(DataBatch* dataBatch);
......@@ -267,7 +267,7 @@ public:
void setPending(bool pending) { pending_ = pending; }
protected:
protected:
virtual void asyncLoadBatch();
void insertOneBatch(DataBatch* batch);
......@@ -290,7 +290,7 @@ protected:
* one is for input, one is for label.
*/
class DataProvider {
public:
public:
static ClassRegistrar<DataProvider, DataConfig, ModelConfig, bool> registrar_;
static DataProvider* create(const DataConfig& config,
const ModelConfig& modelConfig,
......@@ -359,7 +359,7 @@ public:
*/
virtual int64_t getNextBatchInternal(int64_t size, DataBatch* batch) = 0;
protected:
protected:
DataConfig config_;
bool skipShuffle_;
float usageRatio_;
......@@ -382,7 +382,7 @@ protected:
* necessary configurations such as stream_names
*/
class DummyDataProvider : public DataProvider {
public:
public:
DummyDataProvider(const DataConfig& config, bool useGpu)
: DataProvider(config, useGpu) {}
virtual void shuffle() {}
......@@ -399,7 +399,7 @@ public:
* Data provider for one input and one integer label.
*/
class SimpleDataProviderBase : public DataProvider {
protected:
protected:
/// sample feature dimension
int64_t sampleDim_;
/// the number of samples
......@@ -425,7 +425,7 @@ protected:
RWLock lock_;
public:
public:
SimpleDataProviderBase(const DataConfig& config, bool useGpu, bool withInfo);
~SimpleDataProviderBase() {}
......@@ -440,7 +440,7 @@ public:
/// return the number of samples in the buffer
int64_t fillBuffer();
protected:
protected:
/**
* @brief Fill at most size samples into data and label.
*
......@@ -458,12 +458,12 @@ protected:
};
class SimpleDataProvider : public SimpleDataProviderBase {
public:
public:
SimpleDataProvider(const DataConfig& config, bool useGpu);
~SimpleDataProvider();
virtual void reset();
protected:
protected:
void loadData(const std::string& fileName);
void loadDataFile(const std::string& fileName);
virtual int64_t fillBufferImp(real* data,
......@@ -471,7 +471,7 @@ protected:
int* info,
int64_t size);
protected:
protected:
size_t currentSampleIndex_;
std::vector<int> labels_;
std::vector<real> data_;
......
paddle/gserver/dataproviders/DataProviderGroup.h
浏览文件 @ 60d827a8
......@@ -20,7 +20,7 @@ namespace paddle {
template <class T>
class DataProviderGroup : public DataProvider {
protected:
protected:
typedef T ProviderType;
typedef std::shared_ptr<ProviderType> ProviderPtrType;
ProviderPtrType provider_;
......@@ -29,7 +29,7 @@ protected:
std::mutex lock_;
std::unique_ptr<MultiThreadWorker<ProviderType>> loader_;
public:
public:
DataProviderGroup(const DataConfig& config, bool useGpu);
~DataProviderGroup() {}
......@@ -38,7 +38,7 @@ public:
virtual int64_t getSize() { return -1; }
virtual int64_t getNextBatchInternal(int64_t size, DataBatch* batch);
private:
private:
void startLoader();
void stopLoader();
void forceStopLoader();
......
paddle/gserver/dataproviders/MultiDataProvider.h
浏览文件 @ 60d827a8
......@@ -19,10 +19,10 @@ limitations under the License. */
namespace paddle {
class MultiDataProvider : public DataProvider {
protected:
protected:
std::vector<std::unique_ptr<DataProvider>> subDataProviders_;
public:
public:
MultiDataProvider(const DataConfig& config,
const ModelConfig& modelConfig,
bool useGpu);
......@@ -33,7 +33,7 @@ public:
virtual int64_t getNextBatchInternal(int64_t size, DataBatch* batch);
bool isTestMode() const { return isTestMode_; }
private:
private:
int totalDataRatio_;
bool isTestMode_;
};
......
paddle/gserver/dataproviders/ProtoReader.h
浏览文件 @ 60d827a8
......@@ -28,7 +28,7 @@ namespace paddle {
* messages from/to i/ostream.
*/
class ProtoReader {
public:
public:
explicit ProtoReader(std::istream* s, bool dataCompression = false) {
CHECK(s) << "istream pointer is nullptr";
istreamInput_.reset(new google::protobuf::io::IstreamInputStream(s));
......@@ -109,7 +109,7 @@ public:
return true;
}
protected:
protected:
std::unique_ptr<google::protobuf::io::ZeroCopyInputStream> istreamInput_;
std::unique_ptr<google::protobuf::io::GzipInputStream> gzipInput_;
std::unique_ptr<google::protobuf::io::CodedInputStream> codedInput_;
......@@ -144,7 +144,7 @@ protected:
};
class ProtoWriter {
public:
public:
explicit ProtoWriter(std::ostream* s, bool dataCompression = false) {
CHECK(s) << "ostream pointer is nullptr";
ostreamOutput_.reset(new google::protobuf::io::OstreamOutputStream(s));
......@@ -168,7 +168,7 @@ public:
return ret;
}
protected:
protected:
std::unique_ptr<google::protobuf::io::ZeroCopyOutputStream> ostreamOutput_;
std::unique_ptr<google::protobuf::io::GzipOutputStream> gzipOutput_;
std::unique_ptr<google::protobuf::io::CodedOutputStream> codedOutput_;
......
paddle/gserver/dataproviders/PyDataProvider.h
浏览文件 @ 60d827a8
......@@ -23,7 +23,7 @@ limitations under the License. */
namespace paddle {
class PyDataProvider : public DataProvider {
public:
public:
PyDataProvider(const DataConfig& config,
bool useGpu,
bool loadDataAll = true);
......@@ -40,7 +40,7 @@ public:
virtual int64_t getNextBatchInternal(int64_t size, DataBatch* batch);
protected:
protected:
struct ProtoSlot;
// return false if each each sample is one sequence, i.e., independent
// of other samples.
......@@ -73,7 +73,7 @@ protected:
void resetSlots();
void loadData(const std::vector<std::string>& fileList);
protected:
protected:
struct ProtoSlot {
SlotDef::SlotType type;
int dim;
......
paddle/gserver/dataproviders/PyDataProvider2.cpp
浏览文件 @ 60d827a8
......@@ -93,7 +93,7 @@ inline std::ostream& operator<<(std::ostream& os, const SlotHeader& header) {
* prepare step, fill data into argument during fill step.
*/
class IFieldScanner {
public:
public:
DISABLE_COPY(IFieldScanner);
/**
* Ctor.
......@@ -146,7 +146,7 @@ public:
*/
static IFieldScanner* create(SlotHeader* header);
protected:
protected:
SlotHeader* headerPtr_;
};
......@@ -154,7 +154,7 @@ protected:
* Py Data Provider Cache Interface.
*/
class IPyDataProviderCache {
public:
public:
virtual ~IPyDataProviderCache() {}
/**
......@@ -193,7 +193,7 @@ public:
* data. And it support cache strategies.
*/
class PyDataProvider2 : public DataProvider {
public:
public:
/**
* Ctor
*/
......@@ -234,7 +234,7 @@ public:
*/
virtual ~PyDataProvider2() { resetImpl(false); }
private:
private:
void createPyDataObj(const std::string& model,
const std::string& className,
const std::string& fileListName,
......@@ -435,7 +435,7 @@ private:
exit_ = false;
}
private:
private:
std::unique_ptr<std::thread> loadThread_;
std::atomic<bool> exit_;
std::deque<PyObjectPtr> callingContexts_;
......@@ -461,7 +461,7 @@ private:
static PyObjectPtr zeroTuple_;
class PositionRandom {
public:
public:
inline explicit PositionRandom(bool skipRand)
: eng_(ThreadLocalRandomEngine::get()), skipRand_(skipRand) {}
......@@ -476,14 +476,14 @@ private:
}
}
private:
private:
std::default_random_engine& eng_;
std::unique_ptr<std::uniform_int_distribution<size_t>> dist_;
bool skipRand_;
};
// DataProvider interface
public:
public:
/**
* Resetting the PyDataProvider. May start reading thread here.
*/
......@@ -666,7 +666,7 @@ REGISTER_DATA_PROVIDER_EX(py2, PyDataProvider2);
* Scanner for dense slot.
*/
class DenseScanner : public IFieldScanner {
public:
public:
explicit DenseScanner(SlotHeader* ptr) : IFieldScanner(ptr), height_(0) {}
/**
......@@ -708,7 +708,7 @@ public:
++height_;
}
private:
private:
size_t height_;
};
......@@ -716,7 +716,7 @@ private:
* Scanner for index slot
*/
class IndexScanner : public IFieldScanner {
public:
public:
explicit IndexScanner(SlotHeader* ptr) : IFieldScanner(ptr), cnt_(0) {}
/**
......@@ -740,12 +740,12 @@ public:
CHECK(ok) << "Cannot cast int " << py::repr(obj);
}
private:
private:
size_t cnt_;
};
class SparseNonValueScanner : public IFieldScanner {
public:
public:
explicit SparseNonValueScanner(SlotHeader* ptr)
: IFieldScanner(ptr), nnz_(0), height_(0) {}
......@@ -790,7 +790,7 @@ public:
++height_;
}
protected:
protected:
/**
* Set a single sparse index and value.
* @param [out] col sparse index
......@@ -809,7 +809,7 @@ protected:
};
class SparseValueScanner : public SparseNonValueScanner {
public:
public:
explicit SparseValueScanner(SlotHeader* ptr) : SparseNonValueScanner(ptr) {}
virtual void finishPrepare(Argument& argument) {
......@@ -817,7 +817,7 @@ public:
argument.value, height_, headerPtr_->dim, nnz_, FLOAT_VALUE);
}
protected:
protected:
virtual void setData(int* col, real* dat, PyObject* obj) {
py::SequenceHelper s(obj);
SparseNonValueScanner::setData(col, dat, s[0]);
......@@ -829,7 +829,7 @@ protected:
* Sequence Scanner. Scanner for sequence or sub-sequence.
*/
class SequenceScanner : public IFieldScanner {
public:
public:
/**
* Ctor
* @param innerScanner inner scanner for each timestep or sub-sequence.
......@@ -902,7 +902,7 @@ public:
*/
virtual void finishFill(Argument& argument) { inner_->finishFill(argument); }
protected:
protected:
size_t getSize(PyObject* obj) {
py::SequenceHelper s(obj);
auto sc = dynamic_cast<SequenceScanner*>(inner_.get());
......@@ -917,7 +917,7 @@ protected:
}
}
private:
private:
std::unique_ptr<IFieldScanner> inner_;
size_t cnt_;
std::function<ICpuGpuVectorPtr&(Argument&)> getSeqStartPos_;
......@@ -969,7 +969,7 @@ IFieldScanner* IFieldScanner::create(SlotHeader* header) {
* python every pass.
*/
class NoCacheStrategy : public IPyDataProviderCache {
public:
public:
virtual bool reset() { return true; }
virtual void drop(std::deque<PyObjectPtr>* data) { data->clear(); }
......@@ -984,7 +984,7 @@ public:
* The rest passes, will load data from memory.
*/
class CacheOnePassInMemory : public IPyDataProviderCache {
public:
public:
CacheOnePassInMemory()
: objPool_(new std::deque<PyObjectPtr>()),
droppedPool_(new std::deque<PyObjectPtr>()) {}
......@@ -1011,7 +1011,7 @@ public:
virtual std::deque<PyObjectPtr>* load() { return objPool_.get(); }
private:
private:
std::unique_ptr<std::deque<PyObjectPtr>> objPool_;
std::unique_ptr<std::deque<PyObjectPtr>> droppedPool_;
};
......
paddle/gserver/evaluators/CTCErrorEvaluator.cpp
浏览文件 @ 60d827a8
......@@ -22,7 +22,7 @@ namespace paddle {
* calculate sequence-to-sequence edit distance
*/
class CTCErrorEvaluator : public Evaluator {
private:
private:
MatrixPtr outActivations_;
int numTimes_, numClasses_, numSequences_, blank_;
real deletions_, insertions_, substitutions_;
......@@ -197,7 +197,7 @@ private:
(real)seqClassficationError_ / numSequences_;
}
public:
public:
CTCErrorEvaluator()
: numTimes_(0),
numClasses_(0),
......
paddle/gserver/evaluators/ChunkEvaluator.cpp
浏览文件 @ 60d827a8
......@@ -77,7 +77,7 @@ class ChunkEvaluator : public Evaluator {
std::set<int> excludedChunkTypes_;
mutable std::unordered_map<std::string, real> values_;
public:
public:
virtual void init(const EvaluatorConfig& config) {
Evaluator::init(config);
if (config.chunk_scheme() == "IOB") {
......@@ -276,7 +276,7 @@ public:
return "chunk";
}
private:
private:
void storeLocalValues() const {
CHECK_GE(numOutputSegments_, 0);
CHECK_GE(numLabelSegments_, 0);
......
paddle/gserver/evaluators/DetectionMAPEvaluator.cpp
浏览文件 @ 60d827a8
......@@ -28,7 +28,7 @@ namespace paddle {
* The config file api is detection_map_evaluator.
*/
class DetectionMAPEvaluator : public Evaluator {
public:
public:
DetectionMAPEvaluator()
: evaluateDifficult_(false), cpuOutput_(nullptr), cpuLabel_(nullptr) {}
......@@ -132,7 +132,7 @@ public:
LOG(FATAL) << "Distribute detection evaluation not implemented.";
}
protected:
protected:
void calcTFPos(const size_t batchSize,
const vector<map<size_t, vector<NormalizedBBox>>>& allGTBBoxes,
const vector<map<size_t, vector<pair<real, NormalizedBBox>>>>&
......@@ -287,7 +287,7 @@ protected:
real getValueImpl() const { return calcMAP(); }
private:
private:
real overlapThreshold_; // overlap threshold when determining whether matched
bool evaluateDifficult_; // whether evaluate difficult ground truth
size_t backgroundId_; // class index of background
......
paddle/gserver/evaluators/Evaluator.cpp
浏览文件 @ 60d827a8
......@@ -38,7 +38,7 @@ void Evaluator::eval(const NeuralNetwork& nn) {
* The config file api is classification_error_evaluator.
*/
class ClassificationErrorEvaluator : public Evaluator {
public:
public:
/*
ClassificationErrorEvaluator() : totalScore2_(0) {}
......@@ -124,7 +124,7 @@ public:
}
// Evaluator interface
protected:
protected:
std::string getTypeImpl() const { return "classification_error"; }
};
......@@ -135,7 +135,7 @@ protected:
*/
class SequenceClassificationErrorEvaluator
: public ClassificationErrorEvaluator {
public:
public:
virtual void updateSamplesNum(const std::vector<Argument>& arguments) {
numSamples_ += arguments[0].getNumSequences();
}
......@@ -166,7 +166,7 @@ public:
}
// Evaluator interface
protected:
protected:
std::string getTypeImpl() const { return "seq_classification_error"; }
};
REGISTER_EVALUATOR(seq_classification_error,
......@@ -178,7 +178,7 @@ REGISTER_EVALUATOR(seq_classification_error,
* The config file api is sum_evaluator.
*/
class SumEvaluator : public Evaluator {
public:
public:
SumEvaluator() : cpuLabel_(nullptr), cpuWeight_(nullptr) {}
virtual void updateSamplesNum(const std::vector<Argument>& arguments) {
......@@ -255,12 +255,12 @@ public:
mergeResultsOfAllClients(client);
}
private:
private:
IVectorPtr cpuLabel_;
MatrixPtr cpuWeight_;
// Evaluator interface
protected:
protected:
std::string getTypeImpl() const { return "sum"; }
};
/**
......@@ -274,7 +274,7 @@ protected:
*
*/
class ColumnSumEvaluator : public Evaluator {
public:
public:
explicit ColumnSumEvaluator(int32_t colIdx)
: colIdx_(colIdx), colNum_(0), sum_(nullptr) {}
......@@ -368,13 +368,13 @@ public:
client->reduce(&numSamples_, &numSamples_, 1, FLAGS_trainer_id, 0);
}
private:
private:
int32_t colIdx_;
size_t colNum_;
MatrixPtr sum_; /* cpu matrix */
// Evaluator interface
protected:
protected:
std::string getTypeImpl() const {
if (colIdx_ == -1)
return "last-column-sum";
......@@ -1018,7 +1018,7 @@ static InitFunction __reg_type_auc_sum__([]() {
* The config file api is value_printer_evaluator.
*/
class ValuePrinter : public NotGetableEvaluator {
public:
public:
virtual void eval(const NeuralNetwork& nn) {
for (const std::string& name : config_.input_layers()) {
nn.getLayer(name)->getOutput().printValueString(LOG(INFO),
......@@ -1038,7 +1038,7 @@ REGISTER_EVALUATOR(value_printer, ValuePrinter);
* The config file api is gradient_printer_evaluator.
*/
class GradientPrinter : public NotGetableEvaluator {
public:
public:
virtual void eval(const NeuralNetwork& nn) {
for (const std::string& name : config_.input_layers()) {
const Argument& argu = nn.getLayer(name)->getOutput();
......@@ -1061,11 +1061,11 @@ REGISTER_EVALUATOR(gradient_printer, GradientPrinter);
* The config file api is maxid_printer_evaluator.
*/
class MaxIdPrinter : public NotGetableEvaluator {
private:
private:
IVectorPtr maxIds_;
MatrixPtr maxValues_;
public:
public:
MaxIdPrinter() {}
virtual void eval(const NeuralNetwork& nn) {
......@@ -1103,12 +1103,12 @@ REGISTER_EVALUATOR(max_id_printer, MaxIdPrinter);
* The config file api is maxframe_printer_evaluator.
*/
class MaxFramePrinter : public NotGetableEvaluator {
private:
private:
IVectorPtr maxIds_;
MatrixPtr maxValues_;
MatrixPtr value_;
public:
public:
MaxFramePrinter() {
value_ =
Matrix::create(nullptr, /* height= */ 1, 1, /* trans= */ false, false);
......@@ -1190,7 +1190,7 @@ REGISTER_EVALUATOR(max_frame_printer, MaxFramePrinter);
*
*/
class SequenceTextPrinter : public NotGetableEvaluator {
private:
private:
/// dict_file, which contains a list of tokens
std::vector<std::string> dict_;
/// result_file, which is the output file
......@@ -1203,7 +1203,7 @@ private:
/// store the probability associated with each sequence
std::vector<MatrixPtr> cpuIn_;
public:
public:
SequenceTextPrinter() {}
virtual void init(const EvaluatorConfig& config) {
......@@ -1334,7 +1334,7 @@ REGISTER_EVALUATOR(seq_text_printer, SequenceTextPrinter);
* The config file api is classification_error_printer_evaluator.
*/
class ClassificationErrorPrinter : public ClassificationErrorEvaluator {
public:
public:
virtual void updateSamplesNum(const std::vector<Argument>& arguments) {}
virtual real evalImp(std::vector<Argument>& arguments) {
......
paddle/gserver/gradientmachines/GradientMachine.h
浏览文件 @ 60d827a8
......@@ -73,7 +73,7 @@ class GradientMachine;
typedef std::shared_ptr<GradientMachine> GradientMachinePtr;
class GradientMachine {
public:
public:
enum CreateMode {
kNormal = 0,
kSgdSparseCpuTraining = 3,
......@@ -240,7 +240,7 @@ public:
*/
virtual void releaseOutput() {}
protected:
protected:
virtual void onLoadParameter() {}
std::vector<ParameterPtr> parameters_;
......
paddle/gserver/gradientmachines/GradientMachineMode.h
浏览文件 @ 60d827a8
......@@ -19,14 +19,14 @@ limitations under the License. */
namespace paddle {
class IGradientMachineMode {
public:
public:
virtual ~IGradientMachineMode() {}
public: // interfaces
/**
* @brief create current mode's gradient machine by model config.
* @param config model config
*/
public: // interfaces
/**
* @brief create current mode's gradient machine by model config.
* @param config model config
*/
virtual GradientMachine* create(const ModelConfig& config) = 0;
/**
......@@ -55,14 +55,14 @@ public: // interfaces
*/
virtual bool needTrainWholeDataInOneBatch() const = 0;
public: // static methods.
/**
* @brief register a custom gradient machine mode.
* @note For user to register a custom gradient machine mode, id should >=
* kCustom.
* @param mode mode id.
* @param ptr mode description object.
*/
public: // static methods.
/**
* @brief register a custom gradient machine mode.
* @note For user to register a custom gradient machine mode, id should >=
* kCustom.
* @param mode mode id.
* @param ptr mode description object.
*/
static void regGradientMachineMode(
int32_t mode, std::unique_ptr<IGradientMachineMode>&& ptr) {
modes_.insert(std::make_pair(mode, std::move(ptr)));
......@@ -141,7 +141,7 @@ public: // static methods.
}
}
private:
private:
static std::unordered_map<int32_t, std::unique_ptr<IGradientMachineMode>>
modes_;
};
......
paddle/gserver/layers/AddtoLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/AgentLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/CRFLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/CTCLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/ClipLayer.cpp
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/Conv3DLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/CosSimLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/CostLayer.cpp
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/CostLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/CropLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/DataLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/ExpandLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/GruCompute.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/Layer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/LstmCompute.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/LstmLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/MKLDNNBase.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/MKLDNNLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/MaxLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/MaxOutLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/MixedLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/NCELayer.cpp
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/NormLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/Operator.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/PadLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/Pool3DLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/PoolLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/PriorBox.cpp
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/Projection.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/RotateLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/TensorLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/layers/TransLayer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/gserver/tests/MKLDNNTester.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/Allocator.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/BaseMatrix.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/CpuSparseMatrix.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/ExecViaCpu.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/MKLDNNMatrix.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/MathFunctions.cpp
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/Matrix.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/MatrixBitCode.cpp
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/MemoryHandle.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/PoolAllocator.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/RowBuffer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/SparseMatrix.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/SparseRowMatrix.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/Storage.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/TensorApply.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/TensorAssign.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/TensorExpression.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/Vector.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/tests/TensorCheck.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/math/tests/TestUtils.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/optimizer/adam_optimizer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/optimizer/lr_policy.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/optimizer/sgd_optimizer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/optimizer/tensor.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/parameter/AverageOptimizer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/parameter/Parameter.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/parameter/Regularizer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/parameter/Weight.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/pserver/BaseClient.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/pserver/LightNetwork.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/pserver/ParameterClient2.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/pserver/ParameterServer2.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/pserver/ProtoServer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/pserver/SocketChannel.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/pserver/test/SocketTest.cpp
浏览文件 @ 60d827a8
此差异已折叠。
paddle/trainer/ParamUtil.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/trainer/ParameterUpdater.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/trainer/Tester.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/trainer/Trainer.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/trainer/TrainerInternal.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/trainer/tests/picojson.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/utils/ClassRegistrar.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/utils/CpuId.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/utils/CustomStackTrace.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/utils/Error.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/utils/GlobalConstants.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/utils/Locks.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/utils/PythonUtil.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/utils/Queue.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/utils/Stat.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/utils/Thread.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/utils/ThreadLocal.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/utils/Util.h
浏览文件 @ 60d827a8
此差异已折叠。
paddle/utils/arch/linux/Locks.cpp
浏览文件 @ 60d827a8
此差异已折叠。
paddle/utils/arch/osx/Locks.cpp
浏览文件 @ 60d827a8
此差异已折叠。
tools/codestyle/docstring_checker.py 0 → 100644
浏览文件 @ 60d827a8
此差异已折叠。
tools/codestyle/pylint_pre_commit.hook 0 → 100755
浏览文件 @ 60d827a8
此差异已折叠。
tools/codestyle/test_docstring_checker.py 0 → 100644
浏览文件 @ 60d827a8
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或