Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
PaddlePaddle
Paddle
提交
70782e63
P
Paddle
项目概览
PaddlePaddle
/
Paddle
大约 2 年 前同步成功
通知
2325
Star
20933
Fork
5424
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
1423
列表
看板
标记
里程碑
合并请求
543
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
P
Paddle
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
1,423
Issue
1,423
列表
看板
标记
里程碑
合并请求
543
合并请求
543
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
未验证
提交
70782e63
编写于
4月 03, 2020
作者:
Z
Zhaolong Xing
提交者:
GitHub
4月 03, 2020
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[Inference doc]: refine paddle_api.h doc (#23354)
* refine paddle api doc test=develop * fix comments test=develop
上级
bcafe317
变更
1
隐藏空白更改
内联
并排
Showing
1 changed file
with
82 addition
and
61 deletion
+82
-61
paddle/fluid/inference/api/paddle_api.h
paddle/fluid/inference/api/paddle_api.h
+82
-61
未找到文件。
paddle/fluid/inference/api/paddle_api.h
浏览文件 @
70782e63
...
@@ -143,42 +143,69 @@ struct PaddleTensor {
...
@@ -143,42 +143,69 @@ struct PaddleTensor {
enum
class
PaddlePlace
{
kUNK
=
-
1
,
kCPU
,
kGPU
};
enum
class
PaddlePlace
{
kUNK
=
-
1
,
kCPU
,
kGPU
};
/** Tensor without copy, currently only supports `AnalysisPredictor`.
/// \brief Represents an n-dimensional array of values.
*/
/// The ZeroCopyTensor is used to store the input or output of the network.
/// Zero copy means that the tensor supports direct copy of host or device data
/// to device,
/// eliminating additional CPU copy. ZeroCopyTensor is only used in the
/// AnalysisPredictor.
/// It is obtained through PaddlePredictor::GetinputTensor()
/// and PaddlePredictor::GetOutputTensor() interface.
class
ZeroCopyTensor
{
class
ZeroCopyTensor
{
public:
public:
/// \brief Reset the shape of the tensor.
/// Generally it's only used for the input tensor.
/// Reshape must be called before calling mutable_data() or copy_from_cpu()
/// \param shape The shape to set.
void
Reshape
(
const
std
::
vector
<
int
>&
shape
);
void
Reshape
(
const
std
::
vector
<
int
>&
shape
);
/** Get the memory in CPU or GPU with specific data type, should Reshape first
/// \brief Get the memory pointer in CPU or GPU with specific data type.
* to tell the data size.
/// Please Reshape the tensor first before call this.
* One can directly call this data to feed the data.
/// It's usually used to get input data pointer.
* This is for writing the input tensor.
/// \param place The place of the tensor.
*/
template
<
typename
T
>
template
<
typename
T
>
T
*
mutable_data
(
PaddlePlace
place
);
T
*
mutable_data
(
PaddlePlace
place
);
/** Get the memory directly, will return the place and element size by
* pointer.
/// \brief Get the memory pointer directly.
* This is for reading the output tensor.
/// It's usually used to get the output data pointer.
*/
/// \param[out] place To get the device type of the tensor.
/// \param[out] size To get the data size of the tensor.
/// \return The tensor data buffer pointer.
template
<
typename
T
>
template
<
typename
T
>
T
*
data
(
PaddlePlace
*
place
,
int
*
size
)
const
;
T
*
data
(
PaddlePlace
*
place
,
int
*
size
)
const
;
/// \brief Copy the host memory to tensor data.
/// It's usually used to set the input tensor data.
/// \param data The pointer of the data, from which the tensor will copy.
template
<
typename
T
>
template
<
typename
T
>
void
copy_from_cpu
(
const
T
*
data
);
void
copy_from_cpu
(
const
T
*
data
);
/// \brief Copy the tensor data to the host memory.
/// It's usually used to get the output tensor data.
/// \param[out] data The tensor will copy the data to the address.
template
<
typename
T
>
template
<
typename
T
>
void
copy_to_cpu
(
T
*
data
);
void
copy_to_cpu
(
T
*
data
);
/// \brief Return the shape of the Tensor.
std
::
vector
<
int
>
shape
()
const
;
std
::
vector
<
int
>
shape
()
const
;
/// \brief Set lod info of the tensor.
/// More about LOD can be seen here:
/// https://www.paddlepaddle.org.cn/documentation/docs/zh/beginners_guide/basic_concept/lod_tensor.html#lodtensor
/// \param x the lod info.
void
SetLoD
(
const
std
::
vector
<
std
::
vector
<
size_t
>>&
x
);
void
SetLoD
(
const
std
::
vector
<
std
::
vector
<
size_t
>>&
x
);
/// \brief Return the lod info of the tensor.
std
::
vector
<
std
::
vector
<
size_t
>>
lod
()
const
;
std
::
vector
<
std
::
vector
<
size_t
>>
lod
()
const
;
/// \brief Return the name of the tensor.
const
std
::
string
&
name
()
const
{
return
name_
;
}
const
std
::
string
&
name
()
const
{
return
name_
;
}
void
SetPlace
(
PaddlePlace
place
,
int
device
=
-
1
)
{
void
SetPlace
(
PaddlePlace
place
,
int
device
=
-
1
)
{
place_
=
place
;
place_
=
place
;
device_
=
device
;
device_
=
device
;
}
}
/// \brief Return the data type of the tensor.
/// It's usually used to get the output tensor data type.
/// \return The data type of the tensor.
PaddleDType
type
()
const
;
PaddleDType
type
()
const
;
protected:
protected:
...
@@ -199,8 +226,8 @@ class ZeroCopyTensor {
...
@@ -199,8 +226,8 @@ class ZeroCopyTensor {
int
device_
;
int
device_
;
};
};
/
** A simple Inference API for Paddle
.
/
// \brief A Predictor for executing inference on a model
.
*/
/// Base class for AnalysisPredictor and NativePaddlePredictor.
class
PaddlePredictor
{
class
PaddlePredictor
{
public:
public:
struct
Config
;
struct
Config
;
...
@@ -208,85 +235,79 @@ class PaddlePredictor {
...
@@ -208,85 +235,79 @@ class PaddlePredictor {
PaddlePredictor
(
const
PaddlePredictor
&
)
=
delete
;
PaddlePredictor
(
const
PaddlePredictor
&
)
=
delete
;
PaddlePredictor
&
operator
=
(
const
PaddlePredictor
&
)
=
delete
;
PaddlePredictor
&
operator
=
(
const
PaddlePredictor
&
)
=
delete
;
/** Predict an record.
/// \brief This interface takes input and runs the network.
* The caller should be responsible for allocating and releasing the memory of
/// There are redundant copies of data between hosts in this operation,
* `inputs`. `inputs` should be available until Run returns. Caller should be
/// so it is more recommended to use the zecopyrun interface
* responsible for the output tensor's buffer, either allocated or passed from
/// \param[in] inputs An list of PaddleTensor as the input to the network.
* outside.
/// \param[out] output_data Pointer to the tensor list, which holds the output
*/
/// paddletensor
/// \param[in] batch_size This setting has been discarded and can be ignored.
/// \return Whether the run is successful
virtual
bool
Run
(
const
std
::
vector
<
PaddleTensor
>&
inputs
,
virtual
bool
Run
(
const
std
::
vector
<
PaddleTensor
>&
inputs
,
std
::
vector
<
PaddleTensor
>*
output_data
,
std
::
vector
<
PaddleTensor
>*
output_data
,
int
batch_size
=
-
1
)
=
0
;
int
batch_size
=
-
1
)
=
0
;
/** \brief Get input names of the model
/// \brief Used to get the name of the network input.
*/
/// Be inherited by AnalysisPredictor, Only used in ZeroCopy scenarios.
/// \return Input tensor names.
virtual
std
::
vector
<
std
::
string
>
GetInputNames
()
{
return
{};
}
virtual
std
::
vector
<
std
::
string
>
GetInputNames
()
{
return
{};
}
/
** \brief Get input shapes of the model
/
// \brief Get the input shape of the model.
*/
/// \return A map contains all the input names and shape defined in the model.
virtual
std
::
map
<
std
::
string
,
std
::
vector
<
int64_t
>>
GetInputTensorShape
()
{
virtual
std
::
map
<
std
::
string
,
std
::
vector
<
int64_t
>>
GetInputTensorShape
()
{
return
{};
return
{};
}
}
/** \brief Get output names of the model
/// \brief Used to get the name of the network output.
*/
/// Be inherited by AnalysisPredictor, Only used in ZeroCopy scenarios.
/// \return Output tensor names.
virtual
std
::
vector
<
std
::
string
>
GetOutputNames
()
{
return
{};
}
virtual
std
::
vector
<
std
::
string
>
GetOutputNames
()
{
return
{};
}
/** \brief Get a mutable tensor directly.
/// \brief Get the input ZeroCopyTensor by name.
*
/// Be inherited by AnalysisPredictor, Only used in ZeroCopy scenarios.
* NOTE Only works in AnalysisPredictor.
/// The name is obtained from the GetInputNames() interface.
*
/// \param name The input tensor name.
* One can also use this to modify any temporary variable related tensors in
/// \return Return the corresponding input ZeroCopyTensor.
* the predictor.
*
*/
virtual
std
::
unique_ptr
<
ZeroCopyTensor
>
GetInputTensor
(
virtual
std
::
unique_ptr
<
ZeroCopyTensor
>
GetInputTensor
(
const
std
::
string
&
name
)
{
const
std
::
string
&
name
)
{
return
nullptr
;
return
nullptr
;
}
}
/**
* \brief Get an immutable tensor without copy.
/// \brief Get the output ZeroCopyTensor by name.
*
/// Be inherited by AnalysisPredictor, Only used in ZeroCopy scenarios.
* NOTE Only works in AnalysisPredictor.
/// The name is obtained from the GetOutputNames() interface.
* One can use this API to get any temporary tensors in the predictor and
/// \param name The output tensor name.
* read it.
/// \return Return the corresponding output ZeroCopyTensor.
*/
virtual
std
::
unique_ptr
<
ZeroCopyTensor
>
GetOutputTensor
(
virtual
std
::
unique_ptr
<
ZeroCopyTensor
>
GetOutputTensor
(
const
std
::
string
&
name
)
{
const
std
::
string
&
name
)
{
return
nullptr
;
return
nullptr
;
}
}
/**
/// \brief Run the network with zero-copied inputs and outputs.
* \brief Run the predictor with zero-copied inputs and outputs.
/// Be inherited by AnalysisPredictor and only used in ZeroCopy scenarios.
*
/// This will save the IO copy for transfering inputs and outputs to predictor
* NOTE Only works in AnalysisPredictor.
/// workspace
*
/// and get some performance improvement.
* This will save the IO copy for transfering inputs and outputs to predictor
/// To use it, one should call the AnalysisConfig.SwitchUseFeedFetchOp(true)
* workspace and get some performance improvement.
/// and then use the `GetInputTensor` and `GetOutputTensor`
* To use it, one should call the `AnalysisConfig.SwitchUseFeedFetchOp(true)`
/// to directly write or read the input/output tensors.
* and then use the `GetInputTensor` and `GetOutputTensor` to directly write
/// \return Whether the run is successful
* or read the input/output tensors.
*/
virtual
bool
ZeroCopyRun
()
{
return
false
;
}
virtual
bool
ZeroCopyRun
()
{
return
false
;
}
/** Clone a predictor that share the model weights, the Cloned predictor
/// \brief Clone an existing predictor
* should be thread-safe.
/// When using clone, the same network will be created,
*/
/// and the parameters between them are shared.
/// \return unique_ptr which contains the pointer of predictor
virtual
std
::
unique_ptr
<
PaddlePredictor
>
Clone
()
=
0
;
virtual
std
::
unique_ptr
<
PaddlePredictor
>
Clone
()
=
0
;
/** Destroy the Predictor.
/// \brief Destroy the Predictor.
*/
virtual
~
PaddlePredictor
()
=
default
;
virtual
~
PaddlePredictor
()
=
default
;
/** \brief Get the serialized model program that executes in inference phase.
* Its data type is ProgramDesc, which is a protobuf message.
*/
virtual
std
::
string
GetSerializedProgram
()
const
{
virtual
std
::
string
GetSerializedProgram
()
const
{
assert
(
false
);
// Force raise error.
assert
(
false
);
// Force raise error.
return
"NotImplemented"
;
return
"NotImplemented"
;
}
}
/** The common configs for all the predictors.
/// \brief Base class for NativeConfig and AnalysisConfig.
*/
struct
Config
{
struct
Config
{
std
::
string
model_dir
;
/*!< path to the model directory. */
std
::
string
model_dir
;
/*!< path to the model directory. */
};
};
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录