Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
机器未来
Paddle
提交
70782e63
P
Paddle
项目概览
机器未来
/
Paddle
与 Fork 源项目一致
Fork自
PaddlePaddle / Paddle
通知
1
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
1
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
P
Paddle
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
1
Issue
1
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
未验证
提交
70782e63
编写于
4月 03, 2020
作者:
Z
Zhaolong Xing
提交者:
GitHub
4月 03, 2020
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[Inference doc]: refine paddle_api.h doc (#23354)
* refine paddle api doc test=develop * fix comments test=develop
上级
bcafe317
变更
1
隐藏空白更改
内联
并排
Showing
1 changed file
with
82 addition
and
61 deletion
+82
-61
paddle/fluid/inference/api/paddle_api.h
paddle/fluid/inference/api/paddle_api.h
+82
-61
未找到文件。
paddle/fluid/inference/api/paddle_api.h
浏览文件 @
70782e63
...
@@ -143,42 +143,69 @@ struct PaddleTensor {
...
@@ -143,42 +143,69 @@ struct PaddleTensor {
enum
class
PaddlePlace
{
kUNK
=
-
1
,
kCPU
,
kGPU
};
enum
class
PaddlePlace
{
kUNK
=
-
1
,
kCPU
,
kGPU
};
/** Tensor without copy, currently only supports `AnalysisPredictor`.
/// \brief Represents an n-dimensional array of values.
*/
/// The ZeroCopyTensor is used to store the input or output of the network.
/// Zero copy means that the tensor supports direct copy of host or device data
/// to device,
/// eliminating additional CPU copy. ZeroCopyTensor is only used in the
/// AnalysisPredictor.
/// It is obtained through PaddlePredictor::GetinputTensor()
/// and PaddlePredictor::GetOutputTensor() interface.
class
ZeroCopyTensor
{
class
ZeroCopyTensor
{
public:
public:
/// \brief Reset the shape of the tensor.
/// Generally it's only used for the input tensor.
/// Reshape must be called before calling mutable_data() or copy_from_cpu()
/// \param shape The shape to set.
void
Reshape
(
const
std
::
vector
<
int
>&
shape
);
void
Reshape
(
const
std
::
vector
<
int
>&
shape
);
/** Get the memory in CPU or GPU with specific data type, should Reshape first
/// \brief Get the memory pointer in CPU or GPU with specific data type.
* to tell the data size.
/// Please Reshape the tensor first before call this.
* One can directly call this data to feed the data.
/// It's usually used to get input data pointer.
* This is for writing the input tensor.
/// \param place The place of the tensor.
*/
template
<
typename
T
>
template
<
typename
T
>
T
*
mutable_data
(
PaddlePlace
place
);
T
*
mutable_data
(
PaddlePlace
place
);
/** Get the memory directly, will return the place and element size by
* pointer.
/// \brief Get the memory pointer directly.
* This is for reading the output tensor.
/// It's usually used to get the output data pointer.
*/
/// \param[out] place To get the device type of the tensor.
/// \param[out] size To get the data size of the tensor.
/// \return The tensor data buffer pointer.
template
<
typename
T
>
template
<
typename
T
>
T
*
data
(
PaddlePlace
*
place
,
int
*
size
)
const
;
T
*
data
(
PaddlePlace
*
place
,
int
*
size
)
const
;
/// \brief Copy the host memory to tensor data.
/// It's usually used to set the input tensor data.
/// \param data The pointer of the data, from which the tensor will copy.
template
<
typename
T
>
template
<
typename
T
>
void
copy_from_cpu
(
const
T
*
data
);
void
copy_from_cpu
(
const
T
*
data
);
/// \brief Copy the tensor data to the host memory.
/// It's usually used to get the output tensor data.
/// \param[out] data The tensor will copy the data to the address.
template
<
typename
T
>
template
<
typename
T
>
void
copy_to_cpu
(
T
*
data
);
void
copy_to_cpu
(
T
*
data
);
/// \brief Return the shape of the Tensor.
std
::
vector
<
int
>
shape
()
const
;
std
::
vector
<
int
>
shape
()
const
;
/// \brief Set lod info of the tensor.
/// More about LOD can be seen here:
/// https://www.paddlepaddle.org.cn/documentation/docs/zh/beginners_guide/basic_concept/lod_tensor.html#lodtensor
/// \param x the lod info.
void
SetLoD
(
const
std
::
vector
<
std
::
vector
<
size_t
>>&
x
);
void
SetLoD
(
const
std
::
vector
<
std
::
vector
<
size_t
>>&
x
);
/// \brief Return the lod info of the tensor.
std
::
vector
<
std
::
vector
<
size_t
>>
lod
()
const
;
std
::
vector
<
std
::
vector
<
size_t
>>
lod
()
const
;
/// \brief Return the name of the tensor.
const
std
::
string
&
name
()
const
{
return
name_
;
}
const
std
::
string
&
name
()
const
{
return
name_
;
}
void
SetPlace
(
PaddlePlace
place
,
int
device
=
-
1
)
{
void
SetPlace
(
PaddlePlace
place
,
int
device
=
-
1
)
{
place_
=
place
;
place_
=
place
;
device_
=
device
;
device_
=
device
;
}
}
/// \brief Return the data type of the tensor.
/// It's usually used to get the output tensor data type.
/// \return The data type of the tensor.
PaddleDType
type
()
const
;
PaddleDType
type
()
const
;
protected:
protected:
...
@@ -199,8 +226,8 @@ class ZeroCopyTensor {
...
@@ -199,8 +226,8 @@ class ZeroCopyTensor {
int
device_
;
int
device_
;
};
};
/
** A simple Inference API for Paddle
.
/
// \brief A Predictor for executing inference on a model
.
*/
/// Base class for AnalysisPredictor and NativePaddlePredictor.
class
PaddlePredictor
{
class
PaddlePredictor
{
public:
public:
struct
Config
;
struct
Config
;
...
@@ -208,85 +235,79 @@ class PaddlePredictor {
...
@@ -208,85 +235,79 @@ class PaddlePredictor {
PaddlePredictor
(
const
PaddlePredictor
&
)
=
delete
;
PaddlePredictor
(
const
PaddlePredictor
&
)
=
delete
;
PaddlePredictor
&
operator
=
(
const
PaddlePredictor
&
)
=
delete
;
PaddlePredictor
&
operator
=
(
const
PaddlePredictor
&
)
=
delete
;
/** Predict an record.
/// \brief This interface takes input and runs the network.
* The caller should be responsible for allocating and releasing the memory of
/// There are redundant copies of data between hosts in this operation,
* `inputs`. `inputs` should be available until Run returns. Caller should be
/// so it is more recommended to use the zecopyrun interface
* responsible for the output tensor's buffer, either allocated or passed from
/// \param[in] inputs An list of PaddleTensor as the input to the network.
* outside.
/// \param[out] output_data Pointer to the tensor list, which holds the output
*/
/// paddletensor
/// \param[in] batch_size This setting has been discarded and can be ignored.
/// \return Whether the run is successful
virtual
bool
Run
(
const
std
::
vector
<
PaddleTensor
>&
inputs
,
virtual
bool
Run
(
const
std
::
vector
<
PaddleTensor
>&
inputs
,
std
::
vector
<
PaddleTensor
>*
output_data
,
std
::
vector
<
PaddleTensor
>*
output_data
,
int
batch_size
=
-
1
)
=
0
;
int
batch_size
=
-
1
)
=
0
;
/** \brief Get input names of the model
/// \brief Used to get the name of the network input.
*/
/// Be inherited by AnalysisPredictor, Only used in ZeroCopy scenarios.
/// \return Input tensor names.
virtual
std
::
vector
<
std
::
string
>
GetInputNames
()
{
return
{};
}
virtual
std
::
vector
<
std
::
string
>
GetInputNames
()
{
return
{};
}
/
** \brief Get input shapes of the model
/
// \brief Get the input shape of the model.
*/
/// \return A map contains all the input names and shape defined in the model.
virtual
std
::
map
<
std
::
string
,
std
::
vector
<
int64_t
>>
GetInputTensorShape
()
{
virtual
std
::
map
<
std
::
string
,
std
::
vector
<
int64_t
>>
GetInputTensorShape
()
{
return
{};
return
{};
}
}
/** \brief Get output names of the model
/// \brief Used to get the name of the network output.
*/
/// Be inherited by AnalysisPredictor, Only used in ZeroCopy scenarios.
/// \return Output tensor names.
virtual
std
::
vector
<
std
::
string
>
GetOutputNames
()
{
return
{};
}
virtual
std
::
vector
<
std
::
string
>
GetOutputNames
()
{
return
{};
}
/** \brief Get a mutable tensor directly.
/// \brief Get the input ZeroCopyTensor by name.
*
/// Be inherited by AnalysisPredictor, Only used in ZeroCopy scenarios.
* NOTE Only works in AnalysisPredictor.
/// The name is obtained from the GetInputNames() interface.
*
/// \param name The input tensor name.
* One can also use this to modify any temporary variable related tensors in
/// \return Return the corresponding input ZeroCopyTensor.
* the predictor.
*
*/
virtual
std
::
unique_ptr
<
ZeroCopyTensor
>
GetInputTensor
(
virtual
std
::
unique_ptr
<
ZeroCopyTensor
>
GetInputTensor
(
const
std
::
string
&
name
)
{
const
std
::
string
&
name
)
{
return
nullptr
;
return
nullptr
;
}
}
/**
* \brief Get an immutable tensor without copy.
/// \brief Get the output ZeroCopyTensor by name.
*
/// Be inherited by AnalysisPredictor, Only used in ZeroCopy scenarios.
* NOTE Only works in AnalysisPredictor.
/// The name is obtained from the GetOutputNames() interface.
* One can use this API to get any temporary tensors in the predictor and
/// \param name The output tensor name.
* read it.
/// \return Return the corresponding output ZeroCopyTensor.
*/
virtual
std
::
unique_ptr
<
ZeroCopyTensor
>
GetOutputTensor
(
virtual
std
::
unique_ptr
<
ZeroCopyTensor
>
GetOutputTensor
(
const
std
::
string
&
name
)
{
const
std
::
string
&
name
)
{
return
nullptr
;
return
nullptr
;
}
}
/**
/// \brief Run the network with zero-copied inputs and outputs.
* \brief Run the predictor with zero-copied inputs and outputs.
/// Be inherited by AnalysisPredictor and only used in ZeroCopy scenarios.
*
/// This will save the IO copy for transfering inputs and outputs to predictor
* NOTE Only works in AnalysisPredictor.
/// workspace
*
/// and get some performance improvement.
* This will save the IO copy for transfering inputs and outputs to predictor
/// To use it, one should call the AnalysisConfig.SwitchUseFeedFetchOp(true)
* workspace and get some performance improvement.
/// and then use the `GetInputTensor` and `GetOutputTensor`
* To use it, one should call the `AnalysisConfig.SwitchUseFeedFetchOp(true)`
/// to directly write or read the input/output tensors.
* and then use the `GetInputTensor` and `GetOutputTensor` to directly write
/// \return Whether the run is successful
* or read the input/output tensors.
*/
virtual
bool
ZeroCopyRun
()
{
return
false
;
}
virtual
bool
ZeroCopyRun
()
{
return
false
;
}
/** Clone a predictor that share the model weights, the Cloned predictor
/// \brief Clone an existing predictor
* should be thread-safe.
/// When using clone, the same network will be created,
*/
/// and the parameters between them are shared.
/// \return unique_ptr which contains the pointer of predictor
virtual
std
::
unique_ptr
<
PaddlePredictor
>
Clone
()
=
0
;
virtual
std
::
unique_ptr
<
PaddlePredictor
>
Clone
()
=
0
;
/** Destroy the Predictor.
/// \brief Destroy the Predictor.
*/
virtual
~
PaddlePredictor
()
=
default
;
virtual
~
PaddlePredictor
()
=
default
;
/** \brief Get the serialized model program that executes in inference phase.
* Its data type is ProgramDesc, which is a protobuf message.
*/
virtual
std
::
string
GetSerializedProgram
()
const
{
virtual
std
::
string
GetSerializedProgram
()
const
{
assert
(
false
);
// Force raise error.
assert
(
false
);
// Force raise error.
return
"NotImplemented"
;
return
"NotImplemented"
;
}
}
/** The common configs for all the predictors.
/// \brief Base class for NativeConfig and AnalysisConfig.
*/
struct
Config
{
struct
Config
{
std
::
string
model_dir
;
/*!< path to the model directory. */
std
::
string
model_dir
;
/*!< path to the model directory. */
};
};
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录