提交 7747e01b 编写于 作者: Y yuyang18

Polish LoDTensor API

上级 ab210925
...@@ -19,33 +19,41 @@ __all__ = ['create_lod_tensor', 'create_random_int_lodtensor'] ...@@ -19,33 +19,41 @@ __all__ = ['create_lod_tensor', 'create_random_int_lodtensor']
def create_lod_tensor(data, lod, place): def create_lod_tensor(data, lod, place):
"""Create a lod tensor from a numpy array, a list, or an existing lod tensor. """
Create a lod tensor from a numpy array, a list, or an existing lod tensor.
Create a lod tensor by doing the following: Create a lod tensor by doing the following:
1. Check that the length-based input lod is valid. 1. Check that the length-based input lod is valid.
2. Convert the length-based lod to a offset-based LoD. 2. Convert the length-based lod to a offset-based LoD.
3. Copy the data from a numpy array, a list or a existing lod tensor to 3. Copy the data from a numpy array, a list or a existing lod tensor to
CPU or GPU device (based on input place). CPU or GPU device (based on input place).
4. Set the level of detail (LoD) using the offset-based LoD. 4. Set the level of detail (LoD) using the offset-based LoD.
Use example: Examples:
Suppose we want LoDTensor to hold data for sequences of word, where each word is
represented by an integer. If we want to create a LoDTensor to represent two
sentences, one of 2 words, and one of 3 words.
Then 'data' can be a numpy array of integers with shape (5, 1). Suppose we want LoDTensor to hold data for sequences of word, where each
'lod' will be [[2, 3]], indicating the length(# of words) in each sentence. word is represented by an integer. If we want to create a LoDTensor to
This length-based input lod [[2, 3]] will be converted to offset-based lod [[0, 2, 5]] represent two sentences, one of 2 words, and one of 3 words.
inside the function call.
Please refer to Then :code:`data` can be a numpy array of integers with shape (5, 1).
github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/concepts/lod_tensor.md :code:`lod` will be [[2, 3]], indicating the length(# of words) in each
for more details regarding LoD. sentence. This length-based input lod [[2, 3]] will be converted to
offset-based lod [[0, 2, 5]] inside the function call.
Please reference :ref:`api_guide_low_level_lod_tensor` for more details
regarding LoD.
Args: Args:
data: a numpy array or a LoDTensor or a list holding the data to be copied. data(numpy.ndarray|list|LoDTensor): a numpy array or a LoDTensor or a
lod: a list of lists indicating the length-based LoD info specified by the user. list holding the data to be copied.
place: CPU or GPU place indicating where the data in the new LoDTensor will be stored. lod(list): a list of lists indicating the length-based LoD info
specified by the user.
place(Place): CPU or GPU place indicating where the data in the new
LoDTensor will be stored.
Returns: Returns:
A fluid LoDTensor object with tensor data and lod info. A fluid LoDTensor object with tensor data and lod info.
...@@ -77,31 +85,38 @@ def create_lod_tensor(data, lod, place): ...@@ -77,31 +85,38 @@ def create_lod_tensor(data, lod, place):
def create_random_int_lodtensor(lod, base_shape, place, low, high): def create_random_int_lodtensor(lod, base_shape, place, low, high):
"""Create a LoDTensor containing random integers. """
Create a LoDTensor containing random integers.
This function is frequently used in the book examples. So we revised it based on This function is frequently used in the book examples. So we revised it
the new create_lod_tensor API and put it here in the lod_tensor module to simplify based on the new create_lod_tensor API and put it here in the lod_tensor
the code. module to simplify the code.
The function does the following: The function does the following:
1. Calculate the overall shape of the LoDTensor based on the length-based 'lod' input
and the shape of the basic element in 'base_shape'. 1. Calculate the overall shape of the LoDTensor based on the length-based
:code:`lod` input and the shape of the basic element in
:code:`base_shape`.
2. Create a numpy array of this shape. 2. Create a numpy array of this shape.
3. Create the LoDTensor using create_lod_tensor API. 3. Create the LoDTensor using create_lod_tensor API.
Suppose we want LoDTensor to hold data for sequences of word, where each word is Suppose we want LoDTensor to hold data for sequences of word, where each
represented by an integer. If we want to create a LoDTensor to represent two word is represented by an integer. If we want to create a LoDTensor to
sentences, one of 2 words, and one of 3 words. Then 'base_shape' is [1], input represent two sentences, one of 2 words, and one of 3 words. Then
length-based 'lod' is [[2, 3]]. Then the overall shape of the LoDTensor would be 'base_shape' is [1], input length-based 'lod' is [[2, 3]]. Then the overall
[5, 1], holding 5 words for two sentences. shape of the LoDTensor would be [5, 1], holding 5 words for two sentences.
Args: Args:
data: a numpy array or a LoDTensor holding the data to be copied. lod(list): a list of lists indicating the length-based LoD info
lod: a list of lists indicating the length-based LoD info specified by the user. specified by the user.
base_shape: the shape of the basic element to be held by the LoDTensor. base_shape(list): the shape of the basic element to be held by the
place: CPU or GPU place indicating where the data in the new LoDTensor will be stored. LoDTensor.
low: the lower bound of the random integers. place(Place): CPU or GPU place indicating where the data in the new
high: the upper bound of the random integers. LoDTensor will be stored.
low(int): the lower bound of the random integers.
high(int): the upper bound of the random integers.
Returns: Returns:
A fluid LoDTensor object with tensor data and lod info. A fluid LoDTensor object with tensor data and lod info.
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册