Created by: kexinzhao
Some background can be found here.
Essentially, our level of detail info (LoD) is currently both implemented in C++ and exposed in Python using "offsets". We want to encapsulate the offset
implementation detail in C++ and only expose length
based interface in Python.
What this PR does (See the main change in pybind.cc
here):
-
Previously, the
set_lod
andlod
API inpybind.cc
uses offset_based LoD. This PR modifies the LoDTensor API inpybind.cc
so that the user on the python side can provide or obtain length-based LoD usingset_lod
andlod
, respectively. -
The
init
API is also modified so that user can create a LoDTensor object in Python and initialize it with a length-based LoD. -
A
has_valid_lod
API is added inpybind.cc
so that user can check if the LoD info in a tensor is valid and matches the its data dimension on the python side. -
Utility functions to create LoDTensor in the python side defined here is largely simplified using the updated
set_lod
andlod
in pybind. (See changes here) -
Over 30 tests that uses
set_lod
andlod
has been modified to reflect the updated LoDTensor API inpybind.cc
.