Design Doc: Model Format

Motivation

A model is an output of the training process. One complete model consists of two parts, the topology and the parameters. In order to support industrial deployment, the model format must be self-complete and must not expose any training source code.

As a result, In PaddlePaddle, the topology is represented as a ProgramDesc, which describes the model structure. The parameters contain all the trainable weights in the model. We must support large size parameters and efficient serialization/deserialization of parameters.

Implementation

The topology is saved as a plain text in a detailed self-contain protobuf file.

The parameters are saved as a binary file. As we all know, the protobuf message has a limit of 64M size. We have done a benchmark experiment, which shows that protobuf is not fit for the task.

As a result, we design a particular format for tensor serialization. By default, an arbitrary tensor in Paddle is a LoDTensor, and has a description information proto of LoDTensorDesc. We save the DescProto as the byte string header. It contains all the necessary information, such as the dims, the name of the tensor, and the LoD information in LoDTensor. A tensor stores values in a continuous memory buffer. For speed we dump the raw memory to disk and save it as the byte string content. So, the binary format of one tensor is,

|HeaderLength|ContentLength|LoDTensorDesc|TensorValue|

The table below shows a tensor’s byte view in detail. Note that all the signed values are written in the little-endian format.

[offset] [type]              [description] 
0004     4 bytes integer      HeaderLength, the length of LoDTensorDesc
0008     4 bytes integer      ContentLength, the length of LodTensor Buffer
0009     1 bytes char         TensorDesc
00010    1 bytes char         TensorDesc
...
00100    1 bytes char         TensorValue
00101    1 bytes char         TensorValue
00102    1 bytes char         TensorValue              ..
...

Summary

  • We introduce a model format.
  • The ProgramDesc describe the model topology.
  • A bunch of specified format binary tensors describe the parameters.