# Design Doc: PaddlePaddle Programs

## Compile and Execution

A PaddlePaddle program consists of two parts – the first generates a `ProgramDesc` protobuf message that describes the program, and the second runs this message using a C++ class `Executor`.

A simple example PaddlePaddle program can be found in graph.md:

    x = layer.data("images")
    l = layer.data("label")
    y = layer.fc(x)
    cost = layer.mse(y, l)
    optimize(cost)
    train(cost, reader=mnist.train())

The first five lines of the following PaddlePaddle program generates, or, compiles, the `ProgramDesc` message. The last line runs it.

## Programs and Blocks

The basic structure of a PaddlePaddle program is some nested blocks, as a C++ or Java program.

- program: some nested blocks
- block:
  - some local variable definitions, and
  - a sequence of operators

The concept of block comes from usual programs. For example, the following C++ program has three blocks:

    int main() { // block 0
      int i = 0;
      if (i < 10) { // block 1
        for (int j = 0; j < 10; j++) { // block 2
        }
      }
      return 0;
    }

The following PaddlePaddle program has three blocks:

    import paddle as pd  // block 0

    x = minibatch([10, 20, 30]) # shape=[None, 1]
    y = var(1) # shape=[1], value=1
    z = minibatch([10, 20, 30]) # shape=[None, 1]
    cond = larger_than(x, 15) # [false, true, true]

    ie = pd.ifelse()
    with ie.true_block():  // block 1
        d = pd.layer.add_scalar(x, y)
        ie.output(d, pd.layer.softmax(d))
    with ie.false_block():  // block 2
        d = pd.layer.fc(z)
        ie.output(d, d+1)
    o1, o2 = ie(cond)

## `BlockDesc` and `ProgramDesc`

All protobuf messages are defined in `framework.proto`.

`BlockDesc` is straight-forward – it includes local variable definitions, `vars`, and a sequence of operators, `ops`.

    message BlockDesc {
      required int32 parent = 1;
      repeated VarDesc vars = 2;
      repeated OpDesc ops = 3;
    }

The parent ID indicates the parent block so that operators in a block can refer to variables defined locally and also those defined in their ancestor blocks.

All hierarchical blocks in a program are flattened and stored in an array. The block ID is the index of the block in this array.

    message ProgramDesc {
      repeated BlockDesc blocks = 1;
    }

### Global Block

The global block is the first one in the above array.

## Operators that Use Blocks

In the above example, the operator `IfElseOp` has two blocks – the true branch and the false branch.

The definition of `OpDesc` shows that an operator could have some attributes:

    message OpDesc {
      AttrDesc attrs = 1;
      ...
    }

and an attribute could be of type block, which is, in fact, a block ID as described above:

    message AttrDesc {
      required string name = 1;

      enum AttrType {
        INT = 1,
        STRING = 2,
        ...
        BLOCK = ...
      }
      required AttrType type = 2;

      optional int32 block = 10; // when type == BLOCK
      ...
    }

## InferShape

With this design, the InferShape function should take the following parameters:

    void InferShape(int current_block,
                    int current_operator,
                    ProgramDesc* program // might change VarDesc values.
                    ) {
      ...
    }

where

- `current_block` indices into `ProgramDesc::blocks`,
- `current_operator` indices into `BlockDesc::ops`.