## Interaction between C++ and Python

Users employ API in Python to describe their own network, however, the network construction actually happens in C++. so Protobuf is introduced to send the message between Python and C++.

The Interaction between Python and C++ can be simplified as two steps:

1. C++ tells Python how many Ops there are, and what parameter do users need to offer to initialize a new Op. Python then builds API for each Op at compile time.

2. Users invoke APIs built by Python and provide necessary parameters. These parameters will be sent to C++ for finishing the Op construction task.

### Message from C++ to Python

We define a Protobuf message class `OpProto` to hold message needed in the first step. What should an `OpProto` contain? This question is equivalent to “What message do we need to offer, to build a Python API which is legal and user oriented and can use to describe a whole Op.”

Following message are necessary:

1. Op's name, and its simple comment.
2. Input and output variable number; each variable's name, type, and comment.
3. Op's attributes; each attribute includes name, type, comment, **default value** and **value range**.

So `OpProto` can be defined as follows:

```proto
enum AttrType {
    INT = 1;
    FLOAT = 2;
    STRING = 3;
    INTS = 4;
    FLOATS = 5;
    STRINGS = 6;
};

message AttrValue {
    AttrType type = 1;
    optional int iv = 2;
    optional float fv = 3;
    optional string sv = 4;
    repeated int ivs = 5;
    repeated float fvs = 6;
    repeated string svs = 7;
};

message AttrProto {
    required string name = 1;
    required string comment = 2;
    required AttrType type = 3;
};

message VarProto {
    required string name = 1;
    required string comment = 2;
    required bool is_tensor = 3;
};

message OpProto {
    repeated VarProto inputs = 1;
    repeated VarProto outputs = 2;
    repeated AttrProto attrs = 3;
    required string type = 4;
    required string comment = 5;
};
```

To generate Python code automatically:

```python
def create_python_ops_creatation_functions():
    op_protos = paddle.framework.OpRegistry.get_all_op_proto()
    for type_name in op_protos:
        op_proto = op_protos[type_name]
        def __impl__(**kwargs):  # User must use key word args in Paddle API
            inputs = [kwargs.get(ipt.name, "") for ipt in op_proto.inputs]
            outputs = [kwargs.get(opt.name, "") for opt in op_proto.outputs]
            attrs = [cast_to_op_attr(attr, kwargs.get(attr.name, None)) for attr in op_proto.attrs]
            opdesc = （input, outputs, type_name, attrs）
            return paddle.framework.OpRegistry.CreateOp(opdesc)
        __impl__.__doc__ = create_doc_string(op_proto)
        globals()[type_name] = __impl__

create_python_ops_creatation_functions()
```

### Message from Python to C++

To hold message needed in the above second step, we define Protobuf message class `OpDesc`. It is used to hold user-specified parameters in Op describing.

```proto
message OpDesc {
    required string type = 1;
    repeated string inputs = 2;
    repeated string outputs = 3;
    map<string, AttrValue> attrs = 4;
};
```

## OpProto Register

Every Op has its own `OpProto`. For using convenience, we need to register them and record all their messages. For each `Op` class, we define a corresponding `OpMaker` class, in whose constructor we implement the `OpProto`'s building process. `OpMaker`'s constructor will be invoked by another function `OpRegistry::RegisterOp()`.

```cpp
class OpProtoMaker {
public:
    OpProtoMaker(OpProto* proto): proto_(proto) {}
protected:
    OpProto* proto_;
    void AddInput(const std::string& name, const std::string& desc) {...}
    void AddAttr(const std::string& name, const std::string& desc, TypeId type) {...}
    void AddComment(const std::string& comment) { ... }
};

class OpRegistry {
public:
    using OpCreator = std::function<OperatorBase* (OpDesc& desc)>;

    template <typename OpType, typename OpMaker>
    static void RegisterOp(const std::string& name) {
        gCreators_[name] = [](const OpDesc& desc) {
            return new OpType(desc);
        };
        OpProto& opProto = gProtos_[name];
        OpMaker()(&opProto);
    }

    static map<string, OpCreator> gCreators_;
    static map<string, OpProto> gProtos_;
};

template <typename OpType, typename OpMaker>
class OpRegister {
  public:
    OpRegister(std::string type) {
        OpRegistry::RegisterOp<OpType, OpMaker>(type);
    }
};

#define REGISTER_OP(op_class, op_maker_class, type_name)         \
    class op_class##Register {                                   \
      private:                                                   \
        const static OpRegister<#op_class, #op_maker_class> reg; \
    };                                                           \
    const Register op_class##Register::reg(#type_name);

class CosineOp {
// ...
}

struct CosineOpProtoMaker : public OpProtoMaker {
    CosineOpProtoMaker(OpProto* proto) : OpProtoMaker(proto) {
        AddInput("input", "input of cosine op");
        AddAttr("scale", "scale of cosine op", float).Default(1.0).GreaterThan(0.0);
        AddType("cos");
        AddComment("This is cos op");
    }
}

REGISTER_OP(CosineOp, CosineOpProtoMaker, cos);
```

In `REGISTER_OP(CosineOp, CosineOpProtoMaker, cos)`, we register not only `CosineOp` but also `CosineOpProto`. As fields of `CosineOpProto`, the default value and value range of `scale` are also registered here.

## Python API

Python  APIs are divided into two types, high-level API and low-level API.

### High-Level API

High-level API is called by users directly, so it should keep its style consistent with existing V2 APIs.

Here is a sample about how a define a fc layer:

```python
hd = fc_layer(input=data, size=56, with_bias=True, activation="sigmoid");
```

`hd` is the output of `fc_layer` and it's a `variable`. It can be further sent into other layers as input.

The definition of `fc_layer()`:

```python
def fc_layer(input, size, with_bias, activation):
    attr_map = {"size":size}
    check_attrs(attr_map)
    w = make_variable('w')
    if with_bias:
        b = make_variable('b')
    else:
        b = None
    fc_output = make_variable('fc_output');
    fc_op(input, w, b, fc_output, attr_map)
    act_output = make_variable('sigmod_output');
    if activation == "sigmod":
        sigmod_op(fc_output, act_output);
    elif:
        # ...
    return act_output;
```

### Low Leval API

In above sample, `fc_op` and `sigmod_op` are low-level API. They build `OpDesc` and invoke corresponding C++ code.

*TODO*
