Neural network framework often provides Symbolic api for users to write network topology conveniently. This doc manily focus on Symbolic api in most popular neural network frameworks, and try to find out how to parse Symbolic configuration to a portable file, such as protobuf or json.
The core concept of Symbolic api is `Symbol`. Mxnet implements `Symbol` class in C++, and export to Python using CAPI. Please refer to the comments in Mxnet:
`Symbol` is help class used to represent the operator node in Graph.
`Symbol` acts as an interface for building graphs from different components like Variable, Functor and Group. `Symbol` is also exported to python front-end (while Graph is not) to enable quick test and deployment. Conceptually, symbol is the final operation of a graph and thus including all the information required (the graph) to evaluate its output value.
一个简单的网络定义如下:
A simple network topology wrote by Symbol is as follows:
Varible here is actually a Symbol. Every basic Symbol will correspond to one Node, and every Node has its own NodeAttr. There is a op field in NodeAttr class, when a Symbol represents Variable(often input data), the op field is null.
Symbol contains a data member, std::vector<NodeEntry> outputs, and NodeEntry cantains a poniter to Node. We can follow the Node pointer to get all the Graph.
The core concept of Symbolic api is `Tensor`. Tensorflow defines `Tensor` in Python. Please refer to the comments in TensorFlow:
A `Tensor` is a symbolic handle to one of the outputs of an `Operation`. It does not hold the values of that operation's output, but instead provides a means of computing those values in a TensorFlow @{tf.Session}.
一个简单的使用样例如下:
A simple example is as follows:
```python
# Build a dataflow graph.
...
...
@@ -58,8 +98,9 @@ A `Tensor` is a symbolic handle to one of the outputs of an `Operation`. It does
```
Tensor的一些主要成员变量和接口可以参考如下:
The main method of `Tensor` is as follows:
```python
@property
defop(self):
...
...
@@ -89,82 +130,13 @@ def device(self):
returnself._op.device
```
TensorFlow的Tensor可以作为target被session来run,实际上是Tensor已经包含了所有的Graph信息,可以track data dependency。
<tensorflow.python.framework.ops.Graph object at 0x10f256d50>
```
没有找到Graph的debug string接口,但是可以明确知道配置过程中只存在一个Graph。
### Dynet
The core concept of Symbolic api is `Expression`, and Dynet defines `Expression` class in C++.
A simple example is as follows:
```cpp
ComputationGraphcg;
ExpressionW=parameter(cg,pW);
Expressionin=input(cg,xs[i]);
Expressionlabel=input(cg,ys[i]);
Expressionpred=W*in;
Expressionloss=square(pred-label);
```
The input data and parameter are also represented by Expression. Every basci Expression corresponds to a Node. And input data is also a Node.
Expression has a data member ComputationGraph, and ComputationGraph will be modified in users' configuring process. Expression can be a running target, beacuse Expression contains all dependency.
- dynet
dynet可以在C++中书写配置
Here is a detailed example:
write topology in C++
```
ComputationGraph cg;
...
...
@@ -197,7 +189,7 @@ Expression loss = square(pred - ys[i]);
cg.print_graphviz();
```
编译运行后,得到打印结果:
compile and print
```
# first print
...
...
@@ -229,3 +221,12 @@ digraph G {
N3 -> N4;
}
```
### Conclusion
Actually, Symbol/Tensor/Expression in Mxnet/TensorFlow/Dynet are the same level concepts. We use a unified name Expression here, this level concept has following features:
- Users wirte topoloy with Symbolic api, and all return value is Expression, including input data and parameter.
- Expression corresponds with a global Graph, and Expression can also be composed.
- Expression tracks all dependency and can be taken as a run target