Switching RPC library in new pserver implement
Created by: typhoonzero
Related issue: https://github.com/PaddlePaddle/Paddle/issues/1721
We are using net/rpc
in our implementation currently. In fact, I found that when we writing a client for trainers to call from, we have to implement the c binding first, and then compile it to a shared object(so), and then use swig to export that library, and then using ctypes to import so lib from python. The procedure can be displayed below:
As we can see, we need to implement so many things. The root cause is that net/rpc
in go don't have c or python bindings, we have to convert go-written clients to c and then to python.
I would like to introduce a more simple and efficient way using grpc
instead of net/rpc
as below:
If we all switch to grpc and using python binding to implement the pserver client and master client, it will be much more simpler.
Some etcd3 python client bindings: