Trainer ParameterServer Design Doc related discussion
Created by: dzhwinter
maybe open an issue is the way to make us agree on some complicated point , list these interface in detail. Trainer和ParameterServer交互的接口 备选方案:1
/* get Parameters */
int32_t PullParameters(<map<block_id/*pname*/>*Tensor params>);
/* set Parameters */
int32_t UpdateParameters(<map<block_id/*pname*/>*Tensor params>,float32 leaningrate)
@helin
备选方案 :2
/* get Parameters */
int32_t PullParameters(<map<string/*pname*/>*Tensor params);
/* set Parameters */
int32_t UpdateParameters(<map<string/*pname*/>*Tensor params)
/* setup Update method,calculcate the learningrate policy in optimizer/updater module */
void setUpdater(UpdaterBase)
其他深度学习框架使用了相似实现:mxnet ,
/* Given a key, assume x(left Tensor) is the received (pushed) value and y(right Tensor) is the value stored on the store node. The store updates y by `h(x, &y)`. The default h is ASSIGN, namely `*y = x`. */
typedef std::function<void(Tensor& , Tensor*)> UpdaterFunction
void setUpdater(UpdaterFunction)
e.g.
auto updater = [](Tensor& x, Tensor*y) {
y -= x;
}
@dzhwinter ParameterClient_CAPI C++调用Go ParameterClient library的接口函数
- 选1or2,update的方式
- 对于稀疏更新和正则化等更新的支持,是在ParameterServer端实现,还是在Trainer端实现? *more important