Created by: JepsonWong
在动态图下支持SelectedRows计算,以便在动态图下支持单机单卡的稀疏更新。
在单机单卡下测试transformer模型: 稀疏更新: pass : 0 finished, validation avg loss: [4.259961] pass : 1 finished, validation avg loss: [3.3392217] pass : 2 finished, validation avg loss: [2.993626] pass : 3 finished, validation avg loss: [2.901003] pass : 4 finished, validation avg loss: [2.8706653] pass : 5 finished, validation avg loss: [2.883191] pass : 6 finished, validation avg loss: [2.8904562] pass : 7 finished, validation avg loss: [2.930034] pass : 8 finished, validation avg loss: [2.935658] pass : 9 finished, validation avg loss: [2.895582] pass : 10 finished, validation avg loss: [2.897935] pass : 11 finished, validation avg loss: [2.9117343] pass : 12 finished, validation avg loss: [2.927333] pass : 13 finished, validation avg loss: [2.926609] pass : 14 finished, validation avg loss: [2.9505832] pass : 15 finished, validation avg loss: [2.9838722] pass : 16 finished, validation avg loss: [2.9725804] pass : 17 finished, validation avg loss: [3.0003846] pass : 18 finished, validation avg loss: [2.9929357] pass : 19 finished, validation avg loss: [3.0168884] 非稀疏更新: pass : 0 finished, validation avg loss: [4.277582] pass : 1 finished, validation avg loss: [3.3403788] pass : 2 finished, validation avg loss: [2.994371] pass : 3 finished, validation avg loss: [2.9051983] pass : 4 finished, validation avg loss: [2.8535395] pass : 5 finished, validation avg loss: [2.8552744] pass : 6 finished, validation avg loss: [2.9139636] pass : 7 finished, validation avg loss: [2.9096322] pass : 8 finished, validation avg loss: [2.9088447] pass : 9 finished, validation avg loss: [2.9029431] pass : 10 finished, validation avg loss: [2.8912632] pass : 11 finished, validation avg loss: [2.924397] pass : 12 finished, validation avg loss: [2.9139042] pass : 13 finished, validation avg loss: [2.9138305] pass : 14 finished, validation avg loss: [2.9450483] pass : 15 finished, validation avg loss: [2.9633262] pass : 16 finished, validation avg loss: [2.9701042] pass : 17 finished, validation avg loss: [3.0000627] pass : 18 finished, validation avg loss: [3.0083613] pass : 19 finished, validation avg loss: [3.040544]