提交 e0b8b21d 编写于 作者: J Jiawei Wang 提交者: GitHub

Merge pull request #696 from Mycaster/optimize-quantization-tool

fix quantization-tool bug
...@@ -233,7 +233,7 @@ int compress_parameter_parallel(const char *file1, ...@@ -233,7 +233,7 @@ int compress_parameter_parallel(const char *file1,
greedy_search( greedy_search(
emb_table + k * emb_size, xmin, xmax, loss, emb_size, bits); emb_table + k * emb_size, xmin, xmax, loss, emb_size, bits);
// 得出 loss 最小的时候的 scale // 得出 loss 最小的时候的 scale
float scale = (xmax - xmin) * (pow2bits - 1); float scale = (xmax - xmin) / (pow2bits - 1);
char *min_ptr = tensor_temp; char *min_ptr = tensor_temp;
char *max_ptr = tensor_temp + sizeof(float); char *max_ptr = tensor_temp + sizeof(float);
memcpy(min_ptr, &xmin, sizeof(float)); memcpy(min_ptr, &xmin, sizeof(float));
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册