未验证 提交 7456b5d0 编写于 作者: Q Qiao Longfei 提交者: GitHub

Merge pull request #1491 from jacquesqiao/speedup-reader

speedup python reader
...@@ -35,7 +35,7 @@ class CriteoDataset(Dataset): ...@@ -35,7 +35,7 @@ class CriteoDataset(Dataset):
else: else:
dense_feature.append((float(features[idx]) - self.cont_min_[idx - 1]) / self.cont_diff_[idx - 1]) dense_feature.append((float(features[idx]) - self.cont_min_[idx - 1]) / self.cont_diff_[idx - 1])
for idx in self.categorical_range_: for idx in self.categorical_range_:
sparse_feature.append([hash("%d_%s" % (idx, features[idx])) % self.hash_dim_]) sparse_feature.append([hash(str(idx) + features[idx]) % self.hash_dim_])
label = [int(features[0])] label = [int(features[0])]
yield [dense_feature] + sparse_feature + [label] yield [dense_feature] + sparse_feature + [label]
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册