Skip to content

P
PaddleRec

PaddlePaddle / PaddleRec

通知 68
Star 12
Fork 5
P
PaddleRec
提交 1726a417 编写于 作者: L liuyuhui
浏览文件
操作

fix UT failure

上级 765870b2
隐藏空白更改
内联 并排
Showing with 6 addition and 3 deletion
core/utils/dataloader_instance.py
浏览文件 @ 1726a417
...@@ -49,7 +49,8 @@ def dataloader_by_name(readerclass, ...@@ -49,7 +49,8 @@ def dataloader_by_name(readerclass,
files.sort() files.sort()
# for local cluster: discard some files if files cannot be divided equally between GPUs # for local cluster: discard some files if files cannot be divided equally between GPUs
if (context["device"] == "GPU"): if (context["device"] == "GPU"
) and os.getenv("PADDLEREC_GPU_NUMS") is not None:
selected_gpu_nums = int(os.getenv("PADDLEREC_GPU_NUMS")) selected_gpu_nums = int(os.getenv("PADDLEREC_GPU_NUMS"))
discard_file_nums = len(files) % selected_gpu_nums discard_file_nums = len(files) % selected_gpu_nums
if (discard_file_nums != 0): if (discard_file_nums != 0):
...@@ -121,7 +122,8 @@ def slotdataloader_by_name(readerclass, dataset_name, yaml_file, context): ...@@ -121,7 +122,8 @@ def slotdataloader_by_name(readerclass, dataset_name, yaml_file, context):
files.sort() files.sort()
# for local cluster: discard some files if files cannot be divided equally between GPUs # for local cluster: discard some files if files cannot be divided equally between GPUs
if (context["device"] == "GPU"): if (context["device"] == "GPU"
) and os.getenv("PADDLEREC_GPU_NUMS") is not None:
selected_gpu_nums = int(os.getenv("PADDLEREC_GPU_NUMS")) selected_gpu_nums = int(os.getenv("PADDLEREC_GPU_NUMS"))
discard_file_nums = len(files) % selected_gpu_nums discard_file_nums = len(files) % selected_gpu_nums
if (discard_file_nums != 0): if (discard_file_nums != 0):
...@@ -201,7 +203,8 @@ def slotdataloader(readerclass, train, yaml_file, context): ...@@ -201,7 +203,8 @@ def slotdataloader(readerclass, train, yaml_file, context):
files.sort() files.sort()
# for local cluster: discard some files if files cannot be divided equally between GPUs # for local cluster: discard some files if files cannot be divided equally between GPUs
if (context["device"] == "GPU"): if (context["device"] == "GPU"
) and os.getenv("PADDLEREC_GPU_NUMS") is not None:
selected_gpu_nums = int(os.getenv("PADDLEREC_GPU_NUMS")) selected_gpu_nums = int(os.getenv("PADDLEREC_GPU_NUMS"))
discard_file_nums = len(files) % selected_gpu_nums discard_file_nums = len(files) % selected_gpu_nums
if (discard_file_nums != 0): if (discard_file_nums != 0):
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或