Skip to content

  • 体验新版
    • 正在加载...
  • 登录
  • PaddlePaddle
  • Paddle
  • Issue
  • #21595

P
Paddle
  • 项目概览

PaddlePaddle / Paddle
大约 2 年 前同步成功

通知 2325
Star 20933
Fork 5424
  • 代码
    • 文件
    • 提交
    • 分支
    • Tags
    • 贡献者
    • 分支图
    • Diff
  • Issue 1423
    • 列表
    • 看板
    • 标记
    • 里程碑
  • 合并请求 543
  • Wiki 0
    • Wiki
  • 分析
    • 仓库
    • DevOps
  • 项目成员
  • Pages
P
Paddle
  • 项目概览
    • 项目概览
    • 详情
    • 发布
  • 仓库
    • 仓库
    • 文件
    • 提交
    • 分支
    • 标签
    • 贡献者
    • 分支图
    • 比较
  • Issue 1,423
    • Issue 1,423
    • 列表
    • 看板
    • 标记
    • 里程碑
  • 合并请求 543
    • 合并请求 543
  • Pages
  • 分析
    • 分析
    • 仓库分析
    • DevOps
  • Wiki 0
    • Wiki
  • 成员
    • 成员
  • 收起侧边栏
  • 动态
  • 分支图
  • 创建新Issue
  • 提交
  • Issue看板
已关闭
开放中
Opened 12月 06, 2019 by saxon_zh@saxon_zhGuest

ngraph unit-test fails on 6148 machine

Created by: luotao1

The PR_CI_Coverage is 5117 machine, and we add 6148 machine for nightly jobs.

PR_CI_Manylinux_Coverage

This job use cmake .. -DWITH_GPU=ON.

http://ci.paddlepaddle.org/viewLog.html?buildId=238851&buildTypeId=Paddle_PaddleManylinux_PrCiManylinuxCoverage

  • test_batch_norm_ngraph_op
[18:19:16]	[Step 1/1] 89/98 Test #761: test_batch_norm_ngraph_op .....................***Failed   15.97 sec
[18:19:16]	[Step 1/1] W1205 18:19:07.457509 176318 executor.cc:67] FLAGS_use_ngraph=True, garbage collection strategy is disabled in Executor
[18:19:16]	[Step 1/1] W1205 18:19:11.907506 176318 device_context.cc:236] Please NOTE: device: 0, CUDA Capability: 70, Driver API Version: 10.1, Runtime API Version: 9.0
[18:19:16]	[Step 1/1] W1205 18:19:11.943367 176318 device_context.cc:244] device: 0, cuDNN Version: 7.3.
[18:19:16]	[Step 1/1] test_batch_norm_ngraph_op failed
[18:19:16]	[Step 1/1]  E.
[18:19:16]	[Step 1/1] ======================================================================
[18:19:16]	[Step 1/1] ERROR: test_check_output (paddle.fluid.tests.unittests.test_batch_norm_op.TestBatchNormOpInference)
[18:19:16]	[Step 1/1] ----------------------------------------------------------------------
[18:19:16]	[Step 1/1] Traceback (most recent call last):
[18:19:16]	[Step 1/1]   File "/paddle/build/python/paddle/fluid/tests/unittests/test_batch_norm_op.py", line 282, in test_check_output
[18:19:16]	[Step 1/1]     [2, 3, 4, 5])
[18:19:16]	[Step 1/1]   File "/paddle/build/python/paddle/fluid/tests/unittests/test_batch_norm_op.py", line 221, in check_with_place
[18:19:16]	[Step 1/1]     place)
[18:19:16]	[Step 1/1]   File "/paddle/build/python/paddle/fluid/tests/unittests/test_batch_norm_op.py", line 152, in create_or_get_tensor
[18:19:16]	[Step 1/1]     tensor.set(var, place)
[18:19:16]	[Step 1/1] RuntimeError: 
[18:19:16]	[Step 1/1] 
[18:19:16]	[Step 1/1] --------------------------------------------
[18:19:16]	[Step 1/1] C++ Call Stacks (More useful to developers):
[18:19:16]	[Step 1/1] --------------------------------------------
[18:19:16]	[Step 1/1] 0   std::string paddle::platform::GetTraceBackString<std::string>(std::string&&, char const*, int)
[18:19:16]	[Step 1/1] 1   paddle::memory::allocation::CUDAAllocator::AllocateImpl(unsigned long)
[18:19:16]	[Step 1/1] 2   paddle::memory::allocation::AllocatorFacade::Alloc(paddle::platform::Place const&, unsigned long)
[18:19:16]	[Step 1/1] 3   paddle::memory::allocation::AllocatorFacade::AllocShared(paddle::platform::Place const&, unsigned long)
[18:19:16]	[Step 1/1] 4   paddle::memory::AllocShared(paddle::platform::Place const&, unsigned long)
[18:19:16]	[Step 1/1] 5   paddle::framework::Tensor::mutable_data(paddle::platform::Place const&, paddle::framework::proto::VarType_Type, unsigned long)
[18:19:16]	[Step 1/1] 
[18:19:16]	[Step 1/1] ----------------------
[18:19:16]	[Step 1/1] Error Message Summary:
[18:19:16]	[Step 1/1] ----------------------
[18:19:16]	[Step 1/1] ResourceExhaustedError: 
[18:19:16]	[Step 1/1] 
[18:19:16]	[Step 1/1] Out of memory error on GPU 0. Cannot allocate 480.000000B memory on GPU 0, available memory is only 0.000000B.
[18:19:16]	[Step 1/1] 
[18:19:16]	[Step 1/1] Please check whether there is any other process using GPU 0.
[18:19:16]	[Step 1/1] 1. If yes, please stop them, or start PaddlePaddle on another GPU.
[18:19:16]	[Step 1/1] 2. If no, please decrease the batch size of your model.
[18:19:16]	[Step 1/1]  at (/paddle/paddle/fluid/memory/allocation/cuda_allocator.cc:58)
[18:19:16]	[Step 1/1] 
[18:19:16]	[Step 1/1] 
[18:19:16]	[Step 1/1] ----------------------------------------------------------------------
[18:19:16]	[Step 1/1] Ran 2 tests in 11.102s
[18:19:16]	[Step 1/1] 
[18:19:16]	[Step 1/1] FAILED (errors=1)
[18:19:16]	[Step 1/1] 
[18:19:16]	[Step 1/1] op test forward passed:  CPUPlace NCHW
[18:19:16]	[Step 1/1] op test forward passed:  CPUPlace NHWC
[18:19:16]	[Step 1/1] op test forward passed:  CUDAPlace(0) NCHW
[18:19:16]	[Step 1/1] op test forward passed:  CUDAPlace(0) NHWC
  • test_mean_ngraph_op
[18:19:11]	[Step 1/1] 87/98 Test #777: test_mean_ngraph_op ...........................***Failed    3.94 sec
[18:19:11]	[Step 1/1] Traceback (most recent call last):
[18:19:11]	[Step 1/1]   File "/paddle/tools/test_runner.py", line 20, in <module>
[18:19:11]	[Step 1/1]     import paddle.fluid as fluid
[18:19:11]	[Step 1/1]   File "/paddle/build/python/paddle/__init__.py", line 30, in <module>
[18:19:11]	[Step 1/1]     import paddle.dataset
[18:19:11]	[Step 1/1]   File "/paddle/build/python/paddle/dataset/__init__.py", line 25, in <module>
[18:19:11]	[Step 1/1]     import paddle.dataset.sentiment
[18:19:11]	[Step 1/1]   File "/paddle/build/python/paddle/dataset/sentiment.py", line 29, in <module>
[18:19:11]	[Step 1/1]     import nltk
[18:19:11]	[Step 1/1]   File "/usr/local/python2.7.15/lib/python2.7/site-packages/nltk/__init__.py", line 129, in <module>
[18:19:11]	[Step 1/1]     from nltk.collocations import *
[18:19:11]	[Step 1/1]   File "/usr/local/python2.7.15/lib/python2.7/site-packages/nltk/collocations.py", line 40, in <module>
[18:19:11]	[Step 1/1]     from nltk.metrics import ContingencyMeasures, BigramAssocMeasures, TrigramAssocMeasures
[18:19:11]	[Step 1/1]   File "/usr/local/python2.7.15/lib/python2.7/site-packages/nltk/metrics/__init__.py", line 16, in <module>
[18:19:11]	[Step 1/1]     from nltk.metrics.scores import (
[18:19:11]	[Step 1/1]   File "/usr/local/python2.7.15/lib/python2.7/site-packages/nltk/metrics/scores.py", line 18, in <module>
[18:19:11]	[Step 1/1]     from scipy.stats.stats import betai
[18:19:11]	[Step 1/1]   File "/usr/local/python2.7.15/lib/python2.7/site-packages/scipy/stats/__init__.py", line 367, in <module>
[18:19:11]	[Step 1/1]     from .stats import *
[18:19:11]	[Step 1/1]   File "/usr/local/python2.7.15/lib/python2.7/site-packages/scipy/stats/stats.py", line 173, in <module>
[18:19:11]	[Step 1/1]     from . import distributions
[18:19:11]	[Step 1/1]   File "/usr/local/python2.7.15/lib/python2.7/site-packages/scipy/stats/distributions.py", line 10, in <module>
[18:19:11]	[Step 1/1]     from ._distn_infrastructure import (entropy, rv_discrete, rv_continuous,
[18:19:11]	[Step 1/1]   File "/usr/local/python2.7.15/lib/python2.7/site-packages/scipy/stats/_distn_infrastructure.py", line 16, in <module>
[18:19:11]	[Step 1/1]     from scipy.misc import doccer
[18:19:11]	[Step 1/1]   File "/usr/local/python2.7.15/lib/python2.7/site-packages/scipy/misc/__init__.py", line 100, in <module>
[18:19:11]	[Step 1/1]     from .pilutil import *
[18:19:11]	[Step 1/1]   File "/usr/local/python2.7.15/lib/python2.7/site-packages/scipy/misc/pilutil.py", line 19, in <module>
[18:19:11]	[Step 1/1]     from PIL import Image, ImageFilter
[18:19:11]	[Step 1/1]   File "/usr/local/python2.7.15/lib/python2.7/site-packages/PIL/Image.py", line 151, in <module>
[18:19:11]	[Step 1/1]     from pathlib2 import Path
[18:19:11]	[Step 1/1]   File "/usr/local/python2.7.15/lib/python2.7/site-packages/pathlib2/__init__.py", line 52, in <module>
[18:19:11]	[Step 1/1]     from scandir import scandir as os_scandir
[18:19:11]	[Step 1/1]   File "/usr/local/python2.7.15/lib/python2.7/site-packages/scandir.py", line 452, in <module>
[18:19:11]	[Step 1/1]     libc = ctypes.CDLL(ctypes.util.find_library('c'), use_errno=True)
[18:19:11]	[Step 1/1]   File "/usr/local/python2.7.15/lib/python2.7/ctypes/util.py", line 274, in find_library
[18:19:11]	[Step 1/1]     return _findSoname_ldconfig(name) or _get_soname(_findLib_gcc(name))
[18:19:11]	[Step 1/1]   File "/usr/local/python2.7.15/lib/python2.7/ctypes/util.py", line 103, in _findLib_gcc
[18:19:11]	[Step 1/1]     stdout=subprocess.PIPE)
[18:19:11]	[Step 1/1]   File "/usr/local/python2.7.15/lib/python2.7/subprocess.py", line 394, in __init__
[18:19:11]	[Step 1/1]     errread, errwrite)
[18:19:11]	[Step 1/1]   File "/usr/local/python2.7.15/lib/python2.7/subprocess.py", line 938, in _execute_child
[18:19:11]	[Step 1/1]     self.pid = os.fork()
[18:19:11]	[Step 1/1] OSError: [Errno 12] Cannot allocate memory
指派人
分配到
无
里程碑
无
分配里程碑
工时统计
无
截止日期
无
标识: paddlepaddle/Paddle#21595
渝ICP备2023009037号

京公网安备11010502055752号

网络110报警服务 Powered by GitLab CE v13.7
开源知识
Git 入门 Pro Git 电子书 在线学 Git
Markdown 基础入门 IT 技术知识开源图谱
帮助
使用手册 反馈建议 博客
《GitCode 隐私声明》 《GitCode 服务条款》 关于GitCode
Powered by GitLab CE v13.7