paddle识别卡,出现P40能成功的,2080Ti就不行
Created by: Mr-Nineteen
场景描述: 1、参数设置 export CUDA_VISIBLE_DEVICES="1" export FLAGS_selected_gpus="0" 2、测试代码 import paddle.fluid as fluid gpu_place = fluid.CUDAPlace(0) fluid.install_check.run_check() dev_count=fluid.core.get_cuda_device_count() 3、对比 a、P40 Python 3.7.3 (default, Mar 5 2020, 18:37:12) [GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux Type "help", "copyright", "credits" or "license" for more information.
import paddle.fluid as fluid gpu_place = fluid.CUDAPlace(0) fluid.install_check.run_check()
Running Verify Fluid Program ... W0507 10:01:45.536321 23 device_context.cc:261] Please NOTE: device: 0, CUDA Capability: 61, Driver API Version: 10.1, Runtime API Version: 9.0 W0507 10:01:45.543630 23 device_context.cc:269] device: 0, cuDNN Version: 7.0. Your Paddle Fluid is installed successfully! Let's start deep Learning with Paddle Fluid now
b、2080TI Python 3.7.3 (default, Mar 5 2020, 18:37:12) [GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux Type "help", "copyright", "credits" or "license" for more information.
import paddle.fluid as fluid gpu_place = fluid.CUDAPlace(0) fluid.install_check.run_check()
Running Verify Fluid Program ... W0507 10:05:24.886061 1215 device_context.cc:261] Please NOTE: device: 0, CUDA Capability: 75, Driver API Version: 10.1, Runtime API Version: 9.0 W0507 10:05:24.893177 1215 device_context.cc:269] device: 0, cuDNN Version: 7.0. Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python3.7/site-packages/paddle/fluid/install_check.py", line 65, in run_check fetch_list=[out.name, param_grads[1].name]) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/executor.py", line 565, in run use_program_cache=use_program_cache) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/executor.py", line 642, in run exe.run(program.desc, scope, 0, True, True, fetch_var_name) paddle.fluid.core.EnforceNotMet: Invoke operator mul error. Python Callstacks: File "/usr/local/lib/python3.7/site-packages/paddle/fluid/framework.py", line 1654, in append_op attrs=kwargs.get("attrs", None)) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layer_object_helper.py", line 52, in append_op stop_gradient=stop_gradient) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/nn.py", line 919, in forward "y_num_col_dims": 1 File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 158, in call outputs = self.forward(inputs) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/install_check.py", line 37, in forward x = self._fc1(inputs) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 158, in call outputs = self.forward(inputs) File "/usr/local/lib/python3.7/site-packages/paddle/fluid/install_check.py", line 58, in run_check out = simple_layer(inp) File "", line 1, in C++ Callstacks: CUBLAS: execution failed, at [/paddle/paddle/fluid/operators/math/blas_impl.cu.h:34] PaddlePaddle Call Stacks: 0 0x2b3805b3ad60p void paddle::platform::EnforceNotMet::Init<char const>(char const, char const*, int) + 352 1 0x2b3805b3b0d9p paddle::platform::EnforceNotMet::EnforceNotMet(std::exception_ptr::exception_ptr, char const*, int) + 137 2 0x2b3805da61b7p void paddle::operators::math::Blaspaddle::platform::CUDADeviceContext::GEMM(CBLAS_TRANSPOSE, CBLAS_TRANSPOSE, int, int, int, float, float const*, float const*, float, float*) const + 1815 3 0x2b3806220b8cp void paddle::operators::math::Blaspaddle::platform::CUDADeviceContext::MatMul(paddle::framework::Tensor const&, bool, paddle::framework::Tensor const&, bool, float, paddle::framework::Tensor*, float) const + 844 4 0x2b3806220fc6p paddle::operators::MulKernel<paddle::platform::CUDADeviceContext, float>::Compute(paddle::framework::ExecutionContext const&) const + 662 5 0x2b38062211b3p std::Function_handler<void (paddle::framework::ExecutionContext const&), paddle::framework::OpKernelRegistrarFunctor<paddle::platform::CUDAPlace, false, 0ul, paddle::operators::MulKernel<paddle::platform::CUDADeviceContext, float>, paddle::operators::MulKernel<paddle::platform::CUDADeviceContext, double>, paddle::operators::MulKernel<paddle::platform::CUDADeviceContext, paddle::platform::float16> >::operator()(char const*, char const*, int) const::{lambda(paddle::framework::ExecutionContext const&)#1 (closed)}>::M_invoke(std::Any_data const&, paddle::framework::ExecutionContext const&) + 35 6 0x2b38077ca446p paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void, boost::detail::variant::void, boost::detail::variant::void, boost::detail::variant::void, boost::detail::variant::void, boost::detail::variant::void, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&, paddle::framework::RuntimeContext*) const + 662 7 0x2b38077cabb4p paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&) const + 292 8 0x2b38077c84dcp paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&) + 332 9 0x2b3805cad4bep paddle::framework::Executor::RunPreparedContext(paddle::framework::ExecutorPrepareContext*, paddle::framework::Scope*, bool, bool, bool) + 382 10 0x2b3805cae2ffp paddle::framework::Executor::Run(paddle::framework::ProgramDesc const&, paddle::framework::Scope*, int, bool, bool, std::vector<std::string, std::allocatorstd::string > const&, bool) + 143 11 0x2b3805b2a96ep 12 0x2b3805b6d76ep 13 0x2b378cde1c8bp _PyMethodDef_RawFastCallKeywords + 795 14 0x2b378cde1d05p _PyCFunction_FastCallKeywords + 37 15 0x2b378cdba7c2p _PyEval_EvalFrameDefault + 27218 16 0x2b378cec8a68p _PyEval_EvalCodeWithName + 2584 17 0x2b378cde1758p _PyFunction_FastCallKeywords + 168 18 0x2b378cdbbdbcp _PyEval_EvalFrameDefault + 32844 19 0x2b378cec8a68p _PyEval_EvalCodeWithName + 2584 20 0x2b378cde1758p _PyFunction_FastCallKeywords + 168 21 0x2b378cdbbdbcp _PyEval_EvalFrameDefault + 32844 22 0x2b378cdb2b80p 23 0x2b378cdbcd00p _PyEval_EvalFrameDefault + 36752 24 0x2b378cec8a68p _PyEval_EvalCodeWithName + 2584 25 0x2b378cec8bbdp PyEval_EvalCodeEx + 109 26 0x2b378cec8c0bp PyEval_EvalCode + 59 27 0x2b378cf037d3p 28 0x2b378cf03a76p PyRun_InteractiveLoopFlags + 118 29 0x2b378cf0432ep PyRun_AnyFileExFlags + 62 30 0x2b378cf26a91p 31 0x2b378cf26cadp _Py_UnixMain + 45 32 0x2b378de2ab35p __libc_start_main + 245 33 0x40072ep