Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
MegEngine 天元
MegEngine
提交
fd83e025
MegEngine
项目概览
MegEngine 天元
/
MegEngine
1 年多 前同步成功
通知
403
Star
4705
Fork
582
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
MegEngine
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
提交
fd83e025
编写于
4月 02, 2020
作者:
M
Megvii Engine Team
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
fix(mge/test): fix reproducibility of test_correctness.py
GitOrigin-RevId: 70bd43bbabd022ce9538d92291ddb1c93a596e0a
上级
b8d27934
变更
3
隐藏空白更改
内联
并排
Showing
3 changed file
with
51 addition
and
8 deletion
+51
-8
python_module/test/integration/mnist_model_with_test.mge
python_module/test/integration/mnist_model_with_test.mge
+0
-0
python_module/test/integration/mnist_model_with_test_cpu.mge
python_module/test/integration/mnist_model_with_test_cpu.mge
+0
-0
python_module/test/integration/test_correctness.py
python_module/test/integration/test_correctness.py
+51
-8
未找到文件。
python_module/test/integration/mnist_model_with_test.mge
浏览文件 @
fd83e025
无法预览此类型文件
python_module/test/integration/mnist_model_with_test_cpu.mge
浏览文件 @
fd83e025
无法预览此类型文件
python_module/test/integration/test_correctness.py
浏览文件 @
fd83e025
...
...
@@ -7,6 +7,8 @@
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT ARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
import
os
import
re
import
subprocess
import
sys
import
numpy
as
np
...
...
@@ -15,18 +17,48 @@ import megengine as mge
import
megengine.functional
as
F
from
megengine
import
jit
,
tensor
from
megengine.functional.debug_param
import
set_conv_execution_strategy
from
megengine.module
import
BatchNorm2d
,
Conv2d
,
Linear
,
MaxPool2d
,
Module
from
megengine.module
import
AvgPool2d
,
BatchNorm2d
,
Conv2d
,
Linear
,
Module
from
megengine.optimizer
import
SGD
from
megengine.test
import
assertTensorClose
def
get_gpu_name
():
try
:
gpu_info
=
subprocess
.
check_output
(
[
"nvidia-smi"
,
"--query-gpu=gpu_name"
,
"--format=csv,noheader"
]
)
gpu_info
=
gpu_info
.
decode
(
"ascii"
).
split
(
"
\n
"
)[
0
]
except
:
gpu_info
=
"None"
return
gpu_info
def
get_cpu_name
():
cpu_info
=
"None"
try
:
cpu_info
=
subprocess
.
check_output
([
"cat"
,
"/proc/cpuinfo"
]).
decode
(
"ascii"
)
for
line
in
cpu_info
.
split
(
"
\n
"
):
if
"model name"
in
line
:
return
re
.
sub
(
".*model name.*:"
,
""
,
line
,
1
).
strip
()
except
:
pass
return
cpu_info
def
get_xpu_name
():
if
mge
.
is_cuda_available
():
return
get_gpu_name
()
else
:
return
get_cpu_name
()
class
MnistNet
(
Module
):
def
__init__
(
self
,
has_bn
=
False
):
super
().
__init__
()
self
.
conv0
=
Conv2d
(
1
,
20
,
kernel_size
=
5
,
bias
=
True
)
self
.
pool0
=
Max
Pool2d
(
2
)
self
.
pool0
=
Avg
Pool2d
(
2
)
self
.
conv1
=
Conv2d
(
20
,
20
,
kernel_size
=
5
,
bias
=
True
)
self
.
pool1
=
Max
Pool2d
(
2
)
self
.
pool1
=
Avg
Pool2d
(
2
)
self
.
fc0
=
Linear
(
20
*
4
*
4
,
500
,
bias
=
True
)
self
.
fc1
=
Linear
(
500
,
10
,
bias
=
True
)
self
.
bn0
=
None
...
...
@@ -67,6 +99,13 @@ def update_model(model_path):
The model with pre-trained weights is trained for one iter with the test data attached.
The loss and updated net state dict is dumped.
.. code-block:: python
from test_correctness import update_model
update_model('mnist_model_with_test.mge') # for gpu
update_model('mnist_model_with_test_cpu.mge') # for cpu
"""
net
=
MnistNet
(
has_bn
=
True
)
checkpoint
=
mge
.
load
(
model_path
)
...
...
@@ -83,7 +122,11 @@ def update_model(model_path):
loss
=
train
(
data
,
label
,
net
=
net
,
opt
=
opt
)
opt
.
step
()
checkpoint
.
update
({
"net_updated"
:
net
.
state_dict
(),
"loss"
:
loss
.
numpy
()})
xpu_name
=
get_xpu_name
()
checkpoint
.
update
(
{
"net_updated"
:
net
.
state_dict
(),
"loss"
:
loss
.
numpy
(),
"xpu"
:
xpu_name
}
)
mge
.
save
(
checkpoint
,
model_path
)
...
...
@@ -92,9 +135,9 @@ def run_test(model_path, use_jit, use_symbolic):
"""
Load the model with test cases and run the training for one iter.
The loss and updated weights are compared with reference value to verify the correctness.
Dump a new file with updated result by calling update_model
if you think the test fails due to numerical rounding errors instead of bugs.
Dump a new file with updated result by calling update_model
if you think the test fails due to numerical rounding errors instead of bugs.
Please think twice before you do so.
"""
...
...
@@ -109,7 +152,7 @@ def run_test(model_path, use_jit, use_symbolic):
data
.
set_value
(
checkpoint
[
"data"
])
label
.
set_value
(
checkpoint
[
"label"
])
max_err
=
1e-
1
max_err
=
1e-
5
train_func
=
train
if
use_jit
:
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录