Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
Oneflow-Inc
OneFlow-Benchmark
提交
64939ea9
O
OneFlow-Benchmark
项目概览
Oneflow-Inc
/
OneFlow-Benchmark
上一次同步 2 年多
通知
1
Star
92
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
O
OneFlow-Benchmark
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
前往新版Gitcode,体验更适合开发者的 AI 搜索 >>
提交
64939ea9
编写于
10月 28, 2021
作者:
S
ShawnXuan
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
test
上级
ce08ec2b
变更
1
隐藏空白更改
内联
并排
Showing
1 changed file
with
59 addition
and
64 deletion
+59
-64
.github/workflows/cnn_e2e.yml
.github/workflows/cnn_e2e.yml
+59
-64
未找到文件。
.github/workflows/cnn_e2e.yml
浏览文件 @
64939ea9
...
...
@@ -147,7 +147,7 @@ jobs:
python3 ${{ env.ONEFLOW_SRC }}/tools/create_pip_index.py --dir_key ${oss_dir} -b oneflow-staging --index_key=${oss_branch_dir}/index.html --index_key=${oss_dir}/index.html --index_key=${{ needs.find-oss-wheel.outputs.ONEFLOW_WHEEL_PATH }}/index.html
test
:
name
:
Test
suite
name
:
Test
ResNet50
needs
:
[
build-oneflow
]
if
:
always()
runs-on
:
[
'
self-hosted'
,
'
linux'
,
'
x64'
,
'
gpu-8-titan-v'
]
...
...
@@ -155,68 +155,63 @@ jobs:
TEST_CONTAINER_NAME
:
"
oneflow_benchmark-run-id-${{
github.run_id
}}-${{
matrix.entry
}}-test"
TEST_WITH_TORCH_IMG_TAG
:
registry.cn-beijing.aliyuncs.com/oneflow/test-with-pytorch-1.9.0:e7a497b41d8b7f1bce055b1f23d027f93b1557ae
steps
:
-
name
:
just a test
run
:
|
echo ${TEST_CONTAINER_NAME}
echo ${TEST_WITH_TORCH_IMG_TAG}
echo ${{ needs.build-oneflow.outputs.ONEFLOW_WHEEL_PATH }}
# - name: Fix permissions
# run: |
# set -x
# docker run --rm -v $PWD:/p -w /p busybox chown -R $(id -u):$(id -g) .
# - name: Checkout Oneflow-Inc/OneFlow-Benchmark
# uses: actions/checkout@v2
# - name: Remove container
# timeout-minutes: 45
# run: |
# docker rm -f ${{ env.TEST_CONTAINER_NAME }} || true
# - name: Enable Pytorch container
# run: |
# echo "TEST_IMG_TAG=${TEST_WITH_TORCH_IMG_TAG}" >> $GITHUB_ENV
# - name: Start container
# run: |
# docker pull ${{ env.TEST_IMG_TAG }}
# docker run -d --rm --privileged --network host --shm-size=8g \
# --cap-add=SYS_PTRACE --security-opt seccomp=unconfined \
# --runtime=nvidia \
# -v /DATA/disk1:/dataset:ro \
# -e ONEFLOW_WHEEL_PATH=${{ env.ONEFLOW_WHEEL_PATH }} \
# -v $PWD:$PWD \
# -w $PWD \
# -e E2E_NUM_EPOCHS=${{ github.event.inputs.num_epochs }} \
# -e E2E_GPU_NUM_PER_NODE=${{ github.event.inputs.gpu_num_per_node }} \
# -e E2E_NODE_NUM=1 \
# -e E2E_BATCH_SIZE=32 \
# -e E2E_LEARNING_RATE=1.536 \
# -e E2E_SRC_ROOT=Classification/cnns \
# -e E2E_DATA_ROOT=/dataset/ImageNet/ofrecord \
# --name ${TEST_CONTAINER_NAME} \
# ${{ env.TEST_IMG_TAG }} \
# sleep 3600
# # -e ONEFLOW_CI=1 \
# # -v /model_zoo:/model_zoo:ro \
# # -v $HOME/test-container-cache/dot-local:/root/.local \
# # -v $HOME/test-container-cache/dot-cache:/root/.cache \
# # -e ONEFLOW_BIN_PATH=${ONEFLOW_BIN_PATH} \
# # -v ${ONEFLOW_WHEEL_PATH}:${ONEFLOW_WHEEL_PATH}:ro \
# # -v ${ONEFLOW_BIN_PATH}:${ONEFLOW_BIN_PATH}:ro \
# # -v ${ONEFLOW_TEST_CACHE_DIR}:${ONEFLOW_TEST_CACHE_DIR} \
# # -e ONEFLOW_TEST_CACHE_DIR=${ONEFLOW_TEST_CACHE_DIR} \
# - name: Install OneFlow
# run: |
# docker exec ${TEST_CONTAINER_NAME} python3 --version
# docker exec ${TEST_CONTAINER_NAME} python3 -m pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
# docker exec ${TEST_CONTAINER_NAME} python3 -m pip install --find-links=http://oneflow-staging.oss-cn-beijing.aliyuncs.com/${{ needs.build-oneflow.outputs.ONEFLOW_WHEEL_PATH }} oneflow
# - name: Test container
# run: |
# docker exec ${{ env.TEST_CONTAINER_NAME }} bash ci/test/resnet50_e2e.sh
# - name: Upload log
# uses: ./.github/actions/upload_oss
# with:
# src_path: log
# oss_dst_path: oss://oneflow-log/OneFlow-Benchmark/${{ github.ref }}.${GITHUB_SHA::7}/oneflow/${{ github.event.inputs.of_branch_or_commit }}/${{github.run_id}}/log
# oss_access_key_id: ${{ secrets.OSS_ACCESS_KEY_ID }}
# oss_access_key_secret: ${{ secrets.OSS_ACCESS_KEY_SECRET }}
# upload_core: false
-
name
:
Fix permissions
run
:
|
set -x
docker run --rm -v $PWD:/p -w /p busybox chown -R $(id -u):$(id -g) .
-
name
:
Checkout Oneflow-Inc/OneFlow-Benchmark
uses
:
actions/checkout@v2
-
name
:
Remove container
timeout-minutes
:
45
run
:
|
docker rm -f ${{ env.TEST_CONTAINER_NAME }} || true
-
name
:
Enable Pytorch container
run
:
|
echo "TEST_IMG_TAG=${TEST_WITH_TORCH_IMG_TAG}" >> $GITHUB_ENV
-
name
:
Start container
run
:
|
docker pull ${{ env.TEST_IMG_TAG }}
docker run -d --rm --privileged --network host --shm-size=8g \
--cap-add=SYS_PTRACE --security-opt seccomp=unconfined \
--runtime=nvidia \
-v /DATA/disk1:/dataset:ro \
-e ONEFLOW_WHEEL_PATH=${{ env.ONEFLOW_WHEEL_PATH }} \
-v $PWD:$PWD \
-w $PWD \
-e E2E_NUM_EPOCHS=${{ github.event.inputs.num_epochs }} \
-e E2E_GPU_NUM_PER_NODE=${{ github.event.inputs.gpu_num_per_node }} \
-e E2E_NODE_NUM=1 \
-e E2E_BATCH_SIZE=32 \
-e E2E_LEARNING_RATE=1.536 \
-e E2E_SRC_ROOT=Classification/cnns \
-e E2E_DATA_ROOT=/dataset/ImageNet/ofrecord \
--name ${TEST_CONTAINER_NAME} \
${{ env.TEST_IMG_TAG }} \
sleep 3600
# -e ONEFLOW_CI=1 \
# -v /model_zoo:/model_zoo:ro \
# -v $HOME/test-container-cache/dot-local:/root/.local \
# -v $HOME/test-container-cache/dot-cache:/root/.cache \
# -e ONEFLOW_BIN_PATH=${ONEFLOW_BIN_PATH} \
# -v ${ONEFLOW_WHEEL_PATH}:${ONEFLOW_WHEEL_PATH}:ro \
# -v ${ONEFLOW_BIN_PATH}:${ONEFLOW_BIN_PATH}:ro \
# -v ${ONEFLOW_TEST_CACHE_DIR}:${ONEFLOW_TEST_CACHE_DIR} \
# -e ONEFLOW_TEST_CACHE_DIR=${ONEFLOW_TEST_CACHE_DIR} \
-
name
:
Install OneFlow
run
:
|
docker exec ${TEST_CONTAINER_NAME} python3 --version
docker exec ${TEST_CONTAINER_NAME} python3 -m pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
docker exec ${TEST_CONTAINER_NAME} python3 -m pip install --find-links=http://oneflow-staging.oss-cn-beijing.aliyuncs.com/${{ needs.build-oneflow.outputs.ONEFLOW_WHEEL_PATH }} oneflow
-
name
:
Test container
run
:
|
docker exec ${{ env.TEST_CONTAINER_NAME }} bash ci/test/resnet50_e2e.sh
-
name
:
Upload log
uses
:
./.github/actions/upload_oss
with
:
src_path
:
log
oss_dst_path
:
oss://oneflow-log/OneFlow-Benchmark/${{ github.ref }}.${GITHUB_SHA::7}/oneflow/${{ github.event.inputs.of_branch_or_commit }}/${{github.run_id}}/log
oss_access_key_id
:
${{ secrets.OSS_ACCESS_KEY_ID }}
oss_access_key_secret
:
${{ secrets.OSS_ACCESS_KEY_SECRET }}
upload_core
:
false
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录