Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
Greenplum
DeepSpeed
提交
3347460e
D
DeepSpeed
项目概览
Greenplum
/
DeepSpeed
上一次同步 大约 1 年
通知
10
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
D
DeepSpeed
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
前往新版Gitcode,体验更适合开发者的 AI 搜索 >>
未验证
提交
3347460e
编写于
11月 24, 2020
作者:
J
Jeff Rasley
提交者:
GitHub
11月 24, 2020
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Switch to CI to GitHub Actions (#556)
上级
c18fb0de
变更
3
显示空白变更内容
内联
并排
Showing
3 changed file
with
10 addition
and
164 deletion
+10
-164
.github/workflows/main.yml
.github/workflows/main.yml
+10
-7
azure-pipelines-docker.yml
azure-pipelines-docker.yml
+0
-36
azure-pipelines.yml
azure-pipelines.yml
+0
-121
未找到文件。
.github/workflows/main.yml
浏览文件 @
3347460e
...
...
@@ -29,8 +29,12 @@ jobs:
-
name
:
environment
run
:
|
nvidia-smi
echo $GITHUB_WORKSPACE
pwd
which python
python --version
which nvcc
nvcc --version
python -c "import torch; print('torch:', torch.__version__, torch)"
python -c "import torch; print('CUDA available:', torch.cuda.is_available())"
# Runs a set of commands using the runners shell
-
name
:
Install deepspeed
...
...
@@ -40,11 +44,10 @@ jobs:
-
name
:
Formatting checks
run
:
|
pip install pre-commit
pre-commit run --all-files
# Runs a set of commands using the runners shell
-
name
:
Run u
nit tests
-
name
:
U
nit tests
run
:
|
if [[ -d ./torch-extensions ]]; then rm -rf ./torch-extensions; fi
TORCH_EXTENSIONS_DIR=./torch-extensions pytest --durations=0 --forked --verbose -x tests/unit/
azure-pipelines-docker.yml
已删除
100644 → 0
浏览文件 @
c18fb0de
# Docker
# Build a Docker image
# https://docs.microsoft.com/azure/devops/pipelines/languages/docker
trigger
:
-
master
resources
:
-
repo
:
self
variables
:
tag
:
'
$(Build.BuildId)'
stages
:
-
stage
:
Build
displayName
:
Build image
jobs
:
-
job
:
Build
displayName
:
Build
pool
:
vmImage
:
'
ubuntu-latest'
steps
:
-
task
:
Docker@2
displayName
:
Login to Docker Hub
inputs
:
command
:
login
containerRegistry
:
DeepSpeedDocker
-
task
:
Docker@2
displayName
:
Build and Push
inputs
:
command
:
buildAndPush
dockerfile
:
'
$(Build.SourcesDirectory)/docker/Dockerfile'
repository
:
deepspeed/deepspeed
tags
:
|
$(tag)
latest
azure-pipelines.yml
已删除
100644 → 0
浏览文件 @
c18fb0de
trigger
:
paths
:
exclude
:
-
docs
-
azure
jobs
:
-
job
:
DeepSpeed_Tests
timeoutInMinutes
:
360
pool
:
name
:
'
DS_testing'
strategy
:
matrix
:
PyTorch12-CUDA100
:
python.version
:
'
3.6'
cuda.version
:
'
10.0'
pytorch.version
:
'
1.2'
torchvision.version
:
'
0.4.0'
runmodeltests
:
false
#PyTorch15-CUDA101:
# python.version: '3.7'
# cuda.version: '10.1'
# pytorch.version: '1.5.0+cu101'
# torchvision.version: '0.6.0+cu101'
# runmodeltests: true
##PyTorch15-CUDA102:
# python.version: '3.7'
# cuda.version: '10.2'
# pytorch.version: '1.5'
# torchvision.version: '0.6.1'
# runmodeltests: true
variables
:
conda_env
:
'
ds_test_py$(python.version)_cuda$(cuda.version)_pytorch$(pytorch.version)'
steps
:
# Unfortunately nvidia's nvcc_linux-64=<version> seems to install 10.1 regardless?
# Most of this complexity is a workaround to get the compiler toolchain to match the
# cudatoolkit runtime
-
script
:
|
conda create --force --yes -n $(conda_env) python=$(python.version) cudatoolkit=$(cuda.version)
source activate $(conda_env)
conda install -q --yes conda
conda install -q --yes pip
conda install -q --yes gxx_linux-64
echo "PATH=$PATH, LD_LIBRARY_PATH=$LD_LIBRARY_PATH"
displayName
:
'
Setup
environment
python=$(python.version)
pytorch=$(pytorch.version)
cuda=$(cuda.version)'
# Manually install torch/torchvision first to enforce versioning.
-
script
:
|
source activate $(conda_env)
pip install --progress-bar=off torch==$(pytorch.version) torchvision==$(torchvision.version)
pip install .[dev]
ds_report
displayName
:
'
Install
DeepSpeed'
-
script
:
|
source activate $(conda_env)
which python
python --version
which nvcc
nvcc --version
which deepspeed
python -c "import torch; print('torch:', torch.__version__, torch)"
python -c "import torch; print('CUDA available:', torch.cuda.is_available())"
python -c "import deepspeed; print('deepspeed:', deepspeed.__version__)"
displayName
:
'
Show
environment'
-
script
:
|
source activate $(conda_env)
if [[ -d ./torch-extensions ]]; then rm -rf ./torch-extensions; fi
TORCH_EXTENSIONS_DIR=./torch-extensions pytest --durations=0 --forked --verbose -x tests/unit/
displayName
:
'
Unit
tests'
# - script: |
# source activate $(conda_env)
# ln -s /data/Megatron-LM/data DeepSpeedExamples/Megatron-LM/
# pip install --progress-bar=off -r DeepSpeedExamples/Megatron-LM/requirements.txt
# cd tests/model/
# rm -rf BingBertSquad/baseline
# rm -rf Megatron_GPT2/baseline
# pytest --durations=0 -s run_sanity_check.py
# condition: and(succeeded(), eq(variables['runmodeltests'], true))
# displayName: 'Model tests'
# #BingBertSquad logs
# - task: PublishPipelineArtifact@1
# inputs:
# targetPath: '$(Build.SourcesDirectory)/tests/model/BingBertSquad/test/'
# artifactName: BingBertSquad_logs
# displayName: 'BingBertSquad log uploads'
# condition: eq(variables['runmodeltests'], true)
-
job
:
Code_Quality_Checks
pool
:
vmImage
:
'
ubuntu-latest'
variables
:
conda_env
:
'
ds_codetest'
steps
:
-
bash
:
echo "##vso[task.prependpath]$CONDA/bin"
displayName
:
Add conda to PATH
-
script
:
|
conda create --force --yes -n $(conda_env) python=3.7
source activate $(conda_env)
displayName
:
'
Create
code
test
environment'
-
script
:
|
source activate $(conda_env)
pip install pre-commit
pre-commit run --all-files
displayName
:
'
Formatting
checks'
-
script
:
|
source activate $(conda_env)
pip install pylint
pylint --exit-zero deepspeed/
displayName
:
'
Code
linter'
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录