提交 ce9ad07a 编写于 作者: H huangxinda

feat(ci): update ci and readme

上级 88486570
...@@ -23,12 +23,13 @@ jobs: ...@@ -23,12 +23,13 @@ jobs:
runs-on: self-hosted runs-on: self-hosted
needs: [check-commit] needs: [check-commit]
container: container:
image: localhost:5000/megengine-ci:latest image: localhost:5000/megengine-ci:v1
steps: steps:
- name: Checkout MegEngine - name: Checkout MegEngine
uses: actions/checkout@v2 uses: actions/checkout@v2
- name: Checkout submodules - name: Checkout submodules
run: | run: |
apt update&&apt install ninja-build
./third_party/prepare.sh ./third_party/prepare.sh
./third_party/install-mkl.sh ./third_party/install-mkl.sh
- name: Build MegEngine - name: Build MegEngine
...@@ -46,9 +47,9 @@ jobs: ...@@ -46,9 +47,9 @@ jobs:
runs-on: self-hosted runs-on: self-hosted
needs: [check-commit] needs: [check-commit]
container: container:
image: localhost:5000/megengine-ci:latest image: localhost:5000/megengine-ci:v1
volumes: volumes:
- /usr/local/cuda-10.1-libs:/usr/local/cuda-10.1-libs - /usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs
options: --gpus all --shm-size 1g options: --gpus all --shm-size 1g
env: env:
NCCL_LAUNCH_MODE: PARALLEL NCCL_LAUNCH_MODE: PARALLEL
...@@ -57,6 +58,7 @@ jobs: ...@@ -57,6 +58,7 @@ jobs:
uses: actions/checkout@v2 uses: actions/checkout@v2
- name: Checkout submodules - name: Checkout submodules
run: | run: |
apt update&&apt install ninja-build
./third_party/prepare.sh ./third_party/prepare.sh
./third_party/install-mkl.sh ./third_party/install-mkl.sh
- name: Build MegEngine - name: Build MegEngine
...@@ -72,8 +74,10 @@ jobs: ...@@ -72,8 +74,10 @@ jobs:
run: ./ci/run_cpp_test.sh cuda run: ./ci/run_cpp_test.sh cuda
auto-merge: auto-merge:
if: ${{ github.ref == 'refs/heads/try-import' }} if: ${{ github.ref == 'refs/heads/try-import' }}
runs-on: ubuntu-latest runs-on: self-hosted
needs: [cpu-test, gpu-test] needs: [cpu-test, gpu-test]
container:
image: localhost:5000/megengine-ci:v1
steps: steps:
- name: Checkout MegEngine - name: Checkout MegEngine
uses: actions/checkout@v2 uses: actions/checkout@v2
......
...@@ -12,7 +12,7 @@ MegEngine is a fast, scalable and easy-to-use deep learning framework, with auto ...@@ -12,7 +12,7 @@ MegEngine is a fast, scalable and easy-to-use deep learning framework, with auto
## Installation ## Installation
**NOTE:** MegEngine now supports Python installation on Linux-64bit/Windows-64bit/MacOS(CPU-Only)-10.14+ platforms with Python from 3.5 to 3.8. On Windows 10 you can either install the Linux distribution through [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl) or install the Windows distribution directly. Many other platforms are supported for inference. **NOTE:** MegEngine now supports Python installation on Linux-64bit/Windows-64bit/MacOS(CPU-Only)-10.14+/Android 7+(CPU-Only) platforms with Python from 3.5 to 3.8. On Windows 10 you can either install the Linux distribution through [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl) or install the Windows distribution directly. Many other platforms are supported for inference.
### Binaries ### Binaries
......
...@@ -13,7 +13,7 @@ MegEngine 是一个快速、可拓展、易于使用且支持自动求导的深 ...@@ -13,7 +13,7 @@ MegEngine 是一个快速、可拓展、易于使用且支持自动求导的深
## 安装说明 ## 安装说明
**注意:** MegEngine 现在支持在 Linux-64bit/Windows-64bit/macos-10.14及其以上 (MacOS只支持cpu) 等平台上安装 Python 包,支持Python3.5 到 Python3.8。对于 Windows 10 用户,可以通过安装 [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl) 进行体验,同时我们也原生支持Windows。MegEngine 也支持在很多其它平台上进行推理运算。 **注意:** MegEngine 现在支持在 Linux-64bit/Windows-64bit/macos-10.14/Android 7+ 及其以上 (MacOS/Android只支持cpu) 等平台上安装 Python 包,支持Python3.5 到 Python3.8。对于 Windows 10 用户,可以通过安装 [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl) 进行体验,同时我们也原生支持Windows。MegEngine 也支持在很多其它平台上进行推理运算。
### 通过包管理器安装 ### 通过包管理器安装
...@@ -26,8 +26,8 @@ python3 -m pip install megengine -f https://megengine.org.cn/whl/mge.html ...@@ -26,8 +26,8 @@ python3 -m pip install megengine -f https://megengine.org.cn/whl/mge.html
## 通过源码编译安装 ## 通过源码编译安装
* CMake编译细节请参考 [BUILD_README.md](scripts/cmake-build/BUILD_README.md) * CMake 编译细节请参考 [BUILD_README.md](scripts/cmake-build/BUILD_README.md)
* Python绑定编译细节请参考 [BUILD_PYTHON_WHL_README.md](scripts/whl/BUILD_PYTHON_WHL_README.md) * Python 绑定编译细节请参考 [BUILD_PYTHON_WHL_README.md](scripts/whl/BUILD_PYTHON_WHL_README.md)
## 如何参与贡献 ## 如何参与贡献
......
...@@ -27,7 +27,8 @@ function build() { ...@@ -27,7 +27,8 @@ function build() {
-DMGE_WITH_DISTRIBUTED=${DMGE_WITH_DISTRIBUTED} \ -DMGE_WITH_DISTRIBUTED=${DMGE_WITH_DISTRIBUTED} \
-DMGE_WITH_CUDA=${DMGE_WITH_CUDA} \ -DMGE_WITH_CUDA=${DMGE_WITH_CUDA} \
-DMGE_WITH_TEST=ON \ -DMGE_WITH_TEST=ON \
-DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DMGE_WITH_CUSTOM_OP=ON
make -j$(($(nproc) * 2)) -I ${build_dir} make -j$(($(nproc) * 2)) -I ${build_dir}
make develop make develop
popd >/dev/null popd >/dev/null
......
...@@ -24,17 +24,27 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ ...@@ -24,17 +24,27 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
swig \ swig \
vim \ vim \
wget \ wget \
libgl1-mesa-glx \
libsm6 \
libxext6 \
zlib1g-dev \ zlib1g-dev \
# GitLab Runner need Git 2.18 or higher to create a local Git repository # GitLab Runner need Git 2.18 or higher to create a local Git repository
&& add-apt-repository ppa:git-core/ppa -y && apt-get install --no-install-recommends -y git \ && add-apt-repository ppa:git-core/ppa -y && apt-get install --no-install-recommends -y git \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
RUN cd /tmp ; wget https://cmake.org/files/v3.14/cmake-3.14.4.tar.gz;tar -xzvf cmake-3.14.4.tar.gz;cd cmake-3.14.4;./configure; make -j32; make install RUN cd /tmp ; wget https://cmake.org/files/v3.15/cmake-3.15.2.tar.gz;tar -xzvf cmake-3.15.2.tar.gz;cd cmake-3.15.2;./configure; make -j32; make install
RUN git lfs install RUN git lfs install
ENV PATH=${PATH}:/usr/local/cuda/bin \ RUN pip3 install --upgrade pip
LIBRARY_PATH=${LIBRARY_PATH}:/usr/local/cuda/lib:/usr/local/cuda/lib64:/usr/local/cuda/lib/stubs:/usr/local/cuda/lib64/stubs:/usr/local/cuda-10.1-libs/cudnn-v7.6.0/lib:/usr/local/cuda-10.1-libs/cudnn-v7.6.0/lib64:/usr/local/cuda-10.1-libs/TensorRT-5.1.5.0/lib:/usr/local/cuda-10.1-libs/TensorRT-5.1.5.0/lib64 \
LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda-10.1-libs/cudnn-v7.6.0/lib:/usr/local/cuda-10.1-libs/cudnn-v7.6.0/lib64:/usr/local/cuda-10.1-libs/TensorRT-5.1.5.0/lib:/usr/local/cuda-10.1-libs/TensorRT-5.1.5.0/lib64:/tmp/build/cuda/dnn/cuda-stub/libcuda.so \ # TODO: set following envs in github environment.
CPATH=${CPATH}:/usr/local/cuda/include:/usr/local/cuda-10.1-libs/cudnn-v7.6.0/include:/usr/local/cuda-10.1-libs/TensorRT-5.1.5.0/include \ ENV CUDA_ROOT_DIR=/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cuda-10.1 \
CUDA_BIN_PATH=/usr/local/cuda TRT_ROOT_DIR=/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/TensorRT-6.0.1.5 \
TENSORRT_ROOT_DIR=${TRT_ROOT_DIR} \
CUDNN_ROOT_DIR=/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cudnn-v7.6.3 \
PATH=/usr/bin:${CUDA_ROOT_DIR}/bin:${CUDA_ROOT_DIR}/nsight-compute-2019.4.0:$PATH \
LIBRARY_PATH=${LIBRARY_PATH}:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cuda-10.1/lib:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cuda-10.1/lib64:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cuda-10.1/lib/stubs:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cuda-10.1/lib64/stubs:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cudnn-v7.6.3/lib:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cudnn-v7.6.3/lib64:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/TensorRT-6.0.1.5/lib:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/TensorRT-6.0.1.5/lib64 \
LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cudnn-v7.6.3/lib:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cudnn-v7.6.3/lib64:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/TensorRT-6.0.1.5/lib:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/TensorRT-6.0.1.5/lib64 \
CPATH=${CPATH}:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cuda-10.1/include:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cudnn-v7.6.3/include:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/TensorRT-6.0.1.5/include \
CUDA_BIN_PATH=/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cuda-10.1
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册