Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
机器未来
Paddle
提交
6f2bb20e
P
Paddle
项目概览
机器未来
/
Paddle
与 Fork 源项目一致
Fork自
PaddlePaddle / Paddle
通知
1
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
1
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
P
Paddle
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
1
Issue
1
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
未验证
提交
6f2bb20e
编写于
12月 07, 2020
作者:
T
tianshuo78520a
提交者:
GitHub
12月 07, 2020
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
update docker nccl version 2.7.8 (#28575)
上级
64e4e17f
变更
8
隐藏空白更改
内联
并排
Showing
8 changed file
with
168 addition
and
45 deletion
+168
-45
tools/dockerfile/Dockerfile.centos
tools/dockerfile/Dockerfile.centos
+3
-5
tools/dockerfile/Dockerfile.ubuntu18
tools/dockerfile/Dockerfile.ubuntu18
+11
-9
tools/dockerfile/build_scripts/build.sh
tools/dockerfile/build_scripts/build.sh
+0
-11
tools/dockerfile/build_scripts/install_nccl2.sh
tools/dockerfile/build_scripts/install_nccl2.sh
+12
-6
tools/dockerfile/build_scripts/install_trt.sh
tools/dockerfile/build_scripts/install_trt.sh
+13
-0
tools/dockerfile/centos6_manylinux.sh
tools/dockerfile/centos6_manylinux.sh
+3
-4
tools/dockerfile/ci_dockerfile.sh
tools/dockerfile/ci_dockerfile.sh
+2
-10
tools/dockerfile/ubuntu18_dev.sh
tools/dockerfile/ubuntu18_dev.sh
+124
-0
未找到文件。
tools/dockerfile/Dockerfile.centos
浏览文件 @
6f2bb20e
...
@@ -13,10 +13,11 @@ ENV PATH /opt/rh/devtoolset-2/root/usr/bin:$PATH
...
@@ -13,10 +13,11 @@ ENV PATH /opt/rh/devtoolset-2/root/usr/bin:$PATH
ENV LD_LIBRARY_PATH /opt/rh/devtoolset-2/root/usr/lib64:/opt/rh/devtoolset-2/root/usr/lib:/usr/local/lib64:/usr/local/lib:${LD_LIBRARY_PATH}
ENV LD_LIBRARY_PATH /opt/rh/devtoolset-2/root/usr/lib64:/opt/rh/devtoolset-2/root/usr/lib:/usr/local/lib64:/usr/local/lib:${LD_LIBRARY_PATH}
ENV PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
ENV PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
RUN yum install -y gettext-devel sqlite-devel zlib-devel openssl-devel pcre-devel vim tk-devel tkinter libtool xz graphviz wget curl-devel
RUN yum install -y
bzip2
gettext-devel sqlite-devel zlib-devel openssl-devel pcre-devel vim tk-devel tkinter libtool xz graphviz wget curl-devel
COPY build_scripts /build_scripts
COPY build_scripts /build_scripts
RUN bash build_scripts/build.sh
RUN bash build_scripts/build.sh
RUN bash build_scripts/install_nccl2.sh
RUN bash build_scripts/install_nccl2.sh
RUN bash build_scripts/install_trt.sh
RUN rm -rf build_scripts
RUN rm -rf build_scripts
ENV SSL_CERT_FILE=/opt/_internal/certs.pem
ENV SSL_CERT_FILE=/opt/_internal/certs.pem
...
@@ -40,7 +41,7 @@ ENV GOROOT=/usr/local/go GOPATH=/root/gopath
...
@@ -40,7 +41,7 @@ ENV GOROOT=/usr/local/go GOPATH=/root/gopath
ENV PATH=${GOROOT}/bin:${GOPATH}/bin:${PATH}
ENV PATH=${GOROOT}/bin:${GOPATH}/bin:${PATH}
# protobuf 3.6.1
# protobuf 3.6.1
RUN cd /opt && wget -q --no-check-certificate https://
github.com/google/protobuf/releases/download/v3.6.1/protobuf-cpp-3.6.1.tar.gz && \
RUN cd /opt && wget -q --no-check-certificate https://
paddle-ci.cdn.bcebos.com/protobuf-cpp-3.6.1.tar.gz && \
tar xzf protobuf-cpp-3.6.1.tar.gz && \
tar xzf protobuf-cpp-3.6.1.tar.gz && \
cd protobuf-3.6.1 && ./configure && make -j4 && make install && cd .. && rm -f protobuf-cpp-3.6.1.tar.gz
cd protobuf-3.6.1 && ./configure && make -j4 && make install && cd .. && rm -f protobuf-cpp-3.6.1.tar.gz
...
@@ -79,7 +80,4 @@ RUN wget https://paddle-ci.gz.bcebos.com/ccache-3.7.9.tar.gz && \
...
@@ -79,7 +80,4 @@ RUN wget https://paddle-ci.gz.bcebos.com/ccache-3.7.9.tar.gz && \
make -j8 && make install && \
make -j8 && make install && \
ln -s /usr/local/ccache-3.7.9/bin/ccache /usr/local/bin/ccache
ln -s /usr/local/ccache-3.7.9/bin/ccache /usr/local/bin/ccache
# Downgrade gcc&&g++
<install_gcc>
CMD ["bash", "/paddle/paddle/scripts/docker/build.sh"]
CMD ["bash", "/paddle/paddle/scripts/docker/build.sh"]
tools/dockerfile/Dockerfile.ubuntu18
浏览文件 @
6f2bb20e
...
@@ -20,18 +20,20 @@ RUN apt-get update && \
...
@@ -20,18 +20,20 @@ RUN apt-get update && \
apt-get install -y software-properties-common && add-apt-repository ppa:deadsnakes/ppa && \
apt-get install -y software-properties-common && add-apt-repository ppa:deadsnakes/ppa && \
apt-get update && \
apt-get update && \
apt-get install -y curl wget vim git unzip unrar tar xz-utils bzip2 gzip \
apt-get install -y curl wget vim git unzip unrar tar xz-utils bzip2 gzip \
coreutils ntp language-pack-zh-hans python-qt4 libsm6 libxext6 libxrender-dev
coreutils ntp language-pack-zh-hans python-qt4 libsm6 libxext6 libxrender-dev \
bison graphviz libjpeg-dev zlib1g-dev automake locales clang-format swig net-tools libtool module-init-tools
# Downgrade gcc&&g++
# Downgrade gcc&&g++
WORKDIR /usr/bin
WORKDIR /usr/bin
COPY tools/dockerfile/build_scripts /build_scripts
COPY tools/dockerfile/build_scripts /build_scripts
RUN bash /build_scripts/install_gcc.sh gcc82 && rm -rf /build_scripts
RUN bash /build_scripts/install_trt.sh
RUN cp gcc gcc.bak && cp g++ g++.bak && rm gcc && rm g++
RUN bash /build_scripts/install_gcc.sh gcc82 && rm -rf /build_scripts
RUN ln -s /usr/local/gcc-8.2/bin/gcc /usr/local/bin/gcc
RUN cp gcc gcc.bak && cp g++ g++.bak && rm gcc && rm g++
RUN ln -s /usr/local/gcc-8.2/bin/g++ /usr/local/bin/g++
RUN ln -s /usr/local/gcc-8.2/bin/gcc /usr/local/bin/gcc
RUN ln -s /usr/local/gcc-8.2/bin/gcc /usr/bin/gcc
RUN ln -s /usr/local/gcc-8.2/bin/g++ /usr/local/bin/g++
RUN ln -s /usr/local/gcc-8.2/bin/g++ /usr/bin/g++
RUN ln -s /usr/local/gcc-8.2/bin/gcc /usr/bin/gcc
ENV PATH=/usr/local/gcc-8.2/bin:$PATH
RUN ln -s /usr/local/gcc-8.2/bin/g++ /usr/bin/g++
ENV PATH=/usr/local/gcc-8.2/bin:$PATH
RUN apt-get update && \
RUN apt-get update && \
apt-get install -y python2.7 python2.7-dev \
apt-get install -y python2.7 python2.7-dev \
...
...
tools/dockerfile/build_scripts/build.sh
浏览文件 @
6f2bb20e
...
@@ -30,8 +30,6 @@ CPYTHON_VERSIONS="3.8.0 3.7.0 3.6.0 3.5.1 2.7.15"
...
@@ -30,8 +30,6 @@ CPYTHON_VERSIONS="3.8.0 3.7.0 3.6.0 3.5.1 2.7.15"
# archive
# archive
OPENSSL_ROOT
=
openssl-1.1.0i
OPENSSL_ROOT
=
openssl-1.1.0i
OPENSSL_HASH
=
ebbfc844a8c8cc0ea5dc10b86c9ce97f401837f3fa08c17b2cdadc118253cf99
OPENSSL_HASH
=
ebbfc844a8c8cc0ea5dc10b86c9ce97f401837f3fa08c17b2cdadc118253cf99
EPEL_RPM_HASH
=
e5ed9ecf22d0c4279e92075a64c757ad2b38049bcf5c16c4f2b75d5f6860dc0d
DEVTOOLS_HASH
=
a8ebeb4bed624700f727179e6ef771dafe47651131a00a78b342251415646acc
PATCHELF_HASH
=
f2aa40a6148cb3b0ca807a1bf836b081793e55ec9e5540a5356d800132be7e0a
PATCHELF_HASH
=
f2aa40a6148cb3b0ca807a1bf836b081793e55ec9e5540a5356d800132be7e0a
CURL_ROOT
=
curl-7.49.1
CURL_ROOT
=
curl-7.49.1
CURL_HASH
=
eb63cec4bef692eab9db459033f409533e6d10e20942f4b060b32819e81885f1
CURL_HASH
=
eb63cec4bef692eab9db459033f409533e6d10e20942f4b060b32819e81885f1
...
@@ -51,15 +49,6 @@ source $MY_DIR/build_utils.sh
...
@@ -51,15 +49,6 @@ source $MY_DIR/build_utils.sh
# EPEL support
# EPEL support
yum
-y
install
wget curl epel-release
yum
-y
install
wget curl epel-release
#curl -sLO https://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
#check_sha256sum epel-release-6-8.noarch.rpm $EPEL_RPM_HASH
# Dev toolset (for LLVM and other projects requiring C++11 support)
#curl -sLO http://people.centos.org/tru/devtools-2/devtools-2.repo
#check_sha256sum devtools-2.repo $DEVTOOLS_HASH
#mv devtools-2.repo /etc/yum.repos.d/devtools-2.repo
#rpm -Uvh --replacepkgs epel-release-6*.rpm
#rm -f epel-release-6*.rpm
# Development tools and libraries
# Development tools and libraries
yum
-y
install
bzip2 make git patch unzip bison yasm diffutils
\
yum
-y
install
bzip2 make git patch unzip bison yasm diffutils
\
...
...
tools/dockerfile/build_scripts/install_nccl2.sh
浏览文件 @
6f2bb20e
...
@@ -17,12 +17,18 @@
...
@@ -17,12 +17,18 @@
VERSION
=
$(
nvcc
--version
|
grep
release |
grep
-oEi
"release ([0-9]+)
\.
([0-9])"
|
sed
"s/release //"
)
VERSION
=
$(
nvcc
--version
|
grep
release |
grep
-oEi
"release ([0-9]+)
\.
([0-9])"
|
sed
"s/release //"
)
if
[
"
$VERSION
"
==
"10.0"
]
;
then
if
[
"
$VERSION
"
==
"10.0"
]
;
then
DEB
=
"nccl-repo-ubuntu1604-2.4.7-ga-cuda10.0_1-1_amd64.deb"
DEB
=
"nccl-repo-ubuntu1604-2.4.7-ga-cuda10.0_1-1_amd64.deb"
elif
[
"
$VERSION
"
==
"11.0"
]
;
then
elif
[
"
$VERSION
"
==
"10.2"
]
||
[
"
$VERSION
"
==
"10.1"
]
||
[
"
$VERSION
"
==
"11.0"
]
;
then
DEB
=
"nccl-repo-ubuntu1604-2.7.8-ga-cuda11.0_1-1_amd64.deb"
if
[
-f
"ls /etc/redhat-release "
]
;
then
elif
[
"
$VERSION
"
==
"10.2"
]
;
then
rm
-f
/usr/local/lib/libnccl.so
DEB
=
"nccl-repo-ubuntu1604-2.4.7-ga-cuda10.0_1-1_amd64.deb"
wget
--no-check-certificate
-q
https://nccl2-deb.cdn.bcebos.com/libnccl-2.7.8-1+cuda10.2.x86_64.rpm
elif
[
"
$VERSION
"
==
"10.1"
]
;
then
wget
--no-check-certificate
-q
https://nccl2-deb.cdn.bcebos.com/libnccl-devel-2.7.8-1+cuda10.2.x86_64.rpm
DEB
=
"nccl-repo-ubuntu1604-2.4.7-ga-cuda10.0_1-1_amd64.deb"
wget
--no-check-certificate
-q
https://nccl2-deb.cdn.bcebos.com/libnccl-static-2.7.8-1+cuda10.2.x86_64.rpm
rpm
-ivh
libnccl-2.7.8-1+cuda10.2.x86_64.rpm
rpm
-ivh
libnccl-devel-2.7.8-1+cuda10.2.x86_64.rpm
rpm
-ivh
libnccl-static-2.7.8-1+cuda10.2.x86_64.rpm
&&
rm
-f
/usr/include/nccl.h
exit
0
fi
DEB
=
"nccl-repo-ubuntu1604-2.7.8-ga-cuda10.2_1-1_amd64.deb"
elif
[
"
$VERSION
"
==
"9.0"
]
;
then
elif
[
"
$VERSION
"
==
"9.0"
]
;
then
DEB
=
"nccl-repo-ubuntu1604-2.3.7-ga-cuda9.0_1-1_amd64.deb"
DEB
=
"nccl-repo-ubuntu1604-2.3.7-ga-cuda9.0_1-1_amd64.deb"
else
else
...
...
tools/dockerfile/build_scripts/install_trt.sh
浏览文件 @
6f2bb20e
...
@@ -20,12 +20,25 @@ if [[ "$VERSION" == "10.1" ]];then
...
@@ -20,12 +20,25 @@ if [[ "$VERSION" == "10.1" ]];then
wget
-q
https://paddle-ci.gz.bcebos.com/TRT/TensorRT6-cuda10.1-cudnn7.tar.gz
--no-check-certificate
wget
-q
https://paddle-ci.gz.bcebos.com/TRT/TensorRT6-cuda10.1-cudnn7.tar.gz
--no-check-certificate
tar
-zxf
TensorRT6-cuda10.1-cudnn7.tar.gz
-C
/usr/local
tar
-zxf
TensorRT6-cuda10.1-cudnn7.tar.gz
-C
/usr/local
cp
-rf
/usr/local/TensorRT6-cuda10.1-cudnn7/include/
*
/usr/include/
&&
cp
-rf
/usr/local/TensorRT6-cuda10.1-cudnn7/lib/
*
/usr/lib/
cp
-rf
/usr/local/TensorRT6-cuda10.1-cudnn7/include/
*
/usr/include/
&&
cp
-rf
/usr/local/TensorRT6-cuda10.1-cudnn7/lib/
*
/usr/lib/
rm
TensorRT6-cuda10.1-cudnn7.tar.gz
elif
[[
"
$VERSION
"
==
"11.0"
]]
;
then
wget
-q
https://paddle-ci.cdn.bcebos.com/TRT/TensorRT-7.1.3.4.Ubuntu-16.04.x86_64-gnu.cuda-11.0.cudnn8.0.tar.gz
--no-check-certificate
tar
-zxf
TensorRT-7.1.3.4.Ubuntu-16.04.x86_64-gnu.cuda-11.0.cudnn8.0.tar.gz
-C
/usr/local
cp
-rf
/usr/local/TensorRT-7.1.3.4/include/
*
/usr/include/
&&
cp
-rf
/usr/local/TensorRT-7.1.3.4/lib/
*
/usr/lib/
rm
TensorRT-7.1.3.4.Ubuntu-16.04.x86_64-gnu.cuda-11.0.cudnn8.0.tar.gz
elif
[[
"
$VERSION
"
==
"10.2"
]]
;
then
wget
-q
https://paddle-ci.cdn.bcebos.com/TRT/TensorRT7-cuda10.2-cudnn7.tar.gz
--no-check-certificate
tar
-zxf
TensorRT7-cuda10.2-cudnn7.tar.gz
-C
/usr/local
cp
-rf
/usr/local/TensorRT-7.0.0.11/include/
*
/usr/include/
&&
cp
-rf
/usr/local/TensorRT-7.0.0.11/lib/
*
/usr/lib/
rm
TensorRT7-cuda10.2-cudnn7.tar.gz
elif
[[
"
$VERSION
"
==
"10.0"
]]
;
then
elif
[[
"
$VERSION
"
==
"10.0"
]]
;
then
wget
-q
https://paddle-ci.gz.bcebos.com/TRT/TensorRT6-cuda10.0-cudnn7.tar.gz
--no-check-certificate
wget
-q
https://paddle-ci.gz.bcebos.com/TRT/TensorRT6-cuda10.0-cudnn7.tar.gz
--no-check-certificate
tar
-zxf
TensorRT6-cuda10.0-cudnn7.tar.gz
-C
/usr/local
tar
-zxf
TensorRT6-cuda10.0-cudnn7.tar.gz
-C
/usr/local
cp
-rf
/usr/local/TensorRT6-cuda10.0-cudnn7/include/
*
/usr/include/
&&
cp
-rf
/usr/local/TensorRT6-cuda10.0-cudnn7/lib/
*
/usr/lib/
cp
-rf
/usr/local/TensorRT6-cuda10.0-cudnn7/include/
*
/usr/include/
&&
cp
-rf
/usr/local/TensorRT6-cuda10.0-cudnn7/lib/
*
/usr/lib/
rm
TensorRT6-cuda10.0-cudnn7.tar.gz
elif
[[
"
$VERSION
"
==
"9.0"
]]
;
then
elif
[[
"
$VERSION
"
==
"9.0"
]]
;
then
wget
-q
https://paddle-ci.gz.bcebos.com/TRT/TensorRT6-cuda9.0-cudnn7.tar.gz
--no-check-certificate
wget
-q
https://paddle-ci.gz.bcebos.com/TRT/TensorRT6-cuda9.0-cudnn7.tar.gz
--no-check-certificate
tar
-zxf
TensorRT6-cuda9.0-cudnn7.tar.gz
-C
/usr/local
tar
-zxf
TensorRT6-cuda9.0-cudnn7.tar.gz
-C
/usr/local
cp
-rf
/usr/local/TensorRT6-cuda9.0-cudnn7/include/
*
/usr/include/
&&
cp
-rf
/usr/local/TensorRT6-cuda9.0-cudnn7/lib/
*
/usr/lib/
cp
-rf
/usr/local/TensorRT6-cuda9.0-cudnn7/include/
*
/usr/include/
&&
cp
-rf
/usr/local/TensorRT6-cuda9.0-cudnn7/lib/
*
/usr/lib/
rm
TensorRT6-cuda9.0-cudnn7.tar.gz
fi
fi
tools/dockerfile/centos6_manylinux.sh
浏览文件 @
6f2bb20e
...
@@ -30,20 +30,19 @@ function make_cuda10cudnn7() {
...
@@ -30,20 +30,19 @@ function make_cuda10cudnn7() {
function
make_cuda101cudnn7
()
{
function
make_cuda101cudnn7
()
{
sed
's/<baseimg>/10.1-cudnn7-devel-centos6/g'
Dockerfile.centos
>
Dockerfile.tmp
sed
's/<baseimg>/10.1-cudnn7-devel-centos6/g'
Dockerfile.centos
>
Dockerfile.tmp
sed
-i
"s#
COPY build_scripts /build_scripts#COPY build_scripts /build_scripts
\n
RUN bash build_scripts/install_gcc.sh gcc82
\n
ENV PATH=/usr/local/gcc-8.2/bin:
\$
PATH#g"
Dockerfile.tmp
sed
-i
"s#
RUN bash build_scripts/build.sh#RUN bash build_scripts/install_gcc.sh gcc82
\n
RUN mv /usr/bin/cc /usr/bin/cc.bak
\&\&
ln -s /usr/local/gcc-8.2/bin/gcc /usr/bin/cc
\n
ENV PATH=/usr/local/gcc-8.2/bin:
\$
PATH
\n
RUN bash build_scripts/build.sh#g"
Dockerfile.tmp
}
}
function
make_cuda102cudnn7
()
{
function
make_cuda102cudnn7
()
{
sed
's/<baseimg>/10.2-cudnn7-devel-centos6/g'
Dockerfile.centos
>
Dockerfile.tmp
sed
's/<baseimg>/10.2-cudnn7-devel-centos6/g'
Dockerfile.centos
>
Dockerfile.tmp
sed
-i
"s#
COPY build_scripts /build_scripts#COPY build_scripts /build_scripts
\n
RUN bash build_scripts/install_gcc.sh gcc82
\n
ENV PATH=/usr/local/gcc-8.2/bin:
\$
PATH
#g"
Dockerfile.tmp
sed
-i
"s#
RUN bash build_scripts/build.sh#RUN bash build_scripts/install_gcc.sh gcc82
\n
RUN mv /usr/bin/cc /usr/bin/cc.bak
\&\&
ln -s /usr/local/gcc-8.2/bin/gcc /usr/bin/cc
\n
ENV PATH=/usr/local/gcc-8.2/bin:
\$
PATH
\n
RUN bash build_scripts/build.sh
#g"
Dockerfile.tmp
}
}
function
make_cuda11cudnn8
()
{
function
make_cuda11cudnn8
()
{
sed
's/<baseimg>/11.0-cudnn8-devel-centos7/g'
Dockerfile.centos
>
Dockerfile.tmp
sed
's/<baseimg>/11.0-cudnn8-devel-centos7/g'
Dockerfile.centos
>
Dockerfile.tmp
sed
-i
"s#
COPY build_scripts /build_scripts#COPY build_scripts /build_scripts
\n
RUN bash build_scripts/install_gcc.sh gcc82
\n
ENV PATH=/usr/local/gcc-8.2/bin:
\$
PATH
#g"
Dockerfile.tmp
sed
-i
"s#
RUN bash build_scripts/build.sh#RUN bash build_scripts/install_gcc.sh gcc82
\n
RUN mv /usr/bin/cc /usr/bin/cc.bak
\&\&
ln -s /usr/local/gcc-8.2/bin/gcc /usr/bin/cc
\n
ENV PATH=/usr/local/gcc-8.2/bin:
\$
PATH
\n
RUN bash build_scripts/build.sh
#g"
Dockerfile.tmp
}
}
function
main
()
{
function
main
()
{
local
CMD
=
$1
local
CMD
=
$1
case
$CMD
in
case
$CMD
in
...
...
tools/dockerfile/ci_dockerfile.sh
浏览文件 @
6f2bb20e
...
@@ -44,17 +44,9 @@ function make_centos_dockerfile(){
...
@@ -44,17 +44,9 @@ function make_centos_dockerfile(){
sed
-i
"
${
dockerfile_line
}
i RUN ln -s /usr/lib64/libz.so /usr/local/lib/libz.so
\\
sed
-i
"
${
dockerfile_line
}
i RUN ln -s /usr/lib64/libz.so /usr/local/lib/libz.so
\\
RUN ln -s /usr/local/lib/libnccl.so /usr/local/cuda/lib64/
\\
RUN ln -s /usr/local/lib/libnccl.so /usr/local/cuda/lib64/
\\
RUN rm -rf /usr/include/NvInfer*"
${
dockerfile_name
}
RUN rm -rf /usr/include/NvInfer*"
${
dockerfile_name
}
sed
-i
$"
${
dockerfile_line
}
i RUN wget --no-check-certificate -q https://paddle-edl.bj.bcebos.com/hadoop-2.7.7.tar.gz
\\
sed
-i
$"
${
dockerfile_line
}
i RUN wget --no-check-certificate -q
https://paddle-edl.bj.bcebos.com/hadoop-2.7.7.tar.gz
\\
RUN tar -xzf hadoop-2.7.7.tar.gz && mv hadoop-2.7.7 /usr/local/"
${
dockerfile_name
}
RUN tar -xzf hadoop-2.7.7.tar.gz && mv hadoop-2.7.7 /usr/local/"
${
dockerfile_name
}
sed
-i
"s#<install_gcc>#WORKDIR /usr/bin
\\
sed
-i
"s#RUN bash build_scripts/build.sh#RUN bash build_scripts/install_gcc.sh gcc82
\n
RUN mv /usr/bin/cc /usr/bin/cc.bak
\&\&
ln -s /usr/local/gcc-8.2/bin/gcc /usr/bin/cc
\n
ENV PATH=/usr/local/gcc-8.2/bin:
\$
PATH
\n
RUN bash build_scripts/build.sh#g"
${
dockerfile_name
}
COPY tools/dockerfile/build_scripts /build_scripts
\\
RUN bash /build_scripts/install_gcc.sh gcc82
\&\&
rm -rf /build_scripts
\\
RUN cp gcc gcc.bak
\&\&
cp g++ g++.bak
\&\&
rm gcc
\&\&
rm g++
\\
RUN ln -s /usr/local/gcc-8.2/bin/gcc /usr/local/bin/gcc
\\
RUN ln -s /usr/local/gcc-8.2/bin/g++ /usr/local/bin/g++
\\
RUN ln -s /usr/local/gcc-8.2/bin/gcc /usr/bin/gcc
\\
RUN ln -s /usr/local/gcc-8.2/bin/g++ /usr/bin/g++
\\
ENV PATH=/usr/local/gcc-8.2/bin:
\$
PATH #g"
${
dockerfile_name
}
}
}
...
...
tools/dockerfile/ubuntu18_dev.sh
0 → 100755
浏览文件 @
6f2bb20e
#!/bin/bash
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
docker_name
=
$1
function
ref_whl
(){
if
[[
${
WITH_GPU
}
==
"ON"
]]
;
then
ref_gpu
=
gpu-cuda
${
ref_CUDA_MAJOR
}
-cudnn
${
CUDNN_MAJOR
}
install_gpu
=
"_gpu"
else
ref_gpu
=
"cpu"
install_gpu
=
""
fi
if
[[
${
WITH_MKL
}
==
"ON"
]]
;
then
ref_mkl
=
mkl
else
ref_mkl
=
openblas
fi
if
[[
${
gcc_version
}
==
"8.2.0"
]]
;
then
ref_gcc
=
_gcc8.2
fi
if
[[
${
ref_CUDA_MAJOR
}
==
"10"
]]
;
then
ref_version
=
.post100
elif
[[
${
ref_CUDA_MAJOR
}
==
"10.1"
]]
;
then
ref_version
=
.post101
elif
[[
${
ref_CUDA_MAJOR
}
==
"10.2"
]]
;
then
ref_version
=
""
elif
[[
${
ref_CUDA_MAJOR
}
==
"9"
]]
;
then
ref_version
=
.post90
fi
ref_web
=
"https://paddle-wheel.bj.bcebos.com/
${
PADDLE_BRANCH
}
-
${
ref_gpu
}
-
${
ref_mkl
}${
ref_gcc
}
"
if
[[
${
PADDLE_VERSION
}
==
"0.0.0"
&&
${
WITH_GPU
}
==
"ON"
]]
;
then
ref_paddle_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}
-cp27-cp27mu-linux_x86_64
.whl
ref_paddle3_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}
-cp35-cp35m-linux_x86_64
.whl
ref_paddle36_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}
-cp36-cp36m-linux_x86_64
.whl
ref_paddle37_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}
-cp37-cp37m-linux_x86_64
.whl
ref_paddle38_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}
-cp38-cp38-linux_x86_64
.whl
else
ref_paddle_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}
-cp27-cp27mu-linux_x86_64
.whl
ref_paddle3_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}
-cp35-cp35m-linux_x86_64
.whl
ref_paddle36_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}
-cp36-cp36m-linux_x86_64
.whl
ref_paddle37_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}
-cp37-cp37m-linux_x86_64
.whl
ref_paddle38_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}
-cp38-cp38-linux_x86_64
.whl
fi
if
[[
${
PADDLE_VERSION
}
!=
"0.0.0"
&&
${
WITH_GPU
}
==
"ON"
]]
;
then
ref_paddle_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}${
ref_version
}
-cp27-cp27mu-linux_x86_64
.whl
ref_paddle3_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}${
ref_version
}
-cp35-cp35m-linux_x86_64
.whl
ref_paddle36_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}${
ref_version
}
-cp36-cp36m-linux_x86_64
.whl
ref_paddle37_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}${
ref_version
}
-cp37-cp37m-linux_x86_64
.whl
ref_paddle38_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}${
ref_version
}
-cp38-cp38-linux_x86_64
.whl
else
ref_paddle_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}
-cp27-cp27mu-linux_x86_64
.whl
ref_paddle3_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}
-cp35-cp35m-linux_x86_64
.whl
ref_paddle36_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}
-cp36-cp36m-linux_x86_64
.whl
ref_paddle37_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}
-cp37-cp37m-linux_x86_64
.whl
ref_paddle38_whl
=
paddlepaddle
${
install_gpu
}
-
${
PADDLE_VERSION
}
-cp38-cp38-linux_x86_64
.whl
fi
}
function
install_whl
(){
dockerfile_line
=
`
wc
-l
Dockerfile.tmp|awk
'{print $1}'
`
sed
-i
"
${
dockerfile_line
}
i RUN wget -q
${
ref_web
}
/
${
ref_paddle_whl
}
&& pip install
${
ref_paddle_whl
}
&& rm -f
${
ref_paddle_whl
}
"
Dockerfile.tmp
sed
-i
"
${
dockerfile_line
}
i RUN wget -q
${
ref_web
}
/
${
ref_paddle3_whl
}
&& pip3.5 install
${
ref_paddle3_whl
}
&& rm -f
${
ref_paddle3_whl
}
"
Dockerfile.tmp
sed
-i
"
${
dockerfile_line
}
i RUN wget -q
${
ref_web
}
/
${
ref_paddle36_whl
}
&& pip3.6 install
${
ref_paddle36_whl
}
&& rm -f
${
ref_paddle36_whl
}
"
Dockerfile.tmp
sed
-i
"
${
dockerfile_line
}
i RUN wget -q
${
ref_web
}
/
${
ref_paddle37_whl
}
&& pip3.7 install
${
ref_paddle37_whl
}
&& rm -f
${
ref_paddle37_whl
}
"
Dockerfile.tmp
sed
-i
"
${
dockerfile_line
}
i RUN wget -q
${
ref_web
}
/
${
ref_paddle38_whl
}
&& pip3.8 install
${
ref_paddle38_whl
}
&& rm -f
${
ref_paddle38_whl
}
"
Dockerfile.tmp
}
function
install_gcc
(){
if
[
"
${
gcc_version
}
"
==
"8.2.0"
]
;
then
sed
-i
's#<install_gcc>#WORKDIR /usr/bin \
COPY tools/dockerfile/build_scripts /build_scripts \
RUN bash /build_scripts/install_trt.sh \
RUN bash /build_scripts/install_gcc.sh gcc82 \&\& rm -rf /build_scripts \
RUN cp gcc gcc.bak \&\& cp g++ g++.bak \&\& rm gcc \&\& rm g++ \
RUN ln -s /usr/local/gcc-8.2/bin/gcc /usr/local/bin/gcc \
RUN ln -s /usr/local/gcc-8.2/bin/g++ /usr/local/bin/g++ \
RUN ln -s /usr/local/gcc-8.2/bin/gcc /usr/bin/gcc \
RUN ln -s /usr/local/gcc-8.2/bin/g++ /usr/bin/g++ \
ENV PATH=/usr/local/gcc-8.2/bin:$PATH #g'
Dockerfile.tmp
else
sed
-i
's#<install_gcc>#RUN apt-get update \
WORKDIR /usr/bin \
RUN apt install -y gcc-4.8 g++-4.8 \&\& cp gcc gcc.bak \&\& cp g++ g++.bak \&\& rm gcc \&\& rm g++ \&\& ln -s gcc-4.8 gcc \&\& ln -s g++-4.8 g++ #g'
Dockerfile.tmp
fi
}
function
make_dockerfile
(){
sed
"s/<baseimg>/
${
docker_name
}
/g"
tools/dockerfile/Dockerfile.ubuntu18
>
Dockerfile.tmp
}
function
main
(){
make_dockerfile
install_gcc
ref_whl
install_whl
}
main
$@
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录