Created by: gangliao
- Add dependencies
- integrate into Paddle
- Fix bug
- clean travis ci
目前测试Mac, Centos, ubuntu已经全部通过。
- 文档clean工作 下一个PR
- docker clean工作 下一个PR
- third_party test code 下一个PR
由于PR比较复杂,我这里梳理一下,帮助review
已解决:
-
glog, gflags, protobuf, gtest, python, numpy, wheel, pip, swig, python-protobuf, zlib, setuptools, six, cython 自动下载, 编译,安装 。python部分最复杂。
-
warpctc自动下载,编译,安装。原版warpctc存在一些bug,比如没有openmp的开关, 没有gpu的开关,make install写的有问题。已经给他们提交PR,并且merge了。https://github.com/baidu-research/warp-ctc/commits/master 后期会继续给他们贡献一个static库版本。
通过观察Travis CI编译和测试时间,发现第三方依赖全部下载,编译,安装所需的时间与之前没有明显区别。
- 之前的Travis CI: Linux 28 min 13 sec
- 目前的Travis CI: Linux 25 min 35 sec
未解决:
目前唯一没有解决的依赖是代数库 openblas. 原因主要是目前paddle有几个函数依赖于lapack,openblas部分代码需要默认使用gfortran编译,在linux系统上面,除了链接libopenblas.a, 最终还需要链接libgfortran.so, 这部分我再考证一下。
本PR带来的好处是:
- 无需预装第三方依赖,一键式自动完成Paddle编译安装。cmake .. && make就行了。
- 所有第三方依赖全部以静态库的方式包进来。
- 同时解决了python解释器和python库不一致的问题
长远好处:
-
Paddle的第三方依赖全部来自CMake自动下载,编译,安装,与系统本身相对隔离,对今后发布二进制有好处。
-
CMake 第三方依赖库的自动下载,编译,安装,为不同操作系统的移植工作,提供了极大的便利。
Travis ci linux最后安装的时候有问题, 可能是pip或者python版本的问题,比较奇怪的是,我fork的版本,一样的代码,travis ci确实没有这个问题的。https://travis-ci.org/gangliao/Paddle @reyoung
First time run paddle, need to install some python dependencies.
[33mThe directory '/home/travis/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.[0m
[33mThe directory '/home/travis/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.[0m
[31mpy_paddle-0.9.0a0-cp27-cp27m-linux_x86_64.whl is not a supported wheel on this platform.[0m
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
pip install wheels failed.
Please use 'sudo paddle' at the first time you use PaddlePaddle
PaddlePaddle will install some python dependencies automatically.