未验证 提交 5f72d835 编写于 作者: Q Qiao Longfei 提交者: GitHub

Support manylinux build (#165)

* support manylinux build
上级 e4bade74
...@@ -5,16 +5,16 @@ ...@@ -5,16 +5,16 @@
# VisualDL (Visualize the Deep Learning) # VisualDL (Visualize the Deep Learning)
## Introduction ## Introduction
VisualDL is a deep learning visualization tool that can help design deep learning jobs. VisualDL is a deep learning visualization tool that can help design deep learning jobs.
It includes features such as scalar, parameter distribution, model structure and image visualization. It includes features such as scalar, parameter distribution, model structure and image visualization.
Currently it is being developed at a high pace. Currently it is being developed at a high pace.
New features will be continuously added. New features will be continuously added.
At present, most DNN frameworks use Python as their primary language. VisualDL supports Python by nature. At present, most DNN frameworks use Python as their primary language. VisualDL supports Python by nature.
Users can get plentiful visualization results by simply add a few lines of Python code into their model before training. Users can get plentiful visualization results by simply add a few lines of Python code into their model before training.
Besides Python SDK, VisualDL was writen in C++ on the low level. It also provides C++ SDK that Besides Python SDK, VisualDL was writen in C++ on the low level. It also provides C++ SDK that
can be integrated into other platforms. can be integrated into other platforms.
...@@ -27,8 +27,8 @@ VisualDL now provides 4 components: ...@@ -27,8 +27,8 @@ VisualDL now provides 4 components:
- histogram - histogram
### Graph ### Graph
Graph is compatible with ONNX(Open Neural Network Exchange)[https://github.com/onnx/onnx], Graph is compatible with ONNX(Open Neural Network Exchange)[https://github.com/onnx/onnx],
Cooperated with Python SDK, VisualDL can be compatible with most major DNN frameworks, including Cooperated with Python SDK, VisualDL can be compatible with most major DNN frameworks, including
PaddlePaddle, PyTorch and MXNet. PaddlePaddle, PyTorch and MXNet.
<p align="center"> <p align="center">
...@@ -75,7 +75,7 @@ logger = LogWriter(dir, sync_cycle=10) ...@@ -75,7 +75,7 @@ logger = LogWriter(dir, sync_cycle=10)
with logger.mode("train"): with logger.mode("train"):
# create a scalar component called 'scalars/scalar0' # create a scalar component called 'scalars/scalar0'
scalar0 = logger.scalar("scalars/scalar0") scalar0 = logger.scalar("scalars/scalar0")
# add some records during DL model running, lets start from another block. # add some records during DL model running, lets start from another block.
with logger.mode("train"): with logger.mode("train"):
...@@ -98,12 +98,12 @@ namepsace cp = visualdl::components; ...@@ -98,12 +98,12 @@ namepsace cp = visualdl::components;
int main() { int main() {
const std::string dir = "./tmp"; const std::string dir = "./tmp";
vs::LogWriter logger(dir, 10); vs::LogWriter logger(dir, 10);
logger.SetMode("train"); logger.SetMode("train");
auto tablet = logger.NewTablet("scalars/scalar0"); auto tablet = logger.NewTablet("scalars/scalar0");
cp::Scalar<float> scalar0(tablet); cp::Scalar<float> scalar0(tablet);
for (int step = 0; step < 1000; step++) { for (int step = 0; step < 1000; step++) {
float v = (float)std::rand() / RAND_MAX; float v = (float)std::rand() / RAND_MAX;
scalar0.AddRecord(step, v); scalar0.AddRecord(step, v);
...@@ -131,7 +131,7 @@ Board also supports the parameters below for remote access: ...@@ -131,7 +131,7 @@ Board also supports the parameters below for remote access:
### How to install ### How to install
``` ```
python setup.py bdist_wheel python setup.py bdist_wheel
pip install --upgrade dist/visualdl-0.0.1-py2-none-any.whl pip install --upgrade dist/visualdl-*.whl
``` ```
### Run a demo from scratch ### Run a demo from scratch
...@@ -144,6 +144,6 @@ that will start a server locally. ...@@ -144,6 +144,6 @@ that will start a server locally.
### Contribute ### Contribute
VisualDL is initially created by [PaddlePaddle](http://www.paddlepaddle.org/) and VisualDL is initially created by [PaddlePaddle](http://www.paddlepaddle.org/) and
[ECharts](http://echarts.baidu.com/). [ECharts](http://echarts.baidu.com/).
We welcome everyone to use, comment and contribute to Visual DL :) We welcome everyone to use, comment and contribute to Visual DL :)
...@@ -23,7 +23,7 @@ build_frontend_fake() { ...@@ -23,7 +23,7 @@ build_frontend_fake() {
build_backend() { build_backend() {
cd $BUILD_DIR cd $BUILD_DIR
cmake .. cmake .. ${PYTHON_FLAGS}
make -j2 make -j2
} }
......
...@@ -24,7 +24,7 @@ VisualDL的安装很简单。请按照VisualDL的[官方网站](https://github.c ...@@ -24,7 +24,7 @@ VisualDL的安装很简单。请按照VisualDL的[官方网站](https://github.c
``` ```
python setup.py bdist_wheel python setup.py bdist_wheel
pip install --upgrade dist/visualdl-0.0.1-py2-none-any.whl pip install --upgrade dist/visualdl-*.whl
``` ```
## 开始编写训练MNIST的程序 ## 开始编写训练MNIST的程序
......
...@@ -6,7 +6,7 @@ from distutils.spawn import find_executable ...@@ -6,7 +6,7 @@ from distutils.spawn import find_executable
from distutils import sysconfig, dep_util, log from distutils import sysconfig, dep_util, log
import setuptools.command.build_py import setuptools.command.build_py
import setuptools import setuptools
from setuptools import setup, find_packages from setuptools import setup, find_packages, Distribution, Extension
import subprocess import subprocess
TOP_DIR = os.path.realpath(os.path.dirname(__file__)) TOP_DIR = os.path.realpath(os.path.dirname(__file__))
...@@ -79,22 +79,25 @@ datas = [] ...@@ -79,22 +79,25 @@ datas = []
data_root = os.path.join(TOP_DIR, 'visualdl/server/dist') data_root = os.path.join(TOP_DIR, 'visualdl/server/dist')
for root, dirs, files in os.walk(data_root): for root, dirs, files in os.walk(data_root):
for filename in files: for filename in files:
path = 'dist/'+os.path.join(root, filename)[len(data_root)+1:] path = 'dist/' + os.path.join(root, filename)[len(data_root) + 1:]
datas.append(path) datas.append(path)
print datas print datas
setup( setup(
name="visualdl", name="visualdl",
version=VERSION_NUMBER, version=VERSION_NUMBER,
author="PaddlePaddle and Echarts team.", author="PaddlePaddle and Echarts team",
description="Visualize Deep Learning.", description="Visualize Deep Learning",
license=LICENSE, license=LICENSE,
keywords="visualization deeplearning", keywords="visualization deeplearning",
long_description=read('README.md'), long_description=read('README.md'),
install_requires=install_requires, install_requires=install_requires,
package_data={'visualdl.server': datas, package_data={
'visualdl':['core.so'], 'visualdl.server': datas,
'visualdl.python':['core.so', 'dog.jpg']}, 'visualdl': ['core.so'],
'visualdl.python': ['core.so', 'dog.jpg']
},
packages=packages, packages=packages,
ext_modules=[Extension('_foo', ['stub.cc'])],
scripts=['visualdl/server/visualDL', 'demo/vdl_scratch.py'], scripts=['visualdl/server/visualDL', 'demo/vdl_scratch.py'],
cmdclass=cmdclass) cmdclass=cmdclass)
...@@ -6,12 +6,13 @@ readonly TOP_DIR=$(pwd) ...@@ -6,12 +6,13 @@ readonly TOP_DIR=$(pwd)
readonly core_path=$TOP_DIR/build/visualdl/logic readonly core_path=$TOP_DIR/build/visualdl/logic
readonly python_path=$TOP_DIR/visualdl/python readonly python_path=$TOP_DIR/visualdl/python
readonly max_file_size=1000000 # 1MB readonly max_file_size=1000000 # 1MB
readonly version_number=`cat VERSION_NUMBER`
sudo="sudo" sudo="sudo"
if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then sudo=""; fi if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then sudo=""; fi
if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then
curl -O http://python-distribute.org/distribute_setup.py curl -O http://python-distribute.org/distribute_setup.py
python distribute_setup.py python distribute_setup.py
curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py
...@@ -41,7 +42,7 @@ package() { ...@@ -41,7 +42,7 @@ package() {
cd $TOP_DIR cd $TOP_DIR
python setup.py bdist_wheel python setup.py bdist_wheel
$sudo pip install dist/visualdl-0.0.1-py2-none-any.whl $sudo pip install dist/visualdl-${version_number}-*.whl
} }
backend_test() { backend_test() {
......
...@@ -18,12 +18,19 @@ add_dependencies(im storage_proto) ...@@ -18,12 +18,19 @@ add_dependencies(im storage_proto)
add_dependencies(sdk entry storage storage_proto eigen3) add_dependencies(sdk entry storage storage_proto eigen3)
## pybind ## pybind
set(OPTIONAL_LINK_FLAGS)
if(NOT APPLE)
set(OPTIONAL_LINK_FLAGS "rt")
endif()
add_library(core SHARED ${PROJECT_SOURCE_DIR}/visualdl/logic/pybind.cc) add_library(core SHARED ${PROJECT_SOURCE_DIR}/visualdl/logic/pybind.cc)
if (NOT ON_RELEASE) if (NOT ON_RELEASE)
add_dependencies(core pybind python im entry tablet storage sdk protobuf glog eigen3) add_dependencies(core pybind python im entry tablet storage sdk protobuf glog eigen3)
target_link_libraries(core PRIVATE pybind entry python im tablet storage sdk protobuf glog) target_link_libraries(core PRIVATE pybind entry python im tablet storage sdk protobuf glog ${OPTIONAL_LINK_FLAGS})
else() else()
add_dependencies(core pybind python im entry tablet storage sdk protobuf eigen3) add_dependencies(core pybind python im entry tablet storage sdk protobuf eigen3)
target_link_libraries(core PRIVATE pybind entry python im tablet storage sdk protobuf) target_link_libraries(core PRIVATE pybind entry python im tablet storage sdk protobuf ${OPTIONAL_LINK_FLAGS})
endif() endif()
set_target_properties(core PROPERTIES PREFIX "" SUFFIX ".so") set_target_properties(core PROPERTIES PREFIX "" SUFFIX ".so")
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册