From c5f321325ea17ded84c4bd0d0123a050f29cedb1 Mon Sep 17 00:00:00 2001 From: daminglu Date: Mon, 23 Apr 2018 23:58:33 -0700 Subject: [PATCH] Translate VisualDL front-end & back-end info (#421) --- docs/develop/how_to_dev_backend_cn.md | 103 +++++++++++++------------ docs/develop/how_to_dev_frontend_cn.md | 68 ++++++++-------- 2 files changed, 87 insertions(+), 84 deletions(-) diff --git a/docs/develop/how_to_dev_backend_cn.md b/docs/develop/how_to_dev_backend_cn.md index bcdeb7e6..a9803351 100644 --- a/docs/develop/how_to_dev_backend_cn.md +++ b/docs/develop/how_to_dev_backend_cn.md @@ -1,18 +1,18 @@ # 后台指南 -VisualDL has three components. -1. The Python/C++ SDK that logs the data during training. -1. The single page client side app that visualizes training data. -1. The server powered by Flask, reads the log data and delivers it to the client side app for displaying graphs (scalars/histograms) and embeddings. +VisualDL有三个功能模块. +1. 在训练过程中用来记录日志数据的 Python/C++ SDK。 +1. 用来可视化训练数据的客户端单页面应用。 +1. 后端 Flask 服务器,用来读取日志数据并提供给前端应用,以便前端可以展示各类图标,如柱状图,嵌入图等等. -This doc will go over Backend / SDK structure with development guide. +这篇文档用来介绍 后端/SDK 的架构,以及指导用户自行开发。 -For front end architecture and development guide, looks [here](https://github.com/PaddlePaddle/VisualDL/blob/develop/docs/how_to_dev_frontend_en.md) +关于前端的架构和开发指南,请看 [这里](https://github.com/PaddlePaddle/VisualDL/blob/develop/docs/how_to_dev_frontend_cn.md) -## Code structure +## 代码结构 -All backend and sdk logic is under visualdl sub directory +所有后端和 SDK 的逻辑都在 visualdl 子文件夹里面 ``` ├── build @@ -27,20 +27,21 @@ All backend and sdk logic is under visualdl sub directory └── utils - Helper function and classes ``` -## Development +## 开发 -Any code changes in ```server``` folder, simply run +任何在 ```server``` 文件夹里代码的改动,都可以通过运行以下命令 ``` python visualdl/server/visualDL --logdir={LOG_DIR} --port=8080 ``` -to restart flask server +来重启 Flask 服务器 -Notice if you install visualDL from pip install wheel, make sure resetting PYTHONPATH to get the command work. +需要注意的是如果你通过 `pip install wheel` 来安装 visualDL, 请确认重置 PYTHONPATH,以便是的上述命令可以运行。 -Any code changes in ```logic```, ```python```, ```storage```, ```utils``` is modified, need to rebuild core.so or sdk to get effect. +任何在 ```logic```, ```python```, ```storage```, ```utils``` 中的代码修改,都需要重新 build core.so 或 SDK 来确保生效。 + +要么运行 VisualDL/build.sh 脚本来重新安装所有组件,要么安装特定的目标组件如下: -Either run VisualDL/build.sh that builds everything or a target like this ``` mkdir {VisualDL_ROOT}/build cd build @@ -49,12 +50,12 @@ make {TARGET MODULE} ``` -## Tests +## 测试 -Modification in SDK API or any major function should be adding test cases. +任何关于 SDK API 或其他核心组件的修改都必须提供相应的测试案例。 -In {VisualDL_ROOT}/CMakeList.txt, we have a list of test files as target +在 {VisualDL_ROOT}/CMakeList.txt 中,我们有一个测试文件清单 ``` add_executable(vl_test @@ -70,7 +71,7 @@ add_executable(vl_test ) ``` -Add new test cases in corresponding files where code is modified or create new test files. You can make vl_test target and just run vl_test executable for all test cases. +在相应被修改的代码文件处添加测试用例或者新的测试文件。你可以制作 vl_test 目标或者运行 vl_test 可执行文件来运行所有测试案例。 ``` mkdir {VisualDL_ROOT}/build @@ -80,45 +81,45 @@ make vl_test ./vl_test ``` -## Backend / SDK architecture +## 后端 / SDK 架构 -### Server +### 服务器 -- Implement a server using a lightweight framework Flask, provides two services: - - Host a main application with support of a front end web app - - Provide a series of HTTP end points, using JSON for front end data communication -- ```visualDL``` : main app entry point - - defines server arguments and config - - provides API and router for front end -- ```lib.py``` : delegate wrapper for function calls - - read and cache log data - - convert data to JSON - - retry mechanism -- ```graph.py``` and ```onnx``` - - graph.py defines the logic to read and parse onnx model file - - create edges and nodes as JSON format for front end to read - - onnx folder contains onnx protobuf interface that ports from onnx repo +- 采用轻量级 Flask 框架来搭建服务器并提供以下两种服务 + - 搭建主程序来支持前端应用 + - 提供一系列 HTTP 访问点,通过返回的 JSON 来和前端沟通 +- ```visualDL``` : 主程序入口 + - 定义了服务器参数 + - 给前端提供 API 和 路由匹配 +- ```lib.py``` : 调用库函数的入口 + - 读取并缓存日志数据 + - 把数据变成 JSON + - 重试机制 +- ```graph.py``` 和 ```onnx``` + - graph.py 定义了读取和解析 ONNX 模型文件的逻辑 + - 给前端提供具有结点和边信息的图模型JSON + - ONNX 文件夹包含从 ONNX 代码库移植来的 ONNX Protobuf 接口 -### Logic & SDK +### 逻辑和 SDK #### Python SDK -- ```pybind.cc``` : create python interface from C++ layer -- ```storage.py``` : ```LogReader``` and ```LogWriter``` APIs of Python SDK that maps to pybind +- ```pybind.cc``` : 为 C++ 创建 Python 接口 +- ```storage.py``` : Python SDK 的 ```LogReader``` 和 ```LogWriter``` API,对应于 pybind #### C++ SDK - ```sdk.h``` - - defines ```LogReader``` and ```LogWriter``` APIs of C++ SDK - - defines feature components such as ```Scalar```, ```Histogram```, ```Image``` and their corresponding component```Reader``` -- ```sdk.cc``` : implements ```Reader``` to use ```Storage``` layer for reading and writing - -#### Information Container -- Currently it just handles logic of sync cycle (when to write) - -#### Storage -- Defines log data format as following: - - ```Storage``` : main set of data with different modes such as "train" or "test", contain multiple ```Tablet``` - - ```Tablet``` : group a single type of data with ```tag```. One tablet will only represent one data type such as scalar, histogram, image and contains multiple ```Record```. For example one tablet represents a line in scalar chart. - - ```Record``` : represent any data structure for any component -- Handles the actual logic to serialize to disk and deserialize from disk -- ```storage.proto```: protobuf file that defines the exact interface to be read/write + - 定义了 ```LogReader``` 和 ```LogWriter``` 的 C++ SDK 的API + - 定义了诸如```Scalar```, ```Histogram```, ```Image``` 的功能模块以及它们相应的 ```Reader``` +- ```sdk.cc``` : 实现了 ```Reader``` 以便使用 ```Storage``` 层来完成读和写 + +#### 信息容器 +- 目前我们仅仅处理写操作时的同步逻辑 + +#### 存储 +- 定义日志数据的格式如下: + - ```Storage``` : 主数据存储地, 包含不同模式,比如 "train" 或 "test", 包含多个 ```Tablet``` + - ```Tablet``` : 把同一个数据类型用 ```tag``` 打包. 一个面板只代表一个数据类型,比如 ‘数值’, ‘柱状图’, ‘图像’。 可以包含多个 ```Record```。 比如,一个面板可以表达在 ‘数值’ 图像中的一条线。 + - ```Record``` : 最小单位,代表任何功能模块中的任何数据结构。 +- 处理和磁盘之间的 序列化/去序列化 传输 +- ```storage.proto```: 用来定义确切 读/写 接口的 protobuf 文件。 diff --git a/docs/develop/how_to_dev_frontend_cn.md b/docs/develop/how_to_dev_frontend_cn.md index c2fd3770..c1616662 100644 --- a/docs/develop/how_to_dev_frontend_cn.md +++ b/docs/develop/how_to_dev_frontend_cn.md @@ -1,16 +1,16 @@ # 前端指南 -VisualDL has three components. -1. The Python/C++ SDK that logs the data during training. -1. The single page client side app that visualized training data. -1. The server (powered on Flask) that reads the logs data and delivers it to the client side app for displaying graphs (scalars/histograms) and embeddings. +VisualDL有三个功能模块. +1. 在训练过程中用来记录日志数据的 Python/C++ SDK。 +1. 用来可视化训练数据的客户端单页面应用。 +1. 后端 Flask 服务器,用来读取日志数据并提供给前端应用,以便前端可以展示各类图标,如柱状图,嵌入图等等. -This article will go over the basic web-app architecture and development guide. +这篇文档用来介绍前端的架构,以及指导用户自行开发。 -## TL;DR -If you are not interested in the details of mumbo jumbo and want to start development as soon as possible, please run the following commands. +## 长话短说 +如果您不关心细节,而是想马上开始开发,请运行以下命令。 -To clone the repo and to prepare the environment +克隆代码库以及准备开发环境 ```bash git clone git@github.com:PaddlePaddle/VisualDL.git cd VisualDL @@ -19,60 +19,62 @@ pip install -r requirements.txt # Set up the node dependencies and the VisualDL SDK ./scripts/setup_dev_env.sh ``` -To run the dev server and start coding +运行开发服务器,开始码代码喽~ ```bash ./scripts/start_dev_server.sh --logdir= ``` -The script will run a dev server at `http://0.0.0.0:8040`. Open a browser and navigate to `http://0.0.0.0:8040/` to see it +这个脚本会在 `http://0.0.0.0:8040` 启动一个开发服务器。打开浏览器,来到 `http://0.0.0.0:8040/`,您就可以发现开发版的 VisualDL 啦~ -## Web App Architecture -The VisualDL Web app uses multiple frameworks to help manage the project. They are +## 前端网络应用架构 -1. webpack: To manage all assets -1. npm: To manage dependencies -1. Vue: Javascript Component framework -1. ECharts: To pilot charts +VisualDL 前端网络应用使用多个框架来管理项目。这些框架包括: + +1. webpack: 管理所有组件 +1. npm: 管理依赖性 +1. Vue: 流行的Javascript组件框架 +1. ECharts: 专业的数据可视化库 ## Webpack -webpack is a module bundler. Its main purpose is to bundle JavaScript files for usage in a browser. +webpack 是一个模块打包框架. 它主要的工作是把前端JavaScript文件带包在一起,提供给浏览器。 -For a long time, web developers need to worry about backward compatibility or how to support different versions of browsers. -webpack can help to transpile the Javascript files to the version where a browser can take. The developers don't need to implement version specific code anymore. -It can also help to minify and uglify Javascript files. In a nutshell, webpack helps to manage all of your assets (.js, .css, .gif, .sass, etc) so you don't have to. +很长一段时间,网络开发者不得不关心如何向后兼容以及如何支持不同的浏览器。Webpack可以把JavaScript代码同步编译到不同的版本,以便适应不同的浏览器。 +网络开发者不必再担心为了特定的浏览器而订制特定的代码版本。 +它也可以 '最小化' 和 '丑化' JavaScript文件。一句话,webpack可以帮助管理你的所有资料(.js, .css, .gif, .sass, etc). -To learn more about [webpack](https://webpack.js.org/). +更多Webpack内容请参考[这里](https://webpack.js.org/). ## npm package -npm is the package manager for JavaScript and is a great tool to manage multiple dependencies. To confirm that you have npm installed. Please run this command +npm 是一个JavaScript的代码包管理器。而且是一个很优秀的管理多种依赖性的工具。请运行以下命令来确保您已经安装了 npm + ``` bash npm -v ``` -If your environment does not have npm already, please [download npm](https://www.npmjs.com/get-npm). +如果您的环境没有npm,请在 [这里](https://www.npmjs.com/get-npm) 下载. -The basic packaging information is stored in `package.json`. It specifies build scripts and dependencies for each environment. -npm reads `package.json` to build different package according to the packaging script. -If you are not familiar with npm, please refer to the [npm documentation](https://docs.npmjs.com/) for more detail. +基本的包裹信息在 `package.json`中。它用来定义构造脚本的依赖关系。 +npm 读取 `package.json` 来根据打包脚本构造不同的包裹。 +如果您对 npm 还不熟,请看 [这里](https://docs.npmjs.com/)。 -Install the necessary dependencies by running +运行如下命令来安装所需的依赖库 ```bash # Navigate to the web-app folder cd VisualDL/frontend npm install ``` -This command will go through `package.json` and install the dependencies in the local node_modules folder. +这个命令会逐行检查`package.json`并且安装相应的依赖库到本地的 `node_modules` 文件夹里 ## Vue -Vue is a JavaScript component framework that helps the developer to implement web component in MVVM architecture pattern. +Vue 是一个很潮的 JavaScript 框架。可以帮助开发者用 MVM 架构模式 开发网络组件。 -Vue allows you to define a self-contained view model in a .vue file and attach view model objects to DOM objects. +Vue 允许开发者自己定义视图模型并保存在 .vue 文件里。然后把视图模型绑定到 DOM 对象上。 -To learn more about [Vue](https://vuejs.org/) +更多内容参见 [这里](https://vuejs.org/) ## ECharts -We use ECharts javascript library to render our charts and graphs. ECharts is a leading open source charting library that supports numerous data visualization. +我们使用一个国内流行的 ECharts 来显示图表和图像。 ECharts 是领先的开源前端数据可视化库。它支持多种多样的数据可视化需求。 -To learn more about ECharts framework, please visit [ECharts](https://ecomfe.github.io/echarts-doc/public/en/index.html). +更多关于 [ECharts](https://ecomfe.github.io/echarts-doc/public/en/index.html)的信息。 -- GitLab