Skip to content

V
VisualDL

PaddlePaddle / VisualDL
大约 1 年 前同步成功

通知 86
Star 4655
Fork 641
V
VisualDL

体验新版 GitCode，发现更多精彩内容 >>

未验证 提交 fe2a1063 编写于 作者: 走神的阿圆's avatar 走神的阿圆 提交者： GitHub
浏览文件
操作

Update api docs to add more detailed descriptions.

上级 de2ec2ea
展开全部 隐藏空白更改
内联 并排
Showing with 1813 addition and 1790 deletion
README.md
浏览文件 @ fe2a1063
......@@ -99,11 +99,9 @@ The Python SDK is provided at the back end of VisualDL, and a logger can be cust
```python
class LogWriter(logdir=None,
comment='',
max_queue=10,
flush_secs=120,
filename_suffix='',
write_to_disk=True,
**kwargs)
```
......@@ -112,14 +110,11 @@ class LogWriter(logdir=None,
| parameters | type | meaning |
| --------------- | ------- | ------------------------------------------------------------ |
| logdir | string | The path location of log file. VisualDL will create a log file under this path to record information generated by the training process. If not specified, the path will be `runs/${CURRENT_TIME}`as default. |
| comment | string | Add a suffix to the log folder name, which is invalid if logdir is already specified. |
| max_queue | int | The maximum capacity of the data generated before recording in a log file. If the capacity is reached, the data are immediately written into the log file. |
| flush_secs | int | The maximum cache time of the data generated before recording in a log file, when this time is reached, the data are immediately written to the log file. |
| max_queue | int | The maximum capacity of the data generated before recording in a log file. Default value is 10. If the capacity is reached, the data are immediately written into the log file. |
| flush_secs | int | The maximum cache time of the data generated before recording in a log file. Default value is 120. When this time is reached, the data are immediately written to the log file. (When the log message queue reaches the maximum cache time or maximum capacity, it will be written to the log file immediately)|
| filename_suffix | string | Add a suffix to the default log file name. |
| write_to_disk | boolean | Write into disk or not. |
| display_name | string | Give a new name to `logdir` when `logdir` is too long or needed to be hidden. If not set, the default name is `logdir`. |
| display_name | string | This parameter is displayed in the location of `Select Data Stream` in the panel. If not set, the default name is `logdir`.(When `logdir` is too long or needed to be hidden). |
| file_name | string | Set the name of the log file. If the file_name already exists, setting the file_name will be new records in the same log file, which will continue to be used. Note that the name should include 'vdlrecords'. |
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/103187556-b9714280-48ff-11eb-9052-008e02a21199.png" width="100%"/>
</p>
......@@ -160,9 +155,9 @@ Parameter details:
| parameters | meaning |
| --------------- | ------------------------------------------------------------ |
| --logdir | Set one or more directories of the log. VisualDL will search the log file recursively under this path to display all the experimental results. |
| --logdir | Set one or more directories of the log. All the logs in the paths or subdirectories will be displayed on the VisualDL Board indepentently. |
| --model | Set a path to the model file (not a directory). VisualDL will visualize the model file in Graph page. PaddlePaddle、ONNX、Keras、Core ML、Caffe and other model formats are supported. Please refer to [Graph - Functional Instructions](./docs/components/UserGuide-en.md#functional-instructions-2). |
| --host | Specify IP address. The default value is `127.0.0.1`. |
| --host | Specify IP address. The default value is `127.0.0.1`. Specify it as `0.0.0.0` or public IP address so that other machines can visit VisualDL Board. |
| --port | Set the port. The default value is `8040`. |
| --cache-timeout | Cache time of the backend. During the cache time, the front end requests the same URL multiple times, and then the returned data are obtained from the cache. The default cache time is 20 seconds. |
| --language | The language of the VisualDL panel. Language can be specified as 'en' or 'zh', and the default is the language used by the browser. |
......@@ -198,9 +193,9 @@ The interface parameters are as follows:
| parameters | type | meaning |
| ------------- | -------------------------------------------------- | ------------------------------------------------------------ |
| logdir | string or list[string_1, string_2, ... , string_n] | Set one or more directories of the log. VisualDL will search the log file recursively under this path to display all the experimental results. |
| logdir | string or list[string_1, string_2, ... , string_n] | Set one or more directories of the log. All the logs in the paths or subdirectories will be displayed on the VisualDL Board indepentently.|
| model | string | Set a path to the model file (not a directory). VisualDL will visualize the model file in Graph page. |
| host | string | Specify IP address. The default value is `127.0.0.1`. |
| host | string | Specify IP address. The default value is `127.0.0.1`. Specify it as `0.0.0.0` or public IP address so that other machines can visit VisualDL Board. |
| port | int | Set the port. The default value is `8040`. |
| cache_timeout | int | Cache time of the backend. During the cache time, the front end requests the same URL multiple times, and then the returned data are obtained from the cache. The default cache time is 20 seconds. |
| language | string | The language of the VisualDL panel. Language can be specified as 'en' or 'zh', and the default is the language used by the browser. |
......
README_CN.md
浏览文件 @ fe2a1063
......@@ -101,11 +101,9 @@ VisualDL的后端提供了Python SDK，可通过LogWriter定制一个日志记
```python
class LogWriter(logdir=None,
comment='',
max_queue=10,
flush_secs=120,
filename_suffix='',
write_to_disk=True,
display_name='',
file_name='',
**kwargs)
......@@ -116,13 +114,11 @@ class LogWriter(logdir=None,
| 参数 | 格式 | 含义 |
| --------------- | ------- | ------------------------------------------------------------ |
| logdir | string | 日志文件所在的路径，VisualDL将在此路径下建立日志文件并进行记录，如果不填则默认为`runs/${CURRENT_TIME}` |
| comment | string | 为日志文件夹名添加后缀，如果制定了logdir则此项无效 |
| max_queue | int | 日志记录消息队列的最大容量，达到此容量则立即写入到日志文件 |
| flush_secs | int | 日志记录消息队列的最大缓存时间，达到此时间则立即写入到日志文件 |
| flush_secs | int | 日志记录消息队列的最大缓存时间，达到此时间则立即写入到日志文件（日志消息队列到达最大缓存时间或最大容量，都会立即写入日志文件） |
| filename_suffix | string | 为默认的日志文件名添加后缀 |
| write_to_disk | boolean | 是否写入到磁盘 |
| display_name | string | 在面板中替换实际显示的`logdir`，当日志所在路径过长或想隐藏日志所在路径时可指定此参数 |
| file_name | string | 指定写入的日志文件名，如果指定的文件名已经存在，则将日志续写在此文件中，文件名必须包括`vdlrecords` |
| display_name | string | 指定面板启动后显示的路径，如不指定此项则显示日志所在的实际路径，当日志所在路径过长或想隐藏日志所在路径时可指定此参数 |
| file_name | string | 指定写入的日志文件名，如果指定的文件名已经存在，则将日志续写在此文件中，因此可通过此参数实现日志续写的功能，文件名必须包括`vdlrecords` |
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/103187556-b9714280-48ff-11eb-9052-008e02a21199.png" width="100%"/>
......@@ -165,7 +161,7 @@ visualdl --logdir <dir_1, dir_2, ... , dir_n> --model <model_file> --host <host>
| --------------- | ------------------------------------------------------------ |
| --logdir | 设定日志所在目录，可以指定多个目录，VisualDL将遍历并且迭代寻找指定目录的子目录，将所有实验结果进行可视化 |
| --model | 设定模型文件路径(非文件夹路径)，VisualDL将在此路径指定的模型文件进行可视化，目前可支持PaddlePaddle、ONNX、Keras、Core ML、Caffe等多种模型结构，详情可查看[graph支持模型种类](./docs/components/README.md#%E5%8A%9F%E8%83%BD%E6%93%8D%E4%BD%9C%E8%AF%B4%E6%98%8E-2) |
| --host | 设定IP，默认为`127.0.0.1` |
| --host | 设定IP，默认为`127.0.0.1`，若想使得本机以外的机器访问启动的VisualDL面板，需指定此项为`0.0.0.0`或自己的公网IP地址 |
| --port | 设定端口，默认为`8040` |
| --cache-timeout | 后端缓存时间，在缓存时间内前端多次请求同一url，返回的数据从缓存中获取，默认为20秒 |
| --language | VisualDL面板语言，可指定为'en'或'zh'，默认为浏览器使用语言 |
......@@ -200,9 +196,9 @@ visualdl.server.app.run(logdir,
| 参数 | 格式 | 含义 |
| ------------- | ------------------------------------------------ | ------------------------------------------------------------ |
| logdir | string或list[string_1, string_2, ... , string_n] | 日志文件所在的路径，VisualDL将在此路径下递归搜索日志文件并进行可视化，可指定单个或多个路径 |
| model | string | 模型文件路径(非文件夹路径)，VisualDL将在此路径指定的模型文件进行可视化 |
| host | string | 指定启动服务的ip，默认为`127.0.0.1` |
| logdir | string或list[string_1, string_2, ... , string_n] | 日志文件所在的路径，VisualDL将在此路径下递归搜索日志文件并进行可视化，可指定单个或多个路径，每个路径中及其子目录中的日志都将视为独立日志展现在前端面板上 |
| model | string | 模型文件路径(非文件夹路径)，VisualDL将在此路径指定的模型文件进行可视化，目前可支持PaddlePaddle、ONNX、Keras、Core ML、Caffe等多种模型结构，详情可查看[graph支持模型种类](./docs/components/README.md#%E5%8A%9F%E8%83%BD%E6%93%8D%E4%BD%9C%E8%AF%B4%E6%98%8E-2) |
| host | string | 设定IP，默认为`127.0.0.1`，若想使得本机以外的机器访问启动的VisualDL面板，需指定此项为`0.0.0.0`或自己的公网IP地址 |
| port | int | 启动服务端口，默认为`8040` |
| cache_timeout | int | 后端缓存时间，在缓存时间内前端多次请求同一url，返回的数据从缓存中获取，默认为20秒 |
| language | string | VisualDL面板语言，可指定为'en'或'zh'，默认为浏览器使用语言 |
......
docs/README-en.md 已删除 100644 → 0
浏览文件 @ de2ec2ea
[**中文**](./README.md)
<p align="center">
<img src="https://raw.githubusercontent.com/PaddlePaddle/VisualDL/develop/frontend/packages/core/public/images/logo-visualdl.svg?sanitize=true" width="70%"/>
</p>
<p align="center">
<a href="https://travis-ci.org/PaddlePaddle/VisualDL"><img src="https://img.shields.io/travis/paddlepaddle/visualdl/develop?style=flat-square" alt="Build Status" /></a>
<a href="https://pypi.org/project/visualdl/"><img src="https://img.shields.io/pypi/v/visualdl?style=flat-square" alt="PyPI" /></a>
<a href="https://pypi.org/project/visualdl/#files"><img src="https://img.shields.io/pypi/dm/visualdl?style=flat-square" alt="Downloads" /></a>
<a href="https://github.com/PaddlePaddle/VisualDL/blob/develop/LICENSE"><img src="https://img.shields.io/github/license/paddlepaddle/visualdl?style=flat-square" alt="License" /></a>
</p>
<p align="center">
<a href="javascript:void(0)"><img src="https://img.shields.io/badge/QQ_Group-1045783368-52B6EF?style=social&logo=tencent-qq&logoColor=000&logoWidth=20" alt="QQ Group" /></a>
</p>
## Introduction
VisualDL, a visualization analysis tool of PaddlePaddle, provides a variety of charts to show the trends of parameters, and visualizes model structures, data samples, histograms of tensors, pr curves and high-dimensional data distributions. It enables users to understand the training process and the model structure more clearly and intuitively so as to optimize models efficiently.
VisualDL provides various visualization functions, including tracking metrics in real-time, visualizing the model structure, displaying the data sample, presenting the changes of distributions of tensors, showing the pr curves, projecting high-dimensional data to a lower dimensional space and more. Additionally, VisualDL provides VDL.service, which enables developers easily to save, track and share visualization results of experiments. For specific guidelines of each function, please refer to [**VisualDL User Guide**](./docs/components/UserGuide-en.md). Currently, VisualDL iterates rapidly and new functions will be continously added.
VisualDL natively supports the use of Python. Developers can retrieve plentiful visualization results by simply adding a few lines of Python code into the model before training.
## Contents
* [Key Highlights](#Key-Highlights)
* [Installation](#Installation)
* [Usage Guideline](#Usage-Guideline)
* [Function Preview](#Function-Preview)
* [Contribution](#Contribution)
* [More Details](#More-Details)
* [Technical Communication](#Technical-Communication)
## Key Highlights
### Easy to Use
The high-level design of API makes it easy to use. Only one click can initiate the visualization of model structures.
### Various Functions
The function contains the visualization of training parameters, data samples, graph structures, histograms of tensors, PR curves and high-dimensional data.
### High Compatibility
VisualDL provides the visualization of the mainstream model structures such as Paddle, ONNX, Caffe, widely supporting visual analysis for diverse users.
### Fully Support
By Integrating into PaddlePaddle and related modules, VisualDL allows developers to use different components unobstructed, and thus have the best experience in the PaddlePaddle ecosystem.
## Installation
### Install by PiP
```shell
python -m pip install visualdl -i https://mirror.baidu.com/pypi/simple
```
### Install by Code
```
git clone https://github.com/PaddlePaddle/VisualDL.git
cd VisualDL
python setup.py bdist_wheel
pip install --upgrade dist/visualdl-*.whl
```
Please note that Python 2 is no longer maintained officially since January 1, 2020. VisualDL now only supports Python 3 in order to ensure the usability of codes.
## Usage Guideline
VisualDL stores the data, parameters and other information of the training process in a log file. Users can launch the panel to observe the visualization results.
### 1. Log
The Python SDK is provided at the back end of VisualDL, and a logger can be customized through LogWriter. The interface description is shown as follows:
```python
class LogWriter(logdir=None,
comment='',
max_queue=10,
flush_secs=120,
filename_suffix='',
write_to_disk=True,
**kwargs)
```
#### Interface Parameters
| parameters | type | meaning |
| --------------- | ------- | ------------------------------------------------------------ |
| logdir | string | The path location of log file. VisualDL will create a log file under this path to record information generated by the training process. If not specified, the path will be `runs/${CURRENT_TIME}`as default. |
| comment | string | Add a suffix to the log folder name, which is invalid if logdir is already specified. |
| max_queue | int | The maximum capacity of the data generated before recording in a log file. If the capacity is reached, the data is immediately written into the log file. |
| flush_secs | int | The maximum cache time of the data generated before recording in a log file, when this time is reached, the data is immediately written to the log file. |
| filename_suffix | string | Add a suffix to the default log file name. |
| write_to_disk | boolean | Write into disk or not. |
| display_name | string | Set the name of different runs when `logdir` is too long or needed to be hidden. If not set, the default name is `logdir`.|
#### Example
Create a log file and record scalar values:
```python
from visualdl import LogWriter
# create a log file under `./log/scalar_test/train`
with LogWriter(logdir="./log/scalar_test/train") as writer:
# use `add_scalar` to record scalar values
writer.add_scalar(tag="acc", step=1, value=0.5678)
writer.add_scalar(tag="acc", step=2, value=0.6878)
writer.add_scalar(tag="acc", step=3, value=0.9878)
```
### 2. Launch Panel
In the above example, the log has recorded three sets of scalar values. Develpers can view the visualization results of the log file through launching the visualDL panel. There are two ways to launch a log file:
#### Launch by Command Line
Use the command line to launch the VisualDL panel：
```python
visualdl --logdir <dir_1, dir_2, ... , dir_n> --host <host> --port <port> --cache-timeout <cache_timeout> --language <language> --public-path <public_path> --api-only
```
Parameter details:
| parameters | meaning |
| --------------- | ------------------------------------------------------------ |
| --logdir | Set one or more directories of the log. VisualDL will search the log file recursively under this path to display the all experimental results. |
| --host | Specify IP address. The default value is·`127.0.0.1`. |
| --port | Set the port. The default value is`8040`. |
| --cache-timeout | Cache time of the backend. During the cache time, the front end requests the same URL multiple times, and then the returned data is obtained from the cache. The default cache time is 20 seconds. |
| --language | The language of the VisualDL panel. Language can be specified as 'en' or 'zh', and the default is the language used by the browser. |
| --public-path | The URL path of the VisualDL panel. The default path is '/app', meaning that the access address is 'http://&lt;host&gt;:&lt;port&gt;/app'. |
| --api-only | Decide whether or not to provide only API. If this parameter is set, VisualDL will only provides API service without displaying the web page, and the API address is 'http://&lt;host&gt;:&lt;port&gt;/&lt;public_path&gt;/api'. Additionally, If the public_path parameter is not specified, the default address is 'http://&lt;host&gt;:&lt;port&gt;/api'. |
To visualize the log file generated in the previous step, developers can launch the panel through the command:
```
visualdl --logdir ./log
```
#### Launch in Python Script
Developers can start the VisualDL panel in Python script as follows:
```python
visualdl.server.app.run(logdir,
host="127.0.0.1",
port=8080,
cache_timeout=20,
language=None,
public_path=None,
api_only=False,
open_browser=False)
```
Please note: since all parameters are indefinite except `logdir`, developers should specify parameter names when using them.
The interface parameters are as follows:
| parameters | type | meaning |
| ------------- | -------------------------------------------------- | ------------------------------------------------------------ |
| logdir | string or list[string_1, string_2, ... , string_n] | Set one or more directories of the log. VisualDL will search the log file recursively under this path to display the all experimental results. |
| host | string | Specify IP address. The default value is·`127.0.0.1`. |
| port | int | Set the port. The default value is`8040`. |
| cache_timeout | int | Cache time of the backend. During the cache time, the front end requests the same URL multiple times, and then the returned data is obtained from the cache. The default cache time is 20 seconds. |
| language | string | The language of the VisualDL panel. Language can be specified as 'en' or 'zh', and the default is the language used by the browser. |
| public_path | string | The URL path of the VisualDL panel. The default path is '/app', meaning that the access address is 'http://&lt;host&gt;:&lt;port&gt;/app'. |
| api_only | boolean | Decide whether or not to provide only API. If this parameter is set, VisualDL will only provides API service without displaying the web page, and the API address is 'http://&lt;host&gt;:&lt;port&gt;/&lt;public_path&gt;/api'. Additionally, If the parameter public_path is not specified, the default address is 'http://&lt;host&gt;:&lt;port&gt;/api'. |
| open_browser | boolean | Whether or not to open the browser. If this parameter is set as True, the browser will be openned automatically and VisualDL panel will be launched at the same time. If parameter api_only is specified as True, parameter open_browser can be ignored. |
To visualize the log file generated in the previous step, developers can launch the panel through the command:
```python
from visualdl.server import app
app.run(logdir="./log")
```
After launching the panel by one of the above methods, developers can see the visualization results on the browser shown as blow:
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/90868674-ba321f00-e3c9-11ea-83c1-f03c6dd19187.png" width="70%"/>
</p>
## Function Preview
### Scalar
**Scalar** makes use of various charts to display how the parameters, such as accuracy, loss and learning rate, change during the training process. In this case, developers can observe not only the single but also the multiple groups of parameters in order to understand the training process and thus speed up the process of model tuning.
#### Dynamic Display
After the launchment of VisualDL Board, the LogReader will continuously record the data to display in the front-end. Hence, the changes of parameters can be visualized in real-time, as shown below:
<p align="center">
<img src="https://visualdl.bj.bcebos.com/images/dynamic_display.gif" width="60%"/>
</p>
#### Comparison of Multiple Experiments
Developers can compare with multiple experiments by specifying and uploading the path of each experiment at the same time so as to visualize the same parameters in the same chart.
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/90869567-fdd95880-e3ca-11ea-9855-6c97ad5c8ae7.gif" width="100%"/>
</p>
### Image
**Image** provides real-time visualizations of the image data during the training process, allowing developers to observe the changes of images in different training stages and to deeply understand the effects of the training process.
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/90869677-22353500-e3cb-11ea-9830-2334bdd8e52e.gif" width="60%"/>
</p>
### Audio
**Audio** aims to allow developers to listen to the audio data in real-time during the training process, helping developers to monitor the process of speech recognition and text-to-speech.
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/90869771-47c23e80-e3cb-11ea-8b2a-a38b6c33d64b.png" width="85%"/>
</p>
### Graph
**Graph** enables developers to visualize model structures by only one click. Moreover, **Graph** allows Developers to explore model attributes, node information, node input and output. aiding them analyze model structure quickly and understand the direction of data flow easily.
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/90869866-6aecee00-e3cb-11ea-8211-b8af070239e6.png" width="85%"/>
</p>
### Histogram
**Histogram** displays how the trend of tensors (weight, bias, gradient, etc.) changes during the training process in the form of histogram. Developers can adjust the model structures accurately by having an in-depth understanding of the effect of each layer.
- Offset Mode
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/90870121-bd2e0f00-e3cb-11ea-89cf-6622cb607b89.png" width="85%"/>
</p>
- Overlay Mode
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/90870194-cfa84880-e3cb-11ea-8a66-bebcad267a10.png" width="85%"/>
</p>
### High Dimensional
**High Dimensional** provides two approaches--T-SNE and PCA--to do the dimensionality reduction, allowing developers to have an in-depth analysis of the relationship between high-dimensional data and to optimize algorithms based on the analysis.
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/90870677-85739700-e3cc-11ea-8653-18fa5c4106a3.GIF" width="85%"/>
</p>
### VDL.service
**VDL.service** enables developers to easily save, track and share visualization results with anyone for free.
<p align="center">
<img src=https://user-images.githubusercontent.com/48054808/93731055-fbeafb00-fbfd-11ea-80f4-bbfd08a0fc35.png
</p>
## Contribution
VisualDL, in which Graph is powered by [Netron](https://github.com/lutzroeder/netron), is an open source project supported by [PaddlePaddle](https://www.paddlepaddle.org/) and [ECharts](https://echarts.apache.org/) . Developers are warmly welcomed to use, comment and contribute.
## More Details
For more details related to the use of VisualDL, please refer to [**VisualDL User Guide**](./docs/components/README.md)。
## Technical Communication
Welcome to join the official QQ group 104578336 to communicate with PaddlePaddle team and other developers.
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/82522691-c2758680-9b5c-11ea-9aee-fca994aba175.png" width="20%"/>
</p>
docs/README.md
浏览文件 @ fe2a1063
此差异已折叠。
docs/README_CN.md 0 → 100644
浏览文件 @ fe2a1063
[**English**](./README.md)
<p align="center">
<img src="https://raw.githubusercontent.com/PaddlePaddle/VisualDL/develop/frontend/packages/core/public/images/logo-visualdl.svg?sanitize=true" width="70%"/>
</p>
<p align="center">
<a href="https://travis-ci.org/PaddlePaddle/VisualDL"><img src="https://img.shields.io/travis/paddlepaddle/visualdl/develop?style=flat-square" alt="Build Status" /></a>
<a href="https://pypi.org/project/visualdl/"><img src="https://img.shields.io/pypi/v/visualdl?style=flat-square" alt="PyPI" /></a>
<a href="https://pypi.org/project/visualdl/#files"><img src="https://img.shields.io/pypi/dm/visualdl?style=flat-square" alt="Downloads" /></a>
<a href="https://github.com/PaddlePaddle/VisualDL/blob/develop/LICENSE"><img src="https://img.shields.io/github/license/paddlepaddle/visualdl?style=flat-square" alt="License" /></a>
</p>
<p align="center">
<a href="https://jq.qq.com/?_wv=1027&k=TyzyVT4C" target="_blank" rel="noreferrer"><img src="https://img.shields.io/badge/QQ_Group-1045783368-52B6EF?style=social&logo=tencent-qq&logoColor=000&logoWidth=20" alt="QQ Group" /></a>
</p>
## 介绍
VisualDL是飞桨可视化分析工具，以丰富的图表呈现训练参数变化趋势、模型结构、数据样本、高维数据分布等。可帮助用户更清晰直观地理解深度学习模型训练过程及模型结构，进而实现高效的模型优化。
VisualDL提供丰富的可视化功能，支持标量、图结构、数据样本可视化、直方图、PR曲线及高维数据降维呈现等诸多功能，同时VisualDL提供可视化结果保存服务，通过VDL.service生成链接，保存并分享可视化结果。具体功能使用方式，请参见 [**VisualDL使用指南**](./components/README.md)。项目正处于高速迭代中，敬请期待新组件的加入。
VisualDL支持浏览器种类：Chrome（81和83）、Safari 13、FireFox（77和78）、Edge（Chromium版）。
VisualDL原生支持python的使用， 通过在模型的Python配置中添加几行代码，便可为训练过程提供丰富的可视化支持。
## 目录
* [核心亮点](#核心亮点)
* [安装方式](#安装方式)
* [使用方式](#使用方式)
* [可视化功能概览](#可视化功能概览)
* [开源贡献](#开源贡献)
* [更多细节](#更多细节)
## 核心亮点
### 简单易用
API设计简洁易懂，使用简单。模型结构一键实现可视化。
### 功能丰富
功能覆盖标量、数据样本、图结构、直方图、PR曲线及数据降维可视化。
### 高兼容性
全面支持Paddle、ONNX、Caffe等市面主流模型结构可视化，广泛支持各类用户进行可视化分析。
### 全面支持
与飞桨服务平台及工具组件全面打通，为您在飞桨生态系统中提供最佳使用体验。
## 安装方式
### 使用pip安装
```shell
python -m pip install visualdl -i https://mirror.baidu.com/pypi/simple
```
### 使用代码安装
```
git clone https://github.com/PaddlePaddle/VisualDL.git
cd VisualDL
python setup.py bdist_wheel
pip install --upgrade dist/visualdl-*.whl
```
需要注意，官方自2020年1月1日起不再维护Python2，为了保障代码可用性，VisualDL现仅支持Python3
## 使用方式
VisualDL将训练过程中的数据、参数等信息储存至日志文件中后，启动面板即可查看可视化结果。
### 1. 记录日志
VisualDL的后端提供了Python SDK，可通过LogWriter定制一个日志记录器，接口如下：
```python
class LogWriter(logdir=None,
comment='',
max_queue=10,
flush_secs=120,
filename_suffix='',
write_to_disk=True,
display_name='',
**kwargs)
```
#### 接口参数
| 参数 | 格式 | 含义 |
| --------------- | ------- | ------------------------------------------------------------ |
| logdir | string | 日志文件所在的路径，VisualDL将在此路径下建立日志文件并进行记录，如果不填则默认为`runs/${CURRENT_TIME}` |
| comment | string | 为日志文件夹名添加后缀，如果制定了logdir则此项无效 |
| max_queue | int | 日志记录消息队列的最大容量，默认值为10，达到此容量则立即写入到日志文件，如果设置为0则不缓存 |
| flush_secs | int | 日志记录消息队列的最大缓存时间，默认值为120，达到此时间则立即写入到日志文件，如果设置为0则不缓存 |
| filename_suffix | string | 为默认的日志文件名添加后缀 |
| write_to_disk | boolean | 是否写入到磁盘 |
| display_name | string | 在面板中`选择数据流`位置显示此参数，如不指定则默认显示日志所在路径（当日志所在路径过长或想隐藏日志所在路径时可指定此参数） |
#### 示例
设置日志文件并记录标量数据：
```python
from visualdl import LogWriter
# 在`./log/scalar_test/train`路径下建立日志文件
with LogWriter(logdir="./log/scalar_test/train") as writer:
# 使用scalar组件记录一个标量数据
writer.add_scalar(tag="acc", step=1, value=0.5678)
writer.add_scalar(tag="acc", step=2, value=0.6878)
writer.add_scalar(tag="acc", step=3, value=0.9878)
```
### 2. 启动面板
在上述示例中，日志已记录三组标量数据，现可启动VisualDL面板查看日志的可视化结果，共有两种启动方式：
#### 在命令行启动
使用命令行启动VisualDL面板，命令格式如下：
```python
visualdl --logdir <dir_1, dir_2, ... , dir_n> --host <host> --port <port> --cache-timeout <cache_timeout> --language <language> --public-path <public_path> --api-only
```
参数详情：
| 参数 | 意义 |
| --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| --logdir | 设定日志所在目录，可以指定多个目录，VisualDL将遍历并且迭代寻找指定目录的子目录，将所有实验结果进行可视化 |
| --model | 设定模型文件路径(非文件夹路径)，VisualDL将在此路径指定的模型文件进行可视化，目前可支持PaddlePaddle、ONNX、Keras、Core ML、Caffe等多种模型结构，详情可查看[graph支持模型种类](./docs/components/README.md#%E5%8A%9F%E8%83%BD%E6%93%8D%E4%BD%9C%E8%AF%B4%E6%98%8E-2) |
| --host | 设定IP，默认为`127.0.0.1` |
| --port | 设定端口，默认为`8040` |
| --cache-timeout | 后端缓存时间，在缓存时间内前端多次请求同一url，返回的数据从缓存中获取，默认为20秒 |
| --language | VisualDL面板语言，可指定为'en'或'zh'，默认为浏览器使用语言 |
| --public-path | VisualDL面板URL路径，默认是'/app'，即访问地址为'http://&lt;host&gt;:&lt;port&gt;/app' |
| --api-only | 是否只提供API，如果设置此参数，则VisualDL不提供页面展示，只提供API服务，此时API地址为'http://&lt;host&gt;:&lt;port&gt;/&lt;public_path&gt;/api'；若没有设置public_path参数，则默认为'http://&lt;host&gt;:&lt;port&gt;/api' |
针对上一步生成的日志，启动命令为：
```
visualdl --logdir ./log
```
#### 在Python脚本中启动
支持在Python脚本中启动VisualDL面板，接口如下：
```python
visualdl.server.app.run(logdir,
host="127.0.0.1",
port=8080,
cache_timeout=20,
language=None,
public_path=None,
api_only=False,
open_browser=False)
```
请注意：除`logdir`外，其他参数均为不定参数，传递时请指明参数名。
接口参数具体如下：
| 参数 | 格式 | 含义 |
| ------------- | ------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| logdir | string或list[string_1, string_2, ... , string_n] | 日志文件所在的路径，VisualDL将在此路径下递归搜索日志文件并进行可视化，可指定单个或多个路径 |
| model | string | 模型文件路径(非文件夹路径)，VisualDL将在此路径指定的模型文件进行可视化 |
| host | string | 指定启动服务的ip，默认为`127.0.0.1` |
| port | int | 启动服务端口，默认为`8040` |
| cache_timeout | int | 后端缓存时间，在缓存时间内前端多次请求同一url，返回的数据从缓存中获取，默认为20秒 |
| language | string | VisualDL面板语言，可指定为'en'或'zh'，默认为浏览器使用语言 |
| public_path | string | VisualDL面板URL路径，默认是'/app'，即访问地址为'http://&lt;host&gt;:&lt;port&gt;/app' |
| api_only | boolean | 是否只提供API，如果设置此参数，则VisualDL不提供页面展示，只提供API服务，此时API地址为'http://&lt;host&gt;:&lt;port&gt;/&lt;public_path&gt;/api'；若没有设置public_path参数，则默认为'http://&lt;host&gt;:&lt;port&gt;/api' |
| open_browser | boolean | 是否打开浏览器，设置为True则在启动后自动打开浏览器并访问VisualDL面板，若设置api_only，则忽略此参数 |
针对上一步生成的日志，我们的启动脚本为：
```python
from visualdl.server import app
app.run(logdir="./log")
```
在使用任意一种方式启动VisualDL面板后，打开浏览器访问VisualDL面板，即可查看日志的可视化结果，如图：
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/82786044-67ae9880-9e96-11ea-8a2b-3a0951a6ec19.png" width="60%"/>
</p>
## 可视化功能概览
### Scalar
以图表形式实时展示训练过程参数，如loss、accuracy。让用户通过观察单组或多组训练参数变化，了解训练过程，加速模型调优。具有两大特点：
#### 动态展示
在启动VisualDL Board后，LogReader将不断增量的读取日志中数据并供前端调用展示，因此能够在训练中同步观测指标变化，如下图：
<p align="center">
<img src="https://visualdl.bj.bcebos.com/images/dynamic_display.gif" width="60%"/>
</p>
#### 多实验对比
只需在启动VisualDL Board的时将每个实验日志所在路径同时传入即可，每个实验中相同tag的指标将绘制在一张图中同步呈现，如下图：
<p align="center">
<img src="https://visualdl.bj.bcebos.com/images/multi_experiments.gif" width="100%"/>
</p>
### Image
实时展示训练过程中的图像数据，用于观察不同训练阶段的图像变化，进而深入了解训练过程及效果。
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/90356439-24715980-e082-11ea-8896-01c27fc2fc9b.gif" width="85%"/>
</p>
### Audio
实时查看训练过程中的音频数据，监控语音识别与合成等任务的训练过程。
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/87659138-b4746880-c78f-11ea-965b-c33804e7c296.png" width="85%"/>
</p>
### Graph
一键可视化模型的网络结构。可查看模型属性、节点信息、节点输入输出等，并支持节点搜索，辅助用户快速分析模型结构与了解数据流向。
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/84483052-5acdd980-accb-11ea-8519-1608da7ee698.png" width="85%"/>
</p>
### Histogram
以直方图形式展示Tensor（weight、bias、gradient等）数据在训练过程中的变化趋势。深入了解模型各层效果，帮助开发者精准调整模型结构。
- Offset模式
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/86551031-86647c80-bf76-11ea-8ec2-8c86826c8137.png" width="85%"/>
</p>
- Overlay模式
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/86551033-882e4000-bf76-11ea-8e6a-af954c662ced.png" width="85%"/>
</p>
### PR Curve
精度-召回率曲线，帮助开发者权衡模型精度和召回率之间的平衡，设定最佳阈值。
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/86738774-ee46c000-c067-11ea-90d2-a98aac445cca.png" width="85%"/>
</p>
### High Dimensional
将高维数据进行降维展示，目前支持T-SNE、PCA两种降维方式，用于深入分析高维数据间的关系，方便用户根据数据特征进行算法优化。
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/82396340-3e4dd100-9a80-11ea-911d-798acdbc9c90.gif" width="85%"/>
</p>
### VDL.service
VisualDL可视化结果保存服务，以链接形式将可视化结果保存下来，方便用户快速、便捷的进行托管与分享。
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/93729521-72382f00-fbf7-11ea-91ff-6b6ab4b41e32.png" width="85%"/>
</p>
## 开源贡献
VisualDL 是由 [PaddlePaddle](https://www.paddlepaddle.org/) 和 [ECharts](https://echarts.apache.org/) 合作推出的开源项目。
Graph 相关功能由 [Netron](https://github.com/lutzroeder/netron) 提供技术支持。
欢迎所有人使用，提意见以及贡献代码。
## 更多细节
想了解更多关于VisualDL可视化功能的使用详情介绍，请查看[**VisualDL使用指南**](./docs/components/README.md)。
## 技术交流
欢迎您加入VisualDL官方QQ群：1045783368 与飞桨团队以及其他用户共同针对VisualDL进行讨论与交流。
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/82522691-c2758680-9b5c-11ea-9aee-fca994aba175.png" width="20%"/>
</p>
docs/components/README.md
浏览文件 @ fe2a1063
此差异已折叠。
docs/components/README_CN.md 0 → 100644
浏览文件 @ fe2a1063
此差异已折叠。
docs/components/UserGuide-en.md 已删除 100644 → 0
浏览文件 @ de2ec2ea
此差异已折叠。
docs/faq.md
浏览文件 @ fe2a1063
# 常见问题
# Frequently Asked Questions
[**English**](./faq_EN.md)
[**中文**](./faq_CN.md)
## 打开浏览器出现空白或者发生错误
可参照下列步骤排除错误原因
1. 确保启动VisualDL时指定的`logdir`路径正确，并确保其下有符合命名要求（文件名包含`vdlrecords`）的日志文件，可进入`logdir`指定的文件夹先行查看。若未解决，转下步。
2. 确认浏览器及版本符合VisualDL要求，建议使用**最新版本**的chrome及chrome内核浏览器、火狐浏览器，尝试更换浏览器及升级到最新版本。若未解决，转下步。
3. 将`--host`参数指定为`0.0.0.0`或`127.0.0.1`，注意后者仅支持本机查看，若在服务器启动并在异地查看，需使用`0.0.0.0`并确保使用的端口可被外网访问。若未解决，转下步。
4. 关闭并重新打开VisualDL前端页面，或强制刷新浏览器并等待15-30秒。若未解决，转下步。
5. 若是windows机器，可查看是否注册表被修改，参考解决[解决注册表修改导致无法查看日志](https://github.com/PaddlePaddle/VisualDL/issues/834)
若经过上述尝试仍无法解决，可在VisualDL GitHub Issue进行提问[VDL Issue](https://github.com/PaddlePaddle/VisualDL/issues)
## How can I do when I see a blank page or error messages in the browser?
To work out bugs, you can follow the steps：
1. You should ensure that the `logdir` path set by VisualDL is correct. In the meanwhile, there must be a log file and its name (including `vdlrecords`) should be correct. And then you can check the folder set by `logdir`. If it is not okay, try the next step.
2. Please check the browser and its version, and ensure it fits VisualDL. We suggest using **the latest versions** of Chrome browser, browsers with Chrome’s kernel, Firefox browser. Please change browsers and upgrade it to the latest version. If it is not okay, try the next step.
3. Please specify `--host`as `0.0.0.0` or `127.0.0.1`. And the latter one only supports the local address. If the server runs and you want to check by other addresses, please use `0.0.0.0` and ensure the end can be visited by outer net. If it is not okay, try the next step.
4. Close and re-open the front-end page of VisualDL. Or do a hard refresh on the browser and wait for 15-30 seconds. If it is not okay, try the next step.
5. If you use windows, please check whether the registry is modified. For users who want to get into more details, please refer to https://github.com/PaddlePaddle/VisualDL/issues/834. If all the steps do not work, please ask questions in VisualDL GitHub Issue. The link is [VDL Issue](https://github.com/PaddlePaddle/VisualDL/issues).
## 使用Image、Audio、Text组件仅显示10个样本
为保证使用体验，防止由于数据量过大造成前端页面卡死或崩溃，在展示Image、Audio、Text组件时使用采样算法进行了数据采样后展示。
尽管数据在前端展示进行了采样，但在日志中保存的数据仍为全部数据，可通过`VisualDL.LogReader`进行全部数据获取，可参考[LogReader使用教程](./components#LogReader)
## When I use modules of Image, Audio and Text, there are only ten samples.
We apply random sampling algorithm to display sampled data, when using modules of Image, Audio and Text. In this way, the front-end page will not be stuck or will not crash because of too much data, and then we can ensure users’ experience.
Though data shown in the front-page are sampled, all the data are saved in the log file. You can obtain all the data by using `VisualDL.LogReader`. For more details, please refer to our [LogReader tutorial](./components#LogReader).
## Why are the curves drawn by Scalar twisty?
Because there are two or more values in a certain step, you will find the curves you draw are like the following picture. Please check your script and find whether you add several values to one step, when using`add_scalar`.
## 为什么Scalar绘制的曲线是迂回的
当你发现绘制的曲线如下图所示，某个step对应的value不止一个值时，请检查你的脚本，是否在使用`add_scalar`时为一个step重复添加了多次value。
<p align="center">
<img src="https://user-images.githubusercontent.com/28444161/99496785-de44d280-29af-11eb-8fbd-ebc7a4919f2f.png" width="40%"/>
</p>
## 官方用例报错，LogWriter对象没有mode属性
## How can I do when confronted with the error of official use cases, saying that the target of LogWriter does not have the mode attribute?
Please check the version of VisualDL you use (which visualdl). According to the error, it is most likely that you are using the VisualDL 1.3, which attributes to Python 2 you use. Python 2 will install the old version of VisualDL automatically.
At present, VisualDL does not support Python 2 any more. And the instructions of existing official documents ae based on VisualDL 2.0, which also will not support Python 2. We suggest upgrading the Python's version to Python 3 and installing the latest version of VisualDL. In this way, the problem will not appear again.
## How to modify an existing log file
Different needs have different solutions
1. If you need to continue to add logs to an existing log file, please specify the file_name parameter as the log name when using LogWriter to obtain the log generator, and this log will be written when subsequent data is added.
2. If you need to add or delete some data in the log, you can read the log through LogReader and process it, and then write it back to a new file. Please refer to LogReader Tutorial.
可以检查一下使用的VisualDL的版本（which visualdl），因为按照此报错来看，大概率是因使用的是Python2，自动安装了VisualDL 1.3版本，但官方示例均基于2.0版本的，故导致此报错。
由于目前VisualDL已经不维护python2了，且现有官方文档上的使用说明都是基于2.0版本的，并且将不再维护旧版本，建议升级至Python3，安装最新版的VisualDL，即不会出现上述问题。
## What are the sampling rules of VisualDL
In order to minimize the CPU resource occupation for sampling, we improve the efficiency of data transmission and make the sampling uniform by using a reservoir sampling algorithm. In this way, all the data will be sampled in the back-end and then transmitted to the front-end. Reservoir Sampling can avoid loading all data through streaming sampling at once. For more details of the sampling theory, please refer to Reservoir Sampling.
\ No newline at end of file
docs/faq_CN.md 0 → 100644
浏览文件 @ fe2a1063
# 常见问题
[**English**](./faq.md)
## 打开浏览器出现空白或者发生错误
可参照下列步骤排除错误原因
1. 确保启动VisualDL时指定的`logdir`路径正确，并确保其下有符合命名要求（文件名包含`vdlrecords`）的日志文件，可进入`logdir`指定的文件夹先行查看。若未解决，转下步。
2. 确认浏览器及版本符合VisualDL要求，建议使用**最新版本**的chrome及chrome内核浏览器、火狐浏览器，尝试更换浏览器及升级到最新版本。若未解决，转下步。
3. 将`--host`参数指定为`0.0.0.0`或`127.0.0.1`，注意后者仅支持本机查看，若在服务器启动并在异地查看，需使用`0.0.0.0`并确保使用的端口可被外网访问。若未解决，转下步。
4. 关闭并重新打开VisualDL前端页面，或强制刷新浏览器并等待15-30秒。若未解决，转下步。
5. 若是windows机器，可查看是否注册表被修改，参考解决[解决注册表修改导致无法查看日志](https://github.com/PaddlePaddle/VisualDL/issues/834)
若经过上述尝试仍无法解决，可在VisualDL GitHub Issue进行提问[VDL Issue](https://github.com/PaddlePaddle/VisualDL/issues)
## 使用Image、Audio、Text组件仅显示10个样本
为保证使用体验，防止由于数据量过大造成前端页面卡死或崩溃，在展示Image、Audio、Text组件时使用采样算法进行了数据采样后展示。
尽管数据在前端展示进行了采样，但在日志中保存的数据仍为全部数据，可通过`VisualDL.LogReader`进行全部数据获取，可参考[LogReader使用教程](./components#LogReader)
## 为什么Scalar绘制的曲线是迂回的
当你发现绘制的曲线如下图所示，某个step对应的value不止一个值时，请检查你的脚本，是否在使用`add_scalar`时为一个step重复添加了多次value。
<p align="center">
<img src="https://user-images.githubusercontent.com/28444161/99496785-de44d280-29af-11eb-8fbd-ebc7a4919f2f.png" width="40%"/>
</p>
## 官方用例报错，LogWriter对象没有mode属性
可以检查一下使用的VisualDL的版本（which visualdl），因为按照此报错来看，大概率是因使用的是Python2，自动安装了VisualDL 1.3版本，但官方示例均基于2.0版本的，故导致此报错。
由于目前VisualDL已经不维护python2了，且现有官方文档上的使用说明都是基于2.0版本的，并且将不再维护旧版本，建议升级至Python3，安装最新版的VisualDL，即不会出现上述问题。
## 如何修改已有的日志文件
不同的需求距有不同的解决思路
1. 如果需要在已有的日志文件中继续添加日志，则在使用LogWriter获取日志生成器时指定`file_name`参数为此日志名，则后续添加数据时都会写入此日志。
2. 如果需要增加或删除日志中某些数据，则可以通过LogReader读取日志并处理后，重新写入到新的文件中，可参考[LogReader使用教程](./components#LogReader)
## VisualDL的采样规则是什么
为了尽量减少数据传输规模且尽可能均匀化采样，同时最小化采样所需的CPU资源占用，VisualDL使用蓄水池采样算法完成对后端所有数据采样后再向前端传输。
关于蓄水池采样的原理和证明，可参考[Reservoir sampling](https://en.wikipedia.org/wiki/Reservoir_sampling)，蓄水池采样能够通过流式采样的方式避免一次性加载所有数据到内存，且保证采样序列尽量平均。
docs/faq_EN.md 已删除 100644 → 0
浏览文件 @ de2ec2ea
# Frequently Asked Questions
[**中文**](./faq.md)
## How can I do when I see a blank page or error messages in the browser?
To work out bugs, you can follow the steps：
1. You should ensure that the `logdir` path set by VisualDL is correct. In the meanwhile, there must be a log file and its name (including `vdlrecords`) should be correct. And then you can check the folder set by `logdir`. If it is not okay, try the next step.
2. Please check the browser and its version, and ensure it fits VisualDL. We suggest using **the latest versions** of Chrome browser, browsers with Chrome’s kernel, Firefox browser. Please change browsers and upgrade it to the latest version. If it is not okay, try the next step.
3. Please specify `--host`as `0.0.0.0` or `127.0.0.1`. And the latter one only supports the local address. If the server runs and you want to check by other addresses, please use `0.0.0.0` and ensure the end can be visited by outer net. If it is not okay, try the next step.
4. Close and re-open the front-end page of VisualDL. Or do a hard refresh on the browser and wait for 15-30 seconds. If it is not okay, try the next step.
5. If you use windows, please check whether the registry is modified. For users who want to get into more details, please refer to https://github.com/PaddlePaddle/VisualDL/issues/834. If all the steps do not work, please ask questions in VisualDL GitHub Issue. The link is [VDL Issue](https://github.com/PaddlePaddle/VisualDL/issues).
## When I use modules of Image, Audio and Text, there are only ten samples.
We apply random sampling algorithm to display sampled data, when using modules of Image, Audio and Text. In this way, the front-end page will not be stuck or will not crash because of too much data, and then we can ensure users’ experience.
Though data shown in the front-page are sampled, all the data are saved in the log file. You can obtain all the data by using `VisualDL.LogReader`. For more details, please refer to our [LogReader tutorial](./components#LogReader).
## Why are the curves drawn by Scalar twisty?
Because there are two or more values in a certain step, you will find the curves you draw are like the following picture. Please check your script and find whether you add several values to one step, when using`add_scalar`.
<p align="center">
<img src="https://user-images.githubusercontent.com/28444161/99496785-de44d280-29af-11eb-8fbd-ebc7a4919f2f.png" width="40%"/>
</p>
## How can I do when confronted with the error of official use cases, saying that the target of LogWriter does not have the mode attribute?
Please check the version of VisualDL you use (which visualdl). According to the error, it is most likely that you are using the VisualDL 1.3, which attributes to Python 2 you use. Python 2 will install the old version of VisualDL automatically.
At present, VisualDL does not support Python 2 any more. And the instructions of existing official documents ae based on VisualDL 2.0, which also will not support Python 2. We suggest upgrading the Python's version to Python 3 and installing the latest version of VisualDL. In this way, the problem will not appear again.
\ No newline at end of file
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或