diff --git a/docs/README-en.md b/docs/README-en.md new file mode 100644 index 0000000000000000000000000000000000000000..1d111b8aae4b41896335cfff1415d0c2601bc110 --- /dev/null +++ b/docs/README-en.md @@ -0,0 +1,279 @@ +[**中文**](./README.md) + +
+ +
+ + + + + + +## Introduction +VisualDL, a visualization analysis tool of PaddlePaddle, provides a variety of charts to show the trends of parameters, and visualizes model structures, data samples, histograms of tensors, pr curves and high-dimensional data distributions. It enables users to understand the training process and the model structure more clearly and intuitively so as to optimize models efficiently. + +VisualDL provides various visualization functions, including tracking metrics in real-time, visualizing the model structure, displaying the data sample, presenting the changes of distributions of tensors, showing the pr curves, projecting high-dimensional data to a lower dimensional space and more. For specific guidelines of each function, please refer to [**VisualDL User Guide**](./docs/components/UserGuide-en.md). Currently, VisualDL iterates rapidly and new functions will be continously added. + +VisualDL natively supports the use of Python. Developers can retrieve plentiful visualization results by simply adding a few lines of Python code into the model before training. + +## Contents + +* [Key Highlights](#Key-Highlights) +* [Installation](#Installation) +* [Usage Guideline](#Usage-Guideline) +* [Function Preview](#Function-Preview) +* [Contribution](#Contribution) +* [More Details](#More-Details) +* [Technical Communication](#Technical-Communication) + + + +## Key Highlights + +### Easy to Use + +The high-level design of API makes it easy to use. Only one click can initiate the visualization of model structures. + +### Various Functions + +The function contains the visualization of training parameters, data samples, graph structures, histograms of tensors, PR curves and high-dimensional data. + +### High Compatibility + +VisualDL provides the visualization of the mainstream model structures such as Paddle, ONNX, Caffe, widely supporting visual analysis for diverse users. + +### Fully Support + +By Integrating into PaddlePaddle and related modules, VisualDL allows developers to use different components unobstructed, and thus have the best experience in the PaddlePaddle ecosystem. + +## Installation + +### Install by PiP + +```shell +python -m pip install visualdl -i https://mirror.baidu.com/pypi/simple +``` +### Install by Code + +``` +git clone https://github.com/PaddlePaddle/VisualDL.git +cd VisualDL + +python setup.py bdist_wheel +pip install --upgrade dist/visualdl-*.whl +``` +Please note that Python 2 is no longer maintained officially since January 1, 2020. VisualDL now only supports Python 3 in order to ensure the usability of codes. + +## Usage Guideline + +VisualDL stores the data, parameters and other information of the training process in a log file. Users can launch the panel to observe the visualization results. + +### 1. Log + +The Python SDK is provided at the back end of VisualDL, and a logger can be customized through LogWriter. The interface description is shown as follows: + +```python +class LogWriter(logdir=None, + comment='', + max_queue=10, + flush_secs=120, + filename_suffix='', + write_to_disk=True, + **kwargs) +``` + +#### Interface Parameters + +| parameters | type | meaning | +| --------------- | ------- | ------------------------------------------------------------ | +| logdir | string | The path location of log file. VisualDL will create a log file under this path to record information generated by the training process. If not specified, the path will be `runs/${CURRENT_TIME}`as default. | +| comment | string | Add a suffix to the log folder name, which is invalid if logdir is already specified. | +| max_queue | int | The maximum capacity of the data generated before recording in a log file. If the capacity is reached, the data is immediately written into the log file. | +| flush_secs | int | The maximum cache time of the data generated before recording in a log file, when this time is reached, the data is immediately written to the log file. | +| filename_suffix | string | Add a suffix to the default log file name. | +| write_to_disk | boolean | Write into disk or not. | + +#### Example + +Create a log file and record scalar values: + +```python +from visualdl import LogWriter + +# create a log file under `./log/scalar_test/train` +with LogWriter(logdir="./log/scalar_test/train") as writer: + # use `add_scalar` to record scalar values + writer.add_scalar(tag="acc", step=1, value=0.5678) + writer.add_scalar(tag="acc", step=2, value=0.6878) + writer.add_scalar(tag="acc", step=3, value=0.9878) +``` + +### 2. Launch Panel + +In the above example, the log has recorded three sets of scalar values. Develpers can view the visualization results of the log file through launching the visualDL panel. There are two ways to launch a log file: + +#### Launch by Command Line + +Use the command line to launch the VisualDL panel: + +```python +visualdl --logdir+ +
+ + + +## Function Preview + +### Scalar +**Scalar** makes use of various charts to display how the parameters, such as accuracy, loss and learning rate, change during the training process. In this case, developers can observe not only the single but also the multiple groups of parameters in order to understand the training process and thus speed up the process of model tuning. + +#### Dynamic Display + +After the launchment of VisualDL Board, the LogReader will continuously record the data to display in the front-end. Hence, the changes of parameters can be visualized in real-time, as shown below: + ++ +
+ + +#### Comparison of Multiple Experiments + +Developers can compare with multiple experiments by specifying and uploading the path of each experiment at the same time so as to visualize the same parameters in the same chart. + ++ +
+ + +### Image +**Image** provides real-time visualizations of the image data during the training process, allowing developers to observe the changes of images in different training stages and to deeply understand the effects of the training process. + ++ +
+ +### Audio +**Audio** aims to allow developers to listen to the audio data in real-time during the training process, helping developers to monitor the process of speech recognition and text-to-speech. + ++ +
+ +### Graph + +**Graph** enables developers to visualize model structures by only one click. Moreover, **Graph** allows Developers to explore model attributes, node information, node input and output. aiding them analyze model structure quickly and understand the direction of data flow easily. + ++ +
+ +### Histogram + +Histogram displays how the trend of tensors (weight, bias, gradient, etc.) changes during the training process in the form of histogram. Developers can adjust the model structures accurately by having an in-depth understanding of the effect of each layer. + +- Offset Mode + ++ +
+ + +- Overlay Mode + ++ +
+ +### High Dimensional + +**High Dimensional** provides two approaches--T-SNE and PCA--to do the dimensionality reduction, allowing developers to have an in-depth analysis of the relationship between high-dimensional data and to optimize algorithms based on the analysis. + ++ +
+ +## Contribution + +VisualDL, in which Graph is powered by [Netron](https://github.com/lutzroeder/netron), is an open source project supported by [PaddlePaddle](https://www.paddlepaddle.org/) and [ECharts](https://echarts.apache.org/) . Developers are warmly welcomed to use, comment and contribute. + + +## More Details + +For more details related to the use of VisualDL, please refer to [**VisualDL User Guide**](./docs/components/README.md)。 + +## Technical Communication + +Welcome to join the official QQ group 104578336 to communicate with PaddlePaddle team and other developers. + ++ +
diff --git a/docs/overview.md b/docs/README.md similarity index 65% rename from docs/overview.md rename to docs/README.md index 49c10219ff23d78ac46e2fc701c9e25f993ac42b..aa1c4cc9e38ce87a33672d2f4e65cb73e21a796d 100644 --- a/docs/overview.md +++ b/docs/README.md @@ -1,17 +1,28 @@ + [**English**](./README-en.md) +-[![Build Status](https://img.shields.io/travis/PaddlePaddle/VisualDL/develop?style=flat-square)](https://travis-ci.org/PaddlePaddle/VisualDL) -[![Documentation Status](https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat-square)](https://github.com/PaddlePaddle/VisualDL/tree/develop/docs) -[![Release](https://img.shields.io/github/release/PaddlePaddle/VisualDL.svg?style=flat-square)](https://github.com/PaddlePaddle/VisualDL/releases) -[![License](https://img.shields.io/badge/license-Apache%202-blue.svg?style=flat-square)](LICENSE) + + + + ## 介绍 -VisualDL是深度学习模型可视化分析工具,以丰富的图表呈现训练参数变化趋势、模型结构、数据样本、高维数据分布等。可帮助用户更清晰直观地理解深度学习模型训练过程及模型结构,进而实现高效的模型优化。 +VisualDL是飞桨可视化分析工具,以丰富的图表呈现训练参数变化趋势、模型结构、数据样本、高维数据分布等。可帮助用户更清晰直观地理解深度学习模型训练过程及模型结构,进而实现高效的模型优化。 + +VisualDL提供丰富的可视化功能,支持标量、图结构、数据样本可视化、直方图、PR曲线及高维数据降维呈现等诸多功能。具体功能使用方式,请参见 [**VisualDL使用指南**](./components/README.md)。项目正处于高速迭代中,敬请期待新组件的加入。 -VisualDL提供丰富的可视化功能,支持实时训练参数分析、图结构、数据样本可视化及高维数据降维呈现等诸多功能。具体功能使用方式,请参见 [**VisualDL使用指南**](./components/README.md)。项目正处于高速迭代中,敬请期待新组件的加入。 +VisualDL支持浏览器种类:Chrome(81和83)、Safari 13、FireFox(77和78)、Edge(Chromium版)。 VisualDL原生支持python的使用, 通过在模型的Python配置中添加几行代码,便可为训练过程提供丰富的可视化支持。 @@ -33,19 +44,19 @@ VisualDL原生支持python的使用, 通过在模型的Python配置中添加 ## 核心亮点 -#### 简单易用 +### 简单易用 API设计简洁易懂,使用简单。模型结构一键实现可视化。 -#### 功能丰富 +### 功能丰富 -功能覆盖训练参数、图结构、数据样本及数据降维可视化。 +功能覆盖标量、数据样本、图结构、直方图、PR曲线及数据降维可视化。 -#### 高兼容性 +### 高兼容性 全面支持Paddle、ONNX、Caffe等市面主流模型结构可视化,广泛支持各类用户进行可视化分析。 -#### 全面支持 +### 全面支持 与飞桨服务平台及工具组件全面打通,为您在飞桨生态系统中提供最佳使用体验。 @@ -53,13 +64,21 @@ API设计简洁易懂,使用简单。模型结构一键实现可视化。 ## 安装方式 -使用pip安装 VisualDL 运行范例: +### 使用pip安装 ```shell -pip install --upgrade visualdl==2.0.0b4 +python -m pip install visualdl -i https://mirror.baidu.com/pypi/simple ``` +### 使用代码安装 +``` +git clone https://github.com/PaddlePaddle/VisualDL.git +cd VisualDL +python setup.py bdist_wheel +pip install --upgrade dist/visualdl-*.whl +``` +需要注意,官方自2020年1月1日起不再维护Python2,为了保障代码可用性,VisualDL现仅支持Python3 ## 使用方式 @@ -81,15 +100,14 @@ class LogWriter(logdir=None, #### 接口参数 -| 参数 | 格式 | 含义 | -| --------------- | ------- | ------------------------------------------------------------------------------------------------------ | +| 参数 | 格式 | 含义 | +| --------------- | ------- | ------------------------------------------------------------ | | logdir | string | 日志文件所在的路径,VisualDL将在此路径下建立日志文件并进行记录,如果不填则默认为`runs/${CURRENT_TIME}` | -| comment | string | 为日志文件夹名添加后缀,如果制定了logdir则此项无效 | -| max_queue | int | 日志记录消息队列的最大容量,达到此容量则立即写入到日志文件 | -| flush_secs | int | 日志记录消息队列的最大缓存时间,达到此时间则立即写入到日志文件 | -| filename_suffix | string | 为默认的日志文件名添加后缀 | -| write_to_disk | boolean | 是否写入到磁盘 | -| display_name | string | 实际展示在面板中的`runs`名称,可隐藏路径信息 | +| comment | string | 为日志文件夹名添加后缀,如果制定了logdir则此项无效 | +| max_queue | int | 日志记录消息队列的最大容量,达到此容量则立即写入到日志文件 | +| flush_secs | int | 日志记录消息队列的最大缓存时间,达到此时间则立即写入到日志文件 | +| filename_suffix | string | 为默认的日志文件名添加后缀 | +| write_to_disk | boolean | 是否写入到磁盘 | #### 示例 @@ -123,13 +141,13 @@ visualdl --logdir
- +
@@ -212,20 +232,67 @@ app.run("./log") +### Audio +实时查看训练过程中的音频数据,监控语音识别与合成等任务的训练过程。 + ++ +
+ +### Graph + +一键可视化模型的网络结构。可查看模型属性、节点信息、节点输入输出等,并支持节点搜索,辅助用户快速分析模型结构与了解数据流向。 ++ +
+ +### Histogram + +以直方图形式展示Tensor(weight、bias、gradient等)数据在训练过程中的变化趋势。深入了解模型各层效果,帮助开发者精准调整模型结构。 + +- Offset模式 + ++ +
+ +- Overlay模式 + ++ +
+ +### PR Curve + +精度-召回率曲线,帮助开发者权衡模型精度和召回率之间的平衡,设定最佳阈值。 + ++ +
### High Dimensional 将高维数据进行降维展示,目前支持T-SNE、PCA两种降维方式,用于深入分析高维数据间的关系,方便用户根据数据特征进行算法优化。- +
## 开源贡献 -VisualDL 是由 [PaddlePaddle](https://www.paddlepaddle.org/) 和 [ECharts](https://echarts.apache.org/) 合作推出的开源项目。欢迎所有人使用,提意见以及贡献代码。 +VisualDL 是由 [PaddlePaddle](https://www.paddlepaddle.org/) 和 [ECharts](https://echarts.apache.org/) 合作推出的开源项目。 +Graph 相关功能由 [Netron](https://github.com/lutzroeder/netron) 提供技术支持。 +欢迎所有人使用,提意见以及贡献代码。 ## 更多细节 -想了解更多关于VisualDL可视化功能的使用详情介绍,请查看[**VisualDL使用指南**](./components/README.md)。 +想了解更多关于VisualDL可视化功能的使用详情介绍,请查看[**VisualDL使用指南**](./docs/components/README.md)。 + +## 技术交流 + +欢迎您加入VisualDL官方QQ群:1045783368 与飞桨团队以及其他用户共同针对VisualDL进行讨论与交流。 + ++ +
diff --git a/docs/components/README.md b/docs/components/README.md index e1a8ecf20d64abb6e262cd51aee5595f70061559..90bf1c9fd5f1fb88790464b592fd32a1136d335d 100644 --- a/docs/components/README.md +++ b/docs/components/README.md @@ -71,7 +71,7 @@ add_scalar(tag, value, step, walltime=None) - 基础使用 -下面展示了使用 Scalar 组件记录数据的示例,代码见[Scalar组件](../../demo/components/scalar_test.py) +下面展示了使用 Scalar 组件记录数据的示例,代码见[Scalar组件](https://github.com/PaddlePaddle/VisualDL/blob/develop/demo/components/scalar_test.py) ```python from visualdl import LogWriter @@ -212,7 +212,7 @@ add_image(tag, img, step, walltime=None) | walltime | int | 记录数据的时间戳,默认为当前时间戳 | ### Demo -下面展示了使用 Image 组件记录数据的示例,代码文件请见[Image组件](../../demo/components/image_test.py) +下面展示了使用 Image 组件记录数据的示例,代码文件请见[Image组件](https://github.com/PaddlePaddle/VisualDL/blob/develop/demo/components/image_test.py) ```python import numpy as np from PIL import Image @@ -579,7 +579,7 @@ add_pr_curve(tag, labels, predictions, step=None, num_thresholds=10) ### Demo -下面展示了使用 PR Curve 组件记录数据的示例,代码见[PR Curve组件](#https://github.com/PaddlePaddle/VisualDL/blob/develop/demo/components/pr_curve_test.py) +下面展示了使用 PR Curve 组件记录数据的示例,代码见[PR Curve组件](https://github.com/PaddlePaddle/VisualDL/blob/develop/demo/components/pr_curve_test.py) ```python from visualdl import LogWriter @@ -674,7 +674,7 @@ add_embeddings(tag, labels, hot_vectors, walltime=None) | walltime | int | 记录数据的时间戳,默认为当前时间戳 | ### Demo -下面展示了使用 High Dimensional 组件记录数据的示例,代码见[High Dimensional组件](../../demo/components/high_dimensional_test.py) +下面展示了使用 High Dimensional 组件记录数据的示例,代码见[High Dimensional组件](https://github.com/PaddlePaddle/VisualDL/blob/develop/demo/components/high_dimensional_test.py) ```python from visualdl import LogWriter diff --git a/docs/conf.py b/docs/conf.py index 6d5f584bb97d889d96a4777fcd6d88d8367ddf1c..6329ef42d31ed78eaab9a529ecd727645a7ce529 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -13,6 +13,7 @@ from recommonmark.parser import CommonMarkParser import os import sys +from visualdl import __version__ sys.path.insert(0, os.path.abspath('../visualdl')) @@ -23,7 +24,7 @@ copyright = '2020, PaddlePaddle' author = 'PaddlePaddle' # The full version, including alpha/beta/rc tags -release = '2.0.0.beta4' +release = __version__ # -- General configuration --------------------------------------------------- diff --git a/docs/index.rst b/docs/index.rst index 6aecddad841bd58411f17b16e53dfaecc77b17c7..f8017a080de6b890a7817b306d979bc1edd5b76c 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -10,7 +10,8 @@ Welcome to VisualDL's documentation! :maxdepth: 2 :caption: Contents: - 概述<../README.md> + + 概述