未验证 提交 fd51dd88 编写于 作者: Y YixinKristy 提交者: GitHub

Add CN&EN instructions of ROC&PR Curve (#900)

* Update README.md

* Update README_CN.md

* Update README.md

* Update README.md

* Update UserGuide-en.md
上级 4681bc3f
......@@ -22,7 +22,7 @@
## Introduction
VisualDL, a visualization analysis tool of PaddlePaddle, provides a variety of charts to show the trends of parameters, and visualizes model structures, data samples, histograms of tensors, pr curves and high-dimensional data distributions. It enables users to understand the training process and the model structure more clearly and intuitively so as to optimize models efficiently.
VisualDL, a visualization analysis tool of PaddlePaddle, provides a variety of charts to show the trends of parameters, and visualizes model structures, data samples, histograms of tensors, PR curves , ROC curves and high-dimensional data distributions. It enables users to understand the training process and the model structure more clearly and intuitively so as to optimize models efficiently.
VisualDL provides various visualization functions, including tracking metrics in real-time, visualizing the model structure, displaying the data sample, presenting the changes of distributions of tensors, showing the pr curves, projecting high-dimensional data to a lower dimensional space and more. Additionally, VisualDL provides VDL.service, which enables developers easily to save, track and share visualization results of experiments. For specific guidelines of each function, please refer to [**VisualDL User Guide**](./docs/components/UserGuide-en.md). Currently, VisualDL iterates rapidly and new functions will be continuously added.
......@@ -308,7 +308,7 @@ Developers can compare with multiple experiments by specifying and uploading the
### Histogram
Histogram displays how the trend of tensors (weight, bias, gradient, etc.) changes during the training process in the form of histogram. Developers can adjust the model structures accurately by having an in-depth understanding of the effect of each layer.
**Histogram** displays how the trend of tensors (weight, bias, gradient, etc.) changes during the training process in the form of histogram. Developers can adjust the model structures accurately by having an in-depth understanding of the effect of each layer.
- Offset Mode
......@@ -324,6 +324,22 @@ Histogram displays how the trend of tensors (weight, bias, gradient, etc.) chang
<img src="https://user-images.githubusercontent.com/48054808/90870194-cfa84880-e3cb-11ea-8a66-bebcad267a10.png" width="85%"/>
</p>
### PR Curve
**PR Curve** displays the precision and recall values under different thresholds, helping developers to find the best threshold efficiently.
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/103274647-1ef72900-49fd-11eb-9284-2a5b63bdf2e3.png" width="85%"/>
</p>
### ROC Curve
**ROC Curve** shows the performance of a classification model at all classification thresholds; the larger the area under the curve, the better the model performs, aiding developers to evaluate the model performance and choose an appropriate threshold.
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/103274711-42ba6f00-49fd-11eb-9452-4dd492682dd8.png" width="85%"/>
</p>
### High Dimensional
......
......@@ -24,7 +24,7 @@
VisualDL是飞桨可视化分析工具,以丰富的图表呈现训练参数变化趋势、模型结构、数据样本、高维数据分布等。可帮助用户更清晰直观地理解深度学习模型训练过程及模型结构,进而实现高效的模型优化。
VisualDL提供丰富的可视化功能,支持标量、图结构、数据样本可视化、直方图、PR曲线及高维数据降维呈现等诸多功能,同时VisualDL提供可视化结果保存服务,通过VDL.service生成链接,保存并分享可视化结果。具体功能使用方式,请参见 [**VisualDL使用指南**](./docs/components/README.md)。项目正处于高速迭代中,敬请期待新组件的加入。
VisualDL提供丰富的可视化功能,支持标量、图结构、数据样本可视化、直方图、PR曲线、ROC曲线及高维数据降维呈现等诸多功能,同时VisualDL提供可视化结果保存服务,通过VDL.service生成链接,保存并分享可视化结果。具体功能使用方式,请参见 [**VisualDL使用指南**](./docs/components/README.md)。项目正处于高速迭代中,敬请期待新组件的加入。
VisualDL支持浏览器:
......@@ -343,6 +343,13 @@ value: 3.1297709941864014
<img src="https://user-images.githubusercontent.com/48054808/86738774-ee46c000-c067-11ea-90d2-a98aac445cca.png" width="85%"/>
</p>
### ROC Curve
展示不同不同阈值下模型的表现(TPR、TNR),曲线下面积越大,模型表现越好,辅助开发者进行阈值选择以及直观的掌握模型训练情况。
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/103275084-51555600-49fe-11eb-8c16-d18d26b724e3.png" width="85%"/>
</p>
### High Dimensional
......
......@@ -6,7 +6,7 @@
VisualDL 是一个面向深度学习任务设计的可视化工具。VisualDL 利用了丰富的图表来展示数据,用户可以更直观、清晰地查看数据的特征与变化趋势,有助于分析数据、及时发现错误,进而改进神经网络模型的设计。
目前,VisualDL 支持 scalar, image, audio,graph, histogram, pr curve, high dimensional 七个组件,项目正处于高速迭代中,敬请期待新组件的加入。
目前,VisualDL 支持 scalar, image, audio,graph, histogram, pr curve, ROC curve, high dimensional 七个组件,项目正处于高速迭代中,敬请期待新组件的加入。
| 组件名称 | 展示图表 | 作用 |
| :----------------------------------------------------------: | :--------: | :----------------------------------------------------------- |
......@@ -16,6 +16,7 @@ VisualDL 是一个面向深度学习任务设计的可视化工具。VisualDL
| [Graph](#Graph--网络结构组件) | 网络结构 | 展示网络结构、节点属性及数据流向,辅助学习、优化网络结构 |
| [Histogram](#Histogram--直方图组件) | 直方图 | 展示训练过程中权重、梯度等张量的分布 |
| [PR Curve](#PR-Curve--PR曲线组件) | 折线图 | 权衡精度与召回率之间的平衡关系 |
| [ROC Curve](#ROC-Curve--ROC曲线组件) | 折线图 | 展示不同阈值下的模型表现 |
| [High Dimensional](#High-Dimensional--数据降维组件) | 数据降维 | 将高维数据映射到 2D/3D 空间来可视化嵌入,便于观察不同数据的相关性 |
同时,VisualDL提供可视化结果保存服务,通过 [VDL.service](#vdlservice) 生成链接,保存并分享可视化结果
......@@ -653,6 +654,65 @@ visualdl --logdir ./log --port 8080
<img src="https://user-images.githubusercontent.com/48054808/86741304-db34ef80-c069-11ea-86eb-787b49ed3705.png" width="30%"/>
</p>
## ROC Curve--ROC曲线组件
### 介绍
ROC曲线展示不同不同阈值下模型的表现(TPR、TNR),曲线下面积越大,模型表现越好,辅助开发者进行阈值选择以及直观的掌握模型训练情况。
### 记录接口
ROC Curve组件的记录接口如下:
```python
add_roc_curve(tag, labels, predictions, step=None, num_thresholds=10)
```
接口参数说明如下:
| 参数 | 格式 | 含义 |
| -------------- | --------------------- | ------------------------------------------- |
| tag | string | 记录指标的标志,如`train/loss`,不能含有`%` |
| labels | numpy.ndarray or list | 以ndarray或list格式表示的实际类别 |
| predictions | numpy.ndarray or list | 以ndarray或list格式表示的预测类别 |
| step | int | 记录的步数 |
| num_thresholds | int | 阈值设置的个数,默认为10,最大值为127 |
| weights | float | 用于设置TP/FP/TN/FN在计算precision和recall时的权重 |
| walltime | int | 记录数据的时间戳,默认为当前时间戳 |
### Demo
下面展示了使用 ROC Curve 组件记录数据的示例,代码见[ROC Curve组件](https://github.com/YixinKristy/VisualDL/blob/develop/demo/components/roc_curve_test.py)
```python
from visualdl import LogWriter
import numpy as np
with LogWriter("./log/roc_curve_test/train") as writer:
for step in range(3):
labels = np.random.randint(2, size=100)
predictions = np.random.rand(100)
writer.add_roc_curve(tag='roc_curve',
labels=labels,
predictions=predictions,
step=step,
num_thresholds=5)
```
运行上述程序后,在命令行执行
```shell
visualdl --logdir ./log --port 8080
```
接着在浏览器打开`http://127.0.0.1:8080`,即可查看ROC Curve
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/103275084-51555600-49fe-11eb-8c16-d18d26b724e3.png" width="80%"/>
</p>
*Note:ROC前端页面使用和PR相同,请参考上述PR Curve的使用说明。
## High Dimensional--数据降维组件
### 介绍
......@@ -661,6 +721,7 @@ High Dimensional 组件将高维数据进行降维展示,用于深入分析高
- PCA : Principle Component Analysis 主成分分析
- t-SNE : t-distributed stochastic neighbor embedding t-分布式随机领域嵌入
- umap: uniform manifold approximation and projection for dimension reduction 流形学习降维算法
### 记录接口
......
......@@ -6,7 +6,7 @@
VisualDL is a visualization tool designed for Deep Learning. VisualDL provides a variety of charts to show the trends of parameters. It enables users to understand the training process and model structures of Deep Learning models more clearly and intuitively so as to optimize models efficiently.
Currently, VisualDL provides seven components: scalar, image, audio, graph, histogram, pr curve and high dimensional. VisualDL iterates rapidly and new functions will be continuously added.
Currently, VisualDL provides seven components: scalar, image, audio, graph, histogram, pr curve, ROC curve and high dimensional. VisualDL iterates rapidly and new functions will be continuously added.
......@@ -19,6 +19,7 @@ Currently, VisualDL provides seven components: scalar, image, audio, graph, hist
| [Graph](#Graph--Network-Structure) | Network Structure | Visualize network structures, node attributes and data flow, assisting developers to learn and to optimize network structures. |
| [Histogram](#Histogram--Distribution-of-Tensors) | Distribution of Tensors | Present the changes of distributions of tensors, such as weights/gradients/bias, during the training process. |
| [PR Curve](#PR-Curve) | Precision & Recall Curve | Display precision-recall curves across training steps, clarifying the tradeoff between precision and recall when comparing models. |
| [ROC Curve](#ROC-Curve) | Receiver Operating Characteristic curve | Shows the performance of a classification model at all classification thresholds |
| [High Dimensional](#High-Dimensional--Data-Dimensionality-Reduction) | Data Dimensionality Reduction | Project high-dimensional data into 2D/3D space for embedding visualization, making it convenient to observe the correlation between data. |
At the same time, VisualDL provides [VDL.service](#vdlservice) , which allows developers to easily save, track and share visualization results of experiments with anyone for free.
......@@ -586,7 +587,7 @@ The interface parameters are described as follows:
### Demo
The following shows an example of how to use High Dimensional component, and script can be found in [PR Curve Demo](../../demo/components/pr_curve_test.py)
The following shows an example of how to use PR Curve component, and script can be found in [PR Curve Demo](../../demo/components/pr_curve_test.py)
```python
from visualdl import LogWriter
......@@ -657,6 +658,66 @@ Then, open the browser and enter the address`http://127.0.0.1:8080` to view:
<img src="https://user-images.githubusercontent.com/48054808/90880354-eace8480-e3da-11ea-921f-20f363eb1b1d.png" width="30%"/>
</p>
## ROC Curve
### Introduction
ROC Curve shows the performance of a classification model at all classification thresholds; the larger the area under the curve, the better the model performs, aiding developers to evaluate the model performance and choose an appropriate threshold.
### Record Interface
The interface of the PR Curve is shown as follows:
```python
add_roc_curve(tag, labels, predictions, step=None, num_thresholds=10)
```
The interface parameters are described as follows:
| parameter | format | meaning |
| -------------- | --------------------- | ------------------------------------------- |
| tag | string | Record the name of the image data,e.g.train/loss. Notice that the name cannot contain `%` |
| values | numpy.ndarray or list | Data is in ndarray or list format |
| predictions | numpy.ndarray or list | Prediction data is in ndarray or list format |
| step | int | Record the training steps |
| num_thresholds | int | Set the number of thresholds, default as 10, maximum as 127 |
| weights | float | Set the weights of TN/FN/TP/FP to calculate precision and recall |
| walltime | int | Record the time-stamp of the data, and the default is the current time-stamp |
### Demo
The following shows an example of how to use ROC curve component, and script can be found in [ROC Curve Demo](../../demo/components/roc_curve_test.py)
```python
from visualdl import LogWriter
import numpy as np
with LogWriter("./log/roc_curve_test/train") as writer:
for step in range(3):
labels = np.random.randint(2, size=100)
predictions = np.random.rand(100)
writer.add_roc_curve(tag='roc_curve',
labels=labels,
predictions=predictions,
step=step,
num_thresholds=5)
```
After running the above program, developers can launch the panel by:
```shell
visualdl --logdir ./log --port 8080
```
Then, open the browser and enter the address`http://127.0.0.1:8080` to view:
<p align="center">
<img src="https://user-images.githubusercontent.com/48054808/103274711-42ba6f00-49fd-11eb-9452-4dd492682dd8.png" width="85%"/>
</p>
*Note: the use of ROC Curve in the frontend is the same as that of PR Curve, please refer to the instructions in PR Curve section if needed.
## High Dimensional--Data Dimensionality Reduction
### Introduction
......@@ -665,6 +726,7 @@ High Dimensional projects high-dimensional data into a low dimensional space, ai
- PCA : Principle Component Analysis
- t-SNE : t-distributed Stochastic Neighbor Embedding
- umap: Uniform Manifold Approximation and Projection
### Record Interface
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册