README.md 4.6 KB
Newer Older
Q
Qiao Longfei 已提交
1 2 3 4 5
[![Build Status](https://travis-ci.org/PaddlePaddle/VisualDL.svg?branch=develop)](https://travis-ci.org/PaddlePaddle/VisualDL)
[![Documentation Status](https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat)](https://github.com/PaddlePaddle/VisualDL/tree/develop/docs)
[![Release](https://img.shields.io/github/release/PaddlePaddle/VisualDL.svg)](https://github.com/PaddlePaddle/VisualDL/releases)
[![License](https://img.shields.io/badge/license-Apache%202-blue.svg)](LICENSE)

D
daminglu 已提交
6
<p align="center">
T
Thuan Nguyen 已提交
7
  <img src="https://raw.githubusercontent.com/PaddlePaddle/VisualDL/develop/docs/images/vs-logo.png" width="60%" />
D
daminglu 已提交
8 9
</p>

T
Thuan Nguyen 已提交
10
#
D
daminglu 已提交
11
## Introduction
Q
Qiao Longfei 已提交
12 13 14
VisualDL is a deep learning visualization tool that can help design deep learning jobs.
It includes features such as scalar, parameter distribution, model structure and image visualization.
Currently it is being developed at a high pace.
D
daminglu 已提交
15 16
New features will be continuously added.

Q
Qiao Longfei 已提交
17
At present, most DNN frameworks use Python as their primary language. VisualDL supports Python by nature.
D
daminglu 已提交
18
Users can get plentiful visualization results by simply add a few lines of Python code into their model before training.
Q
Qiao Longfei 已提交
19 20 21


Besides Python SDK, VisualDL was writen in C++ on the low level. It also provides C++ SDK that
D
daminglu 已提交
22 23 24 25 26 27 28 29 30 31 32 33
can be integrated into other platforms.  


## Component
VisualDL now provides 4 components:

- graph
- scalar
- image
- histogram

### Graph
34
Graph is compatible with ONNX ([Open Neural Network Exchange](https://github.com/onnx/onnx)),
Q
Qiao Longfei 已提交
35
Cooperated with Python SDK, VisualDL can be compatible with most major DNN frameworks, including
D
daminglu 已提交
36 37 38
PaddlePaddle, PyTorch and MXNet.

<p align="center">
T
Thuan Nguyen 已提交
39
  <img src="https://raw.githubusercontent.com/daming-lu/large_files/master/graph_demo.gif" width="60%" />
D
daminglu 已提交
40 41 42 43 44 45 46
</p>

### Scalar
Scalar can be used to show the trends of error during training.


<p align="center">
T
Thuan Nguyen 已提交
47
<img src="https://raw.githubusercontent.com/daming-lu/large_files/master/loss_scalar.gif" width="60%"/>
D
daminglu 已提交
48 49 50 51 52 53
</p>

### Image
Image can be used to visualize any tensor or intermediate generated image.

<p align="center">
T
Thuan Nguyen 已提交
54
<img src="https://raw.githubusercontent.com/daming-lu/large_files/master/loss_image.gif" width="60%"/>
D
daminglu 已提交
55 56 57 58 59 60
</p>

### Histogram
Histogram can be used to visualize parameter distribution and trends for any tensor.

<p align="center">
T
Thuan Nguyen 已提交
61
<img src="https://raw.githubusercontent.com/daming-lu/large_files/master/histogram.gif" width="60%"/>
D
daminglu 已提交
62 63
</p>

Q
Qiao Longfei 已提交
64 65 66 67 68 69 70 71 72 73 74 75
## Quick Start
```
# install
pip install --upgrade visualdl

# run a demo
vdl_scratch.py
visualDL --logdir=scratch_log --port=8080

# visit http://127.0.0.1:8080
```

D
daminglu 已提交
76 77 78 79 80 81 82 83 84 85 86 87
## SDK
VisualDL provides both Python SDK and C++ SDK in order to fit more use cases.


### Python SDK
Below is an example of creating a simple Scalar component and inserting data from different timestamps:

```python
import random
from visualdl import LogWriter

logdir = "./tmp"
J
Jeff Wang 已提交
88
logger = LogWriter(logdir, sync_cycle=10)
D
daminglu 已提交
89 90 91 92 93

# mark the components with 'train' label.
with logger.mode("train"):
    # create a scalar component called 'scalars/scalar0'
    scalar0 = logger.scalar("scalars/scalar0")
Q
Qiao Longfei 已提交
94

Y
Yan Chunwei 已提交
95 96 97
# add some records during DL model running.
for step in range(100):
    scalar0.add_record(step, random.random())
D
daminglu 已提交
98 99 100 101 102 103 104 105
```

### C++ SDK
Here is the C++ SDK identical to the Python SDK example above:

```c++
#include <cstdlib>
#include <string>
106
#include "visualdl/logic/sdk.h"
D
daminglu 已提交
107 108

namespace vs = visualdl;
109
namespace cp = visualdl::components;
D
daminglu 已提交
110 111 112 113

int main() {
  const std::string dir = "./tmp";
  vs::LogWriter logger(dir, 10);
Q
Qiao Longfei 已提交
114

D
daminglu 已提交
115
  logger.SetMode("train");
116
  auto tablet = logger.AddTablet("scalars/scalar0");
Q
Qiao Longfei 已提交
117

D
daminglu 已提交
118
  cp::Scalar<float> scalar0(tablet);
Q
Qiao Longfei 已提交
119

D
daminglu 已提交
120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141
  for (int step = 0; step < 1000; step++) {
    float v = (float)std::rand() / RAND_MAX;
    scalar0.AddRecord(step, v);
  }

  return 0;
}
```

## Launch Board
After some logs have been generated during training, users can launch board to see real-time data visualization.


```
visualDL --logdir <some log dir>
```

Board also supports the parameters below for remote access:

- `--host` set IP
- `--port` set port
- `--model_pb` specify ONNX format for model file
Q
qiaolongfei 已提交
142

Q
Qiao Longfei 已提交
143 144 145 146
### How to install from pypi
```
pip install --upgrade visualdl
```
Q
qiaolongfei 已提交
147

Q
Qiao Longfei 已提交
148
### How to build and install locally
Q
qiaolongfei 已提交
149
```
Q
Qiao Longfei 已提交
150 151 152
git clone https://github.com/PaddlePaddle/VisualDL.git
cd VisualDL

Y
Yan Chunwei 已提交
153
python setup.py bdist_wheel
Q
Qiao Longfei 已提交
154
pip install --upgrade dist/visualdl-*.whl
Q
Qiao Longfei 已提交
155 156
```

Y
Yan Chunwei 已提交
157
### Run a demo from scratch
Q
qiaolongfei 已提交
158
```
Y
Yan Chunwei 已提交
159
vdl_scratch.py
Q
Qiao Longfei 已提交
160
visualDL --logdir=scratch_log --port=8080
Q
Qiao Longfei 已提交
161
```
Q
Qiao Longfei 已提交
162 163
that will start a server locally on port 8080, then
you can visit http://127.0.0.1:8080 the see the visualdl board.
D
daminglu 已提交
164 165 166 167


### Contribute

Q
Qiao Longfei 已提交
168 169
VisualDL is initially created by [PaddlePaddle](http://www.paddlepaddle.org/) and
[ECharts](http://echarts.baidu.com/).
D
daminglu 已提交
170
We welcome everyone to use, comment and contribute to Visual DL :)