README.md 4.0 KB
Newer Older
D
daminglu 已提交
1
<p align="center">
Y
Yan Chunwei 已提交
2
  <img src="./docs/images/vs-logo.png" width="60%" />
D
daminglu 已提交
3 4 5
</p>

## Introduction
Q
Qiao Longfei 已提交
6 7 8
VisualDL is a deep learning visualization tool that can help design deep learning jobs.
It includes features such as scalar, parameter distribution, model structure and image visualization.
Currently it is being developed at a high pace.
D
daminglu 已提交
9 10
New features will be continuously added.

Q
Qiao Longfei 已提交
11
At present, most DNN frameworks use Python as their primary language. VisualDL supports Python by nature.
D
daminglu 已提交
12
Users can get plentiful visualization results by simply add a few lines of Python code into their model before training.
Q
Qiao Longfei 已提交
13 14 15


Besides Python SDK, VisualDL was writen in C++ on the low level. It also provides C++ SDK that
D
daminglu 已提交
16 17 18 19 20 21 22 23 24 25 26 27
can be integrated into other platforms.  


## Component
VisualDL now provides 4 components:

- graph
- scalar
- image
- histogram

### Graph
Q
Qiao Longfei 已提交
28 29
Graph is compatible with ONNX(Open Neural Network Exchange)[https://github.com/onnx/onnx],
Cooperated with Python SDK, VisualDL can be compatible with most major DNN frameworks, including
D
daminglu 已提交
30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75
PaddlePaddle, PyTorch and MXNet.

<p align="center">
  <img src="https://github.com/daming-lu/large_files/blob/master/graph_demo.gif" width="60%" />
</p>

### Scalar
Scalar can be used to show the trends of error during training.


<p align="center">
<img src="https://github.com/daming-lu/large_files/blob/master/loss_scalar.gif" width="60%"/>
</p>

### Image
Image can be used to visualize any tensor or intermediate generated image.

<p align="center">
<img src="https://github.com/daming-lu/large_files/blob/master/loss_image.gif" width="60%"/>
</p>

### Histogram
Histogram can be used to visualize parameter distribution and trends for any tensor.

<p align="center">
<img src="https://github.com/daming-lu/large_files/blob/master/histogram.gif" width="60%"/>
</p>

## SDK
VisualDL provides both Python SDK and C++ SDK in order to fit more use cases.


### Python SDK
Below is an example of creating a simple Scalar component and inserting data from different timestamps:

```python
import random
from visualdl import LogWriter

logdir = "./tmp"
logger = LogWriter(dir, sync_cycle=10)

# mark the components with 'train' label.
with logger.mode("train"):
    # create a scalar component called 'scalars/scalar0'
    scalar0 = logger.scalar("scalars/scalar0")
Q
Qiao Longfei 已提交
76

D
daminglu 已提交
77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98

# add some records during DL model running, lets start from another block.
with logger.mode("train"):
    # add scalars
    for step in range(100):
        scalar0.add_record(step, random.random())
```

### C++ SDK
Here is the C++ SDK identical to the Python SDK example above:

```c++
#include <cstdlib>
#include <string>
#include "visualdl/sdk.h"

namespace vs = visualdl;
namepsace cp = visualdl::components;

int main() {
  const std::string dir = "./tmp";
  vs::LogWriter logger(dir, 10);
Q
Qiao Longfei 已提交
99

D
daminglu 已提交
100 101
  logger.SetMode("train");
  auto tablet = logger.NewTablet("scalars/scalar0");
Q
Qiao Longfei 已提交
102

D
daminglu 已提交
103
  cp::Scalar<float> scalar0(tablet);
Q
Qiao Longfei 已提交
104

D
daminglu 已提交
105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126
  for (int step = 0; step < 1000; step++) {
    float v = (float)std::rand() / RAND_MAX;
    scalar0.AddRecord(step, v);
  }

  return 0;
}
```

## Launch Board
After some logs have been generated during training, users can launch board to see real-time data visualization.


```
visualDL --logdir <some log dir>
```

Board also supports the parameters below for remote access:

- `--host` set IP
- `--port` set port
- `--model_pb` specify ONNX format for model file
Q
qiaolongfei 已提交
127

Q
Qiao Longfei 已提交
128 129 130 131
### How to install from pypi
```
pip install --upgrade visualdl
```
Q
qiaolongfei 已提交
132

Q
Qiao Longfei 已提交
133
### How to build and install locally
Q
qiaolongfei 已提交
134
```
Q
Qiao Longfei 已提交
135 136 137
git clone https://github.com/PaddlePaddle/VisualDL.git
cd VisualDL

Y
Yan Chunwei 已提交
138
python setup.py bdist_wheel
Q
Qiao Longfei 已提交
139
pip install --upgrade dist/visualdl-*.whl
Q
Qiao Longfei 已提交
140 141
```

Y
Yan Chunwei 已提交
142
### Run a demo from scratch
Q
qiaolongfei 已提交
143
```
Y
Yan Chunwei 已提交
144
vdl_scratch.py
Q
Qiao Longfei 已提交
145
visualDL --logdir=scratch_log --port=8080
Q
Qiao Longfei 已提交
146
```
Q
Qiao Longfei 已提交
147 148
that will start a server locally on port 8080, then
you can visit http://127.0.0.1:8080 the see the visualdl board.
D
daminglu 已提交
149 150 151 152


### Contribute

Q
Qiao Longfei 已提交
153 154
VisualDL is initially created by [PaddlePaddle](http://www.paddlepaddle.org/) and
[ECharts](http://echarts.baidu.com/).
D
daminglu 已提交
155
We welcome everyone to use, comment and contribute to Visual DL :)