README.md 3.0 KB
Newer Older
J
Jeff Wang 已提交
1 2 3 4 5 6 7 8 9 10 11
# VisualDL (Visualize the Deep Learning)

## Introduction

VisualDL is a visualization tool for deep learning, including scalar, parameter distribution, model structure, image visualization and other features.
The project is under development, and will provide more features.

Most of the DNN platforms are using Python. VisualDL supports Python out of the box.
By just adding a few lines of configuration to the code, VisualDL can provide a rich visual support for the training process.

In addition to Python SDK, the underlying VisualDL is written in C++, and its exposed C++ SDK can be integrated into other platforms.
T
Thuan Nguyen 已提交
12
Users can access the original features and monitor customized matrix.
J
Jeff Wang 已提交
13 14 15 16 17 18 19 20 21 22 23 24 25

## Components
VisualDL supports four componments:

- graph
- scalar
- image
- histogram

### graph
Compatible with ONNX (Open Neural Network Exchange) [https://github.com/onnx/onnx]. VisualDL is compatible with the mainstream DNN platforms such as PaddlePaddle, Pytorch, and MXNet through Python SDK.

<p align="center">
D
daminglu 已提交
26
<img src="./introduction/graph.png" width="60%"/>
J
Jeff Wang 已提交
27 28 29
</p>

### scalar
T
Thuan Nguyen 已提交
30
Show the error trend throughout the training.
J
Jeff Wang 已提交
31 32

<p align="center">
D
daminglu 已提交
33
<img src="./introduction/scalar.png" width="60%"/>
J
Jeff Wang 已提交
34 35 36 37 38 39
</p>

### image
To visualize any tensor, or model generated images

<p align="center">
D
daminglu 已提交
40
<img src="./introduction/image-gan.png" width="60%"/>
J
Jeff Wang 已提交
41 42 43 44 45 46
</p>

### histogram
To visualize the distribution of elements in any tensor

<p align="center">
D
daminglu 已提交
47
<img src="./introduction/histogram.png" width="60%"/>
J
Jeff Wang 已提交
48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66
</p>

## SDK
VisualDL also provides Python SDK and C++ SDK for different platforms.

### Python SDK
Take the simplest Scalar component, for example, to try to create a scalar component and log the data:

```python
import random
from visualdl import LogWriter

logdir = "./tmp"
logger = LogWriter(dir, sync_cycle=10)

# mark the components with 'train' label.
with logger.mode("train"):
    # create a scalar component called 'scalars/scalar0'
    scalar0 = logger.scalar("scalars/scalar0")
T
Thuan Nguyen 已提交
67

J
Jeff Wang 已提交
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83

# add some records during DL model running, lets start from another block.
with logger.mode("train"):
    # add scalars
    for step in range(100):
        scalar0.add_record(step, random.random())
```

### C++ SDK
The same code for C++ SDK in the above Python SDK is as follows
```c++
#include <cstdlib>
#include <string>
#include "visualdl/sdk.h"

namespace vs = visualdl;
84
namespace cp = visualdl::components;
J
Jeff Wang 已提交
85 86 87 88

int main() {
  const std::string dir = "./tmp";
  vs::LogWriter logger(dir, 10);
T
Thuan Nguyen 已提交
89

J
Jeff Wang 已提交
90
  logger.SetMode("train");
91
  auto tablet = logger.AddTablet("scalars/scalar0");
T
Thuan Nguyen 已提交
92

J
Jeff Wang 已提交
93
  cp::Scalar<float> scalar0(tablet);
T
Thuan Nguyen 已提交
94

J
Jeff Wang 已提交
95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113
  for (int step = 0; step < 1000; step++) {
    float v = (float)std::rand() / RAND_MAX;
    scalar0.AddRecord(step, v);
  }

  return 0;
}
```
## Activate Board
When the log data has been generated during the training process, the board can be started for real-time data visualization.

```
visualDL --logdir <some log dir>
```

the Board supports configuration of remote access.

- `--host` Configure IP
- `--port` Configure port