Skip to content

P
PaddleDetection

s920243400 / PaddleDetection
与 Fork 源项目一致

Fork自 PaddlePaddle / PaddleDetection

通知 2
Star 0
Fork 0
P
PaddleDetection
未验证 提交 c73fe426 编写于 作者: W wangguanzhong 提交者： GitHub
浏览文件
操作

Cherry pick update quick start (#2433) 

* update quick_start

* update get_started

* update doc, test=document_fix
上级 4f08ab9f
隐藏空白更改
内联 并排
Showing with 361 addition and 60 deletion
docs/tutorials/GETTING_STARTED.md 0 → 100644
浏览文件 @ c73fe426
English | [简体中文](GETTING_STARTED_cn.md)
# Getting Started
## Installation
For setting up the running environment, please refer to [installation
instructions](INSTALL_cn.md).
## Data preparation
- Please refer to [PrepareDataSet](PrepareDataSet.md) for data preparation
- Please set the data path for data configuration file in ```configs/datasets```
## Training & Evaluation & Inference
PaddleDetection provides scripts for training, evalution and inference with various features according to different configure.
```bash
# training on single-GPU
export CUDA_VISIBLE_DEVICES=0
python tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml
# training on multi-GPU
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
python -m paddle.distributed.launch --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml
# GPU evaluation
export CUDA_VISIBLE_DEVICES=0
python tools/eval.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml
# Inference
python tools/infer.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml --infer_img=demo/000000570688.jpg
```
### Other argument list
list below can be viewed by `--help`
| FLAG | script supported | description | default | remark |
| :----------------------: | :------------: | :---------------: | :--------------: | :-----------------: |
| -c | ALL | Select config file | None | **required**, such as `-c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml` |
| -o | ALL | Set parameters in configure file | None | `-o` has higher priority to file configured by `-c`. Such as `-o use_gpu=False` |
| --eval | train | Whether to perform evaluation in training | False | set `--eval` if needed |
| -r/--resume_checkpoint | train | Checkpoint path for resuming training | None | such as `-r output/faster_rcnn_r50_1x_coco/10000` |
| --slim_config | ALL | Configure file of slim method | None | such as `--slim_config configs/slim/prune/yolov3_prune_l1_norm.yml` |
| --use_vdl | train/infer | Whether to record the data with [VisualDL](https://github.com/paddlepaddle/visualdl), so as to display in VisualDL | False | VisualDL requires Python>=3.5 |
| --vdl\_log_dir | train/infer | VisualDL logging directory for image | train:`vdl_log_dir/scalar` infer: `vdl_log_dir/image` | VisualDL requires Python>=3.5 |
| --output_eval | eval | Directory for storing the evaluation output | None | such as `--output_eval=eval_output`, default is current directory |
| --json_eval | eval | Whether to evaluate with already existed bbox.json or mask.json | False | set `--json_eval` if needed and json path is set in `--output_eval` |
| --classwise | eval | Whether to eval AP for each class and draw PR curve | False | set `--classwise` if needed |
| --output_dir | infer | Directory for storing the output visualization files | `./output` | such as `--output_dir output` |
| --draw_threshold | infer | Threshold to reserve the result for visualization | 0.5 | such as `--draw_threshold 0.7` |
| --infer_dir | infer | Directory for images to perform inference on | None | One of `infer_dir` and `infer_img` is requied |
| --infer_img | infer | Image path | None | One of `infer_dir` and `infer_img` is requied, `infer_img` has higher priority over `infer_dir` |
## Examples
### Training
- Perform evaluation in training
```bash
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
python -m paddle.distributed.launch --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml --eval
```
Perform training and evalution alternatively and evaluate at each end of epoch. Meanwhile, the best model with highest MAP is saved at each epoch which has the same path as `model_final`.
If evaluation dataset is large, we suggest modifing `snapshot_epoch` in `configs/runtime.yml` to decrease evaluation times or evaluating after training.
- Fine-tune other task
When using pre-trained model to fine-tune other task, pretrain\_weights can be used directly. The parameters with different shape will be ignored automatically. For example:
```bash
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
# If the shape of parameters in program is different from pretrain_weights,
# then PaddleDetection will not use such parameters.
python -m paddle.distributed.launch --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml \
-o pretrain_weights=output/faster_rcnn_r50_1x_coco/model_final \
```
##### NOTES
- `CUDA_VISIBLE_DEVICES` can specify different gpu numbers. Such as: `export CUDA_VISIBLE_DEVICES=0,1,2,3`.
- Dataset will be downloaded automatically and cached in `~/.cache/paddle/dataset` if not be found locally.
- Pretrained model is downloaded automatically and cached in `~/.cache/paddle/weights`.
- Checkpoints are saved in `output` by default, and can be revised from `save_dir` in `configs/runtime.yml`.
### Evaluation
- Evaluate by specified weights path and dataset path
```bash
export CUDA_VISIBLE_DEVICES=0
python -u tools/eval.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml \
-o weights=https://paddledet.bj.bcebos.com/models/faster_rcnn_r50_fpn_1x_coco.pdparams
```
The path of model to be evaluted can be both local path and link in [MODEL_ZOO](../MODEL_ZOO_cn.md).
- Evaluate with json
```bash
export CUDA_VISIBLE_DEVICES=0
python tools/eval.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml \
--json_eval \
-output_eval evaluation/
```
The json file must be named bbox.json or mask.json, placed in the `evaluation/` directory.
### Inference
- Output specified directory && Set up threshold
```bash
export CUDA_VISIBLE_DEVICES=0
python tools/infer.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml \
--infer_img=demo/000000570688.jpg \
--output_dir=infer_output/ \
--draw_threshold=0.5 \
-o weights=output/faster_rcnn_r50_fpn_1x_coco/model_final \
--use_vdl=Ture
```
`--draw_threshold` is an optional argument. Default is 0.5.
Different thresholds will produce different results depending on the calculation of [NMS](https://ieeexplore.ieee.org/document/1699659).
## Deployment
Please refer to [depolyment](../../deploy/README.md)
## Model Compression
Please refer to [slim](../../configs/slim/README.md)
docs/tutorials/GETTING_STARTED_cn.md
浏览文件 @ c73fe426
[English](GETTING_STARTED.md) | 简体中文
# 入门使用
## 安装
......@@ -12,86 +15,129 @@
## 训练/评估/预测
PaddleDetection在[tools](https://github.com/PaddlePaddle/PaddleDetection/tree/master/dygraph/tools)中提供了`训练`/`评估`/`预测`/`导出模型`等功能，支持通过传入不同可选参数实现特定功能
### 参数列表
PaddleDetection提供了`训练`/`评估`/`预测`等功能，支持通过传入不同可选参数实现特定功能
以下列表可以通过`--help`查看
| FLAG | 支持脚本 | 用途 | 默认值 | 备注 |
| :----------------------: | :------------: | :---------------: | :--------------: | :-----------------: |
| -c | ALL | 指定配置文件 | None | **必选**，例如-c configs/ppyolo/ppyolo_r50vd_dcn_1x_coco.yml |
| --eval | train | 是否边训练边测试 | False | 可选，如需指定，直接`--eval`即可 |
| --fleet | train | 是否使用fleet API训练 | False | 可以使用--fleet来指定使用fleet API进行多机训练 |
| --fp16 | train | 是否开启混合精度训练 | False | 可以使用--fp16来指定使用混合精度训练 |
| -o | ALL | 设置或更改配置文件里的参数内容 | None | 可选，例如：`-o use_gpu=False` |
| --slim_config | ALL | 模型压缩策略配置文件 | None | 可选，例如`--slim_config configs/slim/prune/yolov3_prune_l1_norm.yml` |
| --output_dir | infer/export_model | 预测后结果或导出模型保存路径 | `./output` | 可选，例如`--output_dir=output` |
| --draw_threshold | infer | 可视化时分数阈值 | 0.5 | 可选，`--draw_threshold=0.7` |
| --infer_dir | infer | 用于预测的图片文件夹路径 | None | 可选 |
| --infer_img | infer | 用于预测的图片路径 | None | 可选，`--infer_img`和`--infer_dir`必须至少设置一个 |
| --classwise | eval | 是否评估单类AP和绘制单类PR曲线 | False | 可选 |
### 训练
- 单卡训练
```bash
# 通过CUDA_VISIBLE_DEVICES指定GPU卡号
# GPU单卡训练
export CUDA_VISIBLE_DEVICES=0
python tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml
```
- 多卡训练
```bash
# GPU多卡训练
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
python -m paddle.distributed.launch --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml
# GPU评估
export CUDA_VISIBLE_DEVICES=0
python tools/eval.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml
# 预测
python tools/infer.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml --infer_img=demo/000000570688.jpg
```
- 混合精度训练
```bash
export CUDA_VISIBLE_DEVICES=0
python tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml --fp16
```
### 参数列表
- fleet API训练
以下列表可以通过`--help`查看
```bash
# fleet API用于多机训练，启动方式与单机多卡训练方式基本一致，只不过需要使用--ips指定ip列表以及--fleet开启多机训练
python -m paddle.distributed.launch --ips="xx.xx.xx.xx,yy.yy.yy.yy" --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml --fleet
```
| FLAG | 支持脚本 | 用途 | 默认值 | 备注 |
| :----------------------: | :------------: | :---------------: | :--------------: | :-----------------: |
| -c | ALL | 指定配置文件 | None | **必选**，例如-c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml |
| -o | ALL | 设置或更改配置文件里的参数内容 | None | 相较于`-c`设置的配置文件有更高优先级，例如：`-o use_gpu=False` |
| --eval | train | 是否边训练边测试 | False | 如需指定，直接`--eval`即可 |
| -r/--resume_checkpoint | train | 恢复训练加载的权重路径 | None | 例如：`-r output/faster_rcnn_r50_1x_coco/10000` |
| --slim_config | ALL | 模型压缩策略配置文件 | None | 例如`--slim_config configs/slim/prune/yolov3_prune_l1_norm.yml` |
| --use_vdl | train/infer | 是否使用[VisualDL](https://github.com/paddlepaddle/visualdl)记录数据，进而在VisualDL面板中显示 | False | VisualDL需Python>=3.5 |
| --vdl\_log_dir | train/infer | 指定 VisualDL 记录数据的存储路径 | train:`vdl_log_dir/scalar` infer: `vdl_log_dir/image` | VisualDL需Python>=3.5 |
| --output_eval | eval | 评估阶段保存json路径 | None | 例如 `--output_eval=eval_output`, 默认为当前路径 |
| --json_eval | eval | 是否通过已存在的bbox.json或者mask.json进行评估 | False | 如需指定，直接`--json_eval`即可， json文件路径在`--output_eval`中设置 |
| --classwise | eval | 是否评估单类AP和绘制单类PR曲线 | False | 如需指定，直接`--classwise`即可 |
| --output_dir | infer/export_model | 预测后结果或导出模型保存路径 | `./output` | 例如`--output_dir=output` |
| --draw_threshold | infer | 可视化时分数阈值 | 0.5 | 例如`--draw_threshold=0.7` |
| --infer_dir | infer | 用于预测的图片文件夹路径 | None | `--infer_img`和`--infer_dir`必须至少设置一个 |
| --infer_img | infer | 用于预测的图片路径 | None | `--infer_img`和`--infer_dir`必须至少设置一个，`infer_img`具有更高优先级 |
## 使用示例
### 模型训练
- 边训练边评估
```bash
python tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml --eval
```
### 评估
```bash
# 目前只支持单卡评估
CUDA_VISIBLE_DEVICES=0 python tools/eval.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml
```
```bash
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
python -m paddle.distributed.launch --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml --eval
```
### 预测
```bash
CUDA_VISIBLE_DEVICES=0 python tools/infer.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml --infer_img={IMAGE_PATH}
```
在训练中交替执行评估, 评估在每个epoch训练结束后开始。每次评估后还会评出最佳mAP模型保存到`best_model`文件夹下。
## 预测部署
如果验证集很大，测试将会比较耗时，建议调整`configs/runtime.yml` 文件中的 `snapshot_epoch`配置以减少评估次数，或训练完成后再进行评估。
（1）导出模型
```bash
python tools/export_model.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml \
-o weights=output/faster_rcnn_r50_fpn_1x_coco/model_final \
--output_dir=output_inference
```
- Fine-tune其他任务
使用预训练模型fine-tune其他任务时，可以直接加载预训练模型，形状不匹配的参数将自动忽略，例如：
```bash
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
# 如果模型中参数形状与加载权重形状不同，将不会加载这类参数
python -m paddle.distributed.launch --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml \
-o pretrain_weights=output/faster_rcnn_r50_1x_coco/model_final \
```
**提示:**
- `CUDA_VISIBLE_DEVICES` 参数可以指定不同的GPU。例如: `export CUDA_VISIBLE_DEVICES=0,1,2,3`
- 若本地未找到数据集，将自动下载数据集并保存在`~/.cache/paddle/dataset`中。
- 预训练模型自动下载并保存在`〜/.cache/paddle/weights`中。
- 模型checkpoints默认保存在`output`中，可通过修改配置文件`configs/runtime.yml`中`save_dir`进行配置。
### 模型评估
- 指定权重和数据集路径
```bash
export CUDA_VISIBLE_DEVICES=0
python -u tools/eval.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml \
-o weights=https://paddledet.bj.bcebos.com/models/faster_rcnn_r50_fpn_1x_coco.pdparams
```
评估模型可以为本地路径，例如`output/faster_rcnn_r50_1x_coco/model_final`, 也可以是[MODEL_ZOO](../MODEL_ZOO_cn.md)中给出的模型链接。
（2）预测部署
参考[预测部署文档](https://github.com/PaddlePaddle/PaddleDetection/tree/master/dygraph/deploy)。
- 通过json文件评估
```bash
export CUDA_VISIBLE_DEVICES=0
python tools/eval.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml \
--json_eval \
-output_eval evaluation/
```
json文件必须命名为bbox.json或者mask.json，放在`evaluation/`目录下。
### 模型预测
- 设置输出路径 && 设置预测阈值
```bash
export CUDA_VISIBLE_DEVICES=0
python tools/infer.py -c configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.yml \
--infer_img=demo/000000570688.jpg \
--output_dir=infer_output/ \
--draw_threshold=0.5 \
-o weights=output/faster_rcnn_r50_fpn_1x_coco/model_final \
--use_vdl=Ture
```
`--draw_threshold` 是个可选参数. 根据 [NMS](https://ieeexplore.ieee.org/document/1699659) 的计算，
不同阈值会产生不同的结果。
## 预测部署
请参考[预测部署文档](../../deploy/README.md)。
## 模型压缩
参考[模型压缩文档](https://github.com/PaddlePaddle/PaddleDetection/tree/master/dygraph/configs/slim)。
请参考[模型压缩文档](../../configs/slim/README.md)。
docs/tutorials/QUICK_STARTED.md 0 → 100644
浏览文件 @ c73fe426
English | [简体中文](QUICK_STARTED_cn.md)
# Quick Start
In order to enable users to experience PaddleDetection and produce models in a short time, this tutorial introduces the pipeline to get a decent object detection model by finetuning on a small dataset in 10 minutes only. In practical applications, it is recommended that users select a suitable model configuration file for their specific demand.
- **Set GPU**
```bash
export CUDA_VISIBLE_DEVICES=0
```
## Inference Demo with Pre-trained Models
```
# predict an image using PP-YOLO
python tools/infer.py -c configs/ppyolo/ppyolo.yml -o use_gpu=true weights=https://paddlemodels.bj.bcebos.com/object_detection/ppyolo.pdparams --infer_img=demo/000000014439.jpg
```
the result：
![](../images/000000014439.jpg)
## Data preparation
The Dataset is [Kaggle dataset](https://www.kaggle.com/andrewmvd/road-sign-detection) ，including 877 images and 4 data categories: crosswalk, speedlimit, stop, trafficlight. The dataset is divided into training set (701 images) and test set (176 images)，[download link](https://paddlemodels.bj.bcebos.com/object_detection/roadsign_voc.tar).
```
# Note: this command could skip and
# the dataset will be dowloaded automatically at the stage of training.
python dataset/roadsign_voc/download_roadsign_voc.py
```
## Training & Evaluation & Inference
### 1、Training
```
# It will takes about 10 minutes on 1080Ti and 1 hour on CPU
# -c set configuration file
# -o overwrite the settings in the configuration file
# --eval Evaluate while training, and a model named best_model.pdmodel with the most evaluation results will be automatically saved
python tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml --eval -o use_gpu=true
```
If you want to observe the loss change curve in real time through VisualDL, add --use_vdl=true to the training command, and set the log save path through --vdl_log_dir.
**Note: VisualDL need Python>=3.5**
Please install [VisualDL](https://github.com/PaddlePaddle/VisualDL) first
```
python -m pip install visualdl -i https://mirror.baidu.com/pypi/simple
```
```
python -u tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml \
--use_vdl=true \
--vdl_log_dir=vdl_dir/scalar \
--eval
```
View the change curve in real time through the visualdl command:
```
visualdl --logdir vdl_dir/scalar/ --host <host_IP> --port <port_num>
```
### 2、Evaluation
```
# Evaluate best_model by default
# -c set config file
# -o overwrite the settings in the configuration file
python tools/eval.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml -o use_gpu=true
```
### 3、Inference
```
# -c set config file
# -o overwrite the settings in the configuration file
# --infer_img image path
# After the prediction is over, an image of the same name with the prediction result will be generated in the output folder
python tools/infer.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml -o use_gpu=true --infer_img=demo/road554.png
```
The result is as shown below：
![](../images/road554.png)
docs/tutorials/QUICK_STARTED_cn.md
浏览文件 @ c73fe426
[English](QUICK_STARTED.md) | 简体中文
# 快速开始
为了使得用户能够在很短时间内快速产出模型，掌握PaddleDetection的使用方式，这篇教程通过一个预训练检测模型对小数据集进行finetune。在较短时间内即可产出一个效果不错的模型。实际业务中，建议用户根据需要选择合适模型配置文件进行适配。
......@@ -35,7 +37,27 @@ python dataset/roadsign_voc/download_roadsign_voc.py
# -o 参数表示指定配置文件中的全局变量（覆盖配置文件中的设置），这里设置使用gpu
# --eval 参数表示边训练边评估，最后会自动保存一个名为model_final.pdparams的模型
python tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml --eval -o use_gpu=true --weight_type finetune
python tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_roadsign.yml --eval -o use_gpu=true
```
如果想通过VisualDL实时观察loss变化曲线，在训练命令中添加--use_vdl=true，以及通过--vdl_log_dir设置日志保存路径。
**但注意VisualDL需Python>=3.5**
首先安装[VisualDL](https://github.com/PaddlePaddle/VisualDL)
```
python -m pip install visualdl -i https://mirror.baidu.com/pypi/simple
```
```
python -u tools/train.py -c configs/yolov3_mobilenet_v1_roadsign.yml \
--use_vdl=true \
--vdl_log_dir=vdl_dir/scalar \
--eval
```
通过visualdl命令实时查看变化曲线：
```
visualdl --logdir vdl_dir/scalar/ --host <host_IP> --port <port_num>
```
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或