Skip to content

P
PaddleClas

PaddlePaddle / PaddleClas
大约 1 年 前同步成功

通知 115
Star 4999
Fork 1114
P
PaddleClas
提交 bcc6aa3a 编写于 作者: H HydrogenSulfate
浏览文件
操作

add re-ranking option and related description

上级 e2431804
隐藏空白更改
内联 并排
Showing with 91 addition and 14 deletion
docs/en/algorithm_introduction/reid.md
浏览文件 @ bcc6aa3a
...@@ -46,15 +46,15 @@ Early work on person re-identification mainly focused on hand-designed feature e ...@@ -46,15 +46,15 @@ Early work on person re-identification mainly focused on hand-designed feature e
| :---------- | :----: | :----: | :---: | | :---------- | :----: | :----: | :---: |
| VIPeR | 632 | 1264 | 2 | | VIPeR | 632 | 1264 | 2 |
| iLIDS | 119 | 476 | 2 | | iLIDS | 119 | 476 | 2 |
| GRID | 250 | 1275 | 2 | | GRID | 250 | 1275 | 8 |
| PRID2011 | 200 | 1134 | 2 | | PRID2011 | 200 | 1134 | 2 |
| CUHK01 | 971 | 3884 | 2 | | CUHK01 | 971 | 3884 | 2 |
| CUHK02 | 1816 | 7264 | 2 | | CUHK02 | 1816 | 7264 | 10 |
| CUHK03 | 1467 | 13164 | 2 | | CUHK03 | 1467 | 13164 | 2 |
| Market-1501 | 1501 | 32668 | 2 | | Market-1501 | 1501 | 32668 | 6 |
| DukeMTMC | 1404 | 36411 | 2 | | DukeMTMC | 1404 | 36411 | 8 |
| Airport | 39902 | 39902 | 2 | | Airport | 39902 | 39902 | 6 |
| MSMT17 | 126441 | 126441 | 2 | | MSMT17 | 126441 | 126441 | 15 |
#### 2.2 Common metric #### 2.2 Common metric
...@@ -209,6 +209,45 @@ Prepare the `*.pdparams` model parameter file for evaluation. You can use the tr ...@@ -209,6 +209,45 @@ Prepare the `*.pdparams` model parameter file for evaluation. You can use the tr
``` ```
The default evaluation log is saved in `PaddleClas/output/RecModel/eval.log`. You can see that the evaluation indicators of the `softmax_triplet_with_center_pretrained.pdparams` model provided by us on the Market1501 dataset are recall@1=0.94507, recall@5=0.98248 , mAP=0.85827 The default evaluation log is saved in `PaddleClas/output/RecModel/eval.log`. You can see that the evaluation indicators of the `softmax_triplet_with_center_pretrained.pdparams` model provided by us on the Market1501 dataset are recall@1=0.94507, recall@5=0.98248 , mAP=0.85827
- use the re-ranking option to improve the evaluation metrics
The main idea of ​​re-ranking is to use the relationship between the retrieval libraries to further optimize the retrieval results, and the k-reciprocal algorithm is widely used. Turn on re-ranking during evaluation in PaddleClas to improve the final retrieval accuracy.
This can be enabled by adding `-o Global.re_ranking=True` to the evaluation command as shown below.
```bash
python3.7 tools/eval.py \
-c ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml \
-o Global.pretrained_model="pretrained_models/softmax_triplet_with_center_pretrained" \
-o Global.re_ranking=True
```
View the output
```log
...
...
ppcls INFO: unique_endpoints {''}
ppcls INFO: Found /root/.paddleclas/weights/resnet50-19c8e357_torch2paddle.pdparams
ppcls INFO: gallery feature calculation process: [0/125]
ppcls INFO: gallery feature calculation process: [20/125]
ppcls INFO: gallery feature calculation process: [40/125]
ppcls INFO: gallery feature calculation process: [60/125]
ppcls INFO: gallery feature calculation process: [80/125]
ppcls INFO: gallery feature calculation process: [100/125]
ppcls INFO: gallery feature calculation process: [120/125]
ppcls INFO: Build gallery done, all feat shape: [15913, 2048], begin to eval..
ppcls INFO: query feature calculation process: [0/27]
ppcls INFO: query feature calculation process: [20/27]
ppcls INFO: Build query done, all feat shape: [3368, 2048], begin to eval..
ppcls INFO: re_ranking=True
ppcls WARNING: re_ranking=True, Recallk.descending has been set to False
ppcls WARNING: re_ranking=True,mAP.descending has been set to False
ppcls INFO: using GPU to compute original distance
ppcls INFO: starting re_ranking
ppcls INFO: [Eval][Epoch 0][Avg]recall1: 0.95546, recall5: 0.97743, mAP: 0.94252
```
It can be seen that after re-ranking is enabled, the evaluation indicators are recall@1=0.95546, recall@5=0.97743, and mAP=0.94252. It can be found that the algorithm improves the mAP indicator significantly (0.85827->0.94252).
**Note**: The computational complexity of re-ranking is currently high, so it is not enabled by default.
#### 4.2 Model Inference #### 4.2 Model Inference
##### 4.2.1 Inference model preparation ##### 4.2.1 Inference model preparation
......
docs/zh_CN/algorithm_introduction/reid.md
浏览文件 @ bcc6aa3a
...@@ -46,15 +46,15 @@ ...@@ -46,15 +46,15 @@
| :---------- | :----: | :----: | :---: | | :---------- | :----: | :----: | :---: |
| VIPeR | 632 | 1264 | 2 | | VIPeR | 632 | 1264 | 2 |
| iLIDS | 119 | 476 | 2 | | iLIDS | 119 | 476 | 2 |
| GRID | 250 | 1275 | 2 | | GRID | 250 | 1275 | 8 |
| PRID2011 | 200 | 1134 | 2 | | PRID2011 | 200 | 1134 | 2 |
| CUHK01 | 971 | 3884 | 2 | | CUHK01 | 971 | 3884 | 2 |
| CUHK02 | 1816 | 7264 | 2 | | CUHK02 | 1816 | 7264 | 10 |
| CUHK03 | 1467 | 13164 | 2 | | CUHK03 | 1467 | 13164 | 2 |
| Market-1501 | 1501 | 32668 | 2 | | Market-1501 | 1501 | 32668 | 6 |
| DukeMTMC | 1404 | 36411 | 2 | | DukeMTMC | 1404 | 36411 | 8 |
| Airport | 39902 | 39902 | 2 | | Airport | 39902 | 39902 | 6 |
| MSMT17 | 126441 | 126441 | 2 | | MSMT17 | 126441 | 126441 | 15 |
#### 2.2 常用指标 #### 2.2 常用指标
...@@ -95,8 +95,8 @@ ...@@ -95,8 +95,8 @@
以下表格总结了复现的ReID strong-baseline的3种配置在 Market1501 数据集上的精度指标， 以下表格总结了复现的ReID strong-baseline的3种配置在 Market1501 数据集上的精度指标，
| 配置文件 | recall@1(\%) | mAP(\%) | 参考recall@1(\%) | 参考mAP(\%) | 预训练模型下载地址 | inference模型下载地址 | | 配置文件 | recall@1(\%) | mAP(\%) | 参考recall@1(\%) | 参考mAP(\%) | 预训练模型下载地址 | inference模型下载地址 |
| -------------------------------- | ------------ | ------- | ---------------- | ----------- | ----- | ---- | | -------------------------------- | ------------ | ------- | ---------------- | ----------- | --------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| baseline.yaml | 88.45 | 74.37 | 87.7 | 74.0 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/baseline_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/inference/baseline_infer.tar) | | baseline.yaml | 88.45 | 74.37 | 87.7 | 74.0 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/baseline_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/inference/baseline_infer.tar) |
| softmax_triplet.yaml | 94.29 | 85.57 | 94.1 | 85.7 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/softmax_triplet_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/inference/softmax_triplet_infer.tar) | | softmax_triplet.yaml | 94.29 | 85.57 | 94.1 | 85.7 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/softmax_triplet_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/inference/softmax_triplet_infer.tar) |
| softmax_triplet_with_center.yaml | 94.50 | 85.82 | 94.5 | 85.9 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/softmax_triplet_with_center_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/inference/softmax_triplet_with_center_infer.tar) | | softmax_triplet_with_center.yaml | 94.50 | 85.82 | 94.5 | 85.9 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/softmax_triplet_with_center_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/inference/softmax_triplet_with_center_infer.tar) |
...@@ -209,6 +209,44 @@ ...@@ -209,6 +209,44 @@
``` ```
默认评估日志保存在`PaddleClas/output/RecModel/eval.log`中，可以看到我们提供的 `softmax_triplet_with_center_pretrained.pdparams` 模型在 Market1501 数据集上的评估指标为recall@1=0.94507，recall@5=0.98248，mAP=0.85827 默认评估日志保存在`PaddleClas/output/RecModel/eval.log`中，可以看到我们提供的 `softmax_triplet_with_center_pretrained.pdparams` 模型在 Market1501 数据集上的评估指标为recall@1=0.94507，recall@5=0.98248，mAP=0.85827
- 使用re-ranking功能提升评估精度
re-ranking的主要思想是利用检索库之间的相互关系来进一步优化检索结果，比较广泛使用的是k-reciprocal算法。在PaddleClas中在评估时开启re-ranking来提升最终的检索精度。
如下所示，在评估命令中加上 `-o Global.re_ranking=True` 即可开启该功能。
```bash
python3.7 tools/eval.py \
-c ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml \
-o Global.pretrained_model="pretrained_models/softmax_triplet_with_center_pretrained" \
-o Global.re_ranking=True
```
查看输出结果
```log
...
...
ppcls INFO: unique_endpoints {''}
ppcls INFO: Found /root/.paddleclas/weights/resnet50-19c8e357_torch2paddle.pdparams
ppcls INFO: gallery feature calculation process: [0/125]
ppcls INFO: gallery feature calculation process: [20/125]
ppcls INFO: gallery feature calculation process: [40/125]
ppcls INFO: gallery feature calculation process: [60/125]
ppcls INFO: gallery feature calculation process: [80/125]
ppcls INFO: gallery feature calculation process: [100/125]
ppcls INFO: gallery feature calculation process: [120/125]
ppcls INFO: Build gallery done, all feat shape: [15913, 2048], begin to eval..
ppcls INFO: query feature calculation process: [0/27]
ppcls INFO: query feature calculation process: [20/27]
ppcls INFO: Build query done, all feat shape: [3368, 2048], begin to eval..
ppcls INFO: re_ranking=True
ppcls WARNING: re_ranking=True,Recallk.descending has been set to False
ppcls WARNING: re_ranking=True,mAP.descending has been set to False
ppcls INFO: using GPU to compute original distance
ppcls INFO: starting re_ranking
ppcls INFO: [Eval][Epoch 0][Avg]recall1: 0.95546, recall5: 0.97743, mAP: 0.94252
```
可以看到开启re-ranking后，评估指标为recall@1=0.95546，recall@5=0.97743，mAP=0.94252，可以发现该算法对mAP指标的提升比较明显(0.85827->0.94252)。
**注**：目前re-ranking的计算复杂度较高，因此默认不启用。
#### 4.2 模型推理 #### 4.2 模型推理
##### 4.2.1 推理模型准备 ##### 4.2.1 推理模型准备
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或