Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
s920243400
PaddleOCR
提交
f6167acc
P
PaddleOCR
项目概览
s920243400
/
PaddleOCR
与 Fork 源项目一致
Fork自
PaddlePaddle / PaddleOCR
通知
1
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
P
PaddleOCR
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
提交
f6167acc
编写于
7月 14, 2021
作者:
L
LDOUBLEV
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
add test_v7
上级
9d5d552b
变更
3
隐藏空白更改
内联
并排
Showing
3 changed file
with
407 addition
and
0 deletion
+407
-0
tests/params.txt
tests/params.txt
+48
-0
tests/prepare.sh
tests/prepare.sh
+77
-0
tests/test.sh
tests/test.sh
+282
-0
未找到文件。
tests/params.txt
0 → 100644
浏览文件 @
f6167acc
===========================train_params===========================
model_name:ocr_det
python:python3.7
gpu_list:0|0,1
Global.auto_cast:null
Global.epoch_num:2
Global.save_model_dir:./output/
Train.loader.batch_size_per_card:2
Global.use_gpu:
Global.pretrained_model:null
train_model_name:latest
train_infer_img_dir:./train_data/icdar2015/text_localization/ch4_test_images/
null:null
##
trainer:norm_train|pact_train
norm_train:tools/train.py -c configs/det/det_mv3_db.yml -o Global.pretrained_model=./pretrain_models/MobileNetV3_large_x0_5_pretrained
pact_train:deploy/slim/quantization/quant.py -c configs/det/det_mv3_db.yml -o Global.pretrained_model=./pretrain_models/det_mv3_db_v2.0_train/best_accuracy
fpgm_train:null
distill_train:null
null:null
null:null
##
===========================eval_params===========================
eval:null
null:null
##
===========================infer_params===========================
Global.save_inference_dir:./output/
Global.pretrained_model:
norm_export:tools/export_model.py -c configs/det/det_mv3_db.yml -o
quant_export:deploy/slim/quantization/export_model.py -c configs/det/det_mv3_db.yml -o
fpgm_export:deploy/slim/prune/export_prune_model.py
distill_export:null
null:null
null:null
##
inference:tools/infer/predict_det.py
--use_gpu:True|False
--enable_mkldnn:True|False
--cpu_threads:1|6
--rec_batch_num:1
--use_tensorrt:True|False
--precision:fp32|fp16|int8
--det_model_dir:./inference/ch_ppocr_mobile_v2.0_det_infer/
--image_dir:./inference/ch_det_data_50/all-sum-510/
--save_log_path:null
--benchmark:True
null:null
tests/prepare.sh
0 → 100644
浏览文件 @
f6167acc
#!/bin/bash
FILENAME
=
$1
# MODE be one of ['lite_train_infer' 'whole_infer' 'whole_train_infer', 'infer']
MODE
=
$2
dataline
=
$(
cat
${
FILENAME
}
)
# parser params
IFS
=
$'
\n
'
lines
=(
${
dataline
}
)
function
func_parser_key
(){
strs
=
$1
IFS
=
":"
array
=(
${
strs
}
)
tmp
=
${
array
[0]
}
echo
${
tmp
}
}
function
func_parser_value
(){
strs
=
$1
IFS
=
":"
array
=(
${
strs
}
)
tmp
=
${
array
[1]
}
echo
${
tmp
}
}
IFS
=
$'
\n
'
# The training params
model_name
=
$(
func_parser_value
"
${
lines
[0]
}
"
)
train_model_list
=
$(
func_parser_value
"
${
lines
[0]
}
"
)
trainer_list
=
$(
func_parser_value
"
${
lines
[10]
}
"
)
# MODE be one of ['lite_train_infer' 'whole_infer' 'whole_train_infer']
MODE
=
$2
# prepare pretrained weights and dataset
wget
-nc
-P
./pretrain_models/ https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV3_large_x0_5_pretrained.pdparams
wget
-nc
-P
./pretrain_models/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_mv3_db_v2.0_train.tar
cd
pretrain_models
&&
tar
xf det_mv3_db_v2.0_train.tar
&&
cd
../
if
[
${
MODE
}
=
"lite_train_infer"
]
;
then
# pretrain lite train data
rm
-rf
./train_data/icdar2015
wget
-nc
-P
./train_data/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/icdar2015_lite.tar
cd
./train_data/
&&
tar
xf icdar2015_lite.tar
ln
-s
./icdar2015_lite ./icdar2015
cd
../
epoch
=
10
eval_batch_step
=
10
elif
[
${
MODE
}
=
"whole_train_infer"
]
;
then
rm
-rf
./train_data/icdar2015
wget
-nc
-P
./train_data/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/icdar2015.tar
cd
./train_data/
&&
tar
xf icdar2015.tar
&&
cd
../
epoch
=
500
eval_batch_step
=
200
elif
[
${
MODE
}
=
"whole_infer"
]
;
then
rm
-rf
./train_data/icdar2015
wget
-nc
-P
./train_data/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/icdar2015_infer.tar
cd
./train_data/
&&
tar
xf icdar2015_infer.tar
ln
-s
./icdar2015_infer ./icdar2015
cd
../
epoch
=
10
eval_batch_step
=
10
else
rm
-rf
./train_data/icdar2015
wget
-nc
-P
./train_data https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ch_det_data_50.tar
if
[
${
model_name
}
=
"ocr_det"
]
;
then
eval_model_name
=
"ch_ppocr_mobile_v2.0_det_infer"
wget
-nc
-P
./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_infer.tar
cd
./inference
&&
tar
xf
${
eval_model_name
}
.tar
&&
cd
../
else
eval_model_name
=
"ch_ppocr_mobile_v2.0_rec_train"
wget
-nc
-P
./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_rec_train.tar
cd
./inference
&&
tar
xf
${
eval_model_name
}
.tar
&&
cd
../
fi
fi
tests/test.sh
0 → 100644
浏览文件 @
f6167acc
#!/bin/bash
FILENAME
=
$1
# MODE be one of ['lite_train_infer' 'whole_infer' 'whole_train_infer', 'infer']
MODE
=
$2
dataline
=
$(
cat
${
FILENAME
}
)
# parser params
IFS
=
$'
\n
'
lines
=(
${
dataline
}
)
function
func_parser_key
(){
strs
=
$1
IFS
=
":"
array
=(
${
strs
}
)
tmp
=
${
array
[0]
}
echo
${
tmp
}
}
function
func_parser_value
(){
strs
=
$1
IFS
=
":"
array
=(
${
strs
}
)
tmp
=
${
array
[1]
}
echo
${
tmp
}
}
function
func_set_params
(){
key
=
$1
value
=
$2
if
[
${
key
}
=
"null"
]
;
then
echo
" "
elif
[[
${
value
}
=
"null"
]]
||
[[
${
value
}
=
" "
]]
||
[
${#
value
}
-le
0
]
;
then
echo
" "
else
echo
"
${
key
}
=
${
value
}
"
fi
}
function
status_check
(){
last_status
=
$1
# the exit code
run_command
=
$2
run_log
=
$3
if
[
$last_status
-eq
0
]
;
then
echo
-e
"
\0
33[33m Run successfully with command -
${
run_command
}
!
\0
33[0m"
|
tee
-a
${
run_log
}
else
echo
-e
"
\0
33[33m Run failed with command -
${
run_command
}
!
\0
33[0m"
|
tee
-a
${
run_log
}
fi
}
IFS
=
$'
\n
'
# The training params
model_name
=
$(
func_parser_value
"
${
lines
[1]
}
"
)
python
=
$(
func_parser_value
"
${
lines
[2]
}
"
)
gpu_list
=
$(
func_parser_value
"
${
lines
[3]
}
"
)
autocast_list
=
$(
func_parser_value
"
${
lines
[4]
}
"
)
autocast_key
=
$(
func_parser_key
"
${
lines
[4]
}
"
)
epoch_key
=
$(
func_parser_key
"
${
lines
[5]
}
"
)
epoch_num
=
$(
func_parser_value
"
${
lines
[5]
}
"
)
save_model_key
=
$(
func_parser_key
"
${
lines
[6]
}
"
)
train_batch_key
=
$(
func_parser_key
"
${
lines
[7]
}
"
)
train_batch_value
=
$(
func_parser_value
"
${
lines
[7]
}
"
)
train_use_gpu_key
=
$(
func_parser_key
"
${
lines
[8]
}
"
)
pretrain_model_key
=
$(
func_parser_key
"
${
lines
[9]
}
"
)
pretrain_model_value
=
$(
func_parser_value
"
${
lines
[9]
}
"
)
train_model_name
=
$(
func_parser_value
"
${
lines
[10]
}
"
)
train_infer_img_dir
=
$(
func_parser_value
"
${
lines
[11]
}
"
)
train_param_key1
=
$(
func_parser_key
"
${
lines
[12]
}
"
)
train_param_value1
=
$(
func_parser_value
"
${
lines
[12]
}
"
)
trainer_list
=
$(
func_parser_value
"
${
lines
[14]
}
"
)
trainer_norm
=
$(
func_parser_key
"
${
lines
[15]
}
"
)
norm_trainer
=
$(
func_parser_value
"
${
lines
[15]
}
"
)
pact_key
=
$(
func_parser_key
"
${
lines
[16]
}
"
)
pact_trainer
=
$(
func_parser_value
"
${
lines
[16]
}
"
)
fpgm_key
=
$(
func_parser_key
"
${
lines
[17]
}
"
)
fpgm_trainer
=
$(
func_parser_value
"
${
lines
[17]
}
"
)
distill_key
=
$(
func_parser_key
"
${
lines
[18]
}
"
)
distill_trainer
=
$(
func_parser_value
"
${
lines
[18]
}
"
)
trainer_key1
=
$(
func_parser_key
"
${
lines
[19]
}
"
)
trainer_value1
=
$(
func_parser_value
"
${
lines
[19]
}
"
)
trainer_key1
=
$(
func_parser_key
"
${
lines
[20]
}
"
)
trainer_value2
=
$(
func_parser_value
"
${
lines
[20]
}
"
)
eval_py
=
$(
func_parser_value
"
${
lines
[23]
}
"
)
eval_key1
=
$(
func_parser_key
"
${
lines
[24]
}
"
)
eval_value1
=
$(
func_parser_value
"
${
lines
[24]
}
"
)
save_infer_key
=
$(
func_parser_key
"
${
lines
[27]
}
"
)
export_weight
=
$(
func_parser_key
"
${
lines
[28]
}
"
)
norm_export
=
$(
func_parser_value
"
${
lines
[29]
}
"
)
pact_export
=
$(
func_parser_value
"
${
lines
[30]
}
"
)
fpgm_export
=
$(
func_parser_value
"
${
lines
[31]
}
"
)
distill_export
=
$(
func_parser_value
"
${
lines
[32]
}
"
)
export_key1
=
$(
func_parser_key
"
${
lines
[33]
}
"
)
export_value1
=
$(
func_parser_value
"
${
lines
[33]
}
"
)
export_key2
=
$(
func_parser_key
"
${
lines
[34]
}
"
)
export_value2
=
$(
func_parser_value
"
${
lines
[34]
}
"
)
inference_py
=
$(
func_parser_value
"
${
lines
[36]
}
"
)
use_gpu_key
=
$(
func_parser_key
"
${
lines
[37]
}
"
)
use_gpu_list
=
$(
func_parser_value
"
${
lines
[37]
}
"
)
use_mkldnn_key
=
$(
func_parser_key
"
${
lines
[38]
}
"
)
use_mkldnn_list
=
$(
func_parser_value
"
${
lines
[38]
}
"
)
cpu_threads_key
=
$(
func_parser_key
"
${
lines
[39]
}
"
)
cpu_threads_list
=
$(
func_parser_value
"
${
lines
[39]
}
"
)
batch_size_key
=
$(
func_parser_key
"
${
lines
[40]
}
"
)
batch_size_list
=
$(
func_parser_value
"
${
lines
[40]
}
"
)
use_trt_key
=
$(
func_parser_key
"
${
lines
[41]
}
"
)
use_trt_list
=
$(
func_parser_value
"
${
lines
[41]
}
"
)
precision_key
=
$(
func_parser_key
"
${
lines
[42]
}
"
)
precision_list
=
$(
func_parser_value
"
${
lines
[42]
}
"
)
infer_model_key
=
$(
func_parser_key
"
${
lines
[43]
}
"
)
infer_model
=
$(
func_parser_value
"
${
lines
[43]
}
"
)
image_dir_key
=
$(
func_parser_key
"
${
lines
[44]
}
"
)
infer_img_dir
=
$(
func_parser_value
"
${
lines
[44]
}
"
)
save_log_key
=
$(
func_parser_key
"
${
lines
[45]
}
"
)
benchmark_key
=
$(
func_parser_key
"
${
lines
[46]
}
"
)
benchmark_value
=
$(
func_parser_value
"
${
lines
[46]
}
"
)
infer_key2
=
$(
func_parser_key
"
${
lines
[47]
}
"
)
infer_value2
=
$(
func_parser_value
"
${
lines
[47]
}
"
)
LOG_PATH
=
"./tests/output"
mkdir
-p
${
LOG_PATH
}
status_log
=
"
${
LOG_PATH
}
/results.log"
function
func_inference
(){
IFS
=
'|'
_python
=
$1
_script
=
$2
_model_dir
=
$3
_log_path
=
$4
_img_dir
=
$5
_flag_quant
=
$6
# inference
for
use_gpu
in
${
use_gpu_list
[*]
}
;
do
if
[
${
use_gpu
}
=
"False"
]
&&
[
${
_flag_quant
}
=
"True"
]
;
then
continue
fi
if
[
${
use_gpu
}
=
"False"
]
;
then
for
use_mkldnn
in
${
use_mkldnn_list
[*]
}
;
do
for
threads
in
${
cpu_threads_list
[*]
}
;
do
for
batch_size
in
${
batch_size_list
[*]
}
;
do
_save_log_path
=
"
${
_log_path
}
/infer_cpu_usemkldnn_
${
use_mkldnn
}
_threads_
${
threads
}
_batchsize_
${
batch_size
}
.log"
#${image_dir_key}=${_img_dir} ${save_log_key}=${_save_log_path} --benchmark=True
set_infer_data
=
$(
func_set_params
"
${
image_dir_key
}
"
"
${
_img_dir
}
"
)
set_benchmark
=
$(
func_set_params
"
${
benchmark_key
}
"
"
${
benchmark_value
}
"
)
command
=
"
${
_python
}
${
_script
}
${
use_gpu_key
}
=
${
use_gpu
}
${
use_mkldnn_key
}
=
${
use_mkldnn
}
${
cpu_threads_key
}
=
${
threads
}
${
infer_model_key
}
=
${
_model_dir
}
${
batch_size_key
}
=
${
batch_size
}
${
set_infer_data
}
${
set_benchmark
}
>
${
_save_log_path
}
2>&1 "
eval
$command
status_check
$?
"
${
command
}
"
"
${
status_log
}
"
done
done
done
else
for
use_trt
in
${
use_trt_list
[*]
}
;
do
for
precision
in
${
precision_list
[*]
}
;
do
if
[
${
use_trt
}
=
"False"
]
&&
[
${
precision
}
!=
"fp32"
]
;
then
continue
fi
if
[
${
use_trt
}
=
"False"
]
&&
[
${
_flag_quant
}
=
"True"
]
;
then
continue
fi
if
[
${
precision
}
!=
"int8"
]
&&
[
${
_flag_quant
}
=
"True"
]
;
then
continue
fi
for
batch_size
in
${
batch_size_list
[*]
}
;
do
_save_log_path
=
"
${
_log_path
}
/infer_gpu_usetrt_
${
use_trt
}
_precision_
${
precision
}
_batchsize_
${
batch_size
}
.log"
set_infer_data
=
$(
func_set_params
"
${
image_dir_key
}
"
"
${
_img_dir
}
"
)
set_benchmark
=
$(
func_set_params
"
${
benchmark_key
}
"
"
${
benchmark_value
}
"
)
command
=
"
${
_python
}
${
_script
}
${
use_gpu_key
}
=
${
use_gpu
}
${
use_trt_key
}
=
${
use_trt
}
${
precision_key
}
=
${
precision
}
${
infer_model_key
}
=
${
_model_dir
}
${
batch_size_key
}
=
${
batch_size
}
${
set_infer_data
}
${
set_benchmark
}
>
${
_save_log_path
}
2>&1 "
eval
$command
status_check
$?
"
${
command
}
"
"
${
status_log
}
"
done
done
done
fi
done
}
if
[
${
MODE
}
!=
"infer"
]
;
then
IFS
=
"|"
for
gpu
in
${
gpu_list
[*]
}
;
do
use_gpu
=
True
if
[
${
gpu
}
=
"-1"
]
;
then
use_gpu
=
False
env
=
""
elif
[
${#
gpu
}
-le
1
]
;
then
env
=
"export CUDA_VISIBLE_DEVICES=
${
gpu
}
"
eval
${
env
}
elif
[
${#
gpu
}
-le
15
]
;
then
IFS
=
","
array
=(
${
gpu
}
)
env
=
"export CUDA_VISIBLE_DEVICES=
${
array
[0]
}
"
IFS
=
"|"
else
IFS
=
";"
array
=(
${
gpu
}
)
ips
=
${
array
[0]
}
gpu
=
${
array
[1]
}
IFS
=
"|"
env
=
" "
fi
for
autocast
in
${
autocast_list
[*]
}
;
do
for
trainer
in
${
trainer_list
[*]
}
;
do
flag_quant
=
False
if
[
${
trainer
}
=
${
pact_key
}
]
;
then
run_train
=
${
pact_trainer
}
run_export
=
${
pact_export
}
flag_quant
=
True
elif
[
${
trainer
}
=
"
${
fpgm_key
}
"
]
;
then
run_train
=
${
fpgm_trainer
}
run_export
=
${
fpgm_export
}
elif
[
${
trainer
}
=
"
${
distill_key
}
"
]
;
then
run_train
=
${
distill_trainer
}
run_export
=
${
distill_export
}
elif
[
${
trainer
}
=
${
trainer_key1
}
]
;
then
run_train
=
${
trainer_value1
}
run_export
=
${
export_value1
}
elif
[[
${
trainer
}
=
${
trainer_key2
}
]]
;
then
run_train
=
${
trainer_value2
}
run_export
=
${
export_value2
}
else
run_train
=
${
norm_trainer
}
run_export
=
${
norm_export
}
fi
if
[
${
run_train
}
=
"null"
]
;
then
continue
fi
set_autocast
=
$(
func_set_params
"
${
autocast_key
}
"
"
${
autocast
}
"
)
set_autocast
=
$(
func_set_params
"
${
epoch_key
}
"
"
${
epoch_num
}
"
)
set_pretrain
=
$(
func_set_params
"
${
pretrain_model_key
}
"
"
${
pretrain_model_value
}
"
)
set_batchsize
=
$(
func_set_params
"
${
train_batch_key
}
"
"
${
train_batch_value
}
"
)
set_train_params1
=
$(
func_set_params
"
${
train_param_key1
}
"
"
${
train_param_value1
}
"
)
save_log
=
"
${
LOG_PATH
}
/
${
trainer
}
_gpus_
${
gpu
}
_autocast_
${
autocast
}
"
if
[
${#
gpu
}
-le
2
]
;
then
# train with cpu or single gpu
cmd
=
"
${
python
}
${
run_train
}
${
train_use_gpu_key
}
=
${
use_gpu
}
${
save_model_key
}
=
${
save_log
}
${
set_epoch
}
${
set_pretrain
}
${
set_autocast
}
${
set_batchsize
}
${
set_train_params1
}
"
elif
[
${#
gpu
}
-le
15
]
;
then
# train with multi-gpu
cmd
=
"
${
python
}
-m paddle.distributed.launch --gpus=
${
gpu
}
${
run_train
}
${
save_model_key
}
=
${
save_log
}
${
set_epoch
}
${
set_pretrain
}
${
set_autocast
}
${
set_batchsize
}
${
set_train_params1
}
"
else
# train with multi-machine
cmd
=
"
${
python
}
-m paddle.distributed.launch --ips=
${
ips
}
--gpus=
${
gpu
}
${
run_train
}
${
save_model_key
}
=
${
save_log
}
${
set_pretrain
}
${
set_epoch
}
${
set_autocast
}
${
set_batchsize
}
${
set_train_params1
}
"
fi
# run train
eval
$cmd
status_check
$?
"
${
cmd
}
"
"
${
status_log
}
"
# run eval
if
[
${
eval_py
}
!=
"null"
]
;
then
eval_cmd
=
"
${
python
}
${
eval_py
}
${
save_model_key
}
=
${
save_log
}
${
pretrain_model_key
}
=
${
save_log
}
/
${
train_model_name
}
"
eval
$eval_cmd
status_check
$?
"
${
eval_cmd
}
"
"
${
status_log
}
"
fi
if
[
${
run_export
}
!=
"null"
]
;
then
# run export model
save_infer_path
=
"
${
save_log
}
"
export_cmd
=
"
${
python
}
${
run_export
}
${
save_model_key
}
=
${
save_log
}
${
export_weight
}
=
${
save_log
}
/
${
train_model_name
}
${
save_infer_key
}
=
${
save_infer_path
}
"
eval
$export_cmd
status_check
$?
"
${
export_cmd
}
"
"
${
status_log
}
"
#run inference
eval
$env
save_infer_path
=
"
${
save_log
}
"
func_inference
"
${
python
}
"
"
${
inference_py
}
"
"
${
save_infer_path
}
"
"
${
LOG_PATH
}
"
"
${
train_infer_img_dir
}
"
"
${
flag_quant
}
"
eval
"unset CUDA_VISIBLE_DEVICES"
fi
done
done
done
else
GPUID
=
$3
if
[
${#
GPUID
}
-le
0
]
;
then
env
=
" "
else
env
=
"export CUDA_VISIBLE_DEVICES=
${
GPUID
}
"
fi
echo
$env
#run inference
func_inference
"
${
python
}
"
"
${
inference_py
}
"
"
${
infer_model
}
"
"
${
LOG_PATH
}
"
"
${
infer_img_dir
}
"
"False"
fi
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录