Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
PaddlePaddle
PaddleOCR
提交
d40302e6
P
PaddleOCR
项目概览
PaddlePaddle
/
PaddleOCR
大约 1 年 前同步成功
通知
1528
Star
32962
Fork
6643
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
108
列表
看板
标记
里程碑
合并请求
7
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
P
PaddleOCR
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
108
Issue
108
列表
看板
标记
里程碑
合并请求
7
合并请求
7
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
未验证
提交
d40302e6
编写于
7月 27, 2021
作者:
u010070587
提交者:
GitHub
7月 27, 2021
浏览文件
操作
浏览文件
下载
差异文件
Merge pull request #3434 from LDOUBLEV/fix_mem_leak
Fix mem leak
上级
e50f823d
a9bac1ce
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
13 addition
and
22 deletion
+13
-22
ppocr/data/simple_dataset.py
ppocr/data/simple_dataset.py
+12
-21
ppocr/modeling/architectures/distillation_model.py
ppocr/modeling/architectures/distillation_model.py
+1
-1
未找到文件。
ppocr/data/simple_dataset.py
浏览文件 @
d40302e6
...
...
@@ -14,7 +14,6 @@
import
numpy
as
np
import
os
import
random
import
traceback
from
paddle.io
import
Dataset
from
.imaug
import
transform
,
create_operators
...
...
@@ -46,7 +45,6 @@ class SimpleDataSet(Dataset):
self
.
seed
=
seed
logger
.
info
(
"Initialize indexs of datasets:%s"
%
label_file_list
)
self
.
data_lines
=
self
.
get_image_info_list
(
label_file_list
,
ratio_list
)
self
.
check_data
()
self
.
data_idx_order_list
=
list
(
range
(
len
(
self
.
data_lines
)))
if
self
.
mode
==
"train"
and
self
.
do_shuffle
:
self
.
shuffle_data_random
()
...
...
@@ -103,18 +101,25 @@ class SimpleDataSet(Dataset):
def
__getitem__
(
self
,
idx
):
file_idx
=
self
.
data_idx_order_list
[
idx
]
data
=
self
.
data_lines
[
file_idx
]
data
_line
=
self
.
data_lines
[
file_idx
]
try
:
data_line
=
data_line
.
decode
(
'utf-8'
)
substr
=
data_line
.
strip
(
"
\n
"
).
split
(
self
.
delimiter
)
file_name
=
substr
[
0
]
label
=
substr
[
1
]
img_path
=
os
.
path
.
join
(
self
.
data_dir
,
file_name
)
data
=
{
'img_path'
:
img_path
,
'label'
:
label
}
if
not
os
.
path
.
exists
(
img_path
):
raise
Exception
(
"{} does not exist!"
.
format
(
img_path
))
with
open
(
data
[
'img_path'
],
'rb'
)
as
f
:
img
=
f
.
read
()
data
[
'image'
]
=
img
data
[
'ext_data'
]
=
self
.
get_ext_data
()
outs
=
transform
(
data
,
self
.
ops
)
except
:
error_meg
=
traceback
.
format_exc
()
except
Exception
as
e
:
self
.
logger
.
error
(
"When parsing
file {} and label
{}, error happened with msg: {}"
.
format
(
data
[
'img_path'
],
data
[
'label'
],
error_meg
))
"When parsing
line
{}, error happened with msg: {}"
.
format
(
data
_line
,
e
))
outs
=
None
if
outs
is
None
:
# during evaluation, we should fix the idx to get same results for many times of evaluation.
...
...
@@ -125,17 +130,3 @@ class SimpleDataSet(Dataset):
def
__len__
(
self
):
return
len
(
self
.
data_idx_order_list
)
def
check_data
(
self
):
new_data_lines
=
[]
for
data_line
in
self
.
data_lines
:
data_line
=
data_line
.
decode
(
'utf-8'
)
substr
=
data_line
.
strip
(
"
\n
"
).
strip
(
"
\r
"
).
split
(
self
.
delimiter
)
file_name
=
substr
[
0
]
label
=
substr
[
1
]
img_path
=
os
.
path
.
join
(
self
.
data_dir
,
file_name
)
if
os
.
path
.
exists
(
img_path
):
new_data_lines
.
append
({
'img_path'
:
img_path
,
'label'
:
label
})
else
:
self
.
logger
.
info
(
"{} does not exist!"
.
format
(
img_path
))
self
.
data_lines
=
new_data_lines
\ No newline at end of file
ppocr/modeling/architectures/distillation_model.py
浏览文件 @
d40302e6
...
...
@@ -46,7 +46,7 @@ class DistillationModel(nn.Layer):
pretrained
=
model_config
.
pop
(
"pretrained"
)
model
=
BaseModel
(
model_config
)
if
pretrained
is
not
None
:
model
=
load_pretrained_params
(
model
,
pretrained
)
load_pretrained_params
(
model
,
pretrained
)
if
freeze_params
:
for
param
in
model
.
parameters
():
param
.
trainable
=
False
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录