Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
BaiXuePrincess
Paddle
提交
6fc33a0c
P
Paddle
项目概览
BaiXuePrincess
/
Paddle
与 Fork 源项目一致
Fork自
PaddlePaddle / Paddle
通知
1
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
P
Paddle
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
未验证
提交
6fc33a0c
编写于
7月 21, 2021
作者:
K
Kaipeng Deng
提交者:
GitHub
7月 21, 2021
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
fix DataLoader memory leak. test=develop (#34140)
上级
9b63e7f2
变更
1
隐藏空白更改
内联
并排
Showing
1 changed file
with
1 addition
and
19 deletion
+1
-19
python/paddle/fluid/dataloader/dataloader_iter.py
python/paddle/fluid/dataloader/dataloader_iter.py
+1
-19
未找到文件。
python/paddle/fluid/dataloader/dataloader_iter.py
浏览文件 @
6fc33a0c
...
@@ -122,13 +122,6 @@ class _DataLoaderIterSingleProcess(_DataLoaderIterBase):
...
@@ -122,13 +122,6 @@ class _DataLoaderIterSingleProcess(_DataLoaderIterBase):
self
.
_init_thread
()
self
.
_init_thread
()
# if user exit python program when dataloader is still
# iterating, resource may no release safely, so we
# add __del__ function to to CleanupFuncRegistrar
# to make sure __del__ is always called when program
# exit for resoure releasing safely
CleanupFuncRegistrar
.
register
(
self
.
__del__
)
def
_init_thread
(
self
):
def
_init_thread
(
self
):
self
.
_var_names
=
[
v
.
name
for
v
in
self
.
_feed_list
]
self
.
_var_names
=
[
v
.
name
for
v
in
self
.
_feed_list
]
self
.
_shapes
=
[
v
.
shape
for
v
in
self
.
_feed_list
]
self
.
_shapes
=
[
v
.
shape
for
v
in
self
.
_feed_list
]
...
@@ -228,7 +221,7 @@ class _DataLoaderIterSingleProcess(_DataLoaderIterBase):
...
@@ -228,7 +221,7 @@ class _DataLoaderIterSingleProcess(_DataLoaderIterBase):
self
.
_thread_done_event
.
set
()
self
.
_thread_done_event
.
set
()
if
self
.
_thread
is
not
threading
.
current_thread
():
if
self
.
_thread
is
not
threading
.
current_thread
():
self
.
_thread
.
join
()
self
.
_thread
.
join
()
self
.
_thread
=
None
self
.
_thread
=
None
# python2 compatibility
# python2 compatibility
def
next
(
self
):
def
next
(
self
):
...
@@ -284,17 +277,6 @@ class _DataLoaderIterMultiProcess(_DataLoaderIterBase):
...
@@ -284,17 +277,6 @@ class _DataLoaderIterMultiProcess(_DataLoaderIterBase):
self
.
_init_thread
()
self
.
_init_thread
()
self
.
_shutdown
=
False
self
.
_shutdown
=
False
# if user exit python program when dataloader is still
# iterating, resource may no release safely, so we
# add _shutdown_on_exit function to to CleanupFuncRegistrar
# to make sure _try_shutdown_all is always called when program
# exit for resoure releasing safely
# worker join may hang for in _try_shutdown_all call in atexit
# for main process is in atexit state in some OS, so we add
# timeout=1 for shutdown function call in atexit, for shutdown
# function call in __del__, we keep it as it is
CleanupFuncRegistrar
.
register
(
self
.
_shutdown_on_exit
)
def
_init_workers
(
self
):
def
_init_workers
(
self
):
# multiprocess worker and indice queue list initial as empty
# multiprocess worker and indice queue list initial as empty
self
.
_workers
=
[]
self
.
_workers
=
[]
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录