Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
PaddlePaddle
PaddleHub
提交
4dc50007
P
PaddleHub
项目概览
PaddlePaddle
/
PaddleHub
大约 2 年 前同步成功
通知
285
Star
12117
Fork
2091
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
200
列表
看板
标记
里程碑
合并请求
4
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
P
PaddleHub
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
200
Issue
200
列表
看板
标记
里程碑
合并请求
4
合并请求
4
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
提交
4dc50007
编写于
6月 07, 2021
作者:
W
wuzewu
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Fixed the issue that the model does not support training in multi-card scene.
上级
595f4534
变更
1
隐藏空白更改
内联
并排
Showing
1 changed file
with
10 addition
and
3 deletion
+10
-3
paddlehub/finetune/trainer.py
paddlehub/finetune/trainer.py
+10
-3
未找到文件。
paddlehub/finetune/trainer.py
浏览文件 @
4dc50007
...
@@ -18,8 +18,8 @@ import time
...
@@ -18,8 +18,8 @@ import time
from
collections
import
defaultdict
from
collections
import
defaultdict
from
typing
import
Any
,
Callable
,
Generic
,
List
from
typing
import
Any
,
Callable
,
Generic
,
List
import
numpy
as
np
import
paddle
import
paddle
import
numpy
as
np
from
visualdl
import
LogWriter
from
visualdl
import
LogWriter
from
paddlehub.utils.log
import
logger
from
paddlehub.utils.log
import
logger
...
@@ -82,6 +82,7 @@ class Trainer(object):
...
@@ -82,6 +82,7 @@ class Trainer(object):
if
self
.
nranks
>
1
:
if
self
.
nranks
>
1
:
paddle
.
distributed
.
init_parallel_env
()
paddle
.
distributed
.
init_parallel_env
()
self
.
model
=
paddle
.
DataParallel
(
self
.
model
)
self
.
model
=
paddle
.
DataParallel
(
self
.
model
)
self
.
compare_metrics
=
self
.
_compare_metrics
if
not
compare_metrics
else
compare_metrics
self
.
compare_metrics
=
self
.
_compare_metrics
if
not
compare_metrics
else
compare_metrics
self
.
_load_checkpoint
()
self
.
_load_checkpoint
()
...
@@ -178,8 +179,14 @@ class Trainer(object):
...
@@ -178,8 +179,14 @@ class Trainer(object):
collate_fn(callable): function to generate mini-batch data by merging the sample list.
collate_fn(callable): function to generate mini-batch data by merging the sample list.
None for only stack each fields of sample in axis 0(same as :attr::`np.stack(..., axis=0)`). Default None
None for only stack each fields of sample in axis 0(same as :attr::`np.stack(..., axis=0)`). Default None
'''
'''
if
eval_dataset
is
not
None
and
not
hasattr
(
self
.
model
,
'validation_step'
):
if
eval_dataset
is
not
None
:
raise
NotImplementedError
(
'The specified finetuning model does not support evaluation.'
)
if
isinstance
(
self
.
model
,
paddle
.
DataParallel
):
model
=
self
.
model
.
_layers
else
:
model
=
self
.
model
if
not
hasattr
(
model
,
'validation_step'
):
raise
NotImplementedError
(
'The specified finetuning model does not support evaluation.'
)
batch_sampler
=
paddle
.
io
.
DistributedBatchSampler
(
batch_sampler
=
paddle
.
io
.
DistributedBatchSampler
(
train_dataset
,
batch_size
=
batch_size
,
shuffle
=
True
,
drop_last
=
False
)
train_dataset
,
batch_size
=
batch_size
,
shuffle
=
True
,
drop_last
=
False
)
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录