Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
曾经的那一瞬间
Models
提交
913640d4
M
Models
项目概览
曾经的那一瞬间
/
Models
大约 1 年 前同步成功
通知
1
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
M
Models
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
913640d4
编写于
12月 16, 2019
作者:
H
Hongkun Yu
提交者:
A. Unique TensorFlower
12月 16, 2019
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Internal change
PiperOrigin-RevId: 285765110
上级
722d9e57
变更
1
隐藏空白更改
内联
并排
Showing
1 changed file
with
20 addition
and
14 deletion
+20
-14
official/transformer/model/beam_search.py
official/transformer/model/beam_search.py
+20
-14
未找到文件。
official/transformer/model/beam_search.py
浏览文件 @
913640d4
...
...
@@ -323,13 +323,16 @@ class SequenceBeamSearch(object):
new state dictionary.
"""
# Grow alive sequences by one token.
new_seq
,
new_log_probs
,
new_cache
=
self
.
_grow_alive_seq
(
state
)
new_seq
,
new_log_probs
,
topk_ids
,
new_cache
=
self
.
_grow_alive_seq
(
state
)
new_finished_flags
=
tf
.
equal
(
topk_ids
,
self
.
eos_id
)
# Collect top beam_size alive sequences
alive_state
=
self
.
_get_new_alive_state
(
new_seq
,
new_log_probs
,
new_cache
)
alive_state
=
self
.
_get_new_alive_state
(
new_seq
,
new_log_probs
,
new_finished_flags
,
new_cache
)
# Combine newly finished sequences with existing finished sequences, and
# collect the top k scoring sequences.
finished_state
=
self
.
_get_new_finished_state
(
state
,
new_seq
,
new_log_probs
)
finished_state
=
self
.
_get_new_finished_state
(
state
,
new_seq
,
new_log_probs
,
new_finished_flags
)
# Increment loop index and create new state dictionary
new_state
=
{
_StateKeys
.
CUR_INDEX
:
state
[
_StateKeys
.
CUR_INDEX
]
+
1
}
...
...
@@ -407,18 +410,20 @@ class SequenceBeamSearch(object):
tf
.
expand_dims
(
topk_ids
,
axis
=
0
))
topk_seq
=
tf
.
transpose
(
topk_seq
,
perm
=
[
1
,
2
,
0
])
else
:
topk_ids
=
tf
.
expand_dims
(
topk_ids
,
axis
=
2
)
topk_seq
=
tf
.
concat
([
topk_seq
,
topk_ids
],
axis
=
2
)
return
topk_seq
,
topk_log_probs
,
new_cache
topk_seq
=
tf
.
concat
([
topk_seq
,
tf
.
expand_dims
(
topk_ids
,
axis
=
2
)],
axis
=
2
)
return
topk_seq
,
topk_log_probs
,
topk_ids
,
new_cache
def
_get_new_alive_state
(
self
,
new_seq
,
new_log_probs
,
new_cache
):
def
_get_new_alive_state
(
self
,
new_seq
,
new_log_probs
,
new_finished_flags
,
new_cache
):
"""Gather the top k sequences that are still alive.
Args:
new_seq: New sequences generated by growing the current alive sequences
int32 tensor with shape [batch_size, 2 * beam_size, cur_index + 1]
new_log_probs: Log probabilities of new sequences
float32 tensor with shape [batch_size, beam_size]
new_log_probs: Log probabilities of new sequences float32 tensor with
shape [batch_size, beam_size]
new_finished_flags: A boolean Tensor indicates which sequences are live
inside the beam.
new_cache: Dict of cached values for each sequence.
Returns:
...
...
@@ -428,7 +433,6 @@ class SequenceBeamSearch(object):
Dict cache storing decoder states for top alive sequences}
"""
# To prevent finished sequences from being considered, set log probs to -inf
new_finished_flags
=
tf
.
equal
(
new_seq
[:,
:,
-
1
],
self
.
eos_id
)
new_log_probs
+=
tf
.
cast
(
new_finished_flags
,
self
.
dtype
)
*
-
inf
(
self
.
dtype
)
top_alive_seq
,
top_alive_log_probs
,
top_alive_cache
=
_gather_topk_beams
(
...
...
@@ -441,15 +445,18 @@ class SequenceBeamSearch(object):
_StateKeys
.
ALIVE_CACHE
:
top_alive_cache
}
def
_get_new_finished_state
(
self
,
state
,
new_seq
,
new_log_probs
):
def
_get_new_finished_state
(
self
,
state
,
new_seq
,
new_log_probs
,
new_finished_flags
):
"""Combine new and old finished sequences, and gather the top k sequences.
Args:
state: A dictionary with the current loop state.
new_seq: New sequences generated by growing the current alive sequences
int32 tensor with shape [batch_size, beam_size, i + 1]
new_log_probs: Log probabilities of new sequences
float32 tensor with shape [batch_size, beam_size]
new_log_probs: Log probabilities of new sequences float32 tensor with
shape [batch_size, beam_size]
new_finished_flags: A boolean Tensor indicates which sequences are live
inside the beam.
Returns:
Dictionary with finished keys from _StateKeys:
...
...
@@ -476,7 +483,6 @@ class SequenceBeamSearch(object):
new_scores
=
new_log_probs
/
length_norm
# Set the scores of the still-alive seq in new_seq to large negative values.
new_finished_flags
=
tf
.
equal
(
new_seq
[:,
:,
-
1
],
self
.
eos_id
)
new_scores
+=
((
1.
-
tf
.
cast
(
new_finished_flags
,
self
.
dtype
))
*
-
inf
(
self
.
dtype
))
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录