Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
曾经的那一瞬间
Models
提交
8c408bbe
M
Models
项目概览
曾经的那一瞬间
/
Models
大约 1 年 前同步成功
通知
1
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
M
Models
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
8c408bbe
编写于
5月 14, 2020
作者:
C
Chen Chen
提交者:
A. Unique TensorFlower
5月 14, 2020
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Internal change
PiperOrigin-RevId: 311597242
上级
7cdb82e3
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
14 addition
and
9 deletion
+14
-9
official/nlp/modeling/networks/encoder_scaffold.py
official/nlp/modeling/networks/encoder_scaffold.py
+8
-5
official/nlp/modeling/networks/encoder_scaffold_test.py
official/nlp/modeling/networks/encoder_scaffold_test.py
+6
-4
未找到文件。
official/nlp/modeling/networks/encoder_scaffold.py
浏览文件 @
8c408bbe
...
...
@@ -54,9 +54,11 @@ class EncoderScaffold(tf.keras.Model):
pooler_layer_initializer: The initializer for the classification
layer.
embedding_cls: The class or instance to use to embed the input data. This
class or instance defines the inputs to this encoder. If embedding_cls is
not set, a default embedding network (from the original BERT paper) will
be created.
class or instance defines the inputs to this encoder and outputs
(1) embeddings tensor with shape [batch_size, seq_length, hidden_size] and
(2) attention masking with tensor [batch_size, seq_length, seq_length].
If embedding_cls is not set, a default embedding network
(from the original BERT paper) will be created.
embedding_cfg: A dict of kwargs to pass to the embedding_cls, if it needs to
be instantiated. If embedding_cls is not set, a config dict must be
passed to 'embedding_cfg' with the following values:
...
...
@@ -121,7 +123,7 @@ class EncoderScaffold(tf.keras.Model):
else
:
self
.
_embedding_network
=
embedding_cls
inputs
=
self
.
_embedding_network
.
inputs
embeddings
,
mask
=
self
.
_embedding_network
(
inputs
)
embeddings
,
attention_
mask
=
self
.
_embedding_network
(
inputs
)
else
:
self
.
_embedding_network
=
None
word_ids
=
tf
.
keras
.
layers
.
Input
(
...
...
@@ -174,7 +176,8 @@ class EncoderScaffold(tf.keras.Model):
tf
.
keras
.
layers
.
Dropout
(
rate
=
embedding_cfg
[
'dropout_rate'
])(
embeddings
))
attention_mask
=
layers
.
SelfAttentionMask
()([
embeddings
,
mask
])
attention_mask
=
layers
.
SelfAttentionMask
()([
embeddings
,
mask
])
data
=
embeddings
layer_output_data
=
[]
...
...
official/nlp/modeling/networks/encoder_scaffold_test.py
浏览文件 @
8c408bbe
...
...
@@ -211,8 +211,6 @@ class EncoderScaffoldLayerClassTest(keras_parameterized.TestCase):
"kernel_initializer"
:
tf
.
keras
.
initializers
.
TruncatedNormal
(
stddev
=
0.02
),
}
print
(
hidden_cfg
)
print
(
embedding_cfg
)
# Create a small EncoderScaffold for testing.
test_network
=
encoder_scaffold
.
EncoderScaffold
(
num_hidden_instances
=
3
,
...
...
@@ -347,7 +345,9 @@ class EncoderScaffoldEmbeddingNetworkTest(keras_parameterized.TestCase):
initializer
=
tf
.
keras
.
initializers
.
TruncatedNormal
(
stddev
=
0.02
),
name
=
"word_embeddings"
)
word_embeddings
=
embedding_layer
(
word_ids
)
network
=
tf
.
keras
.
Model
([
word_ids
,
mask
],
[
word_embeddings
,
mask
])
attention_mask
=
layers
.
SelfAttentionMask
()([
word_embeddings
,
mask
])
network
=
tf
.
keras
.
Model
([
word_ids
,
mask
],
[
word_embeddings
,
attention_mask
])
hidden_cfg
=
{
"num_attention_heads"
:
...
...
@@ -414,7 +414,9 @@ class EncoderScaffoldEmbeddingNetworkTest(keras_parameterized.TestCase):
initializer
=
tf
.
keras
.
initializers
.
TruncatedNormal
(
stddev
=
0.02
),
name
=
"word_embeddings"
)
word_embeddings
=
embedding_layer
(
word_ids
)
network
=
tf
.
keras
.
Model
([
word_ids
,
mask
],
[
word_embeddings
,
mask
])
attention_mask
=
layers
.
SelfAttentionMask
()([
word_embeddings
,
mask
])
network
=
tf
.
keras
.
Model
([
word_ids
,
mask
],
[
word_embeddings
,
attention_mask
])
hidden_cfg
=
{
"num_attention_heads"
:
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录