encode.sh报错:The shape of Input(Length) should be [batch_size]
Created by: mapingshuo
Hi, 我在使用encode.py来计算一句话的embedding, 正如Readme介绍的那样:https://github.com/PaddlePaddle/ERNIE#faq1-how-to-get-sentencetokens-embedding-of-ernie
我的encode脚本为:
TASK_DATA_PATH=.
MODEL_PATH=/path/to/model
export FLAGS_sync_nccl_allreduce=1
export CUDA_VISIBLE_DEVICES=4
python -u ernie_encoder.py \
--use_cuda true \
--batch_size 3 \
--output_dir "./test" \
--init_pretraining_params ${MODEL_PATH}/trained_chinese/params \
--data_set ${TASK_DATA_PATH}/baidu_input/dev.tsv \
--vocab_path ${MODEL_PATH}/vocab.txt \
--max_seq_len 128 \
--ernie_config_path ${MODEL_PATH}/ernie_config.json
其中,baidu_input/dev.tsv 如下:
label\ttext_a\n
0\t你吃了么\n
1\t我吃过了\n
0\t谢谢你啊\n
模型报错:
Device count: 1
Total num examples: 3
WARNING:root:paddle.fluid.layers.py_reader() may be deprecated in the near future. Please use paddle.fluid.io.PyReader() instead.
Traceback (most recent call last):
File "ernie_encoder.py", line 182, in <module>
main(args)
File "ernie_encoder.py", line 130, in main
args, pyreader_name='reader', ernie_config=ernie_config)
File "ernie_encoder.py", line 77, in create_model
unpad_enc_out = fluid.layers.sequence_unpad(enc_out, length=seq_lens)
File "/home/mapingshuo/paddle_release_home/python-distribute/lib/python2.7/site-packages/paddle/fluid/layers/nn.py", line 4842, in sequence_unpad
outputs={'Out': out})
File "/home/mapingshuo/paddle_release_home/python-distribute/lib/python2.7/site-packages/paddle/fluid/layer_helper.py", line 43, in append_op
return self.main_program.current_block().append_op(*args, **kwargs)
File "/home/mapingshuo/paddle_release_home/python-distribute/lib/python2.7/site-packages/paddle/fluid/framework.py", line 2116, in append_op
attrs=kwargs.get("attrs", None))
File "/home/mapingshuo/paddle_release_home/python-distribute/lib/python2.7/site-packages/paddle/fluid/framework.py", line 1499, in __init__
self.desc.infer_shape(self.block.desc)
paddle.fluid.core_avx.EnforceNotMet:
--------------------------------------------
C++ Call Stacks (More useful to developers):
--------------------------------------------
0 std::string paddle::platform::GetTraceBackString<std::string const&>(std::string const&, char const*, int)
1 paddle::platform::EnforceNotMet::EnforceNotMet(std::string const&, char const*, int)
2 paddle::operators::SequenceUnpadOp::InferShape(paddle::framework::InferShapeContext*) const
3 paddle::framework::OpDesc::InferShape(paddle::framework::BlockDesc const&) const
------------------------------------------
Python Call Stacks (More useful to users):
------------------------------------------
File "/home/mapingshuo/paddle_release_home/python-distribute/lib/python2.7/site-packages/paddle/fluid/framework.py", line 2116, in append_op
attrs=kwargs.get("attrs", None))
File "/home/mapingshuo/paddle_release_home/python-distribute/lib/python2.7/site-packages/paddle/fluid/layer_helper.py", line 43, in append_op
return self.main_program.current_block().append_op(*args, **kwargs)
File "/home/mapingshuo/paddle_release_home/python-distribute/lib/python2.7/site-packages/paddle/fluid/layers/nn.py", line 4842, in sequence_unpad
outputs={'Out': out})
File "ernie_encoder.py", line 77, in create_model
unpad_enc_out = fluid.layers.sequence_unpad(enc_out, length=seq_lens)
File "ernie_encoder.py", line 130, in main
args, pyreader_name='reader', ernie_config=ernie_config)
File "ernie_encoder.py", line 182, in <module>
main(args)
----------------------
Error Message Summary:
----------------------
PaddleCheckError: Expected len_dims.size() == 1, but received len_dims.size():2 != 1:1.
The shape of Input(Length) should be [batch_size]. at [/home/mapingshuo/Paddle/paddle/fluid/operators/sequence_ops/sequence_unpad_op.cc:41]
[operator < sequence_unpad > error]
请问是什么原因,谢谢