serving_client调用rpc预测服务时报错,可服务模型使用paddleNLP的simnet
Created by: lerry-lee
保存可服务模型时,顺便打印了输入输出变量的名称
inference_model_dir = "bow_pairwise/exp1"
serving_client_dir = "temp/serving_client"
serving_server_dir = "temp/serving_server"
feed_var_names, fetch_var_names = inference_model_to_serving(
infer_model=inference_model_dir, serving_client=serving_client_dir, serving_server=serving_server_dir)
print("feed_names:",feed_var_names)
print("fetch_names:",fetch_var_names)
打印结果为
feed_names: dict_keys(['read_file_0.tmp_0', 'read_file_0.tmp_1'])
fetch_names: dict_keys(['fc.tmp_1', 'cos_sim_0.tmp_0'])
于是我按照这个名称来定义输入、获取输出. 启动好rpc预测服务后,调用client测试
vocab = load_vocab('term2id.dict')
str_list = ['你好\t你好\t1']
left, pos_right = Sim_reader(str_list, vocab)
client = Client()
client.load_client_config("temp/serving_server/serving_client_conf.prototxt")
client.connect(["127.0.0.1:9292"])
feed={'read_file_0.tmp_0':left,'read_file_0.tmp_1':pos_right}
fetch=['fc.tmp_1','cos_sim_0.tmp_0']
fetch_res = client.predict(feed=feed, fetch=fetch)
其中Sim_reader是模仿源码里面的数据读取方式写的
def Sim_reader(str_list, vocab):
simnet_process = SimNetProcessor(str_list, vocab)
startup_prog = fluid.Program()
get_test_examples = simnet_process.get_reader()
batch_data = fluid.io.batch(
get_test_examples, 128, drop_last=False)
test_prog = fluid.Program()
inf_pyreader = fluid.layers.py_reader(
capacity=16,
shapes=([-1], [-1]),
dtypes=('int64', 'int64'),
lod_levels=(1, 1),
name='test_reader',
use_double_buffer=False)
inf_pyreader.decorate_paddle_reader(batch_data)
left, right = fluid.layers.read_file(inf_pyreader)
return left, right
其中SimNetProcessor源码里是从文件中一行一行读,我改写成从list中按元素读了. 然后我执行客户端测试,发现报一些错误
fetch_map = client.predict(feed=feed, fetch=fetch)
File "/home/lca/.conda/envs/py36-paddle/lib/python3.6/site-packages/paddle_serving_client/__init__.py", line 296, in predict
int_feed_names, int_shape, fetch_names, result_batch, self.pid)
ValueError: vector::reserve
我尝试打印过left,pos_right,是variable的类型,里面还有lod_tensor,不知道是不是里面没有实际数据...