Skip to content

S
Serving

PaddlePaddle / Serving
接近 2 年 前同步成功

通知 186
Star 833
Fork 253
S
Serving
未验证 提交 9cfb1072 编写于 作者: T TeslaZhao 提交者： GitHub
浏览文件
操作

Merge pull request #1169 from HexToString/v0.6.0 

cherry-pick #1165
上级 debd438b fe90c7c0
隐藏空白更改
内联 并排
Showing with 210 addition and 147 deletion
core/general-server/op/general_detection_op.cpp
浏览文件 @ 9cfb1072
...@@ -22,7 +22,6 @@ ...@@ -22,7 +22,6 @@
#include "core/predictor/framework/resource.h" #include "core/predictor/framework/resource.h"
#include "core/util/include/timer.h" #include "core/util/include/timer.h"
/* /*
#include "opencv2/imgcodecs/legacy/constants_c.h" #include "opencv2/imgcodecs/legacy/constants_c.h"
#include "opencv2/imgproc/types_c.h" #include "opencv2/imgproc/types_c.h"
...@@ -52,18 +51,18 @@ int GeneralDetectionOp::inference() { ...@@ -52,18 +51,18 @@ int GeneralDetectionOp::inference() {
} }
const std::string pre_name = pre_node_names[0]; const std::string pre_name = pre_node_names[0];
const GeneralBlob *input_blob = get_depend_argument<GeneralBlob>(pre_name); const GeneralBlob* input_blob = get_depend_argument<GeneralBlob>(pre_name);
if (!input_blob) { if (!input_blob) {
LOG(ERROR) << "input_blob is nullptr,error"; LOG(ERROR) << "input_blob is nullptr,error";
return -1; return -1;
} }
uint64_t log_id = input_blob->GetLogId(); uint64_t log_id = input_blob->GetLogId();
VLOG(2) << "(logid=" << log_id << ") Get precedent op name: " << pre_name; VLOG(2) << "(logid=" << log_id << ") Get precedent op name: " << pre_name;
GeneralBlob *output_blob = mutable_data<GeneralBlob>(); GeneralBlob* output_blob = mutable_data<GeneralBlob>();
if (!output_blob) { if (!output_blob) {
LOG(ERROR) << "output_blob is nullptr,error"; LOG(ERROR) << "output_blob is nullptr,error";
return -1; return -1;
} }
output_blob->SetLogId(log_id); output_blob->SetLogId(log_id);
...@@ -73,7 +72,7 @@ int GeneralDetectionOp::inference() { ...@@ -73,7 +72,7 @@ int GeneralDetectionOp::inference() {
return -1; return -1;
} }
const TensorVector *in = &input_blob->tensor_vector; const TensorVector* in = &input_blob->tensor_vector;
TensorVector* out = &output_blob->tensor_vector; TensorVector* out = &output_blob->tensor_vector;
int batch_size = input_blob->_batch_size; int batch_size = input_blob->_batch_size;
...@@ -81,38 +80,39 @@ int GeneralDetectionOp::inference() { ...@@ -81,38 +80,39 @@ int GeneralDetectionOp::inference() {
output_blob->_batch_size = batch_size; output_blob->_batch_size = batch_size;
VLOG(2) << "(logid=" << log_id << ") infer batch size: " << batch_size;
std::vector<int> input_shape; std::vector<int> input_shape;
int in_num =0; int in_num = 0;
void* databuf_data = NULL; void* databuf_data = NULL;
char* databuf_char = NULL; char* databuf_char = NULL;
size_t databuf_size = 0; size_t databuf_size = 0;
// now only support single string
char* total_input_ptr = static_cast<char*>(in->at(0).data.data());
std::string base64str = total_input_ptr;
std::string* input_ptr = static_cast<std::string*>(in->at(0).data.data());
std::string base64str = input_ptr[0];
float ratio_h{}; float ratio_h{};
float ratio_w{}; float ratio_w{};
cv::Mat img = Base2Mat(base64str); cv::Mat img = Base2Mat(base64str);
cv::Mat srcimg; cv::Mat srcimg;
cv::Mat resize_img; cv::Mat resize_img;
cv::Mat resize_img_rec; cv::Mat resize_img_rec;
cv::Mat crop_img; cv::Mat crop_img;
img.copyTo(srcimg); img.copyTo(srcimg);
this->resize_op_.Run(img, resize_img, this->max_side_len_, ratio_h, ratio_w, this->resize_op_.Run(img,
resize_img,
this->max_side_len_,
ratio_h,
ratio_w,
this->use_tensorrt_); this->use_tensorrt_);
this->normalize_op_.Run(&resize_img, this->mean_det, this->scale_det, this->normalize_op_.Run(
this->is_scale_); &resize_img, this->mean_det, this->scale_det, this->is_scale_);
std::vector<float> input(1 * 3 * resize_img.rows * resize_img.cols, 0.0f); std::vector<float> input(1 * 3 * resize_img.rows * resize_img.cols, 0.0f);
this->permute_op_.Run(&resize_img, input.data()); this->permute_op_.Run(&resize_img, input.data());
TensorVector* real_in = new TensorVector(); TensorVector* real_in = new TensorVector();
if (!real_in) { if (!real_in) {
LOG(ERROR) << "real_in is nullptr,error"; LOG(ERROR) << "real_in is nullptr,error";
...@@ -121,14 +121,15 @@ int GeneralDetectionOp::inference() { ...@@ -121,14 +121,15 @@ int GeneralDetectionOp::inference() {
for (int i = 0; i < in->size(); ++i) { for (int i = 0; i < in->size(); ++i) {
input_shape = {1, 3, resize_img.rows, resize_img.cols}; input_shape = {1, 3, resize_img.rows, resize_img.cols};
in_num = std::accumulate(input_shape.begin(), input_shape.end(), 1, std::multiplies<int>()); in_num = std::accumulate(
databuf_size = in_num*sizeof(float); input_shape.begin(), input_shape.end(), 1, std::multiplies<int>());
databuf_size = in_num * sizeof(float);
databuf_data = MempoolWrapper::instance().malloc(databuf_size); databuf_data = MempoolWrapper::instance().malloc(databuf_size);
if (!databuf_data) { if (!databuf_data) {
LOG(ERROR) << "Malloc failed, size: " << databuf_size; LOG(ERROR) << "Malloc failed, size: " << databuf_size;
return -1; return -1;
} }
memcpy(databuf_data,input.data(),databuf_size); memcpy(databuf_data, input.data(), databuf_size);
databuf_char = reinterpret_cast<char*>(databuf_data); databuf_char = reinterpret_cast<char*>(databuf_data);
paddle::PaddleBuf paddleBuf(databuf_char, databuf_size); paddle::PaddleBuf paddleBuf(databuf_char, databuf_size);
paddle::PaddleTensor tensor_in; paddle::PaddleTensor tensor_in;
...@@ -143,21 +144,23 @@ int GeneralDetectionOp::inference() { ...@@ -143,21 +144,23 @@ int GeneralDetectionOp::inference() {
Timer timeline; Timer timeline;
int64_t start = timeline.TimeStampUS(); int64_t start = timeline.TimeStampUS();
timeline.Start(); timeline.Start();
if (InferManager::instance().infer( if (InferManager::instance().infer(
engine_name().c_str(), real_in, out, batch_size)) { engine_name().c_str(), real_in, out, batch_size)) {
LOG(ERROR) << "(logid=" << log_id LOG(ERROR) << "(logid=" << log_id
<< ") Failed do infer in fluid model: " << engine_name().c_str(); << ") Failed do infer in fluid model: " << engine_name().c_str();
return -1; return -1;
} }
delete real_in;
std::vector<int> output_shape; std::vector<int> output_shape;
int out_num =0; int out_num = 0;
void* databuf_data_out = NULL; void* databuf_data_out = NULL;
char* databuf_char_out = NULL; char* databuf_char_out = NULL;
size_t databuf_size_out = 0; size_t databuf_size_out = 0;
//this is special add for PaddleOCR postprecess // this is special add for PaddleOCR postprecess
int infer_outnum = out->size(); int infer_outnum = out->size();
for (int k = 0;k <infer_outnum; ++k) { for (int k = 0; k < infer_outnum; ++k) {
int n2 = out->at(k).shape[2]; int n2 = out->at(k).shape[2];
int n3 = out->at(k).shape[3]; int n3 = out->at(k).shape[3];
int n = n2 * n3; int n = n2 * n3;
...@@ -171,17 +174,19 @@ int GeneralDetectionOp::inference() { ...@@ -171,17 +174,19 @@ int GeneralDetectionOp::inference() {
cbuf[i] = (unsigned char)((out_data[i]) * 255); cbuf[i] = (unsigned char)((out_data[i]) * 255);
} }
cv::Mat cbuf_map(n2, n3, CV_8UC1, (unsigned char *)cbuf.data()); cv::Mat cbuf_map(n2, n3, CV_8UC1, (unsigned char*)cbuf.data());
cv::Mat pred_map(n2, n3, CV_32F, (float *)pred.data()); cv::Mat pred_map(n2, n3, CV_32F, (float*)pred.data());
const double threshold = this->det_db_thresh_ * 255; const double threshold = this->det_db_thresh_ * 255;
const double maxvalue = 255; const double maxvalue = 255;
cv::Mat bit_map; cv::Mat bit_map;
cv::threshold(cbuf_map, bit_map, threshold, maxvalue, cv::THRESH_BINARY); cv::threshold(cbuf_map, bit_map, threshold, maxvalue, cv::THRESH_BINARY);
cv::Mat dilation_map; cv::Mat dilation_map;
cv::Mat dila_ele = cv::getStructuringElement(cv::MORPH_RECT, cv::Size(2, 2)); cv::Mat dila_ele =
cv::getStructuringElement(cv::MORPH_RECT, cv::Size(2, 2));
cv::dilate(bit_map, dilation_map, dila_ele); cv::dilate(bit_map, dilation_map, dila_ele);
boxes = post_processor_.BoxesFromBitmap(pred_map, dilation_map, boxes = post_processor_.BoxesFromBitmap(pred_map,
dilation_map,
this->det_db_box_thresh_, this->det_db_box_thresh_,
this->det_db_unclip_ratio_); this->det_db_unclip_ratio_);
...@@ -192,25 +197,28 @@ int GeneralDetectionOp::inference() { ...@@ -192,25 +197,28 @@ int GeneralDetectionOp::inference() {
float wh_ratio = float(crop_img.cols) / float(crop_img.rows); float wh_ratio = float(crop_img.cols) / float(crop_img.rows);
this->resize_op_rec.Run(crop_img, resize_img_rec, wh_ratio, this->use_tensorrt_); this->resize_op_rec.Run(
crop_img, resize_img_rec, wh_ratio, this->use_tensorrt_);
this->normalize_op_.Run(&resize_img_rec, this->mean_rec, this->scale_rec, this->normalize_op_.Run(
this->is_scale_); &resize_img_rec, this->mean_rec, this->scale_rec, this->is_scale_);
std::vector<float> output_rec(1 * 3 * resize_img_rec.rows * resize_img_rec.cols, 0.0f); std::vector<float> output_rec(
1 * 3 * resize_img_rec.rows * resize_img_rec.cols, 0.0f);
this->permute_op_.Run(&resize_img_rec, output_rec.data()); this->permute_op_.Run(&resize_img_rec, output_rec.data());
// Inference. // Inference.
output_shape = {1, 3, resize_img_rec.rows, resize_img_rec.cols}; output_shape = {1, 3, resize_img_rec.rows, resize_img_rec.cols};
out_num = std::accumulate(output_shape.begin(), output_shape.end(), 1, std::multiplies<int>()); out_num = std::accumulate(
databuf_size_out = out_num*sizeof(float); output_shape.begin(), output_shape.end(), 1, std::multiplies<int>());
databuf_size_out = out_num * sizeof(float);
databuf_data_out = MempoolWrapper::instance().malloc(databuf_size_out); databuf_data_out = MempoolWrapper::instance().malloc(databuf_size_out);
if (!databuf_data_out) { if (!databuf_data_out) {
LOG(ERROR) << "Malloc failed, size: " << databuf_size_out; LOG(ERROR) << "Malloc failed, size: " << databuf_size_out;
return -1; return -1;
} }
memcpy(databuf_data_out,output_rec.data(),databuf_size_out); memcpy(databuf_data_out, output_rec.data(), databuf_size_out);
databuf_char_out = reinterpret_cast<char*>(databuf_data_out); databuf_char_out = reinterpret_cast<char*>(databuf_data_out);
paddle::PaddleBuf paddleBuf(databuf_char_out, databuf_size_out); paddle::PaddleBuf paddleBuf(databuf_char_out, databuf_size_out);
paddle::PaddleTensor tensor_out; paddle::PaddleTensor tensor_out;
...@@ -221,9 +229,8 @@ int GeneralDetectionOp::inference() { ...@@ -221,9 +229,8 @@ int GeneralDetectionOp::inference() {
out->push_back(tensor_out); out->push_back(tensor_out);
} }
} }
out->erase(out->begin(),out->begin()+infer_outnum); out->erase(out->begin(), out->begin() + infer_outnum);
int64_t end = timeline.TimeStampUS(); int64_t end = timeline.TimeStampUS();
CopyBlobInfo(input_blob, output_blob); CopyBlobInfo(input_blob, output_blob);
AddBlobInfo(output_blob, start); AddBlobInfo(output_blob, start);
...@@ -231,68 +238,62 @@ int GeneralDetectionOp::inference() { ...@@ -231,68 +238,62 @@ int GeneralDetectionOp::inference() {
return 0; return 0;
} }
cv::Mat GeneralDetectionOp::Base2Mat(std::string &base64_data) cv::Mat GeneralDetectionOp::Base2Mat(std::string& base64_data) {
{ cv::Mat img;
cv::Mat img; std::string s_mat;
std::string s_mat; s_mat = base64Decode(base64_data.data(), base64_data.size());
s_mat = base64Decode(base64_data.data(), base64_data.size()); std::vector<char> base64_img(s_mat.begin(), s_mat.end());
std::vector<char> base64_img(s_mat.begin(), s_mat.end()); img = cv::imdecode(base64_img, cv::IMREAD_COLOR); // CV_LOAD_IMAGE_COLOR
img = cv::imdecode(base64_img, cv::IMREAD_COLOR);//CV_LOAD_IMAGE_COLOR return img;
return img;
} }
std::string GeneralDetectionOp::base64Decode(const char* Data, int DataByte) std::string GeneralDetectionOp::base64Decode(const char* Data, int DataByte) {
{ const char
DecodeTable[] =
const char DecodeTable[] = {
{ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
62, // '+' 62, // '+'
0, 0, 0, 0, 0, 0,
63, // '/' 63, // '/'
52, 53, 54, 55, 56, 57, 58, 59, 60, 61, // '0'-'9' 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, // '0'-'9'
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7,
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22,
13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, // 'A'-'Z' 23, 24, 25, // 'A'-'Z'
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 26, 27, 28, 29, 30, 31, 32, 33, 34,
26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49,
39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, // 'a'-'z' 50, 51, // 'a'-'z'
}; };
std::string strDecode; std::string strDecode;
int nValue; int nValue;
int i = 0; int i = 0;
while (i < DataByte) while (i < DataByte) {
{ if (*Data != '\r' && *Data != '\n') {
if (*Data != '\r' && *Data != '\n') nValue = DecodeTable[*Data++] << 18;
{ nValue += DecodeTable[*Data++] << 12;
nValue = DecodeTable[*Data++] << 18; strDecode += (nValue & 0x00FF0000) >> 16;
nValue += DecodeTable[*Data++] << 12; if (*Data != '=') {
strDecode += (nValue & 0x00FF0000) >> 16; nValue += DecodeTable[*Data++] << 6;
if (*Data != '=') strDecode += (nValue & 0x0000FF00) >> 8;
{ if (*Data != '=') {
nValue += DecodeTable[*Data++] << 6; nValue += DecodeTable[*Data++];
strDecode += (nValue & 0x0000FF00) >> 8; strDecode += nValue & 0x000000FF;
if (*Data != '=') }
{ }
nValue += DecodeTable[*Data++]; i += 4;
strDecode += nValue & 0x000000FF; } else // 回车换行,跳过
} {
} Data++;
i += 4; i++;
} }
else// 回车换行,跳过 }
{ return strDecode;
Data++;
i++;
}
}
return strDecode;
} }
cv::Mat GeneralDetectionOp::GetRotateCropImage(const cv::Mat &srcimage, cv::Mat GeneralDetectionOp::GetRotateCropImage(
std::vector<std::vector<int>> box) { const cv::Mat& srcimage, std::vector<std::vector<int>> box) {
cv::Mat image; cv::Mat image;
srcimage.copyTo(image); srcimage.copyTo(image);
std::vector<std::vector<int>> points = box; std::vector<std::vector<int>> points = box;
...@@ -332,7 +333,9 @@ cv::Mat GeneralDetectionOp::GetRotateCropImage(const cv::Mat &srcimage, ...@@ -332,7 +333,9 @@ cv::Mat GeneralDetectionOp::GetRotateCropImage(const cv::Mat &srcimage,
cv::Mat M = cv::getPerspectiveTransform(pointsf, pts_std); cv::Mat M = cv::getPerspectiveTransform(pointsf, pts_std);
cv::Mat dst_img; cv::Mat dst_img;
cv::warpPerspective(img_crop, dst_img, M, cv::warpPerspective(img_crop,
dst_img,
M,
cv::Size(img_crop_width, img_crop_height), cv::Size(img_crop_width, img_crop_height),
cv::BORDER_REPLICATE); cv::BORDER_REPLICATE);
...@@ -350,4 +353,4 @@ DEFINE_OP(GeneralDetectionOp); ...@@ -350,4 +353,4 @@ DEFINE_OP(GeneralDetectionOp);
} // namespace serving } // namespace serving
} // namespace paddle_serving } // namespace paddle_serving
} // namespace baidu } // namespace baidu
\ No newline at end of file
core/general-server/op/general_reader_op.cpp
浏览文件 @ 9cfb1072
...@@ -77,9 +77,6 @@ int GeneralReaderOp::inference() { ...@@ -77,9 +77,6 @@ int GeneralReaderOp::inference() {
uint64_t log_id = req->log_id(); uint64_t log_id = req->log_id();
int input_var_num = 0; int input_var_num = 0;
std::vector<int64_t> elem_type;
std::vector<int64_t> elem_size;
std::vector<int64_t> databuf_size;
GeneralBlob *res = mutable_data<GeneralBlob>(); GeneralBlob *res = mutable_data<GeneralBlob>();
if (!res) { if (!res) {
...@@ -119,40 +116,44 @@ int GeneralReaderOp::inference() { ...@@ -119,40 +116,44 @@ int GeneralReaderOp::inference() {
} }
*/ */
// package tensor // package tensor
elem_type.resize(var_num);
elem_size.resize(var_num);
databuf_size.resize(var_num);
// prepare basic information for input // prepare basic information for input
// specify the memory needed for output tensor_vector // specify the memory needed for output tensor_vector
// fill the data into output general_blob // fill the data into output general_blob
int data_len = 0; int data_len = 0;
int64_t elem_type = 0;
int64_t elem_size = 0;
int64_t databuf_size = 0;
for (int i = 0; i < var_num; ++i) { for (int i = 0; i < var_num; ++i) {
paddle::PaddleTensor lod_tensor; paddle::PaddleTensor paddleTensor;
const Tensor &tensor = req->insts(0).tensor_array(i); const Tensor &tensor = req->insts(0).tensor_array(i);
data_len = 0; data_len = 0;
elem_type[i] = tensor.elem_type(); elem_type = 0;
VLOG(2) << "var[" << i << "] has elem type: " << elem_type[i]; elem_size = 0;
if (elem_type[i] == P_INT64) { // int64 databuf_size = 0;
elem_size[i] = sizeof(int64_t); elem_type = tensor.elem_type();
lod_tensor.dtype = paddle::PaddleDType::INT64; VLOG(2) << "var[" << i << "] has elem type: " << elem_type;
if (elem_type == P_INT64) { // int64
elem_size = sizeof(int64_t);
paddleTensor.dtype = paddle::PaddleDType::INT64;
data_len = tensor.int64_data_size(); data_len = tensor.int64_data_size();
} else if (elem_type[i] == P_FLOAT32) { } else if (elem_type == P_FLOAT32) {
elem_size[i] = sizeof(float); elem_size = sizeof(float);
lod_tensor.dtype = paddle::PaddleDType::FLOAT32; paddleTensor.dtype = paddle::PaddleDType::FLOAT32;
data_len = tensor.float_data_size(); data_len = tensor.float_data_size();
} else if (elem_type[i] == P_INT32) { } else if (elem_type == P_INT32) {
elem_size[i] = sizeof(int32_t); elem_size = sizeof(int32_t);
lod_tensor.dtype = paddle::PaddleDType::INT32; paddleTensor.dtype = paddle::PaddleDType::INT32;
data_len = tensor.int_data_size(); data_len = tensor.int_data_size();
} else if (elem_type[i] == P_STRING) { } else if (elem_type == P_STRING) {
// use paddle::PaddleDType::UINT8 as for String. // use paddle::PaddleDType::UINT8 as for String.
elem_size[i] = sizeof(uint8_t); elem_size = sizeof(char);
lod_tensor.dtype = paddle::PaddleDType::UINT8; paddleTensor.dtype = paddle::PaddleDType::UINT8;
// this is for vector<String>, cause the databuf_size != // this is for vector<String>, cause the databuf_size !=
// vector<String>.size()*sizeof(char); // vector<String>.size()*sizeof(char);
// data_len should be +1 cause '\0'
// now only support single string
for (int idx = 0; idx < tensor.data_size(); idx++) { for (int idx = 0; idx < tensor.data_size(); idx++) {
data_len += tensor.data()[idx].length(); data_len += tensor.data()[idx].length() + 1;
} }
} }
// implement lod tensor here // implement lod tensor here
...@@ -160,29 +161,29 @@ int GeneralReaderOp::inference() { ...@@ -160,29 +161,29 @@ int GeneralReaderOp::inference() {
// TODO(HexToString): support 2-D lod // TODO(HexToString): support 2-D lod
if (tensor.lod_size() > 0) { if (tensor.lod_size() > 0) {
VLOG(2) << "(logid=" << log_id << ") var[" << i << "] is lod_tensor"; VLOG(2) << "(logid=" << log_id << ") var[" << i << "] is lod_tensor";
lod_tensor.lod.resize(1); paddleTensor.lod.resize(1);
for (int k = 0; k < tensor.lod_size(); ++k) { for (int k = 0; k < tensor.lod_size(); ++k) {
lod_tensor.lod[0].push_back(tensor.lod(k)); paddleTensor.lod[0].push_back(tensor.lod(k));
} }
} }
for (int k = 0; k < tensor.shape_size(); ++k) { for (int k = 0; k < tensor.shape_size(); ++k) {
int dim = tensor.shape(k); int dim = tensor.shape(k);
VLOG(2) << "(logid=" << log_id << ") shape for var[" << i << "]: " << dim; VLOG(2) << "(logid=" << log_id << ") shape for var[" << i << "]: " << dim;
lod_tensor.shape.push_back(dim); paddleTensor.shape.push_back(dim);
} }
lod_tensor.name = model_config->_feed_name[i]; paddleTensor.name = model_config->_feed_name[i];
out->push_back(lod_tensor); out->push_back(paddleTensor);
VLOG(2) << "(logid=" << log_id << ") tensor size for var[" << i VLOG(2) << "(logid=" << log_id << ") tensor size for var[" << i
<< "]: " << data_len; << "]: " << data_len;
databuf_size[i] = data_len * elem_size[i]; databuf_size = data_len * elem_size;
out->at(i).data.Resize(data_len * elem_size[i]); out->at(i).data.Resize(databuf_size);
if (out->at(i).lod.size() > 0) { if (out->at(i).lod.size() > 0) {
VLOG(2) << "(logid=" << log_id << ") var[" << i VLOG(2) << "(logid=" << log_id << ") var[" << i
<< "] has lod_tensor and len=" << out->at(i).lod[0].back(); << "] has lod_tensor and len=" << out->at(i).lod[0].back();
} }
if (elem_type[i] == P_INT64) { if (elem_type == P_INT64) {
int64_t *dst_ptr = static_cast<int64_t *>(out->at(i).data.data()); int64_t *dst_ptr = static_cast<int64_t *>(out->at(i).data.data());
VLOG(2) << "(logid=" << log_id << ") first element data in var[" << i VLOG(2) << "(logid=" << log_id << ") first element data in var[" << i
<< "] is " << tensor.int64_data(0); << "] is " << tensor.int64_data(0);
...@@ -190,14 +191,14 @@ int GeneralReaderOp::inference() { ...@@ -190,14 +191,14 @@ int GeneralReaderOp::inference() {
LOG(ERROR) << "dst_ptr is nullptr"; LOG(ERROR) << "dst_ptr is nullptr";
return -1; return -1;
} }
memcpy(dst_ptr, tensor.int64_data().data(), databuf_size[i]); memcpy(dst_ptr, tensor.int64_data().data(), databuf_size);
/* /*
int elem_num = tensor.int64_data_size(); int elem_num = tensor.int64_data_size();
for (int k = 0; k < elem_num; ++k) { for (int k = 0; k < elem_num; ++k) {
dst_ptr[k] = tensor.int64_data(k); dst_ptr[k] = tensor.int64_data(k);
} }
*/ */
} else if (elem_type[i] == P_FLOAT32) { } else if (elem_type == P_FLOAT32) {
float *dst_ptr = static_cast<float *>(out->at(i).data.data()); float *dst_ptr = static_cast<float *>(out->at(i).data.data());
VLOG(2) << "(logid=" << log_id << ") first element data in var[" << i VLOG(2) << "(logid=" << log_id << ") first element data in var[" << i
<< "] is " << tensor.float_data(0); << "] is " << tensor.float_data(0);
...@@ -205,12 +206,12 @@ int GeneralReaderOp::inference() { ...@@ -205,12 +206,12 @@ int GeneralReaderOp::inference() {
LOG(ERROR) << "dst_ptr is nullptr"; LOG(ERROR) << "dst_ptr is nullptr";
return -1; return -1;
} }
memcpy(dst_ptr, tensor.float_data().data(), databuf_size[i]); memcpy(dst_ptr, tensor.float_data().data(), databuf_size);
/*int elem_num = tensor.float_data_size(); /*int elem_num = tensor.float_data_size();
for (int k = 0; k < elem_num; ++k) { for (int k = 0; k < elem_num; ++k) {
dst_ptr[k] = tensor.float_data(k); dst_ptr[k] = tensor.float_data(k);
}*/ }*/
} else if (elem_type[i] == P_INT32) { } else if (elem_type == P_INT32) {
int32_t *dst_ptr = static_cast<int32_t *>(out->at(i).data.data()); int32_t *dst_ptr = static_cast<int32_t *>(out->at(i).data.data());
VLOG(2) << "(logid=" << log_id << ") first element data in var[" << i VLOG(2) << "(logid=" << log_id << ") first element data in var[" << i
<< "] is " << tensor.int_data(0); << "] is " << tensor.int_data(0);
...@@ -218,15 +219,9 @@ int GeneralReaderOp::inference() { ...@@ -218,15 +219,9 @@ int GeneralReaderOp::inference() {
LOG(ERROR) << "dst_ptr is nullptr"; LOG(ERROR) << "dst_ptr is nullptr";
return -1; return -1;
} }
memcpy(dst_ptr, tensor.int_data().data(), databuf_size[i]); memcpy(dst_ptr, tensor.int_data().data(), databuf_size);
/* } else if (elem_type == P_STRING) {
int elem_num = tensor.int_data_size(); char *dst_ptr = static_cast<char *>(out->at(i).data.data());
for (int k = 0; k < elem_num; ++k) {
dst_ptr[k] = tensor.int_data(k);
}
*/
} else if (elem_type[i] == P_STRING) {
std::string *dst_ptr = static_cast<std::string *>(out->at(i).data.data());
VLOG(2) << "(logid=" << log_id << ") first element data in var[" << i VLOG(2) << "(logid=" << log_id << ") first element data in var[" << i
<< "] is " << tensor.data(0); << "] is " << tensor.data(0);
if (!dst_ptr) { if (!dst_ptr) {
...@@ -234,8 +229,12 @@ int GeneralReaderOp::inference() { ...@@ -234,8 +229,12 @@ int GeneralReaderOp::inference() {
return -1; return -1;
} }
int elem_num = tensor.data_size(); int elem_num = tensor.data_size();
int offset = 0;
for (int k = 0; k < elem_num; ++k) { for (int k = 0; k < elem_num; ++k) {
dst_ptr[k] = tensor.data(k); memcpy(dst_ptr + offset,
tensor.data(k).c_str(),
strlen(tensor.data(k).c_str()) + 1);
offset += strlen(tensor.data(k).c_str()) + 1;
} }
} }
} }
......
doc/LOD.md 0 → 100644
浏览文件 @ 9cfb1072
# Lod Introduction
(English|[简体中文](LOD_CN.md))
## Principle
LoD(Level-of-Detail) Tensor is an advanced feature of paddle and an extension of tensor. LoD Tensor improves training efficiency by sacrificing flexibility.
**Notice：** For most users, there is no need to pay attention to the usage of LoD Tensor. Currently, serving only supports the usage of one-dimensional LOD.
## Use
**Prerequisite：** Your prediction model needs to support variable length tensor input.
Take the visual task as an example. In the visual task, we often need to process video and image. These elements are high-dimensional objects.
Suppose that an existing Mini batch contains three videos, each video contains three frames, one frame and two frames respectively.
If each frame has the same size: 640x480, the mini batch can be expressed as:
```
3 1 2
口口口 口 口口
```
The size of the bottom tenor is (3 + 1 + 2) x640x480, and each 口 represents a 640x480 image.
Then, the shape of tensor is [6,640,480], lod=[0,3,4,6].
Where 0 is the starting value and 3-0 = 3; 4-3=1; 6-4 = 2, these three values just represent your variable length information.
The last element 6 in LOD should be equal to the total length of the first dimension in shape.
The variable length information recorded in LOD and the first dimension information of shape in tensor should be aligned in the above way.
doc/LOD_CN.md 0 → 100644
浏览文件 @ 9cfb1072
# Lod字段说明
(简体中文|[English](LOD.md))
## 概念
LoD(Level-of-Detail) Tensor是Paddle的高级特性，是对Tensor的一种扩充。LoDTensor通过牺牲灵活性来提升训练的效率。
**注：** 对于大部分用户来说，无需关注LoDTensor的用法，目前Serving中仅支持一维Lod的用法。
## 使用
**前提：** 首先您的预测模型需要支持变长Tensor的输入。
以视觉任务为例，在视觉任务中，时常需要处理视频和图像这些元素是高维的对象，假设现存的一个mini-batch包含3个视频，分别有3个，1个和2个帧。
每个帧都具有相同大小：640x480，则这个mini-batch可以被表示为：
```
3 1 2
口口口 口 口口
```
最底层tensor大小为（3+1+2）x640x480，每一个 口 表示一个640x480的图像。
那么此时，Tensor的shape为[6,640,480],lod=[0,3,4,6].
其中0为起始值，3-0=3;4-3=1;6-4=2，这三个值正好表示您的变长信息，lod中的最后一个元素6，应等于shape中第一维度的总长度。
lod中记录的变长信息与Tensor中shape的第一维度的信息应按照上述方式对齐。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或