提交 3d9d9475 编写于 作者: G gineshidalgo99

Body part candidates can be saved

上级 9ce21690
......@@ -162,11 +162,12 @@ Each flag is divided into flag name, default value, and description.
- DEFINE_int32(scale_number, 1, "Number of scales to average.");
- DEFINE_double(scale_gap, 0.3, "Scale gap between scales. No effect unless scale_number > 1. Initial scale is always 1. If you want to change the initial scale, you actually want to multiply the `net_resolution` by your desired initial scale.");
5. OpenPose Body Pose Heatmaps
- DEFINE_bool(heatmaps_add_parts, false, "If true, it will add the body part heatmaps to the final op::Datum::poseHeatMaps array, and analogously face & hand heatmaps to op::Datum::faceHeatMaps & op::Datum::handHeatMaps (program speed will decrease). Not required for our library, enable it only if you intend to process this information later. If more than one `add_heatmaps_X` flag is enabled, it will place then in sequential memory order: body parts + bkg + PAFs. It will follow the order on POSE_BODY_PART_MAPPING in `include/openpose/pose/poseParameters.hpp`.");
5. OpenPose Body Pose Heatmaps and Part Candidates
- DEFINE_bool(heatmaps_add_parts, false, "If true, it will fill op::Datum::poseHeatMaps array with the body part heatmaps, and analogously face & hand heatmaps to op::Datum::faceHeatMaps & op::Datum::handHeatMaps. If more than one `add_heatmaps_X` flag is enabled, it will place then in sequential memory order: body parts + bkg + PAFs. It will follow the order on POSE_BODY_PART_MAPPING in `src/openpose/pose/poseParameters.cpp`. Program speed will considerably decrease. Not required for OpenPose, enable it only if you intend to explicitly use this information later.");
- DEFINE_bool(heatmaps_add_bkg, false, "Same functionality as `add_heatmaps_parts`, but adding the heatmap corresponding to background.");
- DEFINE_bool(heatmaps_add_PAFs, false, "Same functionality as `add_heatmaps_parts`, but adding the PAFs.");
- DEFINE_int32(heatmaps_scale, 2, "Set 0 to scale op::Datum::poseHeatMaps in the range [-1,1], 1 for [0,1]; 2 for integer rounded [0,255]; and 3 for no scaling.");
- DEFINE_bool(part_candidates, false, "Also enable `write_json` in order to save this information. If true, it will fill the op::Datum::poseCandidates array with the body part candidates. Candidates refer to all the detected body parts, before being assembled into people. Note that the number of candidates is equal or higher than the number of final body parts (i.e. after being assembled into people). The empty body parts are filled with 0s. Program speed will slightly decrease. Not required for OpenPose, enable it only if you intend to explicitly use this information.");
6. OpenPose Face
- DEFINE_bool(face, false, "Enables face keypoint detection. It will share some parameters from the body pose, e.g. `model_folder`. Note that this will considerable slow down the performance and increse the required GPU memory. In addition, the greater number of people on the image, the slower OpenPose will be.");
......@@ -209,9 +210,10 @@ Each flag is divided into flag name, default value, and description.
- DEFINE_string(write_images, "", "Directory to write rendered frames in `write_images_format` image format.");
- DEFINE_string(write_images_format, "png", "File extension and format for `write_images`, e.g. png, jpg or bmp. Check the OpenCV function cv::imwrite for all compatible extensions.");
- DEFINE_string(write_video, "", "Full file path to write rendered frames in motion JPEG video format. It might fail if the final path does not finish in `.avi`. It internally uses cv::VideoWriter.");
- DEFINE_string(write_keypoint, "", "Directory to write the people body pose keypoint data. Set format with `write_keypoint_format`.");
- DEFINE_string(write_keypoint_format, "yml", "File extension and format for `write_keypoint`: json, xml, yaml & yml. Json not available for OpenCV < 3.0, use `write_keypoint_json` instead.");
- DEFINE_string(write_keypoint_json, "", "Directory to write people pose data in *.json format, compatible with any OpenCV version.");
- DEFINE_string(write_coco_json, "", "Full file path to write people pose data with *.json COCO validation format.");
- DEFINE_string(write_heatmaps, "", "Directory to write body pose heatmaps in *.png format. At least 1 `add_heatmaps_X` flag must be enabled.");
- DEFINE_string(write_json, "", "Directory to write OpenPose output in JSON format. It includes body, hand, and face pose keypoints, as well as pose candidates (if `--part_candidates` enabled).");
- DEFINE_string(write_coco_json, "", "Full file path to write people pose data with JSON COCO validation format.");
- DEFINE_string(write_heatmaps, "", "Directory to write body pose heatmaps in PNG format. At least 1 `add_heatmaps_X` flag must be enabled.");
- DEFINE_string(write_heatmaps_format, "png", "File extension and format for `write_heatmaps`, analogous to `write_images_format`. For lossless compression, recommended `png` for integer `heatmaps_scale` and `float` for floating values.");
- DEFINE_string(write_keypoint, "", "(Deprecated, use `write_json`) Directory to write the people pose keypoint data. Set format with `write_keypoint_format`.");
- DEFINE_string(write_keypoint_format, "yml", "(Deprecated, use `write_json`) File extension and format for `write_keypoint`: json, xml, yaml & yml. Json not available for OpenCV < 3.0, use `write_keypoint_json` instead.");
- DEFINE_string(write_keypoint_json, "", "(Deprecated, use `write_json`) Directory to write people pose data in JSON format, compatible with any OpenCV version.");
......@@ -33,9 +33,9 @@ OpenPose Library - Release Notes
7. It prints out the error description before throwing the exception (so that it is written on the Windows cmd).
8. Highly improved documentation.
2. Functions or parameters renamed:
1. Flag `write_pose` renamed as `write_keypoint` and it also applies to face and/or hands.
2. Flag `write_pose_json` renamed as `write_keypoint_json` and it also applies to face and/or hands.
3. Flag `write_pose_format` renamed as `write_keypoint_format` and it also applies to face and/or hands.
1. Flag `--write_pose` renamed as `--write_keypoint` and it also applies to face and/or hands.
2. Flag `--write_pose_json` renamed as `--write_keypoint_json` and it also applies to face and/or hands.
3. Flag `--write_pose_format` renamed as `--write_keypoint_format` and it also applies to face and/or hands.
4. PoseSaver and its JSON variant renamed as KeypointSaver.
5. PoseJsonCocoSaver renamed as CocoJsonSaver.
3. Main bugs fixed:
......@@ -55,22 +55,22 @@ OpenPose Library - Release Notes
7. Rendering threshold for pose, face and hands becomes user-configurable.
8. Check() functions give more feedback.
9. WCocoJsonSaver finished and removed its 3599-image limit.
10. Added `camera_fps` so generated video will use that frame rate.
10. Added `--camera_fps` so generated video will use that frame rate.
11. Reduced the number of printed information messages. Default logging priority threshold increased to Priority::Max.
12. Google flags to OpenPose configuration parameters reader moved from each demo to utilities/flagsToOpenPose.
13. Nms classes do not use `numberParts` for `Reshape`, they deduce the value.
14. Improved documentation.
2. Functions or parameters renamed:
1. Render flags renamed in the demo in order to incorporate the CPU/GPU rendering.
2. Keypoints saved in JSON files (`write_keypoint_json`) are now saved as `pose_keypoints`, `face_keypoints`, `hand_left_keypoints`, and `hand_right_keypoints`. They all were previously saved as `body_parts`.
3. Flag `num_scales` renamed as `scale_number`.
4. All hand and pose flags renamed such as they start by `hand_` and `face_` respectively.
2. Keypoints saved in JSON files (`--write_keypoint_json`) are now saved as `pose_keypoints`, `face_keypoints`, `hand_left_keypoints`, and `hand_right_keypoints`. They all were previously saved as `body_parts`.
3. Flag `--num_scales` renamed as `--scale_number`.
4. All hand and pose flags renamed such as they start by `--hand_` and `--face_` respectively.
3. Main bugs fixed:
1. Fixed bug in Array::getConstCvMat() if mVolume=0, now returning empty cv::Mat.
2. Fixed bug: `--process_real_time` threw error with webcam.
3. Fixed bug: Face not working when input and output resolutions are different.
4. Fixed some bugs that prevented debug version to run.
5. Face saved in JSON files were called `body_parts`. Now they are called `face_keypoints`.
5. Face saved in JSON files were called `--body_parts`. Now they are called `--face_keypoints`.
......@@ -129,14 +129,14 @@ OpenPose Library - Release Notes
9. OpenPose successfully compiles if the flags `USE_CAFFE` and/or `USE_CUDA` are not enabled, although it will give an error saying they are required.
10. COCO JSON file outputs 0 as score for non-detected keypoints.
11. Added example for OpenPose for user asynchronous output and cleaned all `tutorial_wrapper/` examples.
12. Added `-1` option for `net_resolution` in order to auto-select the best possible aspect ratio given the user input.
12. Added `-1` option for `--net_resolution` in order to auto-select the best possible aspect ratio given the user input.
13. Net resolution can be dynamically changed (e.g. for images with different size).
14. Added example to add functionality/modules to OpenPose.
15. Added `disable_multi_thread` flag in order to allow debug and/or highly reduce the latency (e.g. when using webcam in real-time).
15. Added `--disable_multi_thread` flag in order to allow debug and/or highly reduce the latency (e.g. when using webcam in real-time).
16. Allowed to output images without any rendering.
2. Functions or parameters renamed:
1. OpenPose able to change its size and initial size dynamically:
1. Flag `resolution` renamed as `output_resolution`.
1. Flag `--resolution` renamed as `--output_resolution`.
2. FrameDisplayer, GuiInfoAdder and Gui constructors arguments modified (gui module).
3. OpOutputToCvMat constructor removed (core module).
4. New Renders classes to split GpuRenderers from CpuRenderers.
......@@ -151,14 +151,17 @@ OpenPose Library - Release Notes
## Current version (future OpenPose 1.3.0)
## Current version (future OpenPose 1.2.1)
1. Main improvements:
1. Heatmaps can be saved in floating format.
2. More efficient non-processing version (i.e. if all keypoint extractors are disabled, and only image extraction and display/saving operations are performed).
3. Heat maps scaling: Added `heatmaps_scale` to OpenPoseDemo, added option not to scale the heatmaps, and added custom `float` format to save heatmaps in floating format.
2. More efficient non-processing version (i.e., if all keypoint extractors are disabled, and only image extraction and display/saving operations are performed).
3. Heat maps scaling: Added `--heatmaps_scale` to OpenPoseDemo, added option not to scale the heatmaps, and added custom `float` format to save heatmaps in floating format.
4. Detector of the number of GPU also considers the initial GPU index given by the user.
5. Added `--write_json` as new version of `--write_keypoint_json`. It includes the body part candidates (if enabled), as well as any extra information added in the future (e.g. person ID).
6. Body part candidates can be retrieved in op::Datum and saved with `--write_json`.
2. Functions or parameters renamed:
1. `PoseParameters` splitted into `PoseParameters` and `PoseParametersRender` and const parameters turned into functions for more clarity.
3. Main bugs fixed:
1. Render working on images > 4K (#324).
2. Cleaned redundant arguments on `getAverageScore` and `getKeypointsArea`.
3. Slight speed up when heatmaps must be returned to the user (not doing a double copy anymore).
......@@ -87,18 +87,26 @@ DEFINE_int32(scale_number, 1, "Number of scales to ave
DEFINE_double(scale_gap, 0.3, "Scale gap between scales. No effect unless scale_number > 1. Initial scale is always 1."
" If you want to change the initial scale, you actually want to multiply the"
" `net_resolution` by your desired initial scale.");
// OpenPose Body Pose Heatmaps
DEFINE_bool(heatmaps_add_parts, false, "If true, it will add the body part heatmaps to the final op::Datum::poseHeatMaps array,"
" and analogously face & hand heatmaps to op::Datum::faceHeatMaps & op::Datum::handHeatMaps"
" (program speed will decrease). Not required for our library, enable it only if you intend"
" to process this information later. If more than one `add_heatmaps_X` flag is enabled, it"
" will place then in sequential memory order: body parts + bkg + PAFs. It will follow the"
" order on POSE_BODY_PART_MAPPING in `include/openpose/pose/poseParameters.hpp`.");
// OpenPose Body Pose Heatmaps and Part Candidates
DEFINE_bool(heatmaps_add_parts, false, "If true, it will fill op::Datum::poseHeatMaps array with the body part heatmaps, and"
" analogously face & hand heatmaps to op::Datum::faceHeatMaps & op::Datum::handHeatMaps."
" If more than one `add_heatmaps_X` flag is enabled, it will place then in sequential"
" memory order: body parts + bkg + PAFs. It will follow the order on"
" POSE_BODY_PART_MAPPING in `src/openpose/pose/poseParameters.cpp`. Program speed will"
" considerably decrease. Not required for OpenPose, enable it only if you intend to"
" explicitly use this information later.");
DEFINE_bool(heatmaps_add_bkg, false, "Same functionality as `add_heatmaps_parts`, but adding the heatmap corresponding to"
" background.");
DEFINE_bool(heatmaps_add_PAFs, false, "Same functionality as `add_heatmaps_parts`, but adding the PAFs.");
DEFINE_int32(heatmaps_scale, 2, "Set 0 to scale op::Datum::poseHeatMaps in the range [-1,1], 1 for [0,1]; 2 for integer"
" rounded [0,255]; and 3 for no scaling.");
DEFINE_bool(part_candidates, false, "Also enable `write_json` in order to save this information. If true, it will fill the"
" op::Datum::poseCandidates array with the body part candidates. Candidates refer to all"
" the detected body parts, before being assembled into people. Note that the number of"
" candidates is equal or higher than the number of final body parts (i.e. after being"
" assembled into people). The empty body parts are filled with 0s. Program speed will"
" slightly decrease. Not required for OpenPose, enable it only if you intend to explicitly"
" use this information.");
// OpenPose Face
DEFINE_bool(face, false, "Enables face keypoint detection. It will share some parameters from the body pose, e.g."
" `model_folder`. Note that this will considerable slow down the performance and increse"
......@@ -166,16 +174,20 @@ DEFINE_string(write_images_format, "png", "File extension and form
" function cv::imwrite for all compatible extensions.");
DEFINE_string(write_video, "", "Full file path to write rendered frames in motion JPEG video format. It might fail if the"
" final path does not finish in `.avi`. It internally uses cv::VideoWriter.");
DEFINE_string(write_keypoint, "", "Directory to write the people body pose keypoint data. Set format with `write_keypoint_format`.");
DEFINE_string(write_keypoint_format, "yml", "File extension and format for `write_keypoint`: json, xml, yaml & yml. Json not available"
" for OpenCV < 3.0, use `write_keypoint_json` instead.");
DEFINE_string(write_keypoint_json, "", "Directory to write people pose data in *.json format, compatible with any OpenCV version.");
DEFINE_string(write_coco_json, "", "Full file path to write people pose data with *.json COCO validation format.");
DEFINE_string(write_heatmaps, "", "Directory to write body pose heatmaps in *.png format. At least 1 `add_heatmaps_X` flag"
DEFINE_string(write_json, "", "Directory to write OpenPose output in JSON format. It includes body, hand, and face pose"
" keypoints, as well as pose candidates (if `--part_candidates` enabled).");
DEFINE_string(write_coco_json, "", "Full file path to write people pose data with JSON COCO validation format.");
DEFINE_string(write_heatmaps, "", "Directory to write body pose heatmaps in PNG format. At least 1 `add_heatmaps_X` flag"
" must be enabled.");
DEFINE_string(write_heatmaps_format, "png", "File extension and format for `write_heatmaps`, analogous to `write_images_format`."
" For lossless compression, recommended `png` for integer `heatmaps_scale` and `float` for"
" floating values.");
DEFINE_string(write_keypoint, "", "(Deprecated, use `write_json`) Directory to write the people pose keypoint data. Set format"
" with `write_keypoint_format`.");
DEFINE_string(write_keypoint_format, "yml", "(Deprecated, use `write_json`) File extension and format for `write_keypoint`: json, xml,"
" yaml & yml. Json not available for OpenCV < 3.0, use `write_keypoint_json` instead.");
DEFINE_string(write_keypoint_json, "", "(Deprecated, use `write_json`) Directory to write people pose data in JSON format,"
" compatible with any OpenCV version.");
int openPoseDemo()
{
......@@ -202,6 +214,11 @@ int openPoseDemo()
FLAGS_camera_resolution, FLAGS_camera_fps);
// poseModel
const auto poseModel = op::flagsToPoseModel(FLAGS_model_pose);
// JSON saving
const auto writeJson = (!FLAGS_write_json.empty() ? FLAGS_write_json : FLAGS_write_keypoint_json);
if (!FLAGS_write_keypoint.empty() || !FLAGS_write_keypoint_json.empty())
op::log("Flags `write_keypoint` and `write_keypoint_json` are deprecated and will eventually be removed."
" Please, use `write_json` instead.", op::Priority::Max);
// keypointScale
const auto keypointScale = op::flagsToScaleMode(FLAGS_keypoint_scale);
// heatmaps to add
......@@ -222,8 +239,9 @@ int openPoseDemo()
(float)FLAGS_scale_gap, op::flagsToRenderMode(FLAGS_render_pose),
poseModel, !FLAGS_disable_blending, (float)FLAGS_alpha_pose,
(float)FLAGS_alpha_heatmap, FLAGS_part_to_show, FLAGS_model_folder,
heatMapTypes, heatMapScale, (float)FLAGS_render_threshold,
enableGoogleLogging, FLAGS_identification};
heatMapTypes, heatMapScale, FLAGS_part_candidates,
(float)FLAGS_render_threshold, enableGoogleLogging,
FLAGS_identification};
// Face configuration (use op::WrapperStructFace{} to disable it)
const op::WrapperStructFace wrapperStructFace{FLAGS_face, faceNetInputSize,
op::flagsToRenderMode(FLAGS_face_render, FLAGS_render_pose),
......@@ -243,7 +261,7 @@ int openPoseDemo()
const op::WrapperStructOutput wrapperStructOutput{!FLAGS_no_display, !FLAGS_no_gui_verbose, FLAGS_fullscreen,
FLAGS_write_keypoint,
op::stringToDataFormat(FLAGS_write_keypoint_format),
FLAGS_write_keypoint_json, FLAGS_write_coco_json,
writeJson, FLAGS_write_coco_json,
FLAGS_write_images, FLAGS_write_images_format, FLAGS_write_video,
FLAGS_write_heatmaps, FLAGS_write_heatmaps_format};
// Configure wrapper
......
......@@ -30,7 +30,7 @@ DEFINE_bool(hand_tracking, false, "");
// Display
DEFINE_bool(no_display, false, "");
// Result Saving
DEFINE_string(write_keypoint_json, "", "");
DEFINE_string(write_json, "", "");
int handFromJsonTest()
{
......@@ -67,7 +67,7 @@ int handFromJsonTest()
op::flagsToRenderMode(1)};
// Configure wrapper
opWrapper.configure(wrapperStructPose, wrapperStructHand, producerSharedPtr, FLAGS_hand_ground_truth,
FLAGS_write_keypoint_json, !FLAGS_no_display);
FLAGS_write_json, !FLAGS_no_display);
// Start processing
op::log("Starting thread(s)", op::Priority::High);
......
......@@ -11,10 +11,10 @@
# Read that script for details about all the paths and change them for your own paths.
# Careful:
# If you are using the NAS, please do not override my files, i.e. please change the output paths (corresponding to the ones indicated by `--write_keypoint_json`, which is ).
# If you are using the NAS, please do not override my files, i.e. please change the output paths (corresponding to the ones indicated by `--write_json`, which is ).
# In order to generate the JSON output:
# Uncomment the commented lines starting by `--write_keypoint_json` and `--no_display`
# Uncomment the commented lines starting by `--write_json` and `--no_display`
# Step 2 - Running JSON output to get accuracy
# Once you have the JSON files, run them with the script Tomas prepared for it, which in my case I use:
......@@ -32,12 +32,12 @@ HAND_TESTING_FOLDER="/media/posefs3b/Users/gines/openpose_train/dataset/hand_tes
IMAGES_FOLDER=${HAND_TESTING_FOLDER}"0_images/"
IMAGES_BB_FOLDER=${HAND_TESTING_FOLDER}"3_images_bounding_box"
HAND_GROUND_TRUTH_FOLDER=${HAND_TESTING_FOLDER}"4_hand_detections"
KEYPOINT_JSON_FOLDER=${HAND_TESTING_FOLDER}"5_keypointJson/"
PEOPLE_JSON_FOLDER=${HAND_TESTING_FOLDER}"5_keypointJson/"
# Variable paths
SCALES=6
SUFFIX="_${SCALES}"
HAND_RESULTS_FOLDER_BASE=${KEYPOINT_JSON_FOLDER}"hand_keypoints_estimated"
HAND_RESULTS_FOLDER_BASE=${PEOPLE_JSON_FOLDER}"hand_keypoints_estimated"
HAND_RESULTS_FOLDER_NO_BB=${HAND_RESULTS_FOLDER_BASE}"_old"${SUFFIX}
HAND_RESULTS_FOLDER_BB=${HAND_RESULTS_FOLDER_BASE}"_BBox"${SUFFIX}
HAND_RESULTS_FOLDER_BODY_59=${HAND_RESULTS_FOLDER_BASE}"_BODY_59"
......@@ -52,7 +52,7 @@ rm -rf $HAND_RESULTS_FOLDER_BB
--hand_scale_number ${SCALES} --hand_scale_range 0.4 \
--image_dir ${IMAGES_BB_FOLDER} \
--hand_ground_truth ${HAND_GROUND_TRUTH_FOLDER} \
--write_keypoint_json $HAND_RESULTS_FOLDER_BB \
--write_json $HAND_RESULTS_FOLDER_BB \
--no_display
......@@ -65,7 +65,7 @@ rm -rf $HAND_RESULTS_FOLDER_NO_BB
--hand \
--hand_scale_number ${SCALES} --hand_scale_range 0.4 \
--image_dir ${IMAGES_FOLDER} \
--write_keypoint_json $HAND_RESULTS_FOLDER_NO_BB \
--write_json $HAND_RESULTS_FOLDER_NO_BB \
--no_display
......@@ -77,5 +77,5 @@ rm -rf $HAND_RESULTS_FOLDER_BODY_59
./build/examples/openpose/openpose.bin \
--model_pose BODY_59 \
--image_dir ${IMAGES_FOLDER} \
--write_keypoint_json $HAND_RESULTS_FOLDER_BODY_59 \
--write_json $HAND_RESULTS_FOLDER_BODY_59 \
--no_display
......@@ -26,7 +26,7 @@ namespace op
const WrapperStructHand& wrapperStructHand,
const std::shared_ptr<Producer>& producerSharedPtr,
const std::string& handGroundTruth,
const std::string& writeKeypointJson,
const std::string& writeJson,
const bool displayGui = false);
/**
......@@ -119,7 +119,7 @@ namespace op
const WrapperStructHand& wrapperStructHand,
const std::shared_ptr<Producer>& producerSharedPtr,
const std::string& handGroundTruth,
const std::string& writeKeypointJson,
const std::string& writeJson,
const bool displayGui)
{
try
......@@ -134,7 +134,7 @@ namespace op
error("The scale gap must be greater than 0 (it has no effect if the number of scales is 1).", __LINE__, __FUNCTION__, __FILE__);
const std::string additionalMessage = " You could also set mThreadManagerMode = mThreadManagerMode::Asynchronous(Out) and/or add your own"
" output worker class before calling this function.";
const auto savingSomething = !writeKeypointJson.empty();
const auto savingSomething = !writeJson.empty();
if (!displayGui && !savingSomething)
{
const auto message = "No output is selected (`no_display`) and no results are generated (no `write_X` flags enabled). Thus,"
......@@ -157,7 +157,7 @@ namespace op
}
// Proper format
const auto writeKeypointJsonCleaned = formatAsDirectory(writeKeypointJson);
const auto writeJsonCleaned = formatAsDirectory(writeJson);
// Common parameters
const auto finalOutputSize = wrapperStructPose.outputSize;
......@@ -243,10 +243,10 @@ namespace op
mOutputWs.clear();
// Write people pose data on disk (json format)
if (!writeKeypointJsonCleaned.empty())
if (!writeJsonCleaned.empty())
{
const auto keypointJsonSaver = std::make_shared<KeypointJsonSaver>(writeKeypointJsonCleaned);
mOutputWs.emplace_back(std::make_shared<WKeypointJsonSaver<TDatumsPtr>>(keypointJsonSaver));
const auto jsonSaver = std::make_shared<PeopleJsonSaver>(writeJsonCleaned);
mOutputWs.emplace_back(std::make_shared<WPeopleJsonSaver<TDatumsPtr>>(jsonSaver));
}
// Minimal graphical user interface (GUI)
spWGui = nullptr;
......
......@@ -96,18 +96,26 @@ DEFINE_int32(scale_number, 1, "Number of scales to ave
DEFINE_double(scale_gap, 0.3, "Scale gap between scales. No effect unless scale_number > 1. Initial scale is always 1."
" If you want to change the initial scale, you actually want to multiply the"
" `net_resolution` by your desired initial scale.");
// OpenPose Body Pose Heatmaps
DEFINE_bool(heatmaps_add_parts, false, "If true, it will add the body part heatmaps to the final op::Datum::poseHeatMaps array,"
" and analogously face & hand heatmaps to op::Datum::faceHeatMaps & op::Datum::handHeatMaps"
" (program speed will decrease). Not required for our library, enable it only if you intend"
" to process this information later. If more than one `add_heatmaps_X` flag is enabled, it"
" will place then in sequential memory order: body parts + bkg + PAFs. It will follow the"
" order on POSE_BODY_PART_MAPPING in `include/openpose/pose/poseParameters.hpp`.");
// OpenPose Body Pose Heatmaps and Part Candidates
DEFINE_bool(heatmaps_add_parts, false, "If true, it will fill op::Datum::poseHeatMaps array with the body part heatmaps, and"
" analogously face & hand heatmaps to op::Datum::faceHeatMaps & op::Datum::handHeatMaps."
" If more than one `add_heatmaps_X` flag is enabled, it will place then in sequential"
" memory order: body parts + bkg + PAFs. It will follow the order on"
" POSE_BODY_PART_MAPPING in `src/openpose/pose/poseParameters.cpp`. Program speed will"
" considerably decrease. Not required for OpenPose, enable it only if you intend to"
" explicitly use this information later.");
DEFINE_bool(heatmaps_add_bkg, false, "Same functionality as `add_heatmaps_parts`, but adding the heatmap corresponding to"
" background.");
DEFINE_bool(heatmaps_add_PAFs, false, "Same functionality as `add_heatmaps_parts`, but adding the PAFs.");
DEFINE_int32(heatmaps_scale, 2, "Set 0 to scale op::Datum::poseHeatMaps in the range [-1,1], 1 for [0,1]; 2 for integer"
" rounded [0,255]; and 3 for no scaling.");
DEFINE_bool(part_candidates, false, "Also enable `write_json` in order to save this information. If true, it will fill the"
" op::Datum::poseCandidates array with the body part candidates. Candidates refer to all"
" the detected body parts, before being assembled into people. Note that the number of"
" candidates is equal or higher than the number of final body parts (i.e. after being"
" assembled into people). The empty body parts are filled with 0s. Program speed will"
" slightly decrease. Not required for OpenPose, enable it only if you intend to explicitly"
" use this information.");
// OpenPose Face
DEFINE_bool(face, false, "Enables face keypoint detection. It will share some parameters from the body pose, e.g."
" `model_folder`. Note that this will considerable slow down the performance and increse"
......@@ -175,16 +183,20 @@ DEFINE_string(write_images_format, "png", "File extension and form
" function cv::imwrite for all compatible extensions.");
DEFINE_string(write_video, "", "Full file path to write rendered frames in motion JPEG video format. It might fail if the"
" final path does not finish in `.avi`. It internally uses cv::VideoWriter.");
DEFINE_string(write_keypoint, "", "Directory to write the people body pose keypoint data. Set format with `write_keypoint_format`.");
DEFINE_string(write_keypoint_format, "yml", "File extension and format for `write_keypoint`: json, xml, yaml & yml. Json not available"
" for OpenCV < 3.0, use `write_keypoint_json` instead.");
DEFINE_string(write_keypoint_json, "", "Directory to write people pose data in *.json format, compatible with any OpenCV version.");
DEFINE_string(write_coco_json, "", "Full file path to write people pose data with *.json COCO validation format.");
DEFINE_string(write_heatmaps, "", "Directory to write body pose heatmaps in *.png format. At least 1 `add_heatmaps_X` flag"
DEFINE_string(write_json, "", "Directory to write OpenPose output in JSON format. It includes body, hand, and face pose"
" keypoints, as well as pose candidates (if `--part_candidates` enabled).");
DEFINE_string(write_coco_json, "", "Full file path to write people pose data with JSON COCO validation format.");
DEFINE_string(write_heatmaps, "", "Directory to write body pose heatmaps in PNG format. At least 1 `add_heatmaps_X` flag"
" must be enabled.");
DEFINE_string(write_heatmaps_format, "png", "File extension and format for `write_heatmaps`, analogous to `write_images_format`."
" For lossless compression, recommended `png` for integer `heatmaps_scale` and `float` for"
" floating values.");
DEFINE_string(write_keypoint, "", "(Deprecated, use `write_json`) Directory to write the people pose keypoint data. Set format"
" with `write_keypoint_format`.");
DEFINE_string(write_keypoint_format, "yml", "(Deprecated, use `write_json`) File extension and format for `write_keypoint`: json, xml,"
" yaml & yml. Json not available for OpenCV < 3.0, use `write_keypoint_json` instead.");
DEFINE_string(write_keypoint_json, "", "(Deprecated, use `write_json`) Directory to write people pose data in JSON format,"
" compatible with any OpenCV version.");
int openPoseTutorialWrapper4()
{
......@@ -211,6 +223,11 @@ int openPoseTutorialWrapper4()
FLAGS_camera_resolution, FLAGS_camera_fps);
// poseModel
const auto poseModel = op::flagsToPoseModel(FLAGS_model_pose);
// JSON saving
const auto writeJson = (!FLAGS_write_json.empty() ? FLAGS_write_json : FLAGS_write_keypoint_json);
if (!FLAGS_write_keypoint.empty() || !FLAGS_write_keypoint_json.empty())
op::log("Flags `write_keypoint` and `write_keypoint_json` are deprecated and will eventually be removed."
" Please, use `write_json` instead.", op::Priority::Max);
// keypointScale
const auto keypointScale = op::flagsToScaleMode(FLAGS_keypoint_scale);
// heatmaps to add
......@@ -231,8 +248,8 @@ int openPoseTutorialWrapper4()
(float)FLAGS_scale_gap, op::flagsToRenderMode(FLAGS_render_pose),
poseModel, !FLAGS_disable_blending, (float)FLAGS_alpha_pose,
(float)FLAGS_alpha_heatmap, FLAGS_part_to_show, FLAGS_model_folder,
heatMapTypes, heatMapScale, (float)FLAGS_render_threshold,
enableGoogleLogging};
heatMapTypes, heatMapScale, FLAGS_part_candidates,
(float)FLAGS_render_threshold, enableGoogleLogging};
// Face configuration (use op::WrapperStructFace{} to disable it)
const op::WrapperStructFace wrapperStructFace{FLAGS_face, faceNetInputSize,
op::flagsToRenderMode(FLAGS_face_render, FLAGS_render_pose),
......@@ -252,7 +269,7 @@ int openPoseTutorialWrapper4()
const op::WrapperStructOutput wrapperStructOutput{!FLAGS_no_display, !FLAGS_no_gui_verbose, FLAGS_fullscreen,
FLAGS_write_keypoint,
op::stringToDataFormat(FLAGS_write_keypoint_format),
FLAGS_write_keypoint_json, FLAGS_write_coco_json,
writeJson, FLAGS_write_coco_json,
FLAGS_write_images, FLAGS_write_images_format, FLAGS_write_video,
FLAGS_write_heatmaps, FLAGS_write_heatmaps_format};
......
......@@ -83,16 +83,13 @@ int openPoseTutorialPose1()
if (FLAGS_scale_gap <= 0. && FLAGS_scale_number > 1)
op::error("Incompatible flag configuration: scale_gap must be greater than 0 or scale_number = 1.",
__LINE__, __FUNCTION__, __FILE__);
// Enabling Google Logging
const bool enableGoogleLogging = true;
// Logging
op::log("", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__);
// Step 3 - Initialize all required classes
op::ScaleAndSizeExtractor scaleAndSizeExtractor(netInputSize, outputSize, FLAGS_scale_number, FLAGS_scale_gap);
op::CvMatToOpInput cvMatToOpInput;
op::CvMatToOpOutput cvMatToOpOutput;
op::PoseExtractorCaffe poseExtractorCaffe{poseModel, FLAGS_model_folder,
FLAGS_num_gpu_start, {}, op::ScaleMode::ZeroToOne, enableGoogleLogging};
op::PoseExtractorCaffe poseExtractorCaffe{poseModel, FLAGS_model_folder, FLAGS_num_gpu_start};
op::PoseCpuRenderer poseRenderer{poseModel, (float)FLAGS_render_threshold, !FLAGS_disable_blending,
(float)FLAGS_alpha_pose};
op::OpOutputToCvMat opOutputToCvMat;
......
......@@ -88,18 +88,14 @@ int openPoseTutorialPose2()
if (FLAGS_scale_gap <= 0. && FLAGS_scale_number > 1)
op::error("Incompatible flag configuration: scale_gap must be greater than 0 or scale_number = 1.",
__LINE__, __FUNCTION__, __FILE__);
// Enabling Google Logging
const bool enableGoogleLogging = true;
// Logging
op::log("", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__);
// Step 3 - Initialize all required classes
op::ScaleAndSizeExtractor scaleAndSizeExtractor(netInputSize, outputSize, FLAGS_scale_number, FLAGS_scale_gap);
op::CvMatToOpInput cvMatToOpInput;
op::CvMatToOpOutput cvMatToOpOutput;
auto poseExtractorPtr = std::make_shared<op::PoseExtractorCaffe>(
poseModel, FLAGS_model_folder, FLAGS_num_gpu_start, std::vector<op::HeatMapType>{}, op::ScaleMode::ZeroToOne,
enableGoogleLogging
);
auto poseExtractorPtr = std::make_shared<op::PoseExtractorCaffe>(poseModel, FLAGS_model_folder,
FLAGS_num_gpu_start);
op::PoseGpuRenderer poseGpuRenderer{poseModel, poseExtractorPtr, (float)FLAGS_render_threshold,
!FLAGS_disable_blending, (float)FLAGS_alpha_pose, (float)FLAGS_alpha_heatmap};
poseGpuRenderer.setElementToRender(FLAGS_part_to_show);
......
......@@ -86,18 +86,26 @@ DEFINE_int32(scale_number, 1, "Number of scales to ave
DEFINE_double(scale_gap, 0.3, "Scale gap between scales. No effect unless scale_number > 1. Initial scale is always 1."
" If you want to change the initial scale, you actually want to multiply the"
" `net_resolution` by your desired initial scale.");
// OpenPose Body Pose Heatmaps
DEFINE_bool(heatmaps_add_parts, false, "If true, it will add the body part heatmaps to the final op::Datum::poseHeatMaps array,"
" and analogously face & hand heatmaps to op::Datum::faceHeatMaps & op::Datum::handHeatMaps"
" (program speed will decrease). Not required for our library, enable it only if you intend"
" to process this information later. If more than one `add_heatmaps_X` flag is enabled, it"
" will place then in sequential memory order: body parts + bkg + PAFs. It will follow the"
" order on POSE_BODY_PART_MAPPING in `include/openpose/pose/poseParameters.hpp`.");
// OpenPose Body Pose Heatmaps and Part Candidates
DEFINE_bool(heatmaps_add_parts, false, "If true, it will fill op::Datum::poseHeatMaps array with the body part heatmaps, and"
" analogously face & hand heatmaps to op::Datum::faceHeatMaps & op::Datum::handHeatMaps."
" If more than one `add_heatmaps_X` flag is enabled, it will place then in sequential"
" memory order: body parts + bkg + PAFs. It will follow the order on"
" POSE_BODY_PART_MAPPING in `src/openpose/pose/poseParameters.cpp`. Program speed will"
" considerably decrease. Not required for OpenPose, enable it only if you intend to"
" explicitly use this information later.");
DEFINE_bool(heatmaps_add_bkg, false, "Same functionality as `add_heatmaps_parts`, but adding the heatmap corresponding to"
" background.");
DEFINE_bool(heatmaps_add_PAFs, false, "Same functionality as `add_heatmaps_parts`, but adding the PAFs.");
DEFINE_int32(heatmaps_scale, 2, "Set 0 to scale op::Datum::poseHeatMaps in the range [-1,1], 1 for [0,1]; 2 for integer"
" rounded [0,255]; and 3 for no scaling.");
DEFINE_bool(part_candidates, false, "Also enable `write_json` in order to save this information. If true, it will fill the"
" op::Datum::poseCandidates array with the body part candidates. Candidates refer to all"
" the detected body parts, before being assembled into people. Note that the number of"
" candidates is equal or higher than the number of final body parts (i.e. after being"
" assembled into people). The empty body parts are filled with 0s. Program speed will"
" slightly decrease. Not required for OpenPose, enable it only if you intend to explicitly"
" use this information.");
// OpenPose Face
DEFINE_bool(face, false, "Enables face keypoint detection. It will share some parameters from the body pose, e.g."
" `model_folder`. Note that this will considerable slow down the performance and increse"
......@@ -159,16 +167,20 @@ DEFINE_string(write_images_format, "png", "File extension and form
" function cv::imwrite for all compatible extensions.");
DEFINE_string(write_video, "", "Full file path to write rendered frames in motion JPEG video format. It might fail if the"
" final path does not finish in `.avi`. It internally uses cv::VideoWriter.");
DEFINE_string(write_keypoint, "", "Directory to write the people body pose keypoint data. Set format with `write_keypoint_format`.");
DEFINE_string(write_keypoint_format, "yml", "File extension and format for `write_keypoint`: json, xml, yaml & yml. Json not available"
" for OpenCV < 3.0, use `write_keypoint_json` instead.");
DEFINE_string(write_keypoint_json, "", "Directory to write people pose data in *.json format, compatible with any OpenCV version.");
DEFINE_string(write_coco_json, "", "Full file path to write people pose data with *.json COCO validation format.");
DEFINE_string(write_heatmaps, "", "Directory to write body pose heatmaps in *.png format. At least 1 `add_heatmaps_X` flag"
DEFINE_string(write_json, "", "Directory to write OpenPose output in JSON format. It includes body, hand, and face pose"
" keypoints, as well as pose candidates (if `--part_candidates` enabled).");
DEFINE_string(write_coco_json, "", "Full file path to write people pose data with JSON COCO validation format.");
DEFINE_string(write_heatmaps, "", "Directory to write body pose heatmaps in PNG format. At least 1 `add_heatmaps_X` flag"
" must be enabled.");
DEFINE_string(write_heatmaps_format, "png", "File extension and format for `write_heatmaps`, analogous to `write_images_format`."
" For lossless compression, recommended `png` for integer `heatmaps_scale` and `float` for"
" floating values.");
DEFINE_string(write_keypoint, "", "(Deprecated, use `write_json`) Directory to write the people pose keypoint data. Set format"
" with `write_keypoint_format`.");
DEFINE_string(write_keypoint_format, "yml", "(Deprecated, use `write_json`) File extension and format for `write_keypoint`: json, xml,"
" yaml & yml. Json not available for OpenCV < 3.0, use `write_keypoint_json` instead.");
DEFINE_string(write_keypoint_json, "", "(Deprecated, use `write_json`) Directory to write people pose data in JSON format,"
" compatible with any OpenCV version.");
// If the user needs his own variables, he can inherit the op::Datum struct and add them
......@@ -287,6 +299,11 @@ int openPoseTutorialWrapper1()
FLAGS_camera_resolution, FLAGS_camera_fps);
// poseModel
const auto poseModel = op::flagsToPoseModel(FLAGS_model_pose);
// JSON saving
const auto writeJson = (!FLAGS_write_json.empty() ? FLAGS_write_json : FLAGS_write_keypoint_json);
if (!FLAGS_write_keypoint.empty() || !FLAGS_write_keypoint_json.empty())
op::log("Flags `write_keypoint` and `write_keypoint_json` are deprecated and will eventually be removed."
" Please, use `write_json` instead.", op::Priority::Max);
// keypointScale
const auto keypointScale = op::flagsToScaleMode(FLAGS_keypoint_scale);
// heatmaps to add
......@@ -307,8 +324,8 @@ int openPoseTutorialWrapper1()
(float)FLAGS_scale_gap, op::flagsToRenderMode(FLAGS_render_pose),
poseModel, !FLAGS_disable_blending, (float)FLAGS_alpha_pose,
(float)FLAGS_alpha_heatmap, FLAGS_part_to_show, FLAGS_model_folder,
heatMapTypes, heatMapScale, (float)FLAGS_render_threshold,
enableGoogleLogging};
heatMapTypes, heatMapScale, FLAGS_part_candidates,
(float)FLAGS_render_threshold, enableGoogleLogging};
// Face configuration (use op::WrapperStructFace{} to disable it)
const op::WrapperStructFace wrapperStructFace{FLAGS_face, faceNetInputSize,
op::flagsToRenderMode(FLAGS_face_render, FLAGS_render_pose),
......@@ -329,8 +346,9 @@ int openPoseTutorialWrapper1()
const bool guiVerbose = false;
const bool fullScreen = false;
const op::WrapperStructOutput wrapperStructOutput{displayGui, guiVerbose, fullScreen, FLAGS_write_keypoint,
op::stringToDataFormat(FLAGS_write_keypoint_format), FLAGS_write_keypoint_json,
FLAGS_write_coco_json, FLAGS_write_images, FLAGS_write_images_format, FLAGS_write_video,
op::stringToDataFormat(FLAGS_write_keypoint_format),
writeJson, FLAGS_write_coco_json,
FLAGS_write_images, FLAGS_write_images_format, FLAGS_write_video,
FLAGS_write_heatmaps, FLAGS_write_heatmaps_format};
// Configure wrapper
opWrapper.configure(wrapperStructPose, wrapperStructFace, wrapperStructHand, wrapperStructInput,
......
......@@ -69,18 +69,26 @@ DEFINE_int32(scale_number, 1, "Number of scales to ave
DEFINE_double(scale_gap, 0.3, "Scale gap between scales. No effect unless scale_number > 1. Initial scale is always 1."
" If you want to change the initial scale, you actually want to multiply the"
" `net_resolution` by your desired initial scale.");
// OpenPose Body Pose Heatmaps
DEFINE_bool(heatmaps_add_parts, false, "If true, it will add the body part heatmaps to the final op::Datum::poseHeatMaps array,"
" and analogously face & hand heatmaps to op::Datum::faceHeatMaps & op::Datum::handHeatMaps"
" (program speed will decrease). Not required for our library, enable it only if you intend"
" to process this information later. If more than one `add_heatmaps_X` flag is enabled, it"
" will place then in sequential memory order: body parts + bkg + PAFs. It will follow the"
" order on POSE_BODY_PART_MAPPING in `include/openpose/pose/poseParameters.hpp`.");
// OpenPose Body Pose Heatmaps and Part Candidates
DEFINE_bool(heatmaps_add_parts, false, "If true, it will fill op::Datum::poseHeatMaps array with the body part heatmaps, and"
" analogously face & hand heatmaps to op::Datum::faceHeatMaps & op::Datum::handHeatMaps."
" If more than one `add_heatmaps_X` flag is enabled, it will place then in sequential"
" memory order: body parts + bkg + PAFs. It will follow the order on"
" POSE_BODY_PART_MAPPING in `src/openpose/pose/poseParameters.cpp`. Program speed will"
" considerably decrease. Not required for OpenPose, enable it only if you intend to"
" explicitly use this information later.");
DEFINE_bool(heatmaps_add_bkg, false, "Same functionality as `add_heatmaps_parts`, but adding the heatmap corresponding to"
" background.");
DEFINE_bool(heatmaps_add_PAFs, false, "Same functionality as `add_heatmaps_parts`, but adding the PAFs.");
DEFINE_int32(heatmaps_scale, 2, "Set 0 to scale op::Datum::poseHeatMaps in the range [-1,1], 1 for [0,1]; 2 for integer"
" rounded [0,255]; and 3 for no scaling.");
DEFINE_bool(part_candidates, false, "Also enable `write_json` in order to save this information. If true, it will fill the"
" op::Datum::poseCandidates array with the body part candidates. Candidates refer to all"
" the detected body parts, before being assembled into people. Note that the number of"
" candidates is equal or higher than the number of final body parts (i.e. after being"
" assembled into people). The empty body parts are filled with 0s. Program speed will"
" slightly decrease. Not required for OpenPose, enable it only if you intend to explicitly"
" use this information.");
// OpenPose Face
DEFINE_bool(face, false, "Enables face keypoint detection. It will share some parameters from the body pose, e.g."
" `model_folder`. Note that this will considerable slow down the performance and increse"
......@@ -142,16 +150,20 @@ DEFINE_string(write_images_format, "png", "File extension and form
" function cv::imwrite for all compatible extensions.");
DEFINE_string(write_video, "", "Full file path to write rendered frames in motion JPEG video format. It might fail if the"
" final path does not finish in `.avi`. It internally uses cv::VideoWriter.");
DEFINE_string(write_keypoint, "", "Directory to write the people body pose keypoint data. Set format with `write_keypoint_format`.");
DEFINE_string(write_keypoint_format, "yml", "File extension and format for `write_keypoint`: json, xml, yaml & yml. Json not available"
" for OpenCV < 3.0, use `write_keypoint_json` instead.");
DEFINE_string(write_keypoint_json, "", "Directory to write people pose data in *.json format, compatible with any OpenCV version.");
DEFINE_string(write_coco_json, "", "Full file path to write people pose data with *.json COCO validation format.");
DEFINE_string(write_heatmaps, "", "Directory to write body pose heatmaps in *.png format. At least 1 `add_heatmaps_X` flag"
DEFINE_string(write_json, "", "Directory to write OpenPose output in JSON format. It includes body, hand, and face pose"
" keypoints, as well as pose candidates (if `--part_candidates` enabled).");
DEFINE_string(write_coco_json, "", "Full file path to write people pose data with JSON COCO validation format.");
DEFINE_string(write_heatmaps, "", "Directory to write body pose heatmaps in PNG format. At least 1 `add_heatmaps_X` flag"
" must be enabled.");
DEFINE_string(write_heatmaps_format, "png", "File extension and format for `write_heatmaps`, analogous to `write_images_format`."
" For lossless compression, recommended `png` for integer `heatmaps_scale` and `float` for"
" floating values.");
DEFINE_string(write_keypoint, "", "(Deprecated, use `write_json`) Directory to write the people pose keypoint data. Set format"
" with `write_keypoint_format`.");
DEFINE_string(write_keypoint_format, "yml", "(Deprecated, use `write_json`) File extension and format for `write_keypoint`: json, xml,"
" yaml & yml. Json not available for OpenCV < 3.0, use `write_keypoint_json` instead.");
DEFINE_string(write_keypoint_json, "", "(Deprecated, use `write_json`) Directory to write people pose data in JSON format,"
" compatible with any OpenCV version.");
// If the user needs his own variables, he can inherit the op::Datum struct and add them
......@@ -367,6 +379,11 @@ int openPoseTutorialWrapper2()
const auto handNetInputSize = op::flagsToPoint(FLAGS_hand_net_resolution, "368x368 (multiples of 16)");
// poseModel
const auto poseModel = op::flagsToPoseModel(FLAGS_model_pose);
// JSON saving
const auto writeJson = (!FLAGS_write_json.empty() ? FLAGS_write_json : FLAGS_write_keypoint_json);
if (!FLAGS_write_keypoint.empty() || !FLAGS_write_keypoint_json.empty())
op::log("Flags `write_keypoint` and `write_keypoint_json` are deprecated and will eventually be removed."
" Please, use `write_json` instead.", op::Priority::Max);
// keypointScale
const auto keypointScale = op::flagsToScaleMode(FLAGS_keypoint_scale);
// heatmaps to add
......@@ -403,8 +420,8 @@ int openPoseTutorialWrapper2()
(float)FLAGS_scale_gap, op::flagsToRenderMode(FLAGS_render_pose),
poseModel, !FLAGS_disable_blending, (float)FLAGS_alpha_pose,
(float)FLAGS_alpha_heatmap, FLAGS_part_to_show, FLAGS_model_folder,
heatMapTypes, heatMapScale, (float)FLAGS_render_threshold,
enableGoogleLogging};
heatMapTypes, heatMapScale, FLAGS_part_candidates,
(float)FLAGS_render_threshold, enableGoogleLogging};
// Face configuration (use op::WrapperStructFace{} to disable it)
const op::WrapperStructFace wrapperStructFace{FLAGS_face, faceNetInputSize,
op::flagsToRenderMode(FLAGS_face_render, FLAGS_render_pose),
......@@ -421,8 +438,9 @@ int openPoseTutorialWrapper2()
const bool guiVerbose = false;
const bool fullScreen = false;
const op::WrapperStructOutput wrapperStructOutput{displayGui, guiVerbose, fullScreen, FLAGS_write_keypoint,
op::stringToDataFormat(FLAGS_write_keypoint_format), FLAGS_write_keypoint_json,
FLAGS_write_coco_json, FLAGS_write_images, FLAGS_write_images_format, FLAGS_write_video,
op::stringToDataFormat(FLAGS_write_keypoint_format),
writeJson, FLAGS_write_coco_json,
FLAGS_write_images, FLAGS_write_images_format, FLAGS_write_video,
FLAGS_write_heatmaps, FLAGS_write_heatmaps_format};
// Configure wrapper
opWrapper.configure(wrapperStructPose, wrapperStructFace, wrapperStructHand, op::WrapperStructInput{},
......
......@@ -69,18 +69,26 @@ DEFINE_int32(scale_number, 1, "Number of scales to ave
DEFINE_double(scale_gap, 0.3, "Scale gap between scales. No effect unless scale_number > 1. Initial scale is always 1."
" If you want to change the initial scale, you actually want to multiply the"
" `net_resolution` by your desired initial scale.");
// OpenPose Body Pose Heatmaps
DEFINE_bool(heatmaps_add_parts, false, "If true, it will add the body part heatmaps to the final op::Datum::poseHeatMaps array,"
" and analogously face & hand heatmaps to op::Datum::faceHeatMaps & op::Datum::handHeatMaps"
" (program speed will decrease). Not required for our library, enable it only if you intend"
" to process this information later. If more than one `add_heatmaps_X` flag is enabled, it"
" will place then in sequential memory order: body parts + bkg + PAFs. It will follow the"
" order on POSE_BODY_PART_MAPPING in `include/openpose/pose/poseParameters.hpp`.");
// OpenPose Body Pose Heatmaps and Part Candidates
DEFINE_bool(heatmaps_add_parts, false, "If true, it will fill op::Datum::poseHeatMaps array with the body part heatmaps, and"
" analogously face & hand heatmaps to op::Datum::faceHeatMaps & op::Datum::handHeatMaps."
" If more than one `add_heatmaps_X` flag is enabled, it will place then in sequential"
" memory order: body parts + bkg + PAFs. It will follow the order on"
" POSE_BODY_PART_MAPPING in `src/openpose/pose/poseParameters.cpp`. Program speed will"
" considerably decrease. Not required for OpenPose, enable it only if you intend to"
" explicitly use this information later.");
DEFINE_bool(heatmaps_add_bkg, false, "Same functionality as `add_heatmaps_parts`, but adding the heatmap corresponding to"
" background.");
DEFINE_bool(heatmaps_add_PAFs, false, "Same functionality as `add_heatmaps_parts`, but adding the PAFs.");
DEFINE_int32(heatmaps_scale, 2, "Set 0 to scale op::Datum::poseHeatMaps in the range [-1,1], 1 for [0,1]; 2 for integer"
" rounded [0,255]; and 3 for no scaling.");
DEFINE_bool(part_candidates, false, "Also enable `write_json` in order to save this information. If true, it will fill the"
" op::Datum::poseCandidates array with the body part candidates. Candidates refer to all"
" the detected body parts, before being assembled into people. Note that the number of"
" candidates is equal or higher than the number of final body parts (i.e. after being"
" assembled into people). The empty body parts are filled with 0s. Program speed will"
" slightly decrease. Not required for OpenPose, enable it only if you intend to explicitly"
" use this information.");
// OpenPose Face
DEFINE_bool(face, false, "Enables face keypoint detection. It will share some parameters from the body pose, e.g."
" `model_folder`. Note that this will considerable slow down the performance and increse"
......@@ -142,16 +150,20 @@ DEFINE_string(write_images_format, "png", "File extension and form
" function cv::imwrite for all compatible extensions.");
DEFINE_string(write_video, "", "Full file path to write rendered frames in motion JPEG video format. It might fail if the"
" final path does not finish in `.avi`. It internally uses cv::VideoWriter.");
DEFINE_string(write_keypoint, "", "Directory to write the people body pose keypoint data. Set format with `write_keypoint_format`.");
DEFINE_string(write_keypoint_format, "yml", "File extension and format for `write_keypoint`: json, xml, yaml & yml. Json not available"
" for OpenCV < 3.0, use `write_keypoint_json` instead.");
DEFINE_string(write_keypoint_json, "", "Directory to write people pose data in *.json format, compatible with any OpenCV version.");
DEFINE_string(write_coco_json, "", "Full file path to write people pose data with *.json COCO validation format.");
DEFINE_string(write_heatmaps, "", "Directory to write body pose heatmaps in *.png format. At least 1 `add_heatmaps_X` flag"
DEFINE_string(write_json, "", "Directory to write OpenPose output in JSON format. It includes body, hand, and face pose"
" keypoints, as well as pose candidates (if `--part_candidates` enabled).");
DEFINE_string(write_coco_json, "", "Full file path to write people pose data with JSON COCO validation format.");
DEFINE_string(write_heatmaps, "", "Directory to write body pose heatmaps in PNG format. At least 1 `add_heatmaps_X` flag"
" must be enabled.");
DEFINE_string(write_heatmaps_format, "png", "File extension and format for `write_heatmaps`, analogous to `write_images_format`."
" For lossless compression, recommended `png` for integer `heatmaps_scale` and `float` for"
" floating values.");
DEFINE_string(write_keypoint, "", "(Deprecated, use `write_json`) Directory to write the people pose keypoint data. Set format"
" with `write_keypoint_format`.");
DEFINE_string(write_keypoint_format, "yml", "(Deprecated, use `write_json`) File extension and format for `write_keypoint`: json, xml,"
" yaml & yml. Json not available for OpenCV < 3.0, use `write_keypoint_json` instead.");
DEFINE_string(write_keypoint_json, "", "(Deprecated, use `write_json`) Directory to write people pose data in JSON format,"
" compatible with any OpenCV version.");
// If the user needs his own variables, he can inherit the op::Datum struct and add them
......@@ -325,6 +337,11 @@ int openPoseTutorialWrapper3()
const auto handNetInputSize = op::flagsToPoint(FLAGS_hand_net_resolution, "368x368 (multiples of 16)");
// poseModel
const auto poseModel = op::flagsToPoseModel(FLAGS_model_pose);
// JSON saving
const auto writeJson = (!FLAGS_write_json.empty() ? FLAGS_write_json : FLAGS_write_keypoint_json);
if (!FLAGS_write_keypoint.empty() || !FLAGS_write_keypoint_json.empty())
op::log("Flags `write_keypoint` and `write_keypoint_json` are deprecated and will eventually be removed."
" Please, use `write_json` instead.", op::Priority::Max);
// keypointScale
const auto keypointScale = op::flagsToScaleMode(FLAGS_keypoint_scale);
// heatmaps to add
......@@ -344,8 +361,8 @@ int openPoseTutorialWrapper3()
(float)FLAGS_scale_gap, op::flagsToRenderMode(FLAGS_render_pose),
poseModel, !FLAGS_disable_blending, (float)FLAGS_alpha_pose,
(float)FLAGS_alpha_heatmap, FLAGS_part_to_show, FLAGS_model_folder,
heatMapTypes, heatMapScale, (float)FLAGS_render_threshold,
enableGoogleLogging};
heatMapTypes, heatMapScale, FLAGS_part_candidates,
(float)FLAGS_render_threshold, enableGoogleLogging};
// Face configuration (use op::WrapperStructFace{} to disable it)
const op::WrapperStructFace wrapperStructFace{FLAGS_face, faceNetInputSize,
op::flagsToRenderMode(FLAGS_face_render, FLAGS_render_pose),
......@@ -362,8 +379,9 @@ int openPoseTutorialWrapper3()
const bool guiVerbose = false;
const bool fullScreen = false;
const op::WrapperStructOutput wrapperStructOutput{displayGui, guiVerbose, fullScreen, FLAGS_write_keypoint,
op::stringToDataFormat(FLAGS_write_keypoint_format), FLAGS_write_keypoint_json,
FLAGS_write_coco_json, FLAGS_write_images, FLAGS_write_images_format, FLAGS_write_video,
op::stringToDataFormat(FLAGS_write_keypoint_format),
writeJson, FLAGS_write_coco_json,
FLAGS_write_images, FLAGS_write_images_format, FLAGS_write_video,
FLAGS_write_heatmaps, FLAGS_write_heatmaps_format};
// Configure wrapper
op::log("Configuring OpenPose wrapper.", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__);
......
......@@ -69,18 +69,26 @@ DEFINE_int32(scale_number, 1, "Number of scales to ave
DEFINE_double(scale_gap, 0.3, "Scale gap between scales. No effect unless scale_number > 1. Initial scale is always 1."
" If you want to change the initial scale, you actually want to multiply the"
" `net_resolution` by your desired initial scale.");
// OpenPose Body Pose Heatmaps
DEFINE_bool(heatmaps_add_parts, false, "If true, it will add the body part heatmaps to the final op::Datum::poseHeatMaps array,"
" and analogously face & hand heatmaps to op::Datum::faceHeatMaps & op::Datum::handHeatMaps"
" (program speed will decrease). Not required for our library, enable it only if you intend"
" to process this information later. If more than one `add_heatmaps_X` flag is enabled, it"
" will place then in sequential memory order: body parts + bkg + PAFs. It will follow the"
" order on POSE_BODY_PART_MAPPING in `include/openpose/pose/poseParameters.hpp`.");
// OpenPose Body Pose Heatmaps and Part Candidates
DEFINE_bool(heatmaps_add_parts, false, "If true, it will fill op::Datum::poseHeatMaps array with the body part heatmaps, and"
" analogously face & hand heatmaps to op::Datum::faceHeatMaps & op::Datum::handHeatMaps."
" If more than one `add_heatmaps_X` flag is enabled, it will place then in sequential"
" memory order: body parts + bkg + PAFs. It will follow the order on"
" POSE_BODY_PART_MAPPING in `src/openpose/pose/poseParameters.cpp`. Program speed will"
" considerably decrease. Not required for OpenPose, enable it only if you intend to"
" explicitly use this information later.");
DEFINE_bool(heatmaps_add_bkg, false, "Same functionality as `add_heatmaps_parts`, but adding the heatmap corresponding to"
" background.");
DEFINE_bool(heatmaps_add_PAFs, false, "Same functionality as `add_heatmaps_parts`, but adding the PAFs.");
DEFINE_int32(heatmaps_scale, 2, "Set 0 to scale op::Datum::poseHeatMaps in the range [-1,1], 1 for [0,1]; 2 for integer"
" rounded [0,255]; and 3 for no scaling.");
DEFINE_bool(part_candidates, false, "Also enable `write_json` in order to save this information. If true, it will fill the"
" op::Datum::poseCandidates array with the body part candidates. Candidates refer to all"
" the detected body parts, before being assembled into people. Note that the number of"
" candidates is equal or higher than the number of final body parts (i.e. after being"
" assembled into people). The empty body parts are filled with 0s. Program speed will"
" slightly decrease. Not required for OpenPose, enable it only if you intend to explicitly"
" use this information.");
// OpenPose Face
DEFINE_bool(face, true, "Enables face keypoint detection. It will share some parameters from the body pose, e.g."
" `model_folder`. Note that this will considerable slow down the performance and increse"
......@@ -143,16 +151,20 @@ DEFINE_string(write_images_format, "png", "File extension and form
" function cv::imwrite for all compatible extensions.");
DEFINE_string(write_video, "", "Full file path to write rendered frames in motion JPEG video format. It might fail if the"
" final path does not finish in `.avi`. It internally uses cv::VideoWriter.");
DEFINE_string(write_keypoint, "", "Directory to write the people body pose keypoint data. Set format with `write_keypoint_format`.");
DEFINE_string(write_keypoint_format, "yml", "File extension and format for `write_keypoint`: json, xml, yaml & yml. Json not available"
" for OpenCV < 3.0, use `write_keypoint_json` instead.");
DEFINE_string(write_keypoint_json, "", "Directory to write people pose data in *.json format, compatible with any OpenCV version.");
DEFINE_string(write_coco_json, "", "Full file path to write people pose data with *.json COCO validation format.");
DEFINE_string(write_heatmaps, "", "Directory to write body pose heatmaps in *.png format. At least 1 `add_heatmaps_X` flag"
DEFINE_string(write_json, "", "Directory to write OpenPose output in JSON format. It includes body, hand, and face pose"
" keypoints, as well as pose candidates (if `--part_candidates` enabled).");
DEFINE_string(write_coco_json, "", "Full file path to write people pose data with JSON COCO validation format.");
DEFINE_string(write_heatmaps, "", "Directory to write body pose heatmaps in PNG format. At least 1 `add_heatmaps_X` flag"
" must be enabled.");
DEFINE_string(write_heatmaps_format, "png", "File extension and format for `write_heatmaps`, analogous to `write_images_format`."
" For lossless compression, recommended `png` for integer `heatmaps_scale` and `float` for"
" floating values.");
DEFINE_string(write_keypoint, "", "(Deprecated, use `write_json`) Directory to write the people pose keypoint data. Set format"
" with `write_keypoint_format`.");
DEFINE_string(write_keypoint_format, "yml", "(Deprecated, use `write_json`) File extension and format for `write_keypoint`: json, xml,"
" yaml & yml. Json not available for OpenCV < 3.0, use `write_keypoint_json` instead.");
DEFINE_string(write_keypoint_json, "", "(Deprecated, use `write_json`) Directory to write people pose data in JSON format,"
" compatible with any OpenCV version.");
int openpose3d()
{
......@@ -176,6 +188,11 @@ int openpose3d()
const auto handNetInputSize = op::flagsToPoint(FLAGS_hand_net_resolution, "368x368 (multiples of 16)");
// poseModel
const auto poseModel = op::flagsToPoseModel(FLAGS_model_pose);
// JSON saving
const auto writeJson = (!FLAGS_write_json.empty() ? FLAGS_write_json : FLAGS_write_keypoint_json);
if (!FLAGS_write_keypoint.empty() || !FLAGS_write_keypoint_json.empty())
op::log("Flags `write_keypoint` and `write_keypoint_json` are deprecated and will eventually be removed."
" Please, use `write_json` instead.", op::Priority::Max);
// keypointScale
const auto keypointScale = op::flagsToScaleMode(FLAGS_keypoint_scale);
// heatmaps to add
......@@ -211,8 +228,8 @@ int openpose3d()
(float)FLAGS_scale_gap, op::flagsToRenderMode(FLAGS_render_pose),
poseModel, !FLAGS_disable_blending, (float)FLAGS_alpha_pose,
(float)FLAGS_alpha_heatmap, FLAGS_part_to_show, FLAGS_model_folder,
heatMapTypes, heatMapScale, (float)FLAGS_render_threshold,
enableGoogleLogging};
heatMapTypes, heatMapScale, FLAGS_part_candidates,
(float)FLAGS_render_threshold, enableGoogleLogging};
// Face configuration (use op::WrapperStructFace{} to disable it)
const op::WrapperStructFace wrapperStructFace{FLAGS_face, faceNetInputSize,
op::flagsToRenderMode(FLAGS_face_render, FLAGS_render_pose),
......@@ -229,8 +246,9 @@ int openpose3d()
const bool guiVerbose = true;
const bool fullScreen = false;
const op::WrapperStructOutput wrapperStructOutput{displayGui, guiVerbose, fullScreen, FLAGS_write_keypoint,
op::stringToDataFormat(FLAGS_write_keypoint_format), FLAGS_write_keypoint_json,
FLAGS_write_coco_json, FLAGS_write_images, FLAGS_write_images_format, FLAGS_write_video,
op::stringToDataFormat(FLAGS_write_keypoint_format),
writeJson, FLAGS_write_coco_json,
FLAGS_write_images, FLAGS_write_images_format, FLAGS_write_video,
FLAGS_write_heatmaps, FLAGS_write_heatmaps_format};
// Configure wrapper
opWrapper.configure(wrapperStructPose, wrapperStructFace, wrapperStructHand, op::WrapperStructInput{},
......
......@@ -85,7 +85,7 @@ namespace op
/**
* Body pose heatmaps (body parts, background and/or PAFs) for the whole image.
* This parameters is by default empty and disabled for performance. Each group (body parts, background and
* This parameter is by default empty and disabled for performance. Each group (body parts, background and
* PAFs) can be individually enabled.
* #heatmaps = #body parts (if enabled) + 1 (if background enabled) + 2 x #PAFs (if enabled). Each PAF has 2
* consecutive channels, one for x- and one for y-coordinates.
......@@ -97,6 +97,17 @@ namespace op
*/
Array<float> poseHeatMaps;
/**
* Body pose candidates for the whole image.
* This parameter is by default empty and disabled for performance. It can be enabled with `candidates_body`.
* Candidates refer to all the detected body parts, before being assembled into people. Note that the number
* of candidates is equal or higher than the number of body parts after being assembled into people.
* Size: #body parts x min(part candidates, POSE_MAX_PEOPLE) x 3 (x,y,score).
* Rather than vector, it should ideally be:
* std::array<std::vector<std::array<float,3>>, #BP> poseCandidates;
*/
std::vector<std::vector<std::array<float,3>>> poseCandidates;
/**
* Face detection locations (x,y,width,height) for each person in the image.
* It is resized to cvInputData.size().
......@@ -140,7 +151,7 @@ namespace op
*/
std::array<Array<float>, 2> handHeatMaps;
// ---------------------------------------- Other parameters ---------------------------------------- //
// ---------------------------------------- Other (internal) parameters ---------------------------------------- //
/**
* Scale ratio between the input Datum::cvInputData and the net input size.
*/
......
......@@ -36,12 +36,15 @@ namespace op
const DataFormat format);
// Json - Saving as *.json not available in OpenCV verions < 3.0, this function is a quick fix
OP_API void saveKeypointsJson(const Array<float>& keypoints, const std::string& keypointName,
const std::string& fileName, const bool humanReadable);
OP_API void savePeopleJson(const Array<float>& keypoints,
const std::vector<std::vector<std::array<float,3>>>& candidates,
const std::string& keypointName, const std::string& fileName,
const bool humanReadable);
// It will save a bunch of Array<float> elements
OP_API void saveKeypointsJson(const std::vector<std::pair<Array<float>, std::string>>& keypointVector,
const std::string& fileName, const bool humanReadable);
OP_API void savePeopleJson(const std::vector<std::pair<Array<float>, std::string>>& keypointVector,
const std::vector<std::vector<std::array<float,3>>>& candidates,
const std::string& fileName, const bool humanReadable);
// Save/load image
OP_API void saveImage(const cv::Mat& cvMat, const std::string& fullFilePath,
......
......@@ -9,15 +9,15 @@
#include <openpose/filestream/heatMapSaver.hpp>
#include <openpose/filestream/imageSaver.hpp>
#include <openpose/filestream/jsonOfstream.hpp>
#include <openpose/filestream/keypointJsonSaver.hpp>
#include <openpose/filestream/keypointSaver.hpp>
#include <openpose/filestream/peopleJsonSaver.hpp>
#include <openpose/filestream/videoSaver.hpp>
#include <openpose/filestream/wCocoJsonSaver.hpp>
#include <openpose/filestream/wFaceSaver.hpp>
#include <openpose/filestream/wHandSaver.hpp>
#include <openpose/filestream/wImageSaver.hpp>
#include <openpose/filestream/wHeatMapSaver.hpp>
#include <openpose/filestream/wKeypointJsonSaver.hpp>
#include <openpose/filestream/wPeopleJsonSaver.hpp>
#include <openpose/filestream/wPoseSaver.hpp>
#include <openpose/filestream/wVideoSaver.hpp>
......
......@@ -21,6 +21,8 @@ namespace op
void arrayClose();
void version(const std::string& version);
void key(const std::string& string);
template <typename T>
......
#ifndef OPENPOSE_FILESTREAM_KEYPOINT_JSON_SAVER_HPP
#define OPENPOSE_FILESTREAM_KEYPOINT_JSON_SAVER_HPP
#ifndef OPENPOSE_FILESTREAM_PEOPLE_JSON_SAVER_HPP
#define OPENPOSE_FILESTREAM_PEOPLE_JSON_SAVER_HPP
#include <openpose/core/common.hpp>
#include <openpose/filestream/fileSaver.hpp>
namespace op
{
class OP_API KeypointJsonSaver : public FileSaver
class OP_API PeopleJsonSaver : public FileSaver
{
public:
KeypointJsonSaver(const std::string& directoryPath);
PeopleJsonSaver(const std::string& directoryPath);
void save(const std::vector<std::pair<Array<float>, std::string>>& keypointVector,
const std::vector<std::vector<std::array<float,3>>>& candidates,
const std::string& fileName, const bool humanReadable = true) const;
};
}
#endif // OPENPOSE_FILESTREAM_KEYPOINT_JSON_SAVER_HPP
#endif // OPENPOSE_FILESTREAM_PEOPLE_JSON_SAVER_HPP
#ifndef OPENPOSE_FILESTREAM_W_KEYPOINT_JSON_SAVER_HPP
#define OPENPOSE_FILESTREAM_W_KEYPOINT_JSON_SAVER_HPP
#ifndef OPENPOSE_FILESTREAM_W_PEOPLE_JSON_SAVER_HPP
#define OPENPOSE_FILESTREAM_W_PEOPLE_JSON_SAVER_HPP
#include <openpose/core/common.hpp>
#include <openpose/filestream/keypointJsonSaver.hpp>
#include <openpose/filestream/peopleJsonSaver.hpp>
#include <openpose/thread/workerConsumer.hpp>
namespace op
{
template<typename TDatums>
class WKeypointJsonSaver : public WorkerConsumer<TDatums>
class WPeopleJsonSaver : public WorkerConsumer<TDatums>
{
public:
explicit WKeypointJsonSaver(const std::shared_ptr<KeypointJsonSaver>& keypointJsonSaver);
explicit WPeopleJsonSaver(const std::shared_ptr<PeopleJsonSaver>& peopleJsonSaver);
void initializationOnThread();
void workConsumer(const TDatums& tDatums);
private:
const std::shared_ptr<KeypointJsonSaver> spKeypointJsonSaver;
const std::shared_ptr<PeopleJsonSaver> spPeopleJsonSaver;
DELETE_COPY(WKeypointJsonSaver);
DELETE_COPY(WPeopleJsonSaver);
};
}
......@@ -33,18 +33,18 @@ namespace op
namespace op
{
template<typename TDatums>
WKeypointJsonSaver<TDatums>::WKeypointJsonSaver(const std::shared_ptr<KeypointJsonSaver>& keypointJsonSaver) :
spKeypointJsonSaver{keypointJsonSaver}
WPeopleJsonSaver<TDatums>::WPeopleJsonSaver(const std::shared_ptr<PeopleJsonSaver>& peopleJsonSaver) :
spPeopleJsonSaver{peopleJsonSaver}
{
}
template<typename TDatums>
void WKeypointJsonSaver<TDatums>::initializationOnThread()
void WPeopleJsonSaver<TDatums>::initializationOnThread()
{
}
template<typename TDatums>
void WKeypointJsonSaver<TDatums>::workConsumer(const TDatums& tDatums)
void WPeopleJsonSaver<TDatums>::workConsumer(const TDatums& tDatums)
{
try
{
......@@ -72,7 +72,7 @@ namespace op
std::make_pair(tDatum.handKeypoints[1], "hand_right_keypoints")
};
// Save keypoints
spKeypointJsonSaver->save(keypointVector, fileName, humanReadable);
spPeopleJsonSaver->save(keypointVector, tDatum.poseCandidates, fileName, humanReadable);
}
// Profiling speed
Profiler::timerEnd(profilerKey);
......@@ -88,7 +88,7 @@ namespace op
}
}
COMPILE_TEMPLATE_DATUM(WKeypointJsonSaver);
COMPILE_TEMPLATE_DATUM(WPeopleJsonSaver);
}
#endif // OPENPOSE_FILESTREAM_W_KEYPOINT_JSON_SAVER_HPP
#endif // OPENPOSE_FILESTREAM_W_PEOPLE_JSON_SAVER_HPP
......@@ -14,7 +14,8 @@ namespace op
public:
PoseExtractor(const PoseModel poseModel,
const std::vector<HeatMapType>& heatMapTypes = {},
const ScaleMode heatMapScale = ScaleMode::ZeroToOne);
const ScaleMode heatMapScale = ScaleMode::ZeroToOne,
const bool addPartCandidates = false);
virtual ~PoseExtractor();
......@@ -23,13 +24,19 @@ namespace op
virtual void forwardPass(const std::vector<Array<float>>& inputNetData, const Point<int>& inputDataSize,
const std::vector<double>& scaleRatios = {1.f}) = 0;
virtual const float* getCandidatesCpuConstPtr() const = 0;
virtual const float* getCandidatesGpuConstPtr() const = 0;
virtual const float* getHeatMapCpuConstPtr() const = 0;
virtual const float* getHeatMapGpuConstPtr() const = 0;
virtual std::vector<int> getHeatMapSize() const = 0;
Array<float> getHeatMaps() const;
Array<float> getHeatMapsCopy() const;
std::vector<std::vector<std::array<float,3>>> getCandidatesCopy() const;
virtual const float* getPoseGpuConstPtr() const = 0;
......@@ -59,6 +66,7 @@ namespace op
private:
const std::vector<HeatMapType> mHeatMapTypes;
const ScaleMode mHeatMapScaleMode;
const bool mAddPartCandidates;
std::array<std::atomic<double>, (int)PoseProperty::Size> mProperties;
std::thread::id mThreadId;
......
......@@ -13,6 +13,7 @@ namespace op
PoseExtractorCaffe(const PoseModel poseModel, const std::string& modelFolder, const int gpuId,
const std::vector<HeatMapType>& heatMapTypes = {},
const ScaleMode heatMapScale = ScaleMode::ZeroToOne,
const bool addPartCandidates = false,
const bool enableGoogleLogging = true);
virtual ~PoseExtractorCaffe();
......@@ -22,6 +23,10 @@ namespace op
void forwardPass(const std::vector<Array<float>>& inputNetData, const Point<int>& inputDataSize,
const std::vector<double>& scaleInputToNetInputs = {1.f});
const float* getCandidatesCpuConstPtr() const;
const float* getCandidatesGpuConstPtr() const;
const float* getHeatMapCpuConstPtr() const;
const float* getHeatMapGpuConstPtr() const;
......
......@@ -61,7 +61,8 @@ namespace op
spPoseExtractor->forwardPass(tDatum.inputNetData,
Point<int>{tDatum.cvInputData.cols, tDatum.cvInputData.rows},
tDatum.scaleInputToNetInputs);
tDatum.poseHeatMaps = spPoseExtractor->getHeatMaps().clone();
tDatum.poseCandidates = spPoseExtractor->getCandidatesCopy();
tDatum.poseHeatMaps = spPoseExtractor->getHeatMapsCopy();
tDatum.poseKeypoints = spPoseExtractor->getPoseKeypoints().clone();
tDatum.poseScores = spPoseExtractor->getPoseScores().clone();
tDatum.scaleNetToOutput = spPoseExtractor->getScaleNetToOutput();
......
......@@ -456,14 +456,14 @@ namespace op
__LINE__, __FUNCTION__, __FILE__);
if (!renderOutput && (!wrapperStructOutput.writeImages.empty() || !wrapperStructOutput.writeVideo.empty()))
{
const auto message = "In order to save the rendered frames (`write_images` or `write_video`), you"
" cannot disable `render_pose`.";
const auto message = "In order to save the rendered frames (`--write_images` or `--write_video`), you"
" cannot disable `--render_pose`.";
log(message, Priority::High);
}
if (!wrapperStructOutput.writeHeatMaps.empty() && wrapperStructPose.heatMapTypes.empty())
{
const auto message = "In order to save the heatmaps (`write_heatmaps`), you need to pick which heat"
" maps you want to save: `heatmaps_add_X` flags or fill the"
const auto message = "In order to save the heatmaps (`--write_heatmaps`), you need to pick which heat"
" maps you want to save: `--heatmaps_add_X` flags or fill the"
" wrapperStructPose.heatMapTypes.";
error(message, __LINE__, __FUNCTION__, __FILE__);
}
......@@ -473,7 +473,7 @@ namespace op
{
const auto message = "In order to save the heatmaps, you must either set"
" wrapperStructPose.heatMapScale to ScaleMode::UnsignedChar (i.e. range [0, 255])"
" or `write_heatmaps_format` to `float` to storage floating numbers in binary"
" or `--write_heatmaps_format` to `float` to storage floating numbers in binary"
" mode.";
error(message, __LINE__, __FUNCTION__, __FILE__);
}
......@@ -486,30 +486,30 @@ namespace op
};
const auto savingSomething = (
!wrapperStructOutput.writeImages.empty() || !wrapperStructOutput.writeVideo.empty()
|| !wrapperStructOutput.writeKeypoint.empty() || !wrapperStructOutput.writeKeypointJson.empty()
|| !wrapperStructOutput.writeKeypoint.empty() || !wrapperStructOutput.writeJson.empty()
|| !wrapperStructOutput.writeCocoJson.empty() || !wrapperStructOutput.writeHeatMaps.empty()
);
if (!wrapperStructOutput.displayGui && !savingSomething)
{
const auto message = "No output is selected (`no_display`) and no results are generated (no"
" `write_X` flags enabled). Thus, no output would be generated."
const auto message = "No output is selected (`--no_display`) and no results are generated (no"
" `--write_X` flags enabled). Thus, no output would be generated."
+ additionalMessage;
error(message, __LINE__, __FUNCTION__, __FILE__);
}
if (wrapperStructInput.framesRepeat && savingSomething)
{
const auto message = "Frames repetition (`frames_repeat`) is enabled as well as some writing"
" function (`write_X`). This program would never stop recording the same"
const auto message = "Frames repetition (`--frames_repeat`) is enabled as well as some writing"
" function (`--write_X`). This program would never stop recording the same"
" frames over and over. Please, disable repetition or remove writing.";
error(message, __LINE__, __FUNCTION__, __FILE__);
}
// Warnings
if ((wrapperStructOutput.displayGui && wrapperStructOutput.guiVerbose) && !renderOutput)
{
const auto message = "No render is enabled (e.g. `render_pose 0`), so you might also want to"
" remove the display (set `no_display` or `no_gui_verbose`). If you simply"
" want to use OpenPose to record video/images without keypoints, you only"
" need to set `num_gpu 0`." + additionalMessage;
const auto message = "No render is enabled (e.g. `--render_pose 0`), so you might also want to"
" remove the display (set `--no_display` or `--no_gui_verbose`). If you"
" simply want to use OpenPose to record video/images without keypoints, you"
" only need to set `--num_gpu 0`." + additionalMessage;
log(message, Priority::High);
}
if (wrapperStructInput.realTimeProcessing && savingSomething)
......@@ -541,8 +541,8 @@ namespace op
// Get total number GPUs
const auto totalGpuNumber = getGpuNumber();
if (totalGpuNumber <= gpuNumberStart)
error("Number of initial GPUs (`number_gpu_start`) must be lower than the total number of used"
" GPUs (`number_gpu`)", __LINE__, __FUNCTION__, __FILE__);
error("Number of initial GPUs (`--number_gpu_start`) must be lower than the total number of used"
" GPUs (`--number_gpu`)", __LINE__, __FUNCTION__, __FILE__);
gpuNumber = totalGpuNumber - gpuNumberStart;
// Reset initial GPU to 0 (we want them all)
// Logging message
......@@ -554,7 +554,7 @@ namespace op
// Proper format
const auto writeImagesCleaned = formatAsDirectory(wrapperStructOutput.writeImages);
const auto writeKeypointCleaned = formatAsDirectory(wrapperStructOutput.writeKeypoint);
const auto writeKeypointJsonCleaned = formatAsDirectory(wrapperStructOutput.writeKeypointJson);
const auto writeJsonCleaned = formatAsDirectory(wrapperStructOutput.writeJson);
const auto writeHeatMapsCleaned = formatAsDirectory(wrapperStructOutput.writeHeatMaps);
const auto modelFolder = formatAsDirectory(wrapperStructPose.modelFolder);
......@@ -623,7 +623,7 @@ namespace op
poseExtractors.emplace_back(std::make_shared<PoseExtractorCaffe>(
wrapperStructPose.poseModel, modelFolder, gpuId + gpuNumberStart,
wrapperStructPose.heatMapTypes, wrapperStructPose.heatMapScale,
wrapperStructPose.enableGoogleLogging
wrapperStructPose.addPartCandidates, wrapperStructPose.enableGoogleLogging
));
// Pose renderers
......@@ -875,11 +875,12 @@ namespace op
if (wrapperStructHand.enable)
mOutputWs.emplace_back(std::make_shared<WHandSaver<TDatumsPtr>>(keypointSaver));
}
// Write people pose data on disk (json format)
if (!writeKeypointJsonCleaned.empty())
// Write OpenPose output data on disk in json format (body/hand/face keypoints, body part locations if
// enabled, etc.)
if (!writeJsonCleaned.empty())
{
const auto keypointJsonSaver = std::make_shared<KeypointJsonSaver>(writeKeypointJsonCleaned);
mOutputWs.emplace_back(std::make_shared<WKeypointJsonSaver<TDatumsPtr>>(keypointJsonSaver));
const auto peopleJsonSaver = std::make_shared<PeopleJsonSaver>(writeJsonCleaned);
mOutputWs.emplace_back(std::make_shared<WPeopleJsonSaver<TDatumsPtr>>(peopleJsonSaver));
}
// Write people pose data on disk (COCO validation json format)
if (!wrapperStructOutput.writeCocoJson.empty())
......@@ -1239,7 +1240,7 @@ namespace op
if (spWPoses.size() > 1)
log("Multi-threading disabled, only 1 thread running. All GPUs have been disabled but the"
" first one, which is defined by gpuNumberStart (e.g. in the OpenPose demo, it is set"
" with the `num_gpu_start` flag).", Priority::High);
" with the `--num_gpu_start` flag).", Priority::High);
mThreadManager.add(mThreadId, spWPoses.at(0), queueIn, queueOut);
}
queueIn++;
......
......@@ -18,13 +18,14 @@ namespace op
bool displayGui;
/**
* Whether to add some information to the frame (number of frame, number people detected, etc.) after it is saved on disk
* and before it is displayed and/or returned to the user.
* Whether to add some information to the frame (number of frame, number people detected, etc.) after it is
* saved on disk and before it is displayed and/or returned to the user.
*/
bool guiVerbose;
/**
* Whether to display the OpenPose small integrated GUI on fullscreen mode. It can be changed by interacting with the GUI itself.
* Whether to display the OpenPose small integrated GUI on fullscreen mode. It can be changed by interacting
* with the GUI itself.
*/
bool fullScreen;
......@@ -43,10 +44,14 @@ namespace op
DataFormat writeKeypointFormat;
/**
* Pose (x, y, score) locations saving folder location in JSON format (e.g. useful when needed JSON but using OpenCV < 3.0).
* Directory to write OpenPose output in JSON format.
* If it is empty (default), it is disabled.
* It includes:
* - `people` field with body, hand, and face pose keypoints in (x, y, score) format.
* - `part_candidates` field with body part candidates in (x, y, score) format (if enabled with
* `--part_candidates`).
*/
std::string writeKeypointJson;
std::string writeJson;
/**
* Pose (x, y, score) locations saving folder location in JSON COCO validation format.
......@@ -93,10 +98,13 @@ namespace op
* It has the recommended and default values we recommend for each element of the struct.
* Since all the elements of the struct are public, they can also be manually filled.
*/
WrapperStructOutput(const bool displayGui = false, const bool guiVerbose = false, const bool fullScreen = false, const std::string& writeKeypoint = "",
const DataFormat writeKeypointFormat = DataFormat::Xml, const std::string& writeKeypointJson = "", const std::string& writeCocoJson = "",
const std::string& writeImages = "", const std::string& writeImagesFormat = "", const std::string& writeVideo = "",
const std::string& writeHeatMaps = "", const std::string& writeHeatMapsFormat = "");
WrapperStructOutput(const bool displayGui = false, const bool guiVerbose = false,
const bool fullScreen = false, const std::string& writeKeypoint = "",
const DataFormat writeKeypointFormat = DataFormat::Xml,
const std::string& writeJson = "", const std::string& writeCocoJson = "",
const std::string& writeImages = "", const std::string& writeImagesFormat = "",
const std::string& writeVideo = "", const std::string& writeHeatMaps = "",
const std::string& writeHeatMapsFormat = "");
};
}
......
......@@ -133,6 +133,12 @@ namespace op
*/
ScaleMode heatMapScale;
/**
* Whether to add the body part candidates.
* Candidates refer to all the detected body parts, before being assembled into people.
*/
bool addPartCandidates;
/**
* Rendering threshold. Only estimated keypoints whose score confidences are higher than this value will be
* rendered. Generally, a high threshold (> 0.5) will only render very clear body parts; while small thresholds
......@@ -170,7 +176,7 @@ namespace op
const float alphaHeatMap = POSE_DEFAULT_ALPHA_HEAT_MAP,
const int defaultPartToRender = 0, const std::string& modelFolder = "models/",
const std::vector<HeatMapType>& heatMapTypes = {},
const ScaleMode heatMapScale = ScaleMode::ZeroToOne,
const ScaleMode heatMapScale = ScaleMode::ZeroToOne, const bool addPartCandidates = false,
const float renderThreshold = 0.05f, const bool enableGoogleLogging = true,
const bool identification = false);
};
......
......@@ -24,6 +24,7 @@ namespace op
poseIds{datum.poseIds},
poseScores{datum.poseScores},
poseHeatMaps{datum.poseHeatMaps},
poseCandidates{datum.poseCandidates},
faceRectangles{datum.faceRectangles},
faceKeypoints{datum.faceKeypoints},
handRectangles{datum.handRectangles},
......@@ -55,6 +56,7 @@ namespace op
poseIds = datum.poseIds,
poseScores = datum.poseScores,
poseHeatMaps = datum.poseHeatMaps,
poseCandidates = datum.poseCandidates,
faceRectangles = datum.faceRectangles,
faceKeypoints = datum.faceKeypoints,
handRectangles = datum.handRectangles,
......@@ -97,6 +99,7 @@ namespace op
std::swap(poseIds, datum.poseIds);
std::swap(poseScores, datum.poseScores);
std::swap(poseHeatMaps, datum.poseHeatMaps);
std::swap(poseCandidates, datum.poseCandidates);
std::swap(faceRectangles, datum.faceRectangles);
std::swap(faceKeypoints, datum.faceKeypoints);
std::swap(handRectangles, datum.handRectangles);
......@@ -130,6 +133,7 @@ namespace op
std::swap(poseIds, datum.poseIds);
std::swap(poseScores, datum.poseScores);
std::swap(poseHeatMaps, datum.poseHeatMaps);
std::swap(poseCandidates, datum.poseCandidates);
std::swap(faceRectangles, datum.faceRectangles);
std::swap(faceKeypoints, datum.faceKeypoints);
std::swap(handRectangles, datum.handRectangles);
......@@ -173,6 +177,7 @@ namespace op
datum.poseIds = poseIds.clone();
datum.poseScores = poseScores.clone();
datum.poseHeatMaps = poseHeatMaps.clone();
datum.poseCandidates = poseCandidates;
datum.faceRectangles = faceRectangles;
datum.faceKeypoints = faceKeypoints.clone();
datum.handRectangles = datum.handRectangles;
......
......@@ -5,8 +5,8 @@ set(SOURCES cocoJsonSaver.cpp
heatMapSaver.cpp
imageSaver.cpp
jsonOfstream.cpp
keypointJsonSaver.cpp
keypointSaver.cpp
peopleJsonSaver.cpp
videoSaver.cpp)
add_library(openpose_filestream ${SOURCES})
......@@ -15,4 +15,4 @@ install(TARGETS openpose_filestream
EXPORT OpenPose
RUNTIME DESTINATION bin
LIBRARY DESTINATION lib
ARCHIVE DESTINATION lib/openpose)
\ No newline at end of file
ARCHIVE DESTINATION lib/openpose)
......@@ -7,7 +7,7 @@ namespace op
DEFINE_TEMPLATE_DATUM(WHandSaver);
DEFINE_TEMPLATE_DATUM(WHeatMapSaver);
DEFINE_TEMPLATE_DATUM(WImageSaver);
DEFINE_TEMPLATE_DATUM(WKeypointJsonSaver);
DEFINE_TEMPLATE_DATUM(WPeopleJsonSaver);
DEFINE_TEMPLATE_DATUM(WPoseSaver);
DEFINE_TEMPLATE_DATUM(WVideoSaver);
}
......@@ -42,6 +42,109 @@ namespace op
return fileNameNoExtension + "." + dataFormatToString(format);
}
void addKeypointsToJson(JsonOfstream& jsonOfstream,
const std::vector<std::pair<Array<float>, std::string>>& keypointVector)
{
try
{
// Security checks
for (const auto& keypointPair : keypointVector)
if (!keypointPair.first.empty() && keypointPair.first.getNumberDimensions() != 3 )
error("keypointVector.getNumberDimensions() != 3.", __LINE__, __FUNCTION__, __FILE__);
// Add people keypoints
jsonOfstream.key("people");
jsonOfstream.arrayOpen();
// Ger max numberPeople
auto numberPeople = 0;
for (auto vectorIndex = 0u ; vectorIndex < keypointVector.size() ; vectorIndex++)
numberPeople = fastMax(numberPeople, keypointVector[vectorIndex].first.getSize(0));
for (auto person = 0 ; person < numberPeople ; person++)
{
jsonOfstream.objectOpen();
for (auto vectorIndex = 0u ; vectorIndex < keypointVector.size() ; vectorIndex++)
{
const auto& keypoints = keypointVector[vectorIndex].first;
const auto& keypointName = keypointVector[vectorIndex].second;
const auto numberBodyParts = keypoints.getSize(1);
jsonOfstream.key(keypointName);
jsonOfstream.arrayOpen();
// Body parts
for (auto bodyPart = 0 ; bodyPart < numberBodyParts ; bodyPart++)
{
const auto finalIndex = 3*(person*numberBodyParts + bodyPart);
jsonOfstream.plainText(keypoints[finalIndex]);
jsonOfstream.comma();
jsonOfstream.plainText(keypoints[finalIndex+1]);
jsonOfstream.comma();
jsonOfstream.plainText(keypoints[finalIndex+2]);
if (bodyPart < numberBodyParts-1)
jsonOfstream.comma();
}
jsonOfstream.arrayClose();
if (vectorIndex < keypointVector.size()-1)
jsonOfstream.comma();
}
jsonOfstream.objectClose();
if (person < numberPeople-1)
{
jsonOfstream.comma();
jsonOfstream.enter();
}
}
// Close bodies array
jsonOfstream.arrayClose();
}
catch (const std::exception& e)
{
error(e.what(), __LINE__, __FUNCTION__, __FILE__);
}
}
void addCandidatesToJson(JsonOfstream& jsonOfstream,
const std::vector<std::vector<std::array<float,3>>>& candidates)
{
try
{
// Add body part candidates
jsonOfstream.key("part_candidates");
jsonOfstream.arrayOpen();
// Ger max numberParts
auto numberParts = candidates.size();
jsonOfstream.objectOpen();
for (auto part = 0u ; part < numberParts ; part++)
{
// Open array
jsonOfstream.key(std::to_string(part));
jsonOfstream.arrayOpen();
// Iterate over part candidates
const auto& partCandidates = candidates[part];
const auto numberPartCandidates = partCandidates.size();
// Body part candidates
for (auto bodyPart = 0u ; bodyPart < numberPartCandidates ; bodyPart++)
{
const auto& candidate = partCandidates[bodyPart];
jsonOfstream.plainText(candidate[0]);
jsonOfstream.comma();
jsonOfstream.plainText(candidate[1]);
jsonOfstream.comma();
jsonOfstream.plainText(candidate[2]);
if (bodyPart < numberPartCandidates-1)
jsonOfstream.comma();
}
jsonOfstream.arrayClose();
if (part < numberParts-1)
jsonOfstream.comma();
}
jsonOfstream.objectClose();
// Close array
jsonOfstream.arrayClose();
}
catch (const std::exception& e)
{
error(e.what(), __LINE__, __FUNCTION__, __FILE__);
}
}
......@@ -171,13 +274,15 @@ namespace op
}
}
void saveKeypointsJson(const Array<float>& keypoints, const std::string& keypointName, const std::string& fileName,
const bool humanReadable)
void savePeopleJson(const Array<float>& keypoints,
const std::vector<std::vector<std::array<float,3>>>& candidates,
const std::string& keypointName, const std::string& fileName,
const bool humanReadable)
{
try
{
saveKeypointsJson(
std::vector<std::pair<Array<float>, std::string>>{std::make_pair(keypoints, keypointName)},
savePeopleJson(
std::vector<std::pair<Array<float>, std::string>>{std::make_pair(keypoints, keypointName)}, candidates,
fileName, humanReadable
);
}
......@@ -187,8 +292,9 @@ namespace op
}
}
void saveKeypointsJson(const std::vector<std::pair<Array<float>, std::string>>& keypointVector,
const std::string& fileName, const bool humanReadable)
void savePeopleJson(const std::vector<std::pair<Array<float>, std::string>>& keypointVector,
const std::vector<std::vector<std::array<float,3>>>& candidates,
const std::string& fileName, const bool humanReadable)
{
try
{
......@@ -199,52 +305,17 @@ namespace op
// Record frame on desired path
JsonOfstream jsonOfstream{fileName, humanReadable};
jsonOfstream.objectOpen();
// Version
jsonOfstream.key("version");
jsonOfstream.plainText("1.0");
// Add version
jsonOfstream.version("1.1");
jsonOfstream.comma();
// Bodies
jsonOfstream.key("people");
jsonOfstream.arrayOpen();
// Ger max numberPeople
auto numberPeople = 0;
for (auto vectorIndex = 0u ; vectorIndex < keypointVector.size() ; vectorIndex++)
numberPeople = fastMax(numberPeople, keypointVector[vectorIndex].first.getSize(0));
for (auto person = 0 ; person < numberPeople ; person++)
// Add people keypoints
addKeypointsToJson(jsonOfstream, keypointVector);
// Add body part candidates
if (!candidates.empty())
{
jsonOfstream.objectOpen();
for (auto vectorIndex = 0u ; vectorIndex < keypointVector.size() ; vectorIndex++)
{
const auto& keypoints = keypointVector[vectorIndex].first;
const auto& keypointName = keypointVector[vectorIndex].second;
const auto numberBodyParts = keypoints.getSize(1);
jsonOfstream.key(keypointName);
jsonOfstream.arrayOpen();
// Body parts
for (auto bodyPart = 0 ; bodyPart < numberBodyParts ; bodyPart++)
{
const auto finalIndex = 3*(person*numberBodyParts + bodyPart);
jsonOfstream.plainText(keypoints[finalIndex]);
jsonOfstream.comma();
jsonOfstream.plainText(keypoints[finalIndex+1]);
jsonOfstream.comma();
jsonOfstream.plainText(keypoints[finalIndex+2]);
if (bodyPart < numberBodyParts-1)
jsonOfstream.comma();
}
jsonOfstream.arrayClose();
if (vectorIndex < keypointVector.size()-1)
jsonOfstream.comma();
}
jsonOfstream.objectClose();
if (person < numberPeople-1)
{
jsonOfstream.comma();
jsonOfstream.enter();
}
jsonOfstream.comma();
addCandidatesToJson(jsonOfstream, candidates);
}
// Close array
jsonOfstream.arrayClose();
// Close object
jsonOfstream.objectClose();
}
......
......@@ -116,6 +116,19 @@ namespace op
}
}
void JsonOfstream::version(const std::string& version)
{
try
{
key("version");
plainText(version);
}
catch (const std::exception& e)
{
error(e.what(), __LINE__, __FUNCTION__, __FILE__);
}
}
void JsonOfstream::key(const std::string& string)
{
try
......
#include <openpose/filestream/fileStream.hpp>
#include <openpose/filestream/keypointJsonSaver.hpp>
#include <openpose/filestream/peopleJsonSaver.hpp>
namespace op
{
KeypointJsonSaver::KeypointJsonSaver(const std::string& directoryPath) :
PeopleJsonSaver::PeopleJsonSaver(const std::string& directoryPath) :
FileSaver{directoryPath}
{
}
void KeypointJsonSaver::save(const std::vector<std::pair<Array<float>, std::string>>& keypointVector,
const std::string& fileName, const bool humanReadable) const
void PeopleJsonSaver::save(const std::vector<std::pair<Array<float>, std::string>>& keypointVector,
const std::vector<std::vector<std::array<float,3>>>& candidates,
const std::string& fileName, const bool humanReadable) const
{
try
{
// Record json
const auto finalFileName = getNextFileName(fileName) + ".json";
saveKeypointsJson(keypointVector, finalFileName, humanReadable);
savePeopleJson(keypointVector, candidates, finalFileName, humanReadable);
}
catch (const std::exception& e)
{
......
......@@ -44,11 +44,12 @@ namespace op
}
PoseExtractor::PoseExtractor(const PoseModel poseModel, const std::vector<HeatMapType>& heatMapTypes,
const ScaleMode heatMapScale) :
const ScaleMode heatMapScale, const bool addPartCandidates) :
mPoseModel{poseModel},
mNetOutputSize{0,0},
mHeatMapTypes{heatMapTypes},
mHeatMapScaleMode{heatMapScale}
mHeatMapScaleMode{heatMapScale},
mAddPartCandidates{addPartCandidates}
{
try
{
......@@ -93,7 +94,7 @@ namespace op
}
}
Array<float> PoseExtractor::getHeatMaps() const
Array<float> PoseExtractor::getHeatMapsCopy() const
{
try
{
......@@ -215,6 +216,43 @@ namespace op
}
}
std::vector<std::vector<std::array<float,3>>> PoseExtractor::getCandidatesCopy() const
{
try
{
// Security check
checkThread();
// Initialization
std::vector<std::vector<std::array<float,3>>> candidates;
// Fill candidates
if (mAddPartCandidates)
{
const auto numberBodyParts = getPoseNumberBodyParts(mPoseModel);
candidates.resize(numberBodyParts);
const auto peaksArea = (POSE_MAX_PEOPLE+1) * 3;
// Memory copy
const auto* candidatesCpuPtr = getCandidatesCpuConstPtr();
for (auto part = 0u ; part < numberBodyParts ; part++)
{
const auto numberPartCandidates = candidatesCpuPtr[part*peaksArea];
candidates[part].resize(numberPartCandidates);
const auto* partCandidatesPtr = &candidatesCpuPtr[part*peaksArea+3];
for (auto candidate = 0 ; candidate < numberPartCandidates ; candidate++)
candidates[part][candidate] = {partCandidatesPtr[3*candidate],
partCandidatesPtr[3*candidate+1],
partCandidatesPtr[3*candidate+2]};
}
}
// Return
return candidates;
}
catch (const std::exception& e)
{
error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return std::vector<std::vector<std::array<float,3>>>{};
}
}
Array<float> PoseExtractor::getPoseKeypoints() const
{
try
......
......@@ -131,8 +131,9 @@ namespace op
PoseExtractorCaffe::PoseExtractorCaffe(const PoseModel poseModel, const std::string& modelFolder,
const int gpuId, const std::vector<HeatMapType>& heatMapTypes,
const ScaleMode heatMapScale, const bool enableGoogleLogging) :
PoseExtractor{poseModel, heatMapTypes, heatMapScale}
const ScaleMode heatMapScale, const bool addPartCandidates,
const bool enableGoogleLogging) :
PoseExtractor{poseModel, heatMapTypes, heatMapScale, addPartCandidates}
#ifdef USE_CAFFE
, upImpl{new ImplPoseExtractorCaffe{poseModel, gpuId, modelFolder, enableGoogleLogging}}
#endif
......@@ -148,6 +149,7 @@ namespace op
UNUSED(gpuId);
UNUSED(heatMapTypes);
UNUSED(heatMapScale);
UNUSED(addPartCandidates);
error("OpenPose must be compiled with the `USE_CAFFE` macro definition in order to use this"
" functionality.", __LINE__, __FUNCTION__, __FILE__);
#endif
......@@ -300,6 +302,42 @@ namespace op
}
}
const float* PoseExtractorCaffe::getCandidatesCpuConstPtr() const
{
try
{
#ifdef USE_CAFFE
checkThread();
return upImpl->spPeaksBlob->cpu_data();
#else
return nullptr;
#endif
}
catch (const std::exception& e)
{
error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return nullptr;
}
}
const float* PoseExtractorCaffe::getCandidatesGpuConstPtr() const
{
try
{
#ifdef USE_CAFFE
checkThread();
return upImpl->spPeaksBlob->gpu_data();
#else
return nullptr;
#endif
}
catch (const std::exception& e)
{
error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return nullptr;
}
}
const float* PoseExtractorCaffe::getHeatMapCpuConstPtr() const
{
try
......
......@@ -2,16 +2,18 @@
namespace op
{
WrapperStructOutput::WrapperStructOutput(const bool displayGui_, const bool guiVerbose_, const bool fullScreen_, const std::string& writeKeypoint_,
const DataFormat writeKeypointFormat_, const std::string& writeKeypointJson_, const std::string& writeCocoJson_,
const std::string& writeImages_, const std::string& writeImagesFormat_, const std::string& writeVideo_,
const std::string& writeHeatMaps_, const std::string& writeHeatMapsFormat_) :
WrapperStructOutput::WrapperStructOutput(const bool displayGui_, const bool guiVerbose_, const bool fullScreen_,
const std::string& writeKeypoint_, const DataFormat writeKeypointFormat_,
const std::string& writeJson_, const std::string& writeCocoJson_,
const std::string& writeImages_, const std::string& writeImagesFormat_,
const std::string& writeVideo_, const std::string& writeHeatMaps_,
const std::string& writeHeatMapsFormat_) :
displayGui{displayGui_},
guiVerbose{guiVerbose_},
fullScreen{fullScreen_},
writeKeypoint{writeKeypoint_},
writeKeypointFormat{writeKeypointFormat_},
writeKeypointJson{writeKeypointJson_},
writeJson{writeJson_},
writeCocoJson{writeCocoJson_},
writeImages{writeImages_},
writeImagesFormat{writeImagesFormat_},
......
......@@ -11,8 +11,9 @@ namespace op
const float alphaHeatMap_, const int defaultPartToRender_,
const std::string& modelFolder_,
const std::vector<HeatMapType>& heatMapTypes_,
const ScaleMode heatMapScale_, const float renderThreshold_,
const bool enableGoogleLogging_, const bool identification_) :
const ScaleMode heatMapScale_, const bool addPartCandidates_,
const float renderThreshold_, const bool enableGoogleLogging_,
const bool identification_) :
enable{enable_},
netInputSize{netInputSize_},
outputSize{outputSize_},
......@@ -30,6 +31,7 @@ namespace op
modelFolder{modelFolder_},
heatMapTypes{heatMapTypes_},
heatMapScale{heatMapScale_},
addPartCandidates{addPartCandidates_},
renderThreshold{renderThreshold_},
enableGoogleLogging{enableGoogleLogging_},
identification{identification_}
......
......@@ -151,15 +151,15 @@
<ClInclude Include="..\..\include\openpose\filestream\heatMapSaver.hpp" />
<ClInclude Include="..\..\include\openpose\filestream\imageSaver.hpp" />
<ClInclude Include="..\..\include\openpose\filestream\jsonOfstream.hpp" />
<ClInclude Include="..\..\include\openpose\filestream\keypointJsonSaver.hpp" />
<ClInclude Include="..\..\include\openpose\filestream\keypointSaver.hpp" />
<ClInclude Include="..\..\include\openpose\filestream\peopleJsonSaver.hpp" />
<ClInclude Include="..\..\include\openpose\filestream\videoSaver.hpp" />
<ClInclude Include="..\..\include\openpose\filestream\wCocoJsonSaver.hpp" />
<ClInclude Include="..\..\include\openpose\filestream\wFaceSaver.hpp" />
<ClInclude Include="..\..\include\openpose\filestream\wHandSaver.hpp" />
<ClInclude Include="..\..\include\openpose\filestream\wHeatMapSaver.hpp" />
<ClInclude Include="..\..\include\openpose\filestream\wImageSaver.hpp" />
<ClInclude Include="..\..\include\openpose\filestream\wKeypointJsonSaver.hpp" />
<ClInclude Include="..\..\include\openpose\filestream\wPeopleJsonSaver.hpp" />
<ClInclude Include="..\..\include\openpose\filestream\wPoseSaver.hpp" />
<ClInclude Include="..\..\include\openpose\filestream\wVideoSaver.hpp" />
<ClInclude Include="..\..\include\openpose\gui\enumClasses.hpp" />
......@@ -290,8 +290,8 @@
<ClCompile Include="..\..\src\openpose\filestream\heatMapSaver.cpp" />
<ClCompile Include="..\..\src\openpose\filestream\imageSaver.cpp" />
<ClCompile Include="..\..\src\openpose\filestream\jsonOfstream.cpp" />
<ClCompile Include="..\..\src\openpose\filestream\keypointJsonSaver.cpp" />
<ClCompile Include="..\..\src\openpose\filestream\keypointSaver.cpp" />
<ClCompile Include="..\..\src\openpose\filestream\peopleJsonSaver.cpp" />
<ClCompile Include="..\..\src\openpose\filestream\videoSaver.cpp" />
<ClCompile Include="..\..\src\openpose\gui\defineTemplates.cpp" />
<ClCompile Include="..\..\src\openpose\gui\frameDisplayer.cpp" />
......
......@@ -248,10 +248,10 @@
<ClInclude Include="..\..\include\openpose\filestream\jsonOfstream.hpp">
<Filter>Header Files\filestream</Filter>
</ClInclude>
<ClInclude Include="..\..\include\openpose\filestream\keypointJsonSaver.hpp">
<ClInclude Include="..\..\include\openpose\filestream\keypointSaver.hpp">
<Filter>Header Files\filestream</Filter>
</ClInclude>
<ClInclude Include="..\..\include\openpose\filestream\keypointSaver.hpp">
<ClInclude Include="..\..\include\openpose\filestream\peopleJsonSaver.hpp">
<Filter>Header Files\filestream</Filter>
</ClInclude>
<ClInclude Include="..\..\include\openpose\filestream\videoSaver.hpp">
......@@ -272,7 +272,7 @@
<ClInclude Include="..\..\include\openpose\filestream\wImageSaver.hpp">
<Filter>Header Files\filestream</Filter>
</ClInclude>
<ClInclude Include="..\..\include\openpose\filestream\wKeypointJsonSaver.hpp">
<ClInclude Include="..\..\include\openpose\filestream\wPeopleJsonSaver.hpp">
<Filter>Header Files\filestream</Filter>
</ClInclude>
<ClInclude Include="..\..\include\openpose\filestream\wPoseSaver.hpp">
......@@ -657,10 +657,10 @@
<ClCompile Include="..\..\src\openpose\filestream\jsonOfstream.cpp">
<Filter>Source Files\filestream</Filter>
</ClCompile>
<ClCompile Include="..\..\src\openpose\filestream\keypointJsonSaver.cpp">
<ClCompile Include="..\..\src\openpose\filestream\keypointSaver.cpp">
<Filter>Source Files\filestream</Filter>
</ClCompile>
<ClCompile Include="..\..\src\openpose\filestream\keypointSaver.cpp">
<ClCompile Include="..\..\src\openpose\filestream\peopleJsonSaver.cpp">
<Filter>Source Files\filestream</Filter>
</ClCompile>
<ClCompile Include="..\..\src\openpose\filestream\videoSaver.cpp">
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册