提交 eae440bd 编写于 作者: D Donglai Xiang 提交者: Gines

Face Detection by OpenCV (#270)

* first try on OpenCV face

* find a way to visualize detected face; enlarge by 1.5x

* revision for first-time pull request

* faceDetector

* fix little mistake

* Update headers.hpp

* Update CMakeLists.txt

* OpenCV face detector thread save & 4x faster

* Added doc
上级 f54a312b
......@@ -160,7 +160,7 @@ ifeq ($(USE_LMDB), 1)
LIBRARIES += lmdb
endif
ifeq ($(USE_OPENCV), 1)
LIBRARIES += opencv_core opencv_highgui opencv_imgproc
LIBRARIES += opencv_core opencv_highgui opencv_imgproc opencv_objdetect
ifeq ($(OPENCV_VERSION), 3)
LIBRARIES += opencv_imgcodecs opencv_videoio
......
......@@ -143,6 +143,7 @@ Each flag is divided into flag name, default value, and description.
- DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) coordinates of the final pose data array, i.e. the scale of the (x,y) coordinates that will be saved with the `write_keypoint` & `write_keypoint_json` flags. Select `0` to scale it to the original source resolution, `1`to scale it to the net output size (set with `net_resolution`), `2` to scale it to the final output size (set with `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related with `scale_number` and `scale_gap`.");
4. OpenPose Body Pose
- DEFINE_bool(body_disable, false, "Disable body keypoint detection. Option only possible for faster (but less accurate) face keypoint detection.");
- DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), `MPI_4_layers` (15 keypoints, even faster but less accurate).");
- DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is decreased, the speed increases. For maximum speed-accuracy balance, it should keep the closest aspect ratio possible to the images or videos to be processed. Using `-1` in any of the dimensions, OP will choose the optimal resolution depending on the other value introduced by the user. E.g. the default `-1x368` is equivalent to `656x368` in 16:9 videos, e.g. full HD (1980x1080) and HD (1280x720) resolutions.");
- DEFINE_int32(scale_number, 1, "Number of scales to average.");
......
......@@ -120,10 +120,11 @@ OpenPose Library - Release Notes
1. FrameDisplayer accepts variable size images by rescaling every time a frame with bigger width or height is displayed (gui module).
2. OpOutputToCvMat & GuiInfoAdder does not require to know the output size at construction time, deduced from each image.
3. CvMatToOutput and Renderers allow to keep input resolution as output for images (core module).
3. Face and hand keypoint detectors now can return each keypoint heatmap.
4. COCO JSON file outputs 0 as score for non-detected keypoints.
5. Added example for OpenPose for user asynchronous output and cleaned all `tutorial_wrapper/` examples.
6. Added `-1` option for `net_resolution` in order to auto-select the best possible aspect ratio given the user input.
3. New standalone face keypoint detector based on OpenCV face detector: much faster if body keypoint detection is not required but much less accurate.
4. Face and hand keypoint detectors now can return each keypoint heatmap.
5. COCO JSON file outputs 0 as score for non-detected keypoints.
6. Added example for OpenPose for user asynchronous output and cleaned all `tutorial_wrapper/` examples.
7. Added `-1` option for `net_resolution` in order to auto-select the best possible aspect ratio given the user input.
2. Functions or parameters renamed:
1. OpenPose able to change its size and initial size:
1. Flag `resolution` renamed as `output_resolution`.
......
......@@ -4,11 +4,17 @@ OpenPose Library - Standalone Face Or Hand Keypoint Detector
In case of hand camera views at which the hands are visible but not the rest of the body, or if you do not need the body keypoint detector and want to considerably speed up the process, you can use the OpenPose face or hand keypoint detectors with your own face or hand detectors, rather than using the body keypoint detector as initial detector for those.
## Standalone Face Keypoint Detector
Note that this method will be much faster than current system, but also much less accurate.
```
./build/examples/openpose/openpose.bin --face --body_disable
```
## Custom Standalone Face Keypoint Detector
There are 2 ways to add the OpenPose face keypoint detector to your own code without using the body pose keypoint extractor as initial face detector:
1. Easiest solution: Forget about the `OpenPose demo` and `wrapper/wrapper.hpp`, and instead use the `include/openpose/face/faceExtractor.hpp` class with the output of your face detector. Recommended if you do not wanna use any other OpenPose functionality.
2. Elegant solution: If you wanna use the whole OpenPose framework, simply copy `include/wrapper/wrapper.hpp` as e.g. `examples/userCode/wrapperFace.hpp`, and change our `FaceDetector` class by your custom face detector class. If you wanna omit the Pose keypoint detection, you can simply delete it from that custom wrapper too.
2. Elegant solution: If you wanna use the whole OpenPose framework, simply copy `include/wrapper/wrapper.hpp` as e.g. `examples/userCode/wrapperFace.hpp`, and change our `FaceDetector` or `FaceDetectorOpenCV` class by your custom face detector class inside your `WrapperFace` class. If you wanna omit the Pose keypoint detection for a big speed up if you do not need it, you can simply use the `body_disable` flag.
## Standalone Hand Keypoint Detector
## Custom Standalone Hand Keypoint Detector
The analogous steps apply to the hand keypoint detector, but modifying `include/openpose/hand/handExtractor.hpp`.
......@@ -64,6 +64,8 @@ DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) co
" `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related"
" with `scale_number` and `scale_gap`.");
// OpenPose Body Pose
DEFINE_bool(body_disable, false, "Disable body keypoint detection. Option only possible for faster (but less accurate) face"
" keypoint detection.");
DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), "
"`MPI_4_layers` (15 keypoints, even faster but less accurate).");
DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is"
......@@ -196,7 +198,7 @@ int openPoseDemo()
op::log("Configuring OpenPose wrapper.", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__);
op::Wrapper<std::vector<op::Datum>> opWrapper;
// Pose configuration (use WrapperStructPose{} for default and recommended configuration)
const op::WrapperStructPose wrapperStructPose{netInputSize, outputSize, keypointScale, FLAGS_num_gpu,
const op::WrapperStructPose wrapperStructPose{!FLAGS_body_disable, netInputSize, outputSize, keypointScale, FLAGS_num_gpu,
FLAGS_num_gpu_start, FLAGS_scale_number, (float)FLAGS_scale_gap,
op::flagsToRenderMode(FLAGS_render_pose), poseModel,
!FLAGS_disable_blending, (float)FLAGS_alpha_pose,
......
......@@ -49,7 +49,7 @@ int handFromJsonTest()
op::log("Configuring OpenPose wrapper.", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__);
op::WrapperHandFromJsonTest<std::vector<op::Datum>> opWrapper;
// Pose configuration (use WrapperStructPose{} for default and recommended configuration)
op::WrapperStructPose wrapperStructPose{op::flagsToPoint("656x368"), op::flagsToPoint("1280x720"),
op::WrapperStructPose wrapperStructPose{true, op::flagsToPoint("656x368"), op::flagsToPoint("1280x720"),
op::ScaleMode::InputResolution, FLAGS_num_gpu, FLAGS_num_gpu_start};
wrapperStructPose.modelFolder = FLAGS_model_folder;
// Hand configuration (use op::WrapperStructHand{} to disable it)
......
......@@ -64,6 +64,8 @@ DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) co
" `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related"
" with `scale_number` and `scale_gap`.");
// OpenPose Body Pose
DEFINE_bool(body_disable, false, "Disable body keypoint detection. Option only possible for faster (but less accurate) face"
" keypoint detection.");
DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), "
"`MPI_4_layers` (15 keypoints, even faster but less accurate).");
DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is"
......@@ -288,7 +290,7 @@ int openPoseTutorialWrapper3()
op::log("Configuring OpenPose wrapper.", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__);
op::Wrapper<std::vector<UserDatum>> opWrapper{op::ThreadManagerMode::AsynchronousOut};
// Pose configuration (use WrapperStructPose{} for default and recommended configuration)
const op::WrapperStructPose wrapperStructPose{netInputSize, outputSize, keypointScale, FLAGS_num_gpu,
const op::WrapperStructPose wrapperStructPose{!FLAGS_body_disable, netInputSize, outputSize, keypointScale, FLAGS_num_gpu,
FLAGS_num_gpu_start, FLAGS_scale_number, (float)FLAGS_scale_gap,
op::flagsToRenderMode(FLAGS_render_pose), poseModel,
!FLAGS_disable_blending, (float)FLAGS_alpha_pose,
......
......@@ -47,6 +47,8 @@ DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) co
" `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related"
" with `scale_number` and `scale_gap`.");
// OpenPose Body Pose
DEFINE_bool(body_disable, false, "Disable body keypoint detection. Option only possible for faster (but less accurate) face"
" keypoint detection.");
DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), "
"`MPI_4_layers` (15 keypoints, even faster but less accurate).");
DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is"
......@@ -380,7 +382,7 @@ int openPoseTutorialWrapper2()
opWrapper.setWorkerOutput(wUserOutput, workerOutputOnNewThread);
// Configure OpenPose
op::log("Configuring OpenPose wrapper.", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__);
const op::WrapperStructPose wrapperStructPose{netInputSize, outputSize, keypointScale, FLAGS_num_gpu,
const op::WrapperStructPose wrapperStructPose{!FLAGS_body_disable, netInputSize, outputSize, keypointScale, FLAGS_num_gpu,
FLAGS_num_gpu_start, FLAGS_scale_number, (float)FLAGS_scale_gap,
op::flagsToRenderMode(FLAGS_render_pose), poseModel,
!FLAGS_disable_blending, (float)FLAGS_alpha_pose,
......
......@@ -47,6 +47,8 @@ DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) co
" `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related"
" with `scale_number` and `scale_gap`.");
// OpenPose Body Pose
DEFINE_bool(body_disable, false, "Disable body keypoint detection. Option only possible for faster (but less accurate) face"
" keypoint detection.");
DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), "
"`MPI_4_layers` (15 keypoints, even faster but less accurate).");
DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is"
......@@ -322,7 +324,7 @@ int openPoseTutorialWrapper1()
// Configure OpenPose
op::Wrapper<std::vector<UserDatum>> opWrapper{op::ThreadManagerMode::Asynchronous};
// Pose configuration (use WrapperStructPose{} for default and recommended configuration)
const op::WrapperStructPose wrapperStructPose{netInputSize, outputSize, keypointScale, FLAGS_num_gpu,
const op::WrapperStructPose wrapperStructPose{!FLAGS_body_disable, netInputSize, outputSize, keypointScale, FLAGS_num_gpu,
FLAGS_num_gpu_start, FLAGS_scale_number, (float)FLAGS_scale_gap,
op::flagsToRenderMode(FLAGS_render_pose), poseModel,
!FLAGS_disable_blending, (float)FLAGS_alpha_pose,
......
......@@ -47,6 +47,8 @@ DEFINE_int32(keypoint_scale, 0, "Scaling of the (x,y) co
" `resolution`), `3` to scale it in the range [0,1], and 4 for range [-1,1]. Non related"
" with `scale_number` and `scale_gap`.");
// OpenPose Body Pose
DEFINE_bool(body_disable, false, "Disable body keypoint detection. Option only possible for faster (but less accurate) face"
" keypoint detection.");
DEFINE_string(model_pose, "COCO", "Model to be used. E.g. `COCO` (18 keypoints), `MPI` (15 keypoints, ~10% faster), "
"`MPI_4_layers` (15 keypoints, even faster but less accurate).");
DEFINE_string(net_resolution, "656x368", "Multiples of 16. If it is increased, the accuracy potentially increases. If it is"
......@@ -191,7 +193,7 @@ int openpose3d()
const auto workerOutputOnNewThread = true;
opWrapper.setWorkerOutput(wRender3D, workerOutputOnNewThread);
// Configure OpenPose
const op::WrapperStructPose wrapperStructPose{netInputSize, outputSize, keypointScale, FLAGS_num_gpu,
const op::WrapperStructPose wrapperStructPose{!FLAGS_body_disable, netInputSize, outputSize, keypointScale, FLAGS_num_gpu,
FLAGS_num_gpu_start, FLAGS_scale_number, (float)FLAGS_scale_gap,
op::flagsToRenderMode(FLAGS_render_pose), poseModel,
!FLAGS_disable_blending, (float)FLAGS_alpha_pose,
......
......@@ -10,21 +10,21 @@
*/
cv::Mat pointGreyToCvMat(const Spinnaker::ImagePtr &imagePtr)
{
try
{
const auto XPadding = imagePtr->GetXPadding();
const auto YPadding = imagePtr->GetYPadding();
const auto rowsize = imagePtr->GetWidth();
const auto colsize = imagePtr->GetHeight();
// image data contains padding. When allocating cv::Mat container size, you need to account for the X,Y image data padding.
return cv::Mat((int)(colsize + YPadding), (int)(rowsize + XPadding), CV_8UC3, imagePtr->GetData(), imagePtr->GetStride());
}
catch (const std::exception& e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return cv::Mat();
}
try
{
const auto XPadding = imagePtr->GetXPadding();
const auto YPadding = imagePtr->GetYPadding();
const auto rowsize = imagePtr->GetWidth();
const auto colsize = imagePtr->GetHeight();
// image data contains padding. When allocating cv::Mat container size, you need to account for the X,Y image data padding.
return cv::Mat((int)(colsize + YPadding), (int)(rowsize + XPadding), CV_8UC3, imagePtr->GetData(), imagePtr->GetStride());
}
catch (const std::exception& e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return cv::Mat();
}
}
// This function configures the camera to use a trigger. First, trigger mode is
......@@ -35,24 +35,24 @@ int configureTrigger(Spinnaker::GenApi::INodeMap &iNodeMap)
{
try
{
int result = 0;
op::log("*** CONFIGURING TRIGGER ***", op::Priority::High);
op::log("Configuring hardware trigger...", op::Priority::High);
int result = 0;
op::log("*** CONFIGURING TRIGGER ***", op::Priority::High);
op::log("Configuring hardware trigger...", op::Priority::High);
// Ensure trigger mode off
// *** NOTES ***
// The trigger must be disabled in order to configure whether the source
// is software or hardware.
Spinnaker::GenApi::CEnumerationPtr ptrTriggerMode = iNodeMap.GetNode("TriggerMode");
if (!Spinnaker::GenApi::IsAvailable(ptrTriggerMode) || !Spinnaker::GenApi::IsReadable(ptrTriggerMode))
op::error("Unable to disable trigger mode (node retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
op::error("Unable to disable trigger mode (node retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
Spinnaker::GenApi::CEnumEntryPtr ptrTriggerModeOff = ptrTriggerMode->GetEntryByName("Off");
if (!Spinnaker::GenApi::IsAvailable(ptrTriggerModeOff) || !Spinnaker::GenApi::IsReadable(ptrTriggerModeOff))
op::error("Unable to disable trigger mode (enum entry retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
op::error("Unable to disable trigger mode (enum entry retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
ptrTriggerMode->SetIntValue(ptrTriggerModeOff->GetValue());
op::log("Trigger mode disabled...", op::Priority::High);
op::log("Trigger mode disabled...", op::Priority::High);
// Select trigger source
// *** NOTES ***
......@@ -60,16 +60,16 @@ int configureTrigger(Spinnaker::GenApi::INodeMap &iNodeMap)
// mode is off.
Spinnaker::GenApi::CEnumerationPtr ptrTriggerSource = iNodeMap.GetNode("TriggerSource");
if (!Spinnaker::GenApi::IsAvailable(ptrTriggerSource) || !Spinnaker::GenApi::IsWritable(ptrTriggerSource))
op::error("Unable to set trigger mode (node retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
op::error("Unable to set trigger mode (node retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
// Set trigger mode to hardware ('Line0')
Spinnaker::GenApi::CEnumEntryPtr ptrTriggerSourceHardware = ptrTriggerSource->GetEntryByName("Line0");
if (!Spinnaker::GenApi::IsAvailable(ptrTriggerSourceHardware) || !Spinnaker::GenApi::IsReadable(ptrTriggerSourceHardware))
op::error("Unable to set trigger mode (enum entry retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
op::error("Unable to set trigger mode (enum entry retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
ptrTriggerSource->SetIntValue(ptrTriggerSourceHardware->GetValue());
op::log("Trigger source set to hardware...", op::Priority::High);
op::log("Trigger source set to hardware...", op::Priority::High);
// Turn trigger mode on
// *** LATER ***
......@@ -78,26 +78,26 @@ int configureTrigger(Spinnaker::GenApi::INodeMap &iNodeMap)
Spinnaker::GenApi::CEnumEntryPtr ptrTriggerModeOn = ptrTriggerMode->GetEntryByName("On");
if (!Spinnaker::GenApi::IsAvailable(ptrTriggerModeOn) || !Spinnaker::GenApi::IsReadable(ptrTriggerModeOn))
{
op::error("Unable to enable trigger mode (enum entry retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
op::error("Unable to enable trigger mode (enum entry retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
return -1;
}
ptrTriggerMode->SetIntValue(ptrTriggerModeOn->GetValue());
op::log("Trigger mode turned back on...", op::Priority::High);
op::log("Trigger mode turned back on...", op::Priority::High);
return result;
return result;
}
catch (const Spinnaker::Exception& e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return -1;
}
catch (const std::exception& e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return -1;
}
catch (const Spinnaker::Exception& e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return -1;
}
catch (const std::exception& e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return -1;
}
}
// This function returns the camera to a normal state by turning off trigger
......@@ -106,7 +106,7 @@ int resetTrigger(Spinnaker::GenApi::INodeMap &iNodeMap)
{
try
{
int result = 0;
int result = 0;
//
// Turn trigger mode back off
//
......@@ -116,28 +116,28 @@ int resetTrigger(Spinnaker::GenApi::INodeMap &iNodeMap)
//
Spinnaker::GenApi::CEnumerationPtr ptrTriggerMode = iNodeMap.GetNode("TriggerMode");
if (!Spinnaker::GenApi::IsAvailable(ptrTriggerMode) || !Spinnaker::GenApi::IsReadable(ptrTriggerMode))
op::error("Unable to disable trigger mode (node retrieval). Non-fatal error...", __LINE__, __FUNCTION__, __FILE__);
op::error("Unable to disable trigger mode (node retrieval). Non-fatal error...", __LINE__, __FUNCTION__, __FILE__);
Spinnaker::GenApi::CEnumEntryPtr ptrTriggerModeOff = ptrTriggerMode->GetEntryByName("Off");
if (!Spinnaker::GenApi::IsAvailable(ptrTriggerModeOff) || !Spinnaker::GenApi::IsReadable(ptrTriggerModeOff))
op::error("Unable to disable trigger mode (enum entry retrieval). Non-fatal error...", __LINE__, __FUNCTION__, __FILE__);
op::error("Unable to disable trigger mode (enum entry retrieval). Non-fatal error...", __LINE__, __FUNCTION__, __FILE__);
ptrTriggerMode->SetIntValue(ptrTriggerModeOff->GetValue());
// op::log("Trigger mode disabled...", op::Priority::High);
// op::log("Trigger mode disabled...", op::Priority::High);
return result;
return result;
}
catch (Spinnaker::Exception &e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return -1;
}
catch (const std::exception& e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return -1;
}
catch (const std::exception& e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return -1;
}
}
}
// This function acquires and displays images from each device.
......@@ -149,7 +149,7 @@ std::vector<cv::Mat> acquireImages(Spinnaker::CameraList &cameraList)
if (cameraList.GetSize() != INTRINSICS.size())
op::checkE(cameraList.GetSize(), INTRINSICS.size(), "The number of cameras must be the same as the INTRINSICS vector size.", __LINE__, __FUNCTION__, __FILE__);
std::vector<cv::Mat> cvMats;
std::vector<cv::Mat> cvMats;
// Retrieve, convert, and return an image for each camera
// In order to work with simultaneous camera streams, nested loops are
......@@ -176,7 +176,7 @@ std::vector<cv::Mat> acquireImages(Spinnaker::CameraList &cameraList)
for (auto i = 0u; i < cameraPtrs.size(); i++)
imagePtrs.at(i) = cameraPtrs.at(i)->GetNextImage();
durationMs = std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::high_resolution_clock::now()-begin).count() * 1e-6;
// op::log("Time extraction (ms): " + std::to_string(durationMs), op::Priority::High, __LINE__, __FUNCTION__, __FILE__);
// op::log("Time extraction (ms): " + std::to_string(durationMs), op::Priority::High);
}
// Original format -> RGB8
......@@ -186,7 +186,7 @@ std::vector<cv::Mat> acquireImages(Spinnaker::CameraList &cameraList)
if (imagePtr->IsIncomplete())
{
op::log("Image incomplete with image status " + std::to_string(imagePtr->GetImageStatus()) + "...",
op::Priority::High, __LINE__, __FUNCTION__, __FILE__);
op::Priority::High, __LINE__, __FUNCTION__, __FILE__);
imagesExtracted = false;
break;
}
......@@ -222,7 +222,7 @@ std::vector<cv::Mat> acquireImages(Spinnaker::CameraList &cameraList)
// imagePtr = imagePtr;
// }
// durationMs = std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::high_resolution_clock::now()-begin).count() * 1e-6;
// op::log("Time conversion (ms): " + std::to_string(durationMs / reps), op::Priority::High, __LINE__, __FUNCTION__, __FILE__);
// op::log("Time conversion (ms): " + std::to_string(durationMs / reps), op::Priority::High);
}
}
......@@ -241,18 +241,18 @@ std::vector<cv::Mat> acquireImages(Spinnaker::CameraList &cameraList)
}
}
return cvMats;
return cvMats;
}
catch (Spinnaker::Exception &e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return {};
}
catch (const std::exception& e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return {};
}
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return {};
}
catch (const std::exception& e)
{
op::error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return {};
}
}
// This function prints the device information of the camera from the transport
......@@ -262,7 +262,7 @@ int printDeviceInfo(Spinnaker::GenApi::INodeMap &iNodeMap, const unsigned int ca
{
int result = 0;
op::log("Printing device information for camera " + std::to_string(camNum) + "...\n", op::Priority::High);
op::log("Printing device information for camera " + std::to_string(camNum) + "...\n", op::Priority::High);
Spinnaker::GenApi::FeatureList_t features;
Spinnaker::GenApi::CCategoryPtr cCategoryPtr = iNodeMap.GetNode("DeviceInformation");
......@@ -275,12 +275,12 @@ int printDeviceInfo(Spinnaker::GenApi::INodeMap &iNodeMap, const unsigned int ca
{
Spinnaker::GenApi::CNodePtr pfeatureNode = *it;
const auto cValuePtr = (Spinnaker::GenApi::CValuePtr)pfeatureNode;
op::log(pfeatureNode->GetName() + " : " + (IsReadable(cValuePtr) ? cValuePtr->ToString() : "Node not readable"), op::Priority::High);
op::log(pfeatureNode->GetName() + " : " + (IsReadable(cValuePtr) ? cValuePtr->ToString() : "Node not readable"), op::Priority::High);
}
}
else
op::log("Device control information not available.", op::Priority::High);
op::log(" ", op::Priority::High);
op::log("Device control information not available.", op::Priority::High);
op::log(" ", op::Priority::High);
return result;
}
......@@ -335,7 +335,7 @@ WPointGrey::~WPointGrey()
// Reset trigger
auto result = resetTrigger(iNodeMap);
if (result < 0)
op::error("Error happened..." + std::to_string(result), __LINE__, __FUNCTION__, __FILE__);
op::error("Error happened..." + std::to_string(result), __LINE__, __FUNCTION__, __FILE__);
// Deinitialize each camera
// Each camera must be deinitialized separately by first
......@@ -343,7 +343,7 @@ WPointGrey::~WPointGrey()
cameraPtr->DeInit();
}
op::log("Completed. Releasing...", op::Priority::High);
op::log("Completed. Releasing...", op::Priority::High);
// Clear camera list before releasing mSystemPtr
mCameraList.Clear();
......@@ -352,7 +352,7 @@ WPointGrey::~WPointGrey()
mSystemPtr->ReleaseInstance();
}
op::log("Done! Exitting...", op::Priority::High);
op::log("Done! Exitting...", op::Priority::High);
}
catch (const Spinnaker::Exception& e)
{
......@@ -371,7 +371,7 @@ void WPointGrey::initializationOnThread()
initialized = true;
// Print application build information
op::log(std::string{ "Application build date: " } + __DATE__ + " " + __TIME__, op::Priority::High, __LINE__, __FUNCTION__, __FILE__);
op::log(std::string{ "Application build date: " } + __DATE__ + " " + __TIME__, op::Priority::High);
// Retrieve singleton reference to mSystemPtr object
mSystemPtr = Spinnaker::System::GetInstance();
......@@ -381,7 +381,7 @@ void WPointGrey::initializationOnThread()
unsigned int numCameras = mCameraList.GetSize();
op::log("Number of cameras detected: " + std::to_string(numCameras), op::Priority::High, __LINE__, __FUNCTION__, __FILE__);
op::log("Number of cameras detected: " + std::to_string(numCameras), op::Priority::High);
// Finish if there are no cameras
if (numCameras == 0)
......@@ -392,12 +392,12 @@ void WPointGrey::initializationOnThread()
// Release mSystemPtr
mSystemPtr->ReleaseInstance();
op::log("Not enough cameras!\nPress Enter to exit...", op::Priority::High);
op::log("Not enough cameras!\nPress Enter to exit...", op::Priority::High);
getchar();
op::error("No cameras detected.", __LINE__, __FUNCTION__, __FILE__);
}
op::log("Camera system initialized...", op::Priority::High);
op::log("Camera system initialized...", op::Priority::High);
//
// Retrieve transport layer nodemaps and print device information for
......@@ -409,7 +409,7 @@ void WPointGrey::initializationOnThread()
// serial number. Rather than caching the nodemap, each nodemap is
// retrieved both times as needed.
//
op::log("\n*** DEVICE INFORMATION ***\n", op::Priority::High);
op::log("\n*** DEVICE INFORMATION ***\n", op::Priority::High);
for (int i = 0; i < mCameraList.GetSize(); i++)
{
......@@ -445,7 +445,7 @@ void WPointGrey::initializationOnThread()
// // Configure trigger
// result = configureTrigger(iNodeMap);
// if (result < 0)
// op::error("Result > 0, error " + std::to_string(result) + " occurred...", __LINE__, __FUNCTION__, __FILE__);
// op::error("Result > 0, error " + std::to_string(result) + " occurred...", __LINE__, __FUNCTION__, __FILE__);
// // Configure chunk data
// result = configureChunkData(iNodeMap);
......@@ -456,11 +456,11 @@ void WPointGrey::initializationOnThread()
Spinnaker::GenApi::INodeMap& snodeMap = cameraPtr->GetTLStreamNodeMap();
Spinnaker::GenApi::CEnumerationPtr ptrBufferHandlingMode = snodeMap.GetNode("StreamBufferHandlingMode");
if (!Spinnaker::GenApi::IsAvailable(ptrBufferHandlingMode) || !Spinnaker::GenApi::IsWritable(ptrBufferHandlingMode))
op::error("Unable to change buffer handling mode", __LINE__, __FUNCTION__, __FILE__);
op::error("Unable to change buffer handling mode", __LINE__, __FUNCTION__, __FILE__);
Spinnaker::GenApi::CEnumEntryPtr ptrBufferHandlingModeNewest = ptrBufferHandlingMode->GetEntryByName("NewestFirstOverwrite");
if (!Spinnaker::GenApi::IsAvailable(ptrBufferHandlingModeNewest) || !IsReadable(ptrBufferHandlingModeNewest))
op::error("Unable to set buffer handling mode to newest (entry 'NewestFirstOverwrite' retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
op::error("Unable to set buffer handling mode to newest (entry 'NewestFirstOverwrite' retrieval). Aborting...", __LINE__, __FUNCTION__, __FILE__);
int64_t bufferHandlingModeNewest = ptrBufferHandlingModeNewest->GetValue();
ptrBufferHandlingMode->SetIntValue(bufferHandlingModeNewest);
......@@ -486,22 +486,22 @@ void WPointGrey::initializationOnThread()
// Set acquisition mode to continuous
Spinnaker::GenApi::CEnumerationPtr ptrAcquisitionMode = cameraPtr->GetNodeMap().GetNode("AcquisitionMode");
if (!Spinnaker::GenApi::IsAvailable(ptrAcquisitionMode) || !Spinnaker::GenApi::IsWritable(ptrAcquisitionMode))
op::error("Unable to set acquisition mode to continuous (node retrieval; camera " + std::to_string(i) + "). Aborting...", __LINE__, __FUNCTION__, __FILE__);
op::error("Unable to set acquisition mode to continuous (node retrieval; camera " + std::to_string(i) + "). Aborting...", __LINE__, __FUNCTION__, __FILE__);
Spinnaker::GenApi::CEnumEntryPtr ptrAcquisitionModeContinuous = ptrAcquisitionMode->GetEntryByName("Continuous");
if (!Spinnaker::GenApi::IsAvailable(ptrAcquisitionModeContinuous) || !Spinnaker::GenApi::IsReadable(ptrAcquisitionModeContinuous))
op::error("Unable to set acquisition mode to continuous (entry 'continuous' retrieval " + std::to_string(i) + "). Aborting...", __LINE__, __FUNCTION__, __FILE__);
op::error("Unable to set acquisition mode to continuous (entry 'continuous' retrieval " + std::to_string(i) + "). Aborting...", __LINE__, __FUNCTION__, __FILE__);
int64_t acquisitionModeContinuous = ptrAcquisitionModeContinuous->GetValue();
ptrAcquisitionMode->SetIntValue(acquisitionModeContinuous);
op::log("Camera " + std::to_string(i) + " acquisition mode set to continuous...", op::Priority::High);
op::log("Camera " + std::to_string(i) + " acquisition mode set to continuous...", op::Priority::High);
// Begin acquiring images
cameraPtr->BeginAcquisition();
op::log("Camera " + std::to_string(i) + " started acquiring images...", op::Priority::High);
op::log("Camera " + std::to_string(i) + " started acquiring images...", op::Priority::High);
// Retrieve device serial number for filename
strSerialNumbers[i] = "";
......@@ -513,10 +513,10 @@ void WPointGrey::initializationOnThread()
strSerialNumbers[i] = ptrStringSerial->GetValue();
op::log("Camera " + std::to_string(i) + " serial number set to " + strSerialNumbers[i].c_str() + "...", op::Priority::High);
}
op::log(" ", op::Priority::High);
op::log(" ", op::Priority::High);
}
op::log("\nRunning for all cameras...\n\n*** IMAGE ACQUISITION ***\n", op::Priority::High);
op::log("\nRunning for all cameras...\n\n*** IMAGE ACQUISITION ***\n", op::Priority::High);
}
catch (const Spinnaker::Exception& e)
{
......
#ifndef OPENPOSE_FACE_FACE_DETECTOR_OPENCV_HPP
#define OPENPOSE_FACE_FACE_DETECTOR_OPENCV_HPP
#include <opencv2/core/core.hpp>
#include <opencv2/objdetect/objdetect.hpp>
#include <openpose/core/common.hpp>
namespace op
{
class OP_API FaceDetectorOpenCV
{
public:
explicit FaceDetectorOpenCV(const std::string& modelFolder);
// No thread-save
std::vector<Rectangle<float>> detectFaces(const cv::Mat& cvInputData);
private:
cv::CascadeClassifier mFaceCascade;
DELETE_COPY(FaceDetectorOpenCV);
};
}
#endif // OPENPOSE_FACE_FACE_DETECTOR_OPENCV_HPP
......@@ -3,6 +3,7 @@
// face module
#include <openpose/face/faceDetector.hpp>
#include <openpose/face/faceDetectorOpenCV.hpp>
#include <openpose/face/faceExtractor.hpp>
#include <openpose/face/faceParameters.hpp>
#include <openpose/face/faceCpuRenderer.hpp>
......@@ -10,6 +11,7 @@
#include <openpose/face/faceRenderer.hpp>
#include <openpose/face/renderFace.hpp>
#include <openpose/face/wFaceDetector.hpp>
#include <openpose/face/wFaceDetectorOpenCV.hpp>
#include <openpose/face/wFaceExtractor.hpp>
#include <openpose/face/wFaceRenderer.hpp>
......
#ifndef OPENPOSE_FACE_W_FACE_EXTRACTOR_OPENCV_HPP
#define OPENPOSE_FACE_W_FACE_EXTRACTOR_OPENCV_HPP
#include <openpose/core/common.hpp>
#include <openpose/face/faceRenderer.hpp>
#include <openpose/thread/worker.hpp>
namespace op
{
template<typename TDatums>
class WFaceDetectorOpenCV : public Worker<TDatums>
{
public:
explicit WFaceDetectorOpenCV(const std::shared_ptr<FaceDetectorOpenCV>& faceDetectorOpenCV);
void initializationOnThread();
void work(TDatums& tDatums);
private:
std::shared_ptr<FaceDetectorOpenCV> spFaceDetectorOpenCV;
DELETE_COPY(WFaceDetectorOpenCV);
};
}
// Implementation
#include <openpose/utilities/pointerContainer.hpp>
namespace op
{
template<typename TDatums>
WFaceDetectorOpenCV<TDatums>::WFaceDetectorOpenCV(const std::shared_ptr<FaceDetectorOpenCV>& faceDetectorOpenCV) :
spFaceDetectorOpenCV{faceDetectorOpenCV}
{
}
template<typename TDatums>
void WFaceDetectorOpenCV<TDatums>::initializationOnThread()
{
}
template<typename TDatums>
void WFaceDetectorOpenCV<TDatums>::work(TDatums& tDatums)
{
try
{
if (checkNoNullNorEmpty(tDatums))
{
// Debugging log
dLog("", Priority::Low, __LINE__, __FUNCTION__, __FILE__);
// Profiling speed
const auto profilerKey = Profiler::timerInit(__LINE__, __FUNCTION__, __FILE__);
// Detect people face
for (auto& tDatum : *tDatums)
tDatum.faceRectangles = spFaceDetectorOpenCV->detectFaces(tDatum.cvInputData);
// Profiling speed
Profiler::timerEnd(profilerKey);
Profiler::printAveragedTimeMsOnIterationX(profilerKey, __LINE__, __FUNCTION__, __FILE__, Profiler::DEFAULT_X);
// Debugging log
dLog("", Priority::Low, __LINE__, __FUNCTION__, __FILE__);
}
}
catch (const std::exception& e)
{
this->stop();
tDatums = nullptr;
error(e.what(), __LINE__, __FUNCTION__, __FILE__);
}
}
COMPILE_TEMPLATE_DATUM(WFaceDetectorOpenCV);
}
#endif // OPENPOSE_FACE_W_FACE_EXTRACTOR_OPENCV_HPP
......@@ -104,7 +104,7 @@ namespace op
const std::string commonMessage{"Input images must be 3-channel BGR."};
if (datum.cvInputData.channels() == 1)
{
log(commonMessage + " Converting your grey image into BGR.", Priority::High, __LINE__, __FUNCTION__, __FILE__);
log(commonMessage + " Converting grey image into BGR.", Priority::High);
cv::cvtColor(datum.cvInputData, datum.cvInputData, CV_GRAY2BGR);
}
else
......
......@@ -524,6 +524,14 @@ namespace op
if (!wrapperStructOutput.writeVideo.empty() && wrapperStructInput.producerSharedPtr == nullptr)
error("Writting video is only available if the OpenPose producer is used (i.e."
" wrapperStructInput.producerSharedPtr cannot be a nullptr).", __LINE__, __FUNCTION__, __FILE__);
if (!wrapperStructPose.enable)
{
if (!wrapperStructFace.enable)
error("Body keypoint detection must be enabled.", __LINE__, __FUNCTION__, __FILE__);
if (wrapperStructHand.enable)
error("Body keypoint detection must be enabled in order to run hand keypoint detection.",
__LINE__, __FUNCTION__, __FILE__);
}
// Get number GPUs
auto gpuNumber = wrapperStructPose.gpuNumber;
......@@ -545,6 +553,7 @@ namespace op
const auto writeKeypointCleaned = formatAsDirectory(wrapperStructOutput.writeKeypoint);
const auto writeKeypointJsonCleaned = formatAsDirectory(wrapperStructOutput.writeKeypointJson);
const auto writeHeatMapsCleaned = formatAsDirectory(wrapperStructOutput.writeHeatMaps);
const auto modelFolder = formatAsDirectory(wrapperStructPose.modelFolder);
// Common parameters
auto finalOutputSize = wrapperStructPose.outputSize;
......@@ -597,46 +606,50 @@ namespace op
else
wDatumProducer = nullptr;
// Pose estimators
// Pose estimators & renderers
const Point<int>& poseNetOutputSize = poseNetInputSize;
std::vector<std::shared_ptr<PoseExtractor>> poseExtractors;
for (auto gpuId = 0; gpuId < gpuNumber; gpuId++)
poseExtractors.emplace_back(std::make_shared<PoseExtractorCaffe>(
poseNetInputSize, poseNetOutputSize, finalOutputSize, wrapperStructPose.scalesNumber,
wrapperStructPose.poseModel, wrapperStructPose.modelFolder, gpuId + gpuNumberStart,
wrapperStructPose.heatMapTypes, wrapperStructPose.heatMapScale
));
// Pose renderers
std::vector<std::shared_ptr<PoseGpuRenderer>> poseGpuRenderers;
std::shared_ptr<PoseCpuRenderer> poseCpuRenderer;
std::vector<TWorker> cpuRenderers;
if (renderOutputGpu || wrapperStructPose.renderMode == RenderMode::Cpu)
if (wrapperStructPose.enable)
{
// If wrapperStructPose.renderMode != RenderMode::Gpu but renderOutput, then we create an alpha = 0
// pose renderer in order to keep the removing background option
const auto alphaKeypoint = (wrapperStructPose.renderMode != RenderMode::None
? wrapperStructPose.alphaKeypoint : 0.f);
const auto alphaHeatMap = (wrapperStructPose.renderMode != RenderMode::None
? wrapperStructPose.alphaHeatMap : 0.f);
// GPU rendering
if (renderOutputGpu)
// Pose estimators
for (auto gpuId = 0; gpuId < gpuNumber; gpuId++)
poseExtractors.emplace_back(std::make_shared<PoseExtractorCaffe>(
poseNetInputSize, poseNetOutputSize, finalOutputSize, wrapperStructPose.scalesNumber,
wrapperStructPose.poseModel, modelFolder, gpuId + gpuNumberStart,
wrapperStructPose.heatMapTypes, wrapperStructPose.heatMapScale
));
// Pose renderers
if (renderOutputGpu || wrapperStructPose.renderMode == RenderMode::Cpu)
{
for (auto gpuId = 0u; gpuId < poseExtractors.size(); gpuId++)
// If wrapperStructPose.renderMode != RenderMode::Gpu but renderOutput, then we create an alpha = 0
// pose renderer in order to keep the removing background option
const auto alphaKeypoint = (wrapperStructPose.renderMode != RenderMode::None
? wrapperStructPose.alphaKeypoint : 0.f);
const auto alphaHeatMap = (wrapperStructPose.renderMode != RenderMode::None
? wrapperStructPose.alphaHeatMap : 0.f);
// GPU rendering
if (renderOutputGpu)
{
poseGpuRenderers.emplace_back(std::make_shared<PoseGpuRenderer>(
poseNetOutputSize, wrapperStructPose.poseModel, poseExtractors[gpuId],
wrapperStructPose.renderThreshold, wrapperStructPose.blendOriginalFrame, alphaKeypoint,
alphaHeatMap, wrapperStructPose.defaultPartToRender
));
for (auto gpuId = 0u; gpuId < poseExtractors.size(); gpuId++)
{
poseGpuRenderers.emplace_back(std::make_shared<PoseGpuRenderer>(
poseNetOutputSize, wrapperStructPose.poseModel, poseExtractors[gpuId],
wrapperStructPose.renderThreshold, wrapperStructPose.blendOriginalFrame, alphaKeypoint,
alphaHeatMap, wrapperStructPose.defaultPartToRender
));
}
}
// CPU rendering
if (wrapperStructPose.renderMode == RenderMode::Cpu)
{
poseCpuRenderer = std::make_shared<PoseCpuRenderer>(wrapperStructPose.poseModel,
wrapperStructPose.blendOriginalFrame);
cpuRenderers.emplace_back(std::make_shared<WPoseRenderer<TDatumsPtr>>(poseCpuRenderer));
}
}
// CPU rendering
if (wrapperStructPose.renderMode == RenderMode::Cpu)
{
poseCpuRenderer = std::make_shared<PoseCpuRenderer>(wrapperStructPose.poseModel,
wrapperStructPose.blendOriginalFrame);
cpuRenderers.emplace_back(std::make_shared<WPoseRenderer<TDatumsPtr>>(poseCpuRenderer));
}
}
log("", Priority::Low, __LINE__, __FUNCTION__, __FILE__);
......@@ -650,22 +663,46 @@ namespace op
spWCvMatToOpOutput = std::make_shared<WCvMatToOpOutput<TDatumsPtr>>(cvMatToOpOutput);
// Pose extractor(s)
spWPoses.resize(poseExtractors.size());
for (auto i = 0u; i < spWPoses.size(); i++)
spWPoses.at(i) = {std::make_shared<WPoseExtractor<TDatumsPtr>>(poseExtractors.at(i))};
if (wrapperStructPose.enable)
{
spWPoses.resize(poseExtractors.size());
for (auto i = 0u; i < spWPoses.size(); i++)
spWPoses.at(i) = {std::make_shared<WPoseExtractor<TDatumsPtr>>(poseExtractors.at(i))};
}
else
spWPoses.resize(gpuNumber);
// Face extractor(s)
if (wrapperStructFace.enable)
{
const auto faceDetector = std::make_shared<FaceDetector>(wrapperStructPose.poseModel);
// Face detector
// OpenPose face detector
if (wrapperStructPose.enable)
{
const auto faceDetector = std::make_shared<FaceDetector>(wrapperStructPose.poseModel);
for (auto gpu = 0u; gpu < spWPoses.size(); gpu++)
spWPoses.at(gpu).emplace_back(std::make_shared<WFaceDetector<TDatumsPtr>>(faceDetector));
}
// OpenCV face detector
else
{
log("Body keypoint detection is disabled. Hence, using OpenCV face detector (much less accurate"
" but faster).", Priority::High);
for (auto gpu = 0u; gpu < spWPoses.size(); gpu++)
{
// 1 FaceDetectorOpenCV per thread, OpenCV face detector is not thread-safe
const auto faceDetectorOpenCV = std::make_shared<FaceDetectorOpenCV>(modelFolder);
spWPoses.at(gpu).emplace_back(std::make_shared<WFaceDetectorOpenCV<TDatumsPtr>>(faceDetectorOpenCV));
}
}
// Face keypoint extractor
for (auto gpu = 0u; gpu < spWPoses.size(); gpu++)
{
// Face detector
spWPoses.at(gpu).emplace_back(std::make_shared<WFaceDetector<TDatumsPtr>>(faceDetector));
// Face keypoint extractor
const auto netOutputSize = wrapperStructFace.netInputSize;
const auto faceExtractor = std::make_shared<FaceExtractor>(
wrapperStructFace.netInputSize, netOutputSize, wrapperStructPose.modelFolder,
wrapperStructFace.netInputSize, netOutputSize, modelFolder,
gpu + gpuNumberStart, wrapperStructPose.heatMapTypes, wrapperStructPose.heatMapScale
);
spWPoses.at(gpu).emplace_back(std::make_shared<WFaceExtractor<TDatumsPtr>>(faceExtractor));
......@@ -690,7 +727,7 @@ namespace op
// Hand keypoint extractor
const auto netOutputSize = wrapperStructHand.netInputSize;
const auto handExtractor = std::make_shared<HandExtractor>(
wrapperStructHand.netInputSize, netOutputSize, wrapperStructPose.modelFolder,
wrapperStructHand.netInputSize, netOutputSize, modelFolder,
gpu + gpuNumberStart, wrapperStructHand.scalesNumber, wrapperStructHand.scaleRange,
wrapperStructPose.heatMapTypes, wrapperStructPose.heatMapScale
);
......
......@@ -10,11 +10,18 @@ namespace op
{
/**
* WrapperStructPose: Pose estimation and rendering configuration struct.
* WrapperStructPose allows the user to set up the pose estimation and rendering parameters that will be used for the OpenPose Wrapper
* class.
* WrapperStructPose allows the user to set up the pose estimation and rendering parameters that will be used for
* the OpenPose Wrapper class.
*/
struct OP_API WrapperStructPose
{
/**
* Whether to extract body.
* It might be optionally disabled if only face keypoint detection is required. Otherwise, it must be always
* true.
*/
bool enable;
/**
* CCN (Conv Net) input size.
* The greater, the slower and more memory it will be needed, but it will potentially increase accuracy.
......@@ -25,21 +32,23 @@ namespace op
/**
* Output size of the final rendered image.
* It barely affects performance compared to netInputSize.
* The final Datum.poseKeypoints can be scaled with respect to outputSize if `keypointScale` is set to ScaleMode::OutputResolution, even if the
* rendering is disabled.
* The final Datum.poseKeypoints can be scaled with respect to outputSize if `keypointScale` is set to
* ScaleMode::OutputResolution, even if the rendering is disabled.
*/
Point<int> outputSize;
/**
* Final scale of the Array<float> Datum.poseKeypoints and the writen pose data.
* The final Datum.poseKeypoints can be scaled with respect to input size (ScaleMode::InputResolution), net output size (ScaleMode::NetOutputResolution),
* output rendering size (ScaleMode::OutputResolution), from 0 to 1 (ScaleMode::ZeroToOne), and -1 to 1 (ScaleMode::PlusMinusOne).
* The final Datum.poseKeypoints can be scaled with respect to input size (ScaleMode::InputResolution), net
* output size (ScaleMode::NetOutputResolution), output rendering size (ScaleMode::OutputResolution), from 0 to
* 1 (ScaleMode::ZeroToOne), and -1 to 1 (ScaleMode::PlusMinusOne).
*/
ScaleMode keypointScale;
/**
* Number of GPUs processing in parallel.
* The greater, the faster the algorithm will run, but potentially higher lag will appear (which only affects in real-time webcam scenarios).
* The greater, the faster the algorithm will run, but potentially higher lag will appear (which only affects
* in real-time webcam scenarios).
*/
int gpuNumber;
......@@ -52,13 +61,15 @@ namespace op
/**
* Number of scales to process.
* The greater, the slower and more memory it will be needed, but it will potentially increase accuracy.
* This parameter is related with scaleGap, such as the final pose estimation will be an average of the predicted results for each scale.
* This parameter is related with scaleGap, such as the final pose estimation will be an average of the
* predicted results for each scale.
*/
int scalesNumber;
/**
* Gap between successive scales.
* The pose estimation will be estimation for the scales in the range [1, 1-scaleGap*scalesNumber], with a gap of scaleGap.
* The pose estimation will be estimation for the scales in the range [1, 1-scaleGap*scalesNumber], with a gap
* of scaleGap.
*/
float scaleGap;
......@@ -70,8 +81,8 @@ namespace op
/**
* Pose model, it affects the number of body parts to render
* Select PoseModel::COCO_18 for 18 body-part COCO, PoseModel::MPI_15 for 15 body-part MPI, PoseModel::MPI_15_4 for faster version
* of MPI, etc.).
* Select PoseModel::COCO_18 for 18 body-part COCO, PoseModel::MPI_15 for 15 body-part MPI, PoseModel::MPI_15_4
* for faster version of MPI, etc.).
*/
PoseModel poseModel;
......@@ -87,7 +98,8 @@ namespace op
float alphaKeypoint;
/**
* Rendering blending alpha value of the heat maps (body part, background or PAF) with respect to the background image.
* Rendering blending alpha value of the heat maps (body part, background or PAF) with respect to the
* background image.
* Value in the range [0, 1]. 0 will only render the background, 1 will only render the heat map.
*/
float alphaHeatMap;
......@@ -95,8 +107,9 @@ namespace op
/**
* Element to initially render.
* Set 0 for pose, [1, #body parts] for each body part following the order on POSE_BODY_PART_MAPPING on
* `include/pose/poseParameters.hpp`, #body parts+1 for background, #body parts+2 for all body parts overlapped,
* #body parts+3 for all PAFs, and [#body parts+4, #body parts+4+#pair pairs] for each PAF following the order on POSE_BODY_PART_PAIRS.
* `include/pose/poseParameters.hpp`, #body parts+1 for background, #body parts+2 for all body parts
* overlapped, #body parts+3 for all PAFs, and [#body parts+4, #body parts+4+#pair pairs] for each PAF
* following the order on POSE_BODY_PART_PAIRS.
*/
int defaultPartToRender;
......@@ -107,21 +120,24 @@ namespace op
/**
* Whether and which heat maps to save on the Array<float> Datum.heatmaps.
* Use HeatMapType::Parts for body parts, HeatMapType::Background for the background, and HeatMapType::PAFs for the Part Affinity Fields.
* Use HeatMapType::Parts for body parts, HeatMapType::Background for the background, and HeatMapType::PAFs for
* the Part Affinity Fields.
*/
std::vector<HeatMapType> heatMapTypes;
/**
* Scale of the Datum.heatmaps.
* Select ScaleMode::ZeroToOne for range [0,1], ScaleMode::PlusMinusOne for [-1,1] and ScaleMode::UnsignedChar for [0, 255]
* Select ScaleMode::ZeroToOne for range [0,1], ScaleMode::PlusMinusOne for [-1,1] and ScaleMode::UnsignedChar
* for [0, 255].
* If heatMapTypes.empty(), then this parameters makes no effect.
*/
ScaleMode heatMapScale;
/**
* Rendering threshold. Only estimated keypoints whose score confidences are higher than this value will be rendered. Generally, a
* high threshold (> 0.5) will only render very clear body parts; while small thresholds (~0.1) will also output guessed and occluded
* keypoints, but also more false positives (i.e. wrong detections).
* Rendering threshold. Only estimated keypoints whose score confidences are higher than this value will be
* rendered. Generally, a high threshold (> 0.5) will only render very clear body parts; while small thresholds
* (~0.1) will also output guessed and occluded keypoints, but also more false positives (i.e. wrong
* detections).
*/
float renderThreshold;
......@@ -130,7 +146,7 @@ namespace op
* It has the recommended and default values we recommend for each element of the struct.
* Since all the elements of the struct are public, they can also be manually filled.
*/
WrapperStructPose(const Point<int>& netInputSize = Point<int>{656, 368},
WrapperStructPose(const bool enable = true, const Point<int>& netInputSize = Point<int>{656, 368},
const Point<int>& outputSize = Point<int>{1280, 720},
const ScaleMode keypointScale = ScaleMode::InputResolution,
const int gpuNumber = -1, const int gpuNumberStart = 0, const int scalesNumber = 1,
......
此差异已折叠。
set(SOURCES
defineTemplates.cpp
faceDetector.cpp
faceDetectorOpenCV.cpp
faceExtractor.cpp
faceCpuRenderer.cpp
faceGpuRenderer.cpp
......
......@@ -5,4 +5,5 @@ namespace op
DEFINE_TEMPLATE_DATUM(WFaceDetector);
DEFINE_TEMPLATE_DATUM(WFaceExtractor);
DEFINE_TEMPLATE_DATUM(WFaceRenderer);
DEFINE_TEMPLATE_DATUM(WFaceDetectorOpenCV);
}
#include <opencv2/imgproc/imgproc.hpp> // cv::COLOR_BGR2GRAY
#include <openpose/pose/poseParameters.hpp>
#include <openpose/face/faceDetectorOpenCV.hpp>
namespace op
{
FaceDetectorOpenCV::FaceDetectorOpenCV(const std::string& modelFolder)
{
try
{
const std::string faceDetectorModelPath{modelFolder + "face/haarcascade_frontalface_alt.xml"};
if (!mFaceCascade.load(faceDetectorModelPath))
error("Face detector model not found at: " + faceDetectorModelPath, __LINE__, __FUNCTION__, __FILE__);
}
catch (const std::exception& e)
{
error(e.what(), __LINE__, __FUNCTION__, __FILE__);
}
}
std::vector<Rectangle<float>> FaceDetectorOpenCV::detectFaces(const cv::Mat& cvInputData)
{
try
{
// Image to grey and pyrDown
cv::Mat frameGray;
cv::cvtColor(cvInputData, frameGray, cv::COLOR_BGR2GRAY);
auto multiplier = 1;
while (frameGray.cols * frameGray.rows > 640*360)
{
cv::pyrDown(frameGray, frameGray);
multiplier *= 2;
}
// Face detection - Example from:
// http://docs.opencv.org/2.4/doc/tutorials/objdetect/cascade_classifier/cascade_classifier.html
std::vector<cv::Rect> detectedFaces;
mFaceCascade.detectMultiScale(frameGray, detectedFaces, 1.2, 3, 0|CV_HAAR_SCALE_IMAGE);
// Rescale rectangles
std::vector<Rectangle<float>> faceRectangles(detectedFaces.size());
for(auto i = 0u; i < detectedFaces.size(); i++)
{
// Enlarge detected rectangle by 1.5x, so that it covers the whole face
faceRectangles.at(i).x = detectedFaces.at(i).x - 0.25*detectedFaces.at(i).width;
faceRectangles.at(i).y = detectedFaces.at(i).y - 0.25*detectedFaces.at(i).height;
faceRectangles.at(i).width = 1.5*detectedFaces.at(i).width;
faceRectangles.at(i).height = 1.5*detectedFaces.at(i).height;
faceRectangles.at(i) *= multiplier;
}
return faceRectangles;
}
catch (const std::exception& e)
{
error(e.what(), __LINE__, __FUNCTION__, __FILE__);
return {};
}
}
}
......@@ -69,7 +69,7 @@ namespace op
// Warnings
if (!mHeatMapTypes.empty())
log("Note only keypoint heatmaps are available with face heatmaps (no background nor PAFs).",
Priority::Low, __LINE__, __FUNCTION__, __FILE__);
Priority::High);
}
catch (const std::exception& e)
{
......
......@@ -156,8 +156,8 @@ namespace op
__LINE__, __FUNCTION__, __FILE__);
// Warnings
if (!mHeatMapTypes.empty())
log("Note only keypoint heatmaps are available with face heatmaps (no background nor PAFs).",
Priority::Low, __LINE__, __FUNCTION__, __FILE__);
log("Note only keypoint heatmaps are available with hand heatmaps (no background nor PAFs).",
Priority::High);
}
catch (const std::exception& e)
{
......
......@@ -2,15 +2,17 @@
namespace op
{
WrapperStructPose::WrapperStructPose(const Point<int>& netInputSize_, const Point<int>& outputSize_,
const ScaleMode keypointScale_, const int gpuNumber_,
const int gpuNumberStart_, const int scalesNumber_, const float scaleGap_,
WrapperStructPose::WrapperStructPose(const bool enable_, const Point<int>& netInputSize_,
const Point<int>& outputSize_, const ScaleMode keypointScale_,
const int gpuNumber_, const int gpuNumberStart_,
const int scalesNumber_, const float scaleGap_,
const RenderMode renderMode_, const PoseModel poseModel_,
const bool blendOriginalFrame_, const float alphaKeypoint_,
const float alphaHeatMap_, const int defaultPartToRender_,
const std::string& modelFolder_,
const std::vector<HeatMapType>& heatMapTypes_,
const ScaleMode heatMapScale_, const float renderThreshold_) :
enable{enable_},
netInputSize{netInputSize_},
outputSize{outputSize_},
keypointScale{keypointScale_},
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册