1.**Add** the **system configuration (all of it!), command and output** if you have some kind of error or performance question.
2.**No duplicated** posts.
3.**No** posts about **questions already answered / clearly explained in** the **documentation** (e.g. **no more low-speed nor out-of-memory questions**).
4. Set a **proper issue title**: add the Ubuntu/Windows word and be specific (e.g. do not simple call it: `Compile error`).
5.**No** questions about **training**. OpenPose only implements testing.
6. Only English comments.
Issues/comments that do not follow this will be **ignored or removed** with no further clarification.
1.**No** questions about **training**. OpenPose only implements testing.
2.**No** questions about **Caffe installation errors/issues**. Check [Caffe](http://caffe.berkeleyvision.org) documentation and help for those errors.
3.**Fill** the **Your System Configuration section (all of it!)** if you have some kind of error or performance question.
4.**No duplicated** posts.
5.**No** posts about **questions already answered / clearly explained in** the **documentation** (e.g. **no more low-speed nor out-of-memory questions**).
6. Set a **proper issue title**: add the Ubuntu/Windows word and be specific (e.g. do not simple call it: `Compile error`).
7. Only English comments.
Issues/comments which do not follow these rules will be **ignored or removed** with no further clarification.
### Issue summary
### Issue Summary
### Executed command (if any)
### Executed Command (if any)
Note: add `--logging_level 0` to get higher debug information.
### OpenPose output (if any)
### OpenPose Output (if any)
### Type of issue
### Type of Issue
You might select multiple topics, delete the rest:
- Compilation/installation error
- Execution error
...
...
@@ -33,13 +34,12 @@ You might select multiple topics, delete the rest:
### Your system configuration
**Installation mode**: CMake or sh script or manual Makefile installation.
### Your System Configuration
**Operating system** (`lsb_release -a` in Ubuntu):
**Installation mode**: CMake, sh script, or manual Makefile installation (Ubuntu); VS2015, VS2017, CMake, ... (Windows)
**CUDA version** (`cat /usr/local/cuda/version.txt` in most cases):
**cuDNN version**:
**GPU model** (`nvidia-smi` in Ubuntu):
**Caffe version**: Default from OpenPose or custom version.
**OpenCV version**: installed with `apt-get install libopencv-dev` (Ubuntu) or default from OpenPose (Windows) or OpenCV 2.X or OpenCV 3.X. Especify **full version** (e.g. 3.1 or 2.4.9)
Generation mode (only for Ubuntu): Makefile + Makefile.config (default, Ubuntu) or CMake (Ubuntu, Windows) or Visual Studio (Windows).
@@ -107,13 +107,13 @@ Note: you should not need to modify the OpenPose source code nor examples. In th
### OpenPose Library
Your case if you want to change internal functions and/or extend its functionality. First, take a look at the [Demo](#demo) and [OpenPose Wrapper](#openpose-wrapper). Second, read the 2 following subsections: OpenPose Overview and Extending Functionality.
Your case if you want to change internal functions and/or extend its functionality.
1.OpenPose Overview: Learn the basics about the library source code in [doc/library_overview.md](doc/library_overview.md).
2. Extending Functionality: Learn how to extend the library in [doc/library_extend_functionality.md](doc/library_extend_functionality.md).
3. Adding An Extra Module: Learn how to add an extra module in [doc/library_add_new_module.md](doc/library_add_new_module.md).
1.Take a look at the [Demo](#demo) and [OpenPose Wrapper](#openpose-wrapper).
2. OpenPose Overview: Learn the basics about the library source code in [doc/library_overview.md](doc/library_overview.md).
3. Extending Functionality: Learn how to extend the library in [doc/library_extend_functionality.md](doc/library_extend_functionality.md).
4. Adding An Extra Module: Learn how to add an extra module in [doc/library_add_new_module.md](doc/library_add_new_module.md).
5. See the Doxygen documentation on [http://cmu-perceptual-computing-lab.github.io/openpose/html/index.html](http://cmu-perceptual-computing-lab.github.io/openpose/html/index.html) or build it from the source code.
-**Nvidia Jetson TX2**, installation instructions in [doc/installation_jetson_tx2](./installation_jetson_tx2).
-**Nvidia Jetson TX2**, installation instructions in [doc/installation_jetson_tx2.md](./installation_jetson_tx2.md).
- OpenPose has also been used on **Windows 7**, **Mac**, **CentOS**, and **Nvidia Jetson (TK1 and TX1)** embedded systems. However, we do not officially support them at the moment.
...
...
@@ -133,7 +133,7 @@ You just need to remove the OpenPose folder, by default called `openpose/`. E.g.
### Installation - Library
1. Install the pre-requisites:
1. Microsoft Visual Studio (VS) 2015 Enterprise Update 3. VS Enterprise Update 1 and VS 2017 will give some compiler errors, while VS 2015 Community has not been tested.
1. Microsoft Visual Studio (VS) 2015 Enterprise Update 3. If Visual Studio 2017 Community is desired, we do not support it, but it might be compiled by firstly [enabling CUDA 8.0](https://stackoverflow.com/questions/43745099/using-cuda-with-visual-studio-2017?answertab=active#tab-top). VS Enterprise Update 1 will give some compiler errors and VS 2015 Community has not been tested.
2.[CUDA 8](https://developer.nvidia.com/cuda-downloads): Install it on the default location, `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0`. Otherwise, modify the Visual Studio project solution accordingly. Install CUDA 8.0 after Visual Studio 2015 is installed to assure that the CUDA installation will generate all necessary files for VS. If CUDA was already installed, re-install it after installing VS!
3.[cuDNN 5.1](https://developer.nvidia.com/cudnn): Once you have downloaded it, just unzip it and copy (merge) the contents on the CUDA folder, `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0`.
2. Download the OpenPose dependencies and models (body, face and hand models) by double-clicking on `{openpose_path}\windows\download_3rdparty_and_models.bat`. Alternatively, you might prefer to download them manually:
...
...
@@ -214,8 +214,8 @@ We only modified some Caffe compilation flags and minor details. You can use you
## Compiling without cuDNN
The [cuDNN](https://developer.nvidia.com/cudnn) library is not mandatory, but required for full keypoint detection accuracy. In case your graphics card is not compatible with cuDNN, you can disable it by:
- Ubuntu: Modifying the `Makefile.config` files in both the OpenPose and `3rdparty/caffe` folders, disabling `USE_CUDNN`.
- Windows: Compiling Caffe by your own with without cuDNN support and removing the `USE_CUDNN` define from the OpenPose project solution in Visual Studio.
- Ubuntu: Disable `USE_CUDNN` in the `Makefile.config` file in `3rdparty/caffe`, and recompiling Caffe.
- Windows: Compiling Caffe by your own with without cuDNN support and replacing the [3rdparty/windows/caffe](../3rdparty/windows/caffe)) folder by your own implementation.
Then, you would have to reduce the `--net_resolution` flag to fit the model into the GPU memory. You can try values like "640x320", "320x240", "320x160", or "160x80" to see your GPU memory capabilities. After finding the maximum approximate resolution that your GPU can handle without throwing an out-of-memory error, adjust the `net_resolution` ratio to your image or video to be processed (see the `--net_resolution` explanation from [doc/demo_overview.md](./demo_overview.md)).
6. Get the last OpenGL Glut library version for the rendering:
- Download the latest `MSVC Package` from http://www.transmissionzero.co.uk/software/freeglut-devel/
- Copy `{freeglutParentDirectory}\freeglut\bin\x64\` as `{OpenPoseDirectory}\3rdparty\windows\freeglut\bin\bin\`.
- Copy `{freeglutParentDirectory}\freeglut\bin\x64\` as `{OpenPoseDirectory}\3rdparty\windows\freeglut\bin\`.
- Copy `{freeglutParentDirectory}\freeglut\include\` as `{OpenPoseDirectory}\3rdparty\windows\freeglut\include\`.
- Copy `{freeglutParentDirectory}\freeglut\lib\x64\` as `{OpenPoseDirectory}\3rdparty\windows\freeglut\lib\`.
...
...
@@ -78,7 +78,6 @@ We did not create an Ubuntu version. We did an very first version for Ubuntu 16
8. Get the required files from `{OpenPose path}/examples_beta/openpose3d/`. Check the Windows VS solution for more details.
9. Create a proper Makefile or CMake file to run it. The following code is part of an old QMake (Qt) file generated for the old version, you can ideally get all the flags and includes from it:
3. CvMatToOutput and Renderers allow to keep input resolution as output for images (core module).
3. New standalone face keypoint detector based on OpenCV face detector: much faster if body keypoint detection is not required but much less accurate.
4. Face and hand keypoint detectors now can return each keypoint heatmap.
5. COCO JSON file outputs 0 as score for non-detected keypoints.
6. Added example for OpenPose for user asynchronous output and cleaned all `tutorial_wrapper/` examples.
7. Added `-1` option for `net_resolution` in order to auto-select the best possible aspect ratio given the user input.
5. The flag `USE_CUDNN` is no longer required; `USE_CAFFE` and `USE_CUDA` (replacing the old `CPU_ONLY`) are no longer required to use the library, only to build it. In addition, Caffe and its dependencies have been removed from the OpenPose header files. Only OpenCV include and lib folders are required when building a project using OpenPose.
6. OpenPose successfully compiles if the flags `USE_CAFFE` and/or `USE_CUDA` are not enabled, although it will give an error saying they are required.
7. COCO JSON file outputs 0 as score for non-detected keypoints.
8. Added example for OpenPose for user asynchronous output and cleaned all `tutorial_wrapper/` examples.
9. Added `-1` option for `net_resolution` in order to auto-select the best possible aspect ratio given the user input.
2. Functions or parameters renamed:
1. OpenPose able to change its size and initial size:
1. Flag `resolution` renamed as `output_resolution`.
2. FrameDisplayer, GuiInfoAdder and Gui constructors arguments modified (gui module).
* Datum: The OpenPose Basic Piece of Information Between Threads
* Datum is one the main OpenPose classes/structs. The workers and threads share by default a std::shared_ptr<std::vector<Datum>>. It contains
* all the parameters that the different workers and threads need to exchange.
* Datum is one the main OpenPose classes/structs. The workers and threads share by default a
* std::shared_ptr<std::vector<Datum>>. It contains all the parameters that the different workers and threads need
* to exchange.
*/
structOP_APIDatum
{
// -------------------------------------------------- ID parameters -------------------------------------------------- //
// ---------------------------------------- ID parameters ---------------------------------------- //
unsignedlonglongid;/**< Datum ID. Internally used to sort the Datums if multi-threading is used. */
std::stringname;/**< Name used when saving the data to disk (e.g. `write_images` or `write_keypoint` flags in the demo). */
/**
* Name used when saving the data to disk (e.g. `write_images` or `write_keypoint` flags in the demo).
*/
std::stringname;
// -------------------------------------------------- Input image and rendered version parameters -------------------------------------------------- //
// ------------------------------ Input image and rendered version parameters ------------------------------ //
/**
* Original image to be processed in cv::Mat uchar format.
* Size: (input_width x input_height) x 3 channels
...
...
@@ -27,8 +31,10 @@ namespace op
/**
* Original image to be processed in Array<float> format.
* It has been resized to the net input resolution, as well as reformatted Array<float> format to be compatible with the net.
* In case of >1 scales, then each scale is right- and bottom-padded to fill the greatest resolution. The scales are sorted from bigger to smaller.
* It has been resized to the net input resolution, as well as reformatted Array<float> format to be compatible
* with the net.
* In case of >1 scales, then each scale is right- and bottom-padded to fill the greatest resolution. The
* scales are sorted from bigger to smaller.
* Size: #scales x 3 x input_net_height x input_net_width
*/
Array<float>inputNetData;
...
...
@@ -49,7 +55,7 @@ namespace op
*/
cv::MatcvOutputData;
// -------------------------------------------------- Resulting Array<float> data parameters -------------------------------------------------- //
// ------------------------------ Resulting Array<float> data parameters ------------------------------ //
/**
* Body pose (x,y,score) locations for each person in the image.
* It has been resized to the desired output resolution (e.g. `resolution` flag in the demo).
...
...
@@ -60,10 +66,14 @@ namespace op
/**
* Body pose heatmaps (body parts, background and/or PAFs) for the whole image.
* This parameters is by default empty and disabled for performance. Each group (body parts, background and PAFs) can be individually enabled.
* #heatmaps = #body parts (if enabled) + 1 (if background enabled) + 2 x #PAFs (if enabled). Each PAF has 2 consecutive channels, one for x- and one for y-coordinates.
* Order heatmaps: body parts + background (as appears in POSE_BODY_PART_MAPPING) + (x,y) channel of each PAF (sorted as appears in POSE_BODY_PART_PAIRS). See `pose/poseParameters.hpp`.
* The user can choose the heatmaps normalization: ranges [0, 1], [-1, 1] or [0, 255]. Check the `heatmaps_scale` flag in the examples/tutorial_wrapper/ for more details.
* This parameters is by default empty and disabled for performance. Each group (body parts, background and
* PAFs) can be individually enabled.
* #heatmaps = #body parts (if enabled) + 1 (if background enabled) + 2 x #PAFs (if enabled). Each PAF has 2
* consecutive channels, one for x- and one for y-coordinates.
* Order heatmaps: body parts + background (as appears in POSE_BODY_PART_MAPPING) + (x,y) channel of each PAF
* (sorted as appears in POSE_BODY_PART_PAIRS). See `pose/poseParameters.hpp`.
* The user can choose the heatmaps normalization: ranges [0, 1], [-1, 1] or [0, 255]. Check the
* `heatmaps_scale` flag in the examples/tutorial_wrapper/ for more details.
* Size: #heatmaps x output_net_height x output_net_width
*/
Array<float>poseHeatMaps;
...
...
@@ -111,29 +121,55 @@ namespace op
*/
std::array<Array<float>,2>handHeatMaps;
// -------------------------------------------------- Other parameters -------------------------------------------------- //
floatscaleInputToOutput;/**< Scale ratio between the input Datum::cvInputData and the output Datum::cvOutputData. */
// ---------------------------------------- Other parameters ---------------------------------------- //
/**
* Scale ratio between the input Datum::cvInputData and the net input size.
*/
std::vector<double>scaleInputToNetInputs;
/**
* Size(s) (width x height) of the image(s) fed to the pose deep net.
* The size of the std::vector corresponds to the number of scales.
*/
std::vector<Point<int>>netInputSizes;
/**
* Scale ratio between the input Datum::cvInputData and the output Datum::cvOutputData.
*/
doublescaleInputToOutput;
floatscaleNetToOutput;/**< Scale ratio between the net output and the final output Datum::cvOutputData. */
/**
* Size (width x height) of the image returned by the deep net.
*/
Point<int>netOutputSize;
std::vector<float>scaleRatios;/**< Scale ratios between each scale (e.g. flag `scale_number`). Used to resize the different scales. */
/**
* Scale ratio between the net output and the final output Datum::cvOutputData.
*/
doublescaleNetToOutput;
std::pair<int,std::string>elementRendered;/**< Pair with the element key id POSE_BODY_PART_MAPPING on `pose/poseParameters.hpp` and its mapped value (e.g. 1 and "Neck"). */
/**
* Pair with the element key id POSE_BODY_PART_MAPPING on `pose/poseParameters.hpp` and its mapped value (e.g.
* It simply initializes the struct, id is temporary set to 0 and each other variable is assigned to its default value.
* It simply initializes the struct, id is temporary set to 0 and each other variable is assigned to its
* default value.
*/
explicitDatum();
/**
* Copy constructor.
* It performs `fast copy`: For performance purpose, copying a Datum or Array<T> or cv::Mat just copies the reference, it still shares the same internal data.
* It performs `fast copy`: For performance purpose, copying a Datum or Array<T> or cv::Mat just copies the
* reference, it still shares the same internal data.
* Modifying the copied element will modify the original one.
* Use clone() for a slower but real copy, similarly to cv::Mat and Array<T>.
* @param datum Datum to be copied.
...
...
@@ -172,7 +208,8 @@ namespace op
/**
* Clone function.
* Similar to cv::Mat::clone and Array<T>::clone.
* It performs a real but slow copy of the data, i.e., even if the copied element is modified, the original one is not.
* It performs a real but slow copy of the data, i.e., even if the copied element is modified, the original
// It mostly follows the Caffe::layer implementation, so Caffe users can easily use it. However, in order to keep the compatibility with any generic Caffe version,
// we keep this 'layer' inside our library rather than in the Caffe code.
// It mostly follows the Caffe::layer implementation, so Caffe users can easily use it. However, in order to keep
// the compatibility with any generic Caffe version, we keep this 'layer' inside our library rather than in the
// It mostly follows the Caffe::layer implementation, so Caffe users can easily use it. However, in order to keep the compatibility with any generic Caffe version,
// we keep this 'layer' inside our library rather than in the Caffe code.
// It mostly follows the Caffe::layer implementation, so Caffe users can easily use it. However, in order to keep
// the compatibility with any generic Caffe version, we keep this 'layer' inside our library rather than in the
// It mostly follows the Caffe::layer implementation, so Caffe users can easily use it. However, in order to keep the compatibility with any generic Caffe version,
// we keep this 'layer' inside our library rather than in the Caffe code.
// It mostly follows the Caffe::layer implementation, so Caffe users can easily use it. However, in order to keep
// the compatibility with any generic Caffe version, we keep this 'layer' inside our library rather than in the
