IP cammera support added

README.md

@@ -11,7 +11,7 @@ OpenPose is a **library for real-time multi-person keypoint detection and multi-
 
 
 ## Latest News
-- Sep 2017: **CMake** installer!
+- Sep 2017: **CMake** installer and **IP camera** support!
 - Jul 2017: [**Windows portable demo**](doc/installation.md#installation---demo)!
 - Jul 2017: **Hands** released!
 - Jun 2017: **Face** released!
@@ -62,7 +62,7 @@ OpenPose is a **library for real-time multi-person keypoint detection and multi-
 ## Introduction
 OpenPose represents the **first real-time system to jointly detect human body, hand and facial keypoints (in total 130 keypoints) on single images**. In addition, the system computational performance on body keypoint estimation is invariant to the number of detected people in the image. It uses Caffe, but it could easily be ported to other frameworks (Tensorflow, Torch, etc.). If you implement any of those, feel free to make a pull request!
 
-OpenPose is authored by [Gines Hidalgo](http://gines-hidalgo.site123.me/), [Zhe Cao](http://www.andrew.cmu.edu/user/zhecao), [Tomas Simon](http://www.cs.cmu.edu/~tsimon/), [Shih-En Wei](https://scholar.google.com/citations?user=sFQD3k4AAAAJ&hl=en), [Hanbyul Joo](http://www.cs.cmu.edu/~hanbyulj/) and [Yaser Sheikh](http://www.cs.cmu.edu/~yaser/). In addition, it is being maintained by [Gines Hidalgo](http://gines-hidalgo.site123.me/) and [Bikramjot Hanzra](https://www.linkedin.com/in/bikz05).
+OpenPose is authored by [Gines Hidalgo](http://gineshidalgo.com/), [Zhe Cao](http://www.andrew.cmu.edu/user/zhecao), [Tomas Simon](http://www.cs.cmu.edu/~tsimon/), [Shih-En Wei](https://scholar.google.com/citations?user=sFQD3k4AAAAJ&hl=en), [Hanbyul Joo](http://www.cs.cmu.edu/~hanbyulj/) and [Yaser Sheikh](http://www.cs.cmu.edu/~yaser/). In addition, it is being maintained by [Gines Hidalgo](http://gineshidalgo.com/) and [Bikramjot Hanzra](https://www.linkedin.com/in/bikz05).
 
 It is freely available for free non-commercial use, and may be redistributed under these conditions. Please, see the [license](LICENSE) for further details. [Interested in a commercial license? Check this link](https://flintbox.com/public/project/47343/). For commercial queries, contact [Yaser Sheikh](http://www.cs.cmu.edu/~yaser/).
 

doc/demo_overview.md

@@ -99,6 +99,7 @@ We enumerate some of the most important flags, check the `Flags Detailed Descrip
 - `--video input.mp4`: Read video.
 - `--camera 3`: Read webcam number 3.
 - `--image_dir path_to_images/`: Run on a folder with images.
+- `--ip_camera http://iris.not.iac.es/axis-cgi/mjpg/video.cgi?resolution=320x240?x.mjpeg`: Run on a streamed IP camera. See examples public IP cameras [here](http://www.webcamxp.com/publicipcams.aspx).
 - `--write_video path.avi`: Save processed images as video.
 - `--write_images folder_path`: Save processed images on a folder.
 - `--write_keypoint path/`: Output JSON, XML or YML files with the people pose data on a folder.
@@ -126,6 +127,7 @@ Each flag is divided into flag name, default value, and description.
 - DEFINE_double(camera_fps,               30.0,           "Frame rate for the webcam (only used when saving video from webcam). Set this value to the minimum value between the OpenPose displayed speed and the webcam real frame rate.");
 - DEFINE_string(video,                    "",             "Use a video file instead of the camera. Use `examples/media/video.avi` for our default example video.");
 - DEFINE_string(image_dir,                "",             "Process a directory of images. Use `examples/media/` for our default example folder with 20 images. Read all standard formats (jpg, png, bmp, etc.).");
+- DEFINE_string(ip_camera,                "",             "String with the IP camera URL. It supports protocols like RTSP and HTTP.");
 - DEFINE_uint64(frame_first,              0,              "Start on desired frame number. Indexes are 0-based, i.e. the first frame has index 0.");
 - DEFINE_uint64(frame_last,               -1,             "Finish on desired frame number. Select -1 to disable. Indexes are 0-based, e.g. if set to 10, it will process 11 frames (0-10).");
 - DEFINE_bool(frame_flip,                 false,          "Flip/mirror each frame (e.g. for real time webcam demonstrations).");

doc/release_notes.md

@@ -115,13 +115,14 @@ OpenPose Library - Release Notes
 
 ## Current version (future OpenPose 1.2.0)
 1. Main improvements:
-    1. COCO JSON file outputs 0 as score for non-detected keypoints.
-    2. Added example for OpenPose for user asynchronous output and cleaned all `tutorial_wrapper/` examples.
-    3. Added `-1` option for `net_resolution` in order to auto-select the best possible aspect ratio given the user input.
-    4. Output images can have the input size, OpenPose able to change its size for each image and not required fixed size anymore.
+    1. Added IP camera support.
+    2. Output images can have the input size, OpenPose able to change its size for each image and not required fixed size anymore.
         1. FrameDisplayer accepts variable size images by rescaling every time a frame with bigger width or height is displayed (gui module).
         2. OpOutputToCvMat & GuiInfoAdder does not require to know the output size at construction time, deduced from each image.
         3. CvMatToOutput and Renderers allow to keep input resolution as output for images (core module).
+    3. COCO JSON file outputs 0 as score for non-detected keypoints.
+    4. Added example for OpenPose for user asynchronous output and cleaned all `tutorial_wrapper/` examples.
+    5. Added `-1` option for `net_resolution` in order to auto-select the best possible aspect ratio given the user input.
 2. Functions or parameters renamed:
     1. OpenPose able to change its size and initial size:
         1. Flag `resolution` renamed as `output_resolution`.

examples/openpose/openpose.cpp

@@ -41,6 +41,7 @@ DEFINE_string(video,                    "",             "Use a video file instea
                                                         " example video.");
 DEFINE_string(image_dir,                "",             "Process a directory of images. Use `examples/media/` for our default example folder with 20"
                                                         " images. Read all standard formats (jpg, png, bmp, etc.).");
+DEFINE_string(ip_camera,                "",             "String with the IP camera URL. It supports protocols like RTSP and HTTP.");
 DEFINE_uint64(frame_first,              0,              "Start on desired frame number. Indexes are 0-based, i.e. the first frame has index 0.");
 DEFINE_uint64(frame_last,               -1,             "Finish on desired frame number. Select -1 to disable. Indexes are 0-based, e.g. if set to"
                                                         " 10, it will process 11 frames (0-10).");
@@ -180,7 +181,8 @@ int openPoseDemo()
     // handNetInputSize
     const auto handNetInputSize = op::flagsToPoint(FLAGS_hand_net_resolution, "368x368 (multiples of 16)");
     // producerType
-    const auto producerSharedPtr = op::flagsToProducer(FLAGS_image_dir, FLAGS_video, FLAGS_camera, FLAGS_camera_resolution, FLAGS_camera_fps);
+    const auto producerSharedPtr = op::flagsToProducer(FLAGS_image_dir, FLAGS_video, FLAGS_ip_camera, FLAGS_camera,
+                                                       FLAGS_camera_resolution, FLAGS_camera_fps);
     // poseModel
     const auto poseModel = op::flagsToPoseModel(FLAGS_model_pose);
     // keypointScale

examples/openpose/openposeBasicDemo.cpp

@@ -22,6 +22,7 @@ DEFINE_string(video,                    "",             "Use a video file instea
                                                         " example video.");
 DEFINE_string(image_dir,                "",             "Process a directory of images. Use `examples/media/` for our default example folder with 20"
                                                         " images. Read all standard formats (jpg, png, bmp, etc.).");
+DEFINE_string(ip_camera,                "",             "String with the IP camera URL. It supports protocols like RTSP and HTTP.");
 // Display
 DEFINE_bool(no_gui_verbose,             false,          "Do not write text on output images on GUI (e.g. number of current frame and people). It"
                                                         " does not affect the pose rendering.");
@@ -34,7 +35,8 @@ int openPoseDemo()
     const auto timerBegin = std::chrono::high_resolution_clock::now();
 
     // Applying user defined configuration - Google flags to program variables
-    const auto producerSharedPtr = op::flagsToProducer(FLAGS_image_dir, FLAGS_video, FLAGS_camera, FLAGS_camera_resolution, FLAGS_camera_fps);
+    const auto producerSharedPtr = op::flagsToProducer(FLAGS_image_dir, FLAGS_video, FLAGS_ip_camera, FLAGS_camera,
+                                                       FLAGS_camera_resolution, FLAGS_camera_fps);
     op::log("", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__);
 
     // OpenPose wrapper

examples/tests/handFromJsonTest.cpp

@@ -42,7 +42,7 @@ int handFromJsonTest()
     // handNetInputSize
     const auto handNetInputSize = op::flagsToPoint(FLAGS_hand_net_resolution, "368x368 (multiples of 16)");
     // producerType
-    const auto producerSharedPtr = op::flagsToProducer(FLAGS_image_dir, "", 0);
+    const auto producerSharedPtr = op::flagsToProducer(FLAGS_image_dir, "", "", 0);
     op::log("", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__);
 
     // OpenPose wrapper

examples/tutorial_thread/1_openpose_read_and_display.cpp

@@ -34,6 +34,7 @@ DEFINE_string(video,                    "",             "Use a video file instea
                                                         " example video.");
 DEFINE_string(image_dir,                "",             "Process a directory of images. Use `examples/media/` for our default example folder with 20"
                                                         " images. Read all standard formats (jpg, png, bmp, etc.).");
+DEFINE_string(ip_camera,                "",             "String with the IP camera URL. It supports protocols like RTSP and HTTP.");
 DEFINE_bool(process_real_time,          false,          "Enable to keep the original source frame rate (e.g. for video). If the processing time is"
                                                         " too long, it will skip frames. If it is too fast, it will slow it down.");
 // OpenPose
@@ -55,7 +56,8 @@ int openPoseTutorialThread1()
     // outputSize
     const auto outputSize = op::flagsToPoint(FLAGS_output_resolution, "-1x-1");
     // producerType
-    const auto producerSharedPtr = op::flagsToProducer(FLAGS_image_dir, FLAGS_video, FLAGS_camera, FLAGS_camera_resolution, FLAGS_camera_fps);
+    const auto producerSharedPtr = op::flagsToProducer(FLAGS_image_dir, FLAGS_video, FLAGS_ip_camera, FLAGS_camera,
+                                                       FLAGS_camera_resolution, FLAGS_camera_fps);
     const auto displayProducerFpsMode = (FLAGS_process_real_time ? op::ProducerFpsMode::OriginalFps : op::ProducerFpsMode::RetrievalFps);
     producerSharedPtr->setProducerFpsMode(displayProducerFpsMode);
     op::log("", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__);

examples/tutorial_thread/2_user_processing_function.cpp

@@ -35,6 +35,7 @@ DEFINE_string(video,                    "",             "Use a video file instea
                                                         " example video.");
 DEFINE_string(image_dir,                "",             "Process a directory of images. Use `examples/media/` for our default example folder with 20"
                                                         " images. Read all standard formats (jpg, png, bmp, etc.).");
+DEFINE_string(ip_camera,                "",             "String with the IP camera URL. It supports protocols like RTSP and HTTP.");
 DEFINE_bool(process_real_time,          false,          "Enable to keep the original source frame rate (e.g. for video). If the processing time is"
                                                         " too long, it will skip frames. If it is too fast, it will slow it down.");
 // OpenPose
@@ -89,7 +90,8 @@ int openPoseTutorialThread2()
     // outputSize
     const auto outputSize = op::flagsToPoint(FLAGS_output_resolution, "-1x-1");
     // producerType
-    const auto producerSharedPtr = op::flagsToProducer(FLAGS_image_dir, FLAGS_video, FLAGS_camera, FLAGS_camera_resolution, FLAGS_camera_fps);
+    const auto producerSharedPtr = op::flagsToProducer(FLAGS_image_dir, FLAGS_video, FLAGS_ip_camera, FLAGS_camera,
+                                                       FLAGS_camera_resolution, FLAGS_camera_fps);
     const auto displayProducerFpsMode = (FLAGS_process_real_time ? op::ProducerFpsMode::OriginalFps : op::ProducerFpsMode::RetrievalFps);
     producerSharedPtr->setProducerFpsMode(displayProducerFpsMode);
     op::log("", op::Priority::Low, __LINE__, __FUNCTION__, __FILE__);

examples/tutorial_wrapper/1_user_asynchronous_output.cpp

@@ -41,6 +41,7 @@ DEFINE_string(video,                    "",             "Use a video file instea
                                                         " example video.");
 DEFINE_string(image_dir,                "",             "Process a directory of images. Use `examples/media/` for our default example folder with 20"
                                                         " images. Read all standard formats (jpg, png, bmp, etc.).");
+DEFINE_string(ip_camera,                "",             "String with the IP camera URL. It supports protocols like RTSP and HTTP.");
 DEFINE_uint64(frame_first,              0,              "Start on desired frame number. Indexes are 0-based, i.e. the first frame has index 0.");
 DEFINE_uint64(frame_last,               -1,             "Finish on desired frame number. Select -1 to disable. Indexes are 0-based, e.g. if set to"
                                                         " 10, it will process 11 frames (0-10).");
@@ -247,7 +248,8 @@ int openPoseTutorialWrapper3()
     // handNetInputSize
     const auto handNetInputSize = op::flagsToPoint(FLAGS_hand_net_resolution, "368x368 (multiples of 16)");
     // producerType
-    const auto producerSharedPtr = op::flagsToProducer(FLAGS_image_dir, FLAGS_video, FLAGS_camera, FLAGS_camera_resolution, FLAGS_camera_fps);
+    const auto producerSharedPtr = op::flagsToProducer(FLAGS_image_dir, FLAGS_video, FLAGS_ip_camera, FLAGS_camera,
+                                                       FLAGS_camera_resolution, FLAGS_camera_fps);
     // poseModel
     const auto poseModel = op::flagsToPoseModel(FLAGS_model_pose);
     // keypointScale

include/openpose/producer/enumClasses.hpp

@@ -25,6 +25,7 @@ namespace op
     enum class ProducerType : unsigned char
     {
         ImageDirectory,     /**< An image directory reader. It is able to read images on a folder with a interface similar to the OpenCV cv::VideoCapture. */
+        IPCamera,           /**< An IP camera frames extractor, extending the functionality of cv::VideoCapture. */
         Video,              /**< A video frames extractor, extending the functionality of cv::VideoCapture. */
         Webcam,             /**< A webcam frames extractor, extending the functionality of cv::VideoCapture. */
         None,               /**< No type defined. Default state when no specific Producer has been picked yet. */

include/openpose/producer/headers.hpp

@@ -4,8 +4,9 @@
 // producer module
 #include <openpose/producer/datumProducer.hpp>
 #include <openpose/producer/enumClasses.hpp>
-#include <openpose/producer/producer.hpp>
 #include <openpose/producer/imageDirectoryReader.hpp>
+#include <openpose/producer/ipCameraReader.hpp>
+#include <openpose/producer/producer.hpp>
 #include <openpose/producer/videoCaptureReader.hpp>
 #include <openpose/producer/videoReader.hpp>
 #include <openpose/producer/webcamReader.hpp>

include/openpose/producer/ipCameraReader.hpp

+#ifndef OPENPOSE_PRODUCER_IP_CAMERA_READER_HPP
+#define OPENPOSE_PRODUCER_IP_CAMERA_READER_HPP
+
+#include <openpose/core/common.hpp>
+#include <openpose/producer/videoCaptureReader.hpp>
+
+namespace op
+{
+    /**
+     * IpCameraReader is a wrapper of the cv::VideoCapture class for IP camera streaming.
+     */
+    class OP_API IpCameraReader : public VideoCaptureReader
+    {
+    public:
+        /**
+         * Constructor of IpCameraReader. It opens the IP camera as a wrapper of cv::VideoCapture.
+         * @param cameraPath const std::string parameter with the full camera IP link.
+         */
+        explicit IpCameraReader(const std::string& cameraPath);
+
+        std::string getFrameName();
+
+        inline double get(const int capProperty)
+        {
+            return VideoCaptureReader::get(capProperty);
+        }
+
+        inline void set(const int capProperty, const double value)
+        {
+            VideoCaptureReader::set(capProperty, value);
+        }
+
+    private:
+        const std::string mPathName;
+
+        cv::Mat getRawFrame();
+
+        DELETE_COPY(IpCameraReader);
+    };
+}
+
+#endif // OPENPOSE_PRODUCER_IP_CAMERA_READER_HPP

include/openpose/producer/videoCaptureReader.hpp

@@ -24,8 +24,9 @@ namespace op
         /**
          * This constructor of VideoCaptureReader wraps cv::VideoCapture(const std::string).
          * @param path const std::string indicating the cv::VideoCapture constructor string argument.
+         * @param producerType const std::string indicating whether the frame source is an IP camera or video.
          */
-        VideoCaptureReader(const std::string& path);
+        explicit VideoCaptureReader(const std::string& path, const ProducerType producerType);
 
         /**
          * Destructor of VideoCaptureReader. It releases the cv::VideoCapture member. It is virtual so that

include/openpose/producer/videoReader.hpp

@@ -7,15 +7,15 @@
 namespace op
 {
     /**
-     * VideoReader is a wrapper of the cv::VideoCapture class for video. It allows controlling a webcam (extracting frames,
-     * setting resolution & fps, etc).
+     * VideoReader is a wrapper of the cv::VideoCapture class for video. It allows controlling a video (e.g. extracting
+     * frames, setting resolution & fps, etc).
      */
     class OP_API VideoReader : public VideoCaptureReader
     {
     public:
         /**
-         * Constructor of VideoReader. It opens the video as a wrapper of cv::VideoCapture. It includes a flag to indicate
-         * whether the video should be repeated once it is completely read.
+         * Constructor of VideoReader. It opens the video as a wrapper of cv::VideoCapture. It includes a flag to
+         * indicate whether the video should be repeated once it is completely read.
          * @param videoPath const std::string parameter with the full video path location.
          */
         explicit VideoReader(const std::string& videoPath);

include/openpose/utilities/flagsToOpenPose.hpp

@@ -13,13 +13,16 @@ namespace op
     OP_API ScaleMode flagsToScaleMode(const int keypointScale);
 
     // Determine type of frame source
-    OP_API ProducerType flagsToProducerType(const std::string& imageDirectory, const std::string& videoPath, const int webcamIndex);
+    OP_API ProducerType flagsToProducerType(const std::string& imageDirectory, const std::string& videoPath,
+                                            const std::string& ipCameraPath, const int webcamIndex);
 
     OP_API std::shared_ptr<Producer> flagsToProducer(const std::string& imageDirectory, const std::string& videoPath,
-                                                     const int webcamIndex, const std::string& webcamResolution = "1280x720",
+                                                     const std::string& ipCameraPath, const int webcamIndex,
+                                                     const std::string& webcamResolution = "1280x720",
                                                      const double webcamFps = 30.);
 
-    OP_API std::vector<HeatMapType> flagsToHeatMaps(const bool heatMapsAddParts = false, const bool heatMapsAddBkg = false,
+    OP_API std::vector<HeatMapType> flagsToHeatMaps(const bool heatMapsAddParts = false,
+                                                    const bool heatMapsAddBkg = false,
                                                     const bool heatMapsAddPAFs = false);
 
     OP_API RenderMode flagsToRenderMode(const int renderFlag, const int renderPoseFlag = -2);

src/openpose/producer/CMakeLists.txt

 set(SOURCES
     defineTemplates.cpp
     imageDirectoryReader.cpp
+    ipCameraReader.cpp
     producer.cpp
     videoCaptureReader.cpp
     videoReader.cpp

src/openpose/producer/ipCameraReader.cpp

+#include <openpose/producer/ipCameraReader.hpp>
+
+namespace op
+{
+    // Public IP cameras for testing (add ?x.mjpeg):
+    // http://iris.not.iac.es/axis-cgi/mjpg/video.cgi?resolution=320x240?x.mjpeg
+    // http://www.webcamxp.com/publicipcams.aspx
+
+    IpCameraReader::IpCameraReader(const std::string & cameraPath) :
+        VideoCaptureReader{cameraPath, ProducerType::IPCamera},
+        mPathName{cameraPath}
+    {
+    }
+
+    std::string IpCameraReader::getFrameName()
+    {
+        try
+        {
+            return mPathName + "_" + VideoCaptureReader::getFrameName();
+        }
+        catch (const std::exception& e)
+        {
+            error(e.what(), __LINE__, __FUNCTION__, __FILE__);
+            return "";
+        }
+    }
+
+    cv::Mat IpCameraReader::getRawFrame()
+    {
+        try
+        {
+            return VideoCaptureReader::getRawFrame();
+        }
+        catch (const std::exception& e)
+        {
+            error(e.what(), __LINE__, __FUNCTION__, __FILE__);
+            return cv::Mat();
+        }
+    }
+}

src/openpose/producer/producer.cpp

@@ -41,7 +41,8 @@ namespace op
 
             if (isOpened())
             {
-                // If ProducerFpsMode::OriginalFps, then force producer to keep the frame rate of the frames producer sources (e.g. a video)
+                // If ProducerFpsMode::OriginalFps, then force producer to keep the frame rate of the frames producer
+                // sources (e.g. a video)
                 keepDesiredFrameRate();
                 // Get frame
                 frame = getRawFrame();
@@ -65,20 +66,24 @@ namespace op
     {
         try
         {
-            check(fpsMode == ProducerFpsMode::RetrievalFps || fpsMode == ProducerFpsMode::OriginalFps, "Unknown ProducerFpsMode.", __LINE__, __FUNCTION__, __FILE__);
-            // For webcam, ProducerFpsMode::OriginalFps == ProducerFpsMode::RetrievalFps, since the internal webcam cache will overwrite frames after it gets full
+            check(fpsMode == ProducerFpsMode::RetrievalFps || fpsMode == ProducerFpsMode::OriginalFps,
+                  "Unknown ProducerFpsMode.", __LINE__, __FUNCTION__, __FILE__);
+            // For webcam, ProducerFpsMode::OriginalFps == ProducerFpsMode::RetrievalFps, since the internal webcam
+            // cache will overwrite frames after it gets full
             if (mType == ProducerType::Webcam)
             {
                 mProducerFpsMode = {ProducerFpsMode::RetrievalFps};
                 if (fpsMode == ProducerFpsMode::OriginalFps)
-                    log("The producer fps mode set to `OriginalFps` (flag `process_real_time` on the demo) is not necessary, it is already assumed for webcam.",
+                    log("The producer fps mode set to `OriginalFps` (flag `process_real_time` on the demo) is not"
+                        " necessary, it is already assumed for webcam.",
                         Priority::Max, __LINE__, __FUNCTION__, __FILE__);
             }
             // If no webcam
             else
             {
                 check(fpsMode == ProducerFpsMode::RetrievalFps || get(CV_CAP_PROP_FPS) > 0,
-                      "Selected to keep the source fps but get(CV_CAP_PROP_FPS) <= 0, i.e. the source did not set its fps property.", __LINE__, __FUNCTION__, __FILE__);
+                      "Selected to keep the source fps but get(CV_CAP_PROP_FPS) <= 0, i.e. the source did not set"
+                      " its fps property.", __LINE__, __FUNCTION__, __FILE__);
                 mProducerFpsMode = {fpsMode};
             }
             reset(mNumberEmptyFrames, mTrackingFps);
@@ -118,12 +123,14 @@ namespace op
                 if (property == ProducerProperty::AutoRepeat)
                 {
                     check(value != 1. || (mType == ProducerType::ImageDirectory || mType == ProducerType::Video),
-                          "ProducerProperty::AutoRepeat only implemented for ProducerType::ImageDirectory and Video.", __LINE__, __FUNCTION__, __FILE__);
+                          "ProducerProperty::AutoRepeat only implemented for ProducerType::ImageDirectory and"
+                          " Video.", __LINE__, __FUNCTION__, __FILE__);
                 }
                 else if (property == ProducerProperty::Rotation)
                 {
                     check(value == 0. || value == 90. || value == 180. || value == 270.,
-                          "ProducerProperty::Rotation only implemented for {0, 90, 180, 270} degrees.", __LINE__, __FUNCTION__, __FILE__);
+                          "ProducerProperty::Rotation only implemented for {0, 90, 180, 270} degrees.",
+                          __LINE__, __FUNCTION__, __FILE__);
                 }
 
                 // Common operation
@@ -152,7 +159,8 @@ namespace op
             {
                 mNumberEmptyFrames = 0;
 
-                if (mType != ProducerType::ImageDirectory && (frame.cols != get(CV_CAP_PROP_FRAME_WIDTH) || frame.rows != get(CV_CAP_PROP_FRAME_HEIGHT)))
+                if (mType != ProducerType::ImageDirectory
+                      && (frame.cols != get(CV_CAP_PROP_FRAME_WIDTH) || frame.rows != get(CV_CAP_PROP_FRAME_HEIGHT)))
                 {
                     log("Frame size changed. Returning empty frame.", Priority::Max, __LINE__, __FUNCTION__, __FILE__);
                     frame = cv::Mat();
@@ -216,9 +224,12 @@ namespace op
         {
             if (isOpened())
             {
-                // OpenCV closing issue: OpenCV goes in the range [1, get(CV_CAP_PROP_FRAME_COUNT) - 1] in some videos (i.e. there is a frame missing),
-                // mNumberEmptyFrames allows the program to be properly closed keeping the 0-index frame counting
-                if (mNumberEmptyFrames > 2 || (mType != ProducerType::Webcam && get(CV_CAP_PROP_POS_FRAMES) >= get(CV_CAP_PROP_FRAME_COUNT)))
+                // OpenCV closing issue: OpenCV goes in the range [1, get(CV_CAP_PROP_FRAME_COUNT) - 1] in some
+                // videos (i.e. there is a frame missing), mNumberEmptyFrames allows the program to be properly
+                // closed keeping the 0-index frame counting
+                if (mNumberEmptyFrames > 2
+                    || (mType != ProducerType::IPCamera && mType != ProducerType::Webcam
+                        && get(CV_CAP_PROP_POS_FRAMES) >= get(CV_CAP_PROP_FRAME_COUNT)))
                 {
                     // Repeat video
                     if (mProperties[(unsigned char)ProducerProperty::AutoRepeat])
@@ -253,7 +264,9 @@ namespace op
                         const auto currentFrames = get(CV_CAP_PROP_POS_FRAMES) - mFirstFrameTrackingFps;
                         // Expected #frames
                         const auto nsPerFrame = 1e9/get(CV_CAP_PROP_FPS);
-                        const auto timeNs = (double)std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::high_resolution_clock::now()-mClockTrackingFps).count();
+                        const auto timeNs = (double)std::chrono::duration_cast<std::chrono::nanoseconds>(
+                            std::chrono::high_resolution_clock::now()-mClockTrackingFps
+                        ).count();
                         const auto expectedFrames = timeNs / nsPerFrame;
 
                         const auto difference = expectedFrames - currentFrames;
@@ -264,7 +277,8 @@ namespace op
                             if (difference > 15)
                             {
                                 set(CV_CAP_PROP_POS_FRAMES, std::floor(expectedFrames) + mFirstFrameTrackingFps);
-                                mNumberSetPositionTrackingFps = fastMin(mNumberSetPositionTrackingFps+1, numberSetPositionThreshold);
+                                mNumberSetPositionTrackingFps = fastMin(mNumberSetPositionTrackingFps+1,
+                                                                        numberSetPositionThreshold);
                             }
                             else
                             {
@@ -273,7 +287,8 @@ namespace op
                                     frame = getRawFrame();
                             }
                         }
-                        // Low down frame extraction - sleep thread unless it is too slow in most frames (using set(frames, X) sets to frame X+delta, due to codecs issues)
+                        // Low down frame extraction - sleep thread unless it is too slow in most frames (using
+                        // set(frames, X) sets to frame X+delta, due to codecs issues)
                         else if (difference < -0.45 && mNumberSetPositionTrackingFps < numberSetPositionThreshold)
                         {
                             const auto sleepMs = intRound( (-difference*nsPerFrame*1e-6)*0.99 );

src/openpose/producer/videoCaptureReader.cpp

@@ -11,7 +11,7 @@ namespace op
     {
         try
         {
-            // assert: make sure video capture was opened
+            // Make sure video capture was opened
             if (throwExceptionIfNoOpened && !isOpened())
                 error("VideoCapture (webcam) could not be opened.", __LINE__, __FUNCTION__, __FILE__);
         }
@@ -21,15 +21,20 @@ namespace op
         }
     }
 
-    VideoCaptureReader::VideoCaptureReader(const std::string& path) :
-        Producer{ProducerType::Video},
+    VideoCaptureReader::VideoCaptureReader(const std::string& path, const ProducerType producerType) :
+        Producer{producerType},
         mVideoCapture{path}
     {
         try
         {
-            // assert: make sure video capture was opened
+            // Make sure only video or IP camera
+            if (producerType != ProducerType::IPCamera && producerType != ProducerType::Video)
+                error("VideoCapture with an input path must be IP camera or video.",
+                      __LINE__, __FUNCTION__, __FILE__);
+            // Make sure video capture was opened
             if (!isOpened())
-                error("VideoCapture (video) could not be opened for path: '" + path + "'.", __LINE__, __FUNCTION__, __FILE__);
+                error("VideoCapture (IP camera/video) could not be opened for path: '" + path + "'.",
+                      __LINE__, __FUNCTION__, __FILE__);
         }
         catch (const std::exception& e)
         {

src/openpose/producer/videoReader.cpp

@@ -4,7 +4,7 @@
 namespace op
 {
     VideoReader::VideoReader(const std::string & videoPath) :
-        VideoCaptureReader{videoPath},
+        VideoCaptureReader{videoPath, ProducerType::Video},
         mPathName{getFileNameNoExtension(videoPath)}
     {
     }

src/openpose/utilities/flagsToOpenPose.cpp

 #include <cstdio> // sscanf
 #include <openpose/producer/imageDirectoryReader.hpp>
+#include <openpose/producer/ipCameraReader.hpp>
 #include <openpose/producer/videoReader.hpp>
 #include <openpose/producer/webcamReader.hpp>
 #include <openpose/utilities/check.hpp>
@@ -25,7 +26,8 @@ namespace op
             else if (poseModeString == "BODY_22")
                 return PoseModel::BODY_22;
             // else
-            error("String does not correspond to any model (COCO, MPI, MPI_4_layers)", __LINE__, __FUNCTION__, __FILE__);
+            error("String does not correspond to any model (COCO, MPI, MPI_4_layers)",
+                  __LINE__, __FUNCTION__, __FILE__);
             return PoseModel::COCO_18;
         }
         catch (const std::exception& e)
@@ -51,8 +53,9 @@ namespace op
             else if (keypointScale == 4)
                 return ScaleMode::PlusMinusOne;
             // else
-            const std::string message = "String does not correspond to any scale mode: (0, 1, 2, 3, 4) for (InputResolution,"
-                                        " NetOutputResolution, OutputResolution, ZeroToOne, PlusMinusOne).";
+            const std::string message = "String does not correspond to any scale mode: (0, 1, 2, 3, 4) for"
+                                        " (InputResolution, NetOutputResolution, OutputResolution, ZeroToOne,"
+                                        " PlusMinusOne).";
             error(message, __LINE__, __FUNCTION__, __FILE__);
             return ScaleMode::InputResolution;
         }
@@ -63,24 +66,33 @@ namespace op
         }
     }
 
-    ProducerType flagsToProducerType(const std::string& imageDirectory, const std::string& videoPath, const int webcamIndex)
+    ProducerType flagsToProducerType(const std::string& imageDirectory, const std::string& videoPath,
+                                     const std::string& ipCameraPath, const int webcamIndex)
     {
         try
         {
             log("", Priority::Low, __LINE__, __FUNCTION__, __FILE__);
             // Avoid duplicates (e.g. selecting at the time camera & video)
-            if (!imageDirectory.empty() && !videoPath.empty())
-                error("Selected simultaneously image directory and video. Please, select only one.", __LINE__, __FUNCTION__, __FILE__);
+            if (int(!imageDirectory.empty()) + int(!videoPath.empty()) + int(!ipCameraPath.empty()) > 1)
+                error("Selected simultaneously image directory, video and/or IP camera. Please, select only one.",
+                      __LINE__, __FUNCTION__, __FILE__);
             else if (!imageDirectory.empty() && webcamIndex > 0)
-                error("Selected simultaneously image directory and webcam. Please, select only one.", __LINE__, __FUNCTION__, __FILE__);
+                error("Selected simultaneously image directory and webcam. Please, select only one.",
+                      __LINE__, __FUNCTION__, __FILE__);
             else if (!videoPath.empty() && webcamIndex > 0)
-                error("Selected simultaneously video and webcam. Please, select only one.", __LINE__, __FUNCTION__, __FILE__);
+                error("Selected simultaneously video and webcam. Please, select only one.",
+                      __LINE__, __FUNCTION__, __FILE__);
+            else if (!ipCameraPath.empty() && webcamIndex > 0)
+                error("Selected simultaneously IP camera and webcam. Please, select only one.",
+                      __LINE__, __FUNCTION__, __FILE__);
 
             // Get desired ProducerType
             if (!imageDirectory.empty())
                 return ProducerType::ImageDirectory;
             else if (!videoPath.empty())
                 return ProducerType::Video;
+            else if (!ipCameraPath.empty())
+                return ProducerType::IPCamera;
             else
                 return ProducerType::Webcam;
         }
@@ -91,18 +103,21 @@ namespace op
         }
     }
 
-    std::shared_ptr<Producer> flagsToProducer(const std::string& imageDirectory, const std::string& videoPath, const int webcamIndex,
+    std::shared_ptr<Producer> flagsToProducer(const std::string& imageDirectory, const std::string& videoPath,
+                                              const std::string& ipCameraPath, const int webcamIndex,
                                               const std::string& webcamResolution, const double webcamFps)
     {
         try
         {
             log("", Priority::Low, __LINE__, __FUNCTION__, __FILE__);
-            const auto type = flagsToProducerType(imageDirectory, videoPath, webcamIndex);
+            const auto type = flagsToProducerType(imageDirectory, videoPath, ipCameraPath, webcamIndex);
 
             if (type == ProducerType::ImageDirectory)
                 return std::make_shared<ImageDirectoryReader>(imageDirectory);
             else if (type == ProducerType::Video)
                 return std::make_shared<VideoReader>(videoPath);
+            else if (type == ProducerType::IPCamera)
+                return std::make_shared<IpCameraReader>(ipCameraPath);
             else if (type == ProducerType::Webcam)
             {
                 // cameraFrameSize
@@ -110,7 +125,8 @@ namespace op
                 if (webcamIndex >= 0)
                 {
                     const auto throwExceptionIfNoOpened = true;
-                    return std::make_shared<WebcamReader>(webcamIndex, webcamFrameSize, webcamFps, throwExceptionIfNoOpened);
+                    return std::make_shared<WebcamReader>(webcamIndex, webcamFrameSize, webcamFps,
+                                                          throwExceptionIfNoOpened);
                 }
                 else
                 {
@@ -118,10 +134,12 @@ namespace op
                     std::shared_ptr<WebcamReader> webcamReader;
                     for (auto index = 0 ; index < 10 ; index++)
                     {
-                        webcamReader = std::make_shared<WebcamReader>(index, webcamFrameSize, webcamFps, throwExceptionIfNoOpened);
+                        webcamReader = std::make_shared<WebcamReader>(index, webcamFrameSize, webcamFps,
+                                                                      throwExceptionIfNoOpened);
                         if (webcamReader->isOpened())
                         {
-                            log("Auto-detecting camera index... Detected and opened camera " + std::to_string(index) + ".", Priority::High);
+                            log("Auto-detecting camera index... Detected and opened camera " + std::to_string(index)
+                                + ".", Priority::High);
                             return webcamReader;
                         }
                     }
@@ -139,7 +157,8 @@ namespace op
         }
     }
 
-    std::vector<HeatMapType> flagsToHeatMaps(const bool heatMapsAddParts, const bool heatMapsAddBkg, const bool heatMapsAddPAFs)
+    std::vector<HeatMapType> flagsToHeatMaps(const bool heatMapsAddParts, const bool heatMapsAddBkg,
+                                             const bool heatMapsAddPAFs)
     {
         try
         {

windows/OpenPose/OpenPose.vcxproj

@@ -295,6 +295,7 @@
     <ClCompile Include="..\..\src\openpose\pose\renderPose.cpp" />
     <ClCompile Include="..\..\src\openpose\producer\defineTemplates.cpp" />
     <ClCompile Include="..\..\src\openpose\producer\imageDirectoryReader.cpp" />
+    <ClCompile Include="..\..\src\openpose\producer\ipCameraReader.cpp" />
     <ClCompile Include="..\..\src\openpose\producer\producer.cpp" />
     <ClCompile Include="..\..\src\openpose\producer\videoCaptureReader.cpp" />
     <ClCompile Include="..\..\src\openpose\producer\videoReader.cpp" />

windows/OpenPose/OpenPose.vcxproj.filters

@@ -544,24 +544,6 @@
     <ClCompile Include="..\..\src\openpose\thread\defineTemplates.cpp">
       <Filter>Source Files\thread</Filter>
     </ClCompile>
-    <ClCompile Include="..\..\src\openpose\producer\defineTemplates.cpp">
-      <Filter>Source Files\producer</Filter>
-    </ClCompile>
-    <ClCompile Include="..\..\src\openpose\producer\imageDirectoryReader.cpp">
-      <Filter>Source Files\producer</Filter>
-    </ClCompile>
-    <ClCompile Include="..\..\src\openpose\producer\producer.cpp">
-      <Filter>Source Files\producer</Filter>
-    </ClCompile>
-    <ClCompile Include="..\..\src\openpose\producer\videoCaptureReader.cpp">
-      <Filter>Source Files\producer</Filter>
-    </ClCompile>
-    <ClCompile Include="..\..\src\openpose\producer\videoReader.cpp">
-      <Filter>Source Files\producer</Filter>
-    </ClCompile>
-    <ClCompile Include="..\..\src\openpose\producer\webcamReader.cpp">
-      <Filter>Source Files\producer</Filter>
-    </ClCompile>
     <ClCompile Include="..\..\src\openpose\gui\defineTemplates.cpp">
       <Filter>Source Files\gui</Filter>
     </ClCompile>
@@ -730,6 +712,27 @@
     <ClCompile Include="..\..\src\openpose\pose\renderPose.cpp">
       <Filter>Source Files\pose</Filter>
     </ClCompile>
+    <ClCompile Include="..\..\src\openpose\producer\defineTemplates.cpp">
+      <Filter>Source Files\producer</Filter>
+    </ClCompile>
+    <ClCompile Include="..\..\src\openpose\producer\imageDirectoryReader.cpp">
+      <Filter>Source Files\producer</Filter>
+    </ClCompile>
+    <ClCompile Include="..\..\src\openpose\producer\ipCameraReader.cpp">
+      <Filter>Source Files\producer</Filter>
+    </ClCompile>
+    <ClCompile Include="..\..\src\openpose\producer\producer.cpp">
+      <Filter>Source Files\producer</Filter>
+    </ClCompile>
+    <ClCompile Include="..\..\src\openpose\producer\videoCaptureReader.cpp">
+      <Filter>Source Files\producer</Filter>
+    </ClCompile>
+    <ClCompile Include="..\..\src\openpose\producer\videoReader.cpp">
+      <Filter>Source Files\producer</Filter>
+    </ClCompile>
+    <ClCompile Include="..\..\src\openpose\producer\webcamReader.cpp">
+      <Filter>Source Files\producer</Filter>
+    </ClCompile>
   </ItemGroup>
   <ItemGroup>
     <None Include="..\..\include\openpose\utilities\cuda.hu">