[G-API] Handle exceptions in streaming executor

* Handle exceptions in streaming executor

* Rethrow exception in non-streaming executor

* Clean up

* Put more tests

* Handle exceptions in IE backend

* Handle exception in IE callbacks

* Handle exception in GExecutor

* Handle all exceptions in IE backend

* Not only (std::exception& e)

* Fix comments to review

* Handle input exception in generic way

* Fix comment

* Clean up

* Apply review comments

* Put more comments
* Fix alignment
* Move test outside of HAVE_NGRAPH

* Fix compilation

G-API: Suppress ngraph warnings for MSVS2015 (4268)

* 4268 are suppressed

* 1900-->>1910 (v140)

* Two more warnings from ngraph include

G-API: Removing G-API test code that is a reflection of ts module

* gapi: don't hijack testing infrastructure

* Removed initDataPath functionality (ts module exists)

* Removed false for ocv_extra data from findDataFile
Co-authored-by: NAlexander Alekhin <alexander.a.alekhin@gmail.com>

[G-API] Enable 2D inputs for GIEBackend

* Enable 2D inputs

* Fix typo

[G-API] Fix bugs in GIEBackend

* Remove inputs/outputs map from IEUnit

* Add test

* Add NV12 test

* Reorganize setBlob function

* Check that backend don't overwrite blob precision

* Stop setting config to global IE::Core

* Replace mutable to const_cast

* Update modules/gapi/test/infer/gapi_infer_ie_test.cpp

* Update modules/gapi/test/infer/gapi_infer_ie_test.cpp

* Make blob parameter as const ref

* Cosmetic fixes

* Fix failed test on inferROI

* Removed double ref for ii

* Disable tests

* Skip tests if device not available

* Use Sync prim under shared_ptr to avoid issue on MAC

* Apply WA for IE::Core

* Apply WA for MAC build

* Try to apply another WA

* Not release IE::Core for apple

* Put comment

* Support PreprocInfo for

* InferROI
* InferList
* InferList2

* Remove empty line

* Fix alignment
Co-authored-by: NMaxim Pashchenkov <maxim.pashchenkov@intel.com>

G-API: Extend ie::Params to specify batch size

* Add cfgBatchSize to ie::Params

* Fix comments to review

G-API: Extend MediaFrame to be able to extract additional info besides access

* Extend MediaFrame to be able to extract additional info besides access

* Add default implementation for blobParams()

* Add comment on the default blobParams()

[G-API] Support generic infer overloads

* Overloads for generic infer

* Fix build

* Refactoring

* Fix docs

* Put extra stuff to detail namespace

* Add doc for usings

* Remove uneccessary template in Priv

G-API: Implement async version for InferList & Infer2

* Implement async version for InferList & Infer2

* Fix warning

* Fix bug with roi ordering

* Post input meta instead of empty

* Fix comments to review

[G-API]: Adding reshape for CNN input.

* Added CNN input IE reshape

* rbs

* Added unordered_set instead vector

* Alignment

[G-API] Support multiple asynchronous requests

* Support nireq option

* Disable tests to check CI

* Fix bug with hanging

* WA to green CI

* Snapshot

* Simplify RequestPool

* Add default values to id

* Fix win warning

[G-API] Async infer request hotfix

* Fix hanging on empty roi list

* Prevent possible data race

* Clean up

[G-API] Implement async infer

* Implement async infer

* Fix typo

 - refactor for consistency

[G-API] Support GFrame for infer

* GInfer(GFrame), currently broken

* Fixed (API only)

* Support GFrame in GIEBackend

* Fix comments to review

* Fix comments to review

* Fix doxygen

* Fix building with different IE versions

* Fix warning on MacOS
Co-authored-by: NDmitry Matveev <dmitry.matveev@intel.com>
Co-authored-by: NSmirnov Alexey <alexey.smirnov@intel.com>

Expand ie::Params to support config

* Add config to IE params

* Add test

* Remove comments from tests

* Rename to pluginConfig

* Add one more overloads for pluginConfig

* Add more tests

[G-API] Introduce generic version for cv::gapi::infer

* Introduce generic infer

* Move Generic to infer.hpp

* Removew num_outs

* Fix windows warnings

* Fix comments to review

* Fix doxygen

* Add comment

* Fix comments to review

* standoalone ifdef in ginfer.cpp

* Fix test

GAPI: Migration to IE Core API

* Migration to IE Core API
 - both versions are maintained
 - checked building with all the OpenVINO versions (2019.R1, R2, R3, 2020.4 (newest))

* commit to awake builders

* Addressing comments
 - migrated to Core API in 'gapi_ie_infer_test.cpp'
 - made Core a singleton object
 - dropped redundant steps

* Addressing comments
 - modified Mutex locking

* Update

* Addressing comments
 - remove getInitMutex()
 - reduce amount of #ifdef by abstracting into functions

* return to single IE::Core

* Divide functions readNet and loadNet to avoid warnings on GCC

* Fix deprecated code warnings

* Fix deprecated code warnings on CMake level

* Functions wrapped
 - All the functions depended on IE version wrapped into a cv::gapi::ie::wrap namesapace
 - All this contained to a new "giebackend/gieapi.hpp" header
 - The header shared with G-API infer tests to avoid code duplications

* Addressing comments
 - Renamed `gieapi.hpp` -> `giewrapper.hpp`, `cv::gapi::ie::wrap` -> `cv::gimpl::ie::wrap`
 - Created new `giewrapper.cpp` source file to avoid potential "multiple definition" problems
 - removed unnecessary step SetLayout() in tests

* Enabling two NN infer teest

* Two-NN infer test change for CI
 - deleted additional network
 - inference of two identical NN used instead

* Fix CI fileNotFound

* Disable MYRIAD test not to fail Custom CI runs

* G-API: Introduce a new gapi::infer2 overload + gaze estimation sample

* G-API/infer2: Introduced static type checking for infer2

- Also added extra tests on the type check routine

* G-API/infer2: Addressed self-review comments in the sample app

- Also fix build on Linux;

* G-API/infer2: Remove incorrect SetLayout(HWC) + dead code

- Also fixed comments in the backend

* G-API/infer2: Continue with self-review

- Fix warnings/compile errors in gaze estimation
- Dropped the use of RTTI (VectorRef::holds()) from the giebackend
- Replaced it with a trait-based enums for GArray<T> and std::vector<T>
- The enums and traits are temporary and need to be unified with
  the S11N when it comes

* G-API/infer2: Final self-review items

- Refactored ROIList test to cover 70% for infer<> and infer2<>;
- Fixed the model data discovery routine to be compatible with new
  OpenVINO;
- Hopefully fixed the final issues (warnings) with the sample.

* G-API/infer2: address review problems

- Fixed typo in comments;
- Fixed public (Doxygen) comment on GArray<GMat> input case for infer2;
- Made model lookup more flexible to allow new & old OMZ dir layouts.

* G-API/infer2: Change the model paths again

* G-API/infer2: Change the lookup path for test data

* G-API/infer2: use randu instead of imread. CI war is over

* G-API-NG/Streaming: Introduced a Streaming API

Now a GComputation can be compiled in a special "streaming" way
and then "played" on a video stream.

Currently only VideoCapture is supported as an input source.

* G-API-NG/Streaming: added threading & real streaming

* G-API-NG/Streaming: Added tests & docs on Copy kernel

- Added very simple pipeline tests, not all data types are covered yet
  (in fact, only GMat is tested now);
- Started testing non-OCV backends in the streaming mode;
- Added required fixes to Fluid backend, likely it works OK now;
- Added required fixes to OCL backend, and now it is likely broken
- Also added a UMat-based (OCL) version of Copy kernel

* G-API-NG/Streaming: Added own concurrent queue class

- Used only if TBB is not available

* G-API-NG/Streaming: Fixing various issues

- Added missing header to CMakeLists.txt
- Fixed various CI issues and warnings

* G-API-NG/Streaming: Fixed a compile-time GScalar queue deadlock

- GStreamingExecutor blindly created island's input queues for
  compile-time (value-initialized) GScalars which didn't have any
  producers, making island actor threads wait there forever

* G-API-NG/Streaming: Dropped own version of Copy kernel

One was added into master already

* G-API-NG/Streaming: Addressed GArray<T> review comments

- Added tests on mov()
- Removed unnecessary changes in garray.hpp

* G-API-NG/Streaming: Added Doxygen comments to new public APIs

Also fixed some other comments in the code

* G-API-NG/Streaming: Removed debug info, added some comments & renamed vars

* G-API-NG/Streaming: Fixed own-vs-cv abstraction leak

- Now every island is triggered with own:: (instead of cv::)
  data objects as inputs;
- Changes in Fluid backend required to support cv::Mat/Scalar were
  reverted;

* G-API-NG/Streaming: use holds_alternative<> instead of index/index_of test

- Also fixed regression test comments
- Also added metadata check comments for GStreamingCompiled

* G-API-NG/Streaming: Made start()/stop() more robust

- Fixed various possible deadlocks
- Unified the shutdown code
- Added more tests covering different corner cases on start/stop

* G-API-NG/Streaming: Finally fixed Windows crashes

In fact the problem hasn't been Windows-only.
Island thread popped data from queues without preserving the Cmd
objects and without taking the ownership over data acquired so when
islands started to process the data, this data may be already freed.

Linux version worked only by occasion.

* G-API-NG/Streaming: Fixed (I hope so) Windows warnings

* G-API-NG/Streaming: fixed typos in internal comments

- Also added some more explanation on Streaming/OpenCL status

* G-API-NG/Streaming: Added more unit tests on streaming

- Various start()/stop()/setSource() call flow combinations

* G-API-NG/Streaming: Added tests on own concurrent bounded queue

* G-API-NG/Streaming: Added more tests on various data types, + more

- Vector/Scalar passed as input;
- Vector/Scalar passed in-between islands;
- Some more assertions;
- Also fixed a deadlock problem when inputs are mixed (1 constant, 1 stream)

* G-API-NG/Streaming: Added tests on output data types handling

- Vector
- Scalar

* G-API-NG/Streaming: Fixed test issues with IE + Windows warnings

* G-API-NG/Streaming: Decoupled G-API from videoio

- Now the core G-API doesn't use a cv::VideoCapture directly,
  it comes in via an abstract interface;
- Polished a little bit the setSource()/start()/stop() semantics,
  now setSource() is mandatory before ANY call to start().

* G-API-NG/Streaming: Fix STANDALONE build (errors brought by render)

* - headers in "infer/" and "infer/ie/" folders are included into gapi_ext_hdrs;
+ because of that a few #includes are required in the headers
- HAVE_INF_ENGINE flag check in headers "infer/ie.hpp" and "infer/ie/util.hpp" is deleted

* - the "ie/util.hpp" header is a private header now as it's used for tests; it's been moved to the scr directory to the place next to the implementation file "ie/giebackend.cpp"
- the path to this header in files "ie/giebackend.cpp" and "test/infer/gapi_infer_ie_test.cpp" is updated
- As it's private header now and explicitly depends on IE, the "HAVE_INF_ENGINE" flag check is returned

* G-API-NG/API: Introduced inference API and IE-based backend

- Very quick-n-dirty implementation
- OpenCV's own DNN module is not used
- No tests so far

* G-API-NG/IE: Refined IE backend, added more tests

* G-API-NG/IE: Fixed various CI warnings & build issues + tests

- Added tests on multi-dimensional own::Mat
- Added tests on GMatDesc with dimensions
- Documentation on infer.hpp
- Fixed more warnings + added a ROI list test
- Fix descr_of clash for vector<Mat> & standalone mode
- Fix build issue with gcc-4.8x
- Addressed review comments

* G-API-NG/IE: Addressed review comments

- Pass `false` to findDataFile()
- Add deprecation warning suppression macros for IE